Problem Statement
Our client is an OEM in the inspection industry and they manufacture high-speed sorting and grading machines for industrial applications. The end users of the sorting machines expect very high throughput with very few sorting defects (Both false positives and false negatives). The cycle time available for completing the process starting from sorting decision to actuation of pneumatic ejectors is < 1 ms. The microcontrollers/DSP in the current generation machines have the processing capacity only to perform simple thresholding techniques in such short cycle time thus restricting the quality of inspection possible in these machines. The problem we had to solve was to improve the image processing capabilities of the Sorting Machines within the same cycle time.
Functional Requirement
Enhance the quality inspection from simple thresholding to colour-based defects, size-based defects and shape-based defects within the same cycle time.
Solution Design
Image Capture:
As this application is high speed and requires high resolution image capture the solution was designed with Line Scan Sensors. The output of the sensor was designed to return 2048 pixels for every line scan at the speed of 10 Khz.
Processor Selection:
One of the key challenges was the selection of an appropriate processor that has the capacity to run advanced vision algorithms within a cycle time of < 1 ms.
In order to run advanced image processing algorithms in such short cycle times we required a hardware-based solution but we wanted to retain the flexibility to upgrade the algorithm. We narrowed down our selection to XILINX Zynq Processor because it combines the parallel nature of FPGA hardware and flexibility of adding software functionality on the ARM engine. We developed and tested the algorithms in higher level language (C++) and used XILINX Vivado HLS tools to synthesis the code to HDL. One more advantage of using XILINX Zynq was that we could use PYNQ to test our algorithms quickly before we deployed a prototype of the platform at the customer site.
Parallelism on FPGA
FPGAs are inherently parallel because it implements the logic required by an application by building separate hardware for each function. This gives FPGAs the speed that results from a hardware design while retaining the reprogrammable flexibility of software at a relatively low cost. This makes FPGAs well suited to image processing, particularly at the low and intermediate levels where they are able to exploit the possibility of pipelining the operators in image processing.
Image Processing Algorithm:
We enhanced the image processing from simple colour thresholding to spectral analysis based on pixel intensity.
The sorting decision is made basis the distribution of the pixel intensity of the object. In this case the object is rejected if there is no single peak and the pixel intensity distribution has more than one peak or if there is significant count of pixels across intensity values.
We have optimized the algorithm to suit FPGA hardware when synthesized using HLS. We have also taken advantage of the inherent parallelism in images to complete the enhanced image processing algorithm within the same cycle time.
Results Achieved
We enhanced the solution to handle 50 K + sorting decisions / sec with improved defect identification capability using Spectral Analysis. The sorting machines are now capable of handling more than one kind of defect within the same cycle time.
For more details on HLS for Image Processing on Zynq check blog post of Adam Taylor on hackster.io
MicroZed Chronicles: xfOpenCV & HLS