Non-Destructive Testing of Fruit Firmness with Real-Time constraints Christopher Mills Supervisors: Dr. Andrew Paplinski Mr Charles Greif.

Non-Destructive Testing of Fruit Firmness with Real-Time constraints

Christopher MillsSupervisors: Dr. Andrew Paplinski

Mr Charles Greif

2

Contents

Research aims

Fruit Firmness

Non-destructive testing (NDT)

Methods

Completed work

Future work

Conclusions

References

3

Project Aims With our background research in Ultrasonic imaging, the

aim is to design a simple system that will grade fruit firmness using NDT

And as part DigSys we are interested in an ASIC application of these algorithms. They can execute up to one hundred times faster in hardware.

Ensure that the system could be used in an industrial setting, i.e. testing fruit on a rapidly moving conveyer belt.

Work within hard real time constraints (ie 10 fruit/sec) Be able to test fruit without actual contact with the skin of fruit (is this

possible?)

4

Fruit FirmnessDefinition of fruit firmness – mechanical rigidity of fruit cell

structure. It can be measured by conventional means; stress testing, Magness-Taylor Probing

Measurement of Fruit Firmness is important because Firmness affects the perception of enjoyment of food. Perception of firmness is linked to freshness and the ripeness of

fruit. Such perception may be of greater importance for the preparation

of fruit for later consumption. (Preservation: canning, preserve/jam, etc)

Humans decide fruit firmness in a variety of ways Feel/look as fruit is consumed. Response to preparation/cooking.

5

Fruit Firmness (cont)

Biological factors of Fruit Firmness Cell size/shape Cell water content Cell organization

Firmness varies with Fruit type (apple, orange) Fruit Age (under ripe, over ripe) Conditions during maturation and storage

The image on the right, shows what apple cells look like at high magnification, the boundaries between the cells are visible.

Image of boiled apple cells at 100x magnification

6

Fruit Firmness (cont)

Ultrasonic reflection can be used to measure firmness. it will be the gaps between cells that will best

respond to ultrasound and describe firmness. The image to the right is a representation of a

fruits internal structure. Fruit firmness varies with ripeness and time,

going from firm and unripe to soft and ripe or overripe.

The reason for this is that chemical changes within the fruit change the way the cells inside interact and the chemical composition within the fruit, eg starch being converted into sugars.

7

Fruit Firmness (cont)

Fruit firmness testing is critical to industries involved in the sorting and grading of fruit. As sorting can be done based on fruit firmness measures.

For the duration of this project, a company called Colour Vision Systems (CVS) will be providing sponsoring for this project. CVS build large scale fruit

sorting machines, including computational circuits for automated sorting based on vision for blemish detection and near infrared for sugar content evaluation.

8

Non-Destructive Testing NDT methods of testing are used on mechanical structures

while they are in use or before use – and the structures can continue to be used post testing.

Various modalities of NDT exist, such as Sound methods (ultrasound, acoustic, etc) Wave energy response (laser, infrared, x-ray) Vision (Video camera’s) Physical Response to small force (Laser air puff, bounce test, micro-

deformation)

Many researchers have attempted to develop methods for fruit firmness testing. For the next few slides I will detail some of these.

9

NDT-Examples, Laser scatter imagingKang et al attempted to use laser-scatter imaging to grade

quality of tomatoes.

The method is reasonably simple, a laser beam is fired through a piece of fruit/vegetable, the scatter of the laser beam is recorded by a camera, and the extent of the scatter is an indication of quality.

10

NDT-Examples, Laser Air PuffMcGlone et al describes a method based on the laser air puff

test.

The laser air puff test uses deformation in the target caused by air under pressure, this deformation is measured by a laser.

It was found that while this method was reasonably accurate on average, there was an issue with confidence and resolution when testing firm fruit due to the decreased measurable deformation.

11

NDT-Examples, Bounce TestDelwiche et al attempted to build a fruit sorter based on the

impact force (or Bounce) testing method.

Based on previous work by the same researchers, built a system where fruit would fall with a speed of 76.7 cm/s.

The force measurement was made by a force transducer mounted vertically on a large steel mass or impact mass. The fruit was dropped from a conveyor belt. Overall, the system could process fruit at 5 fruit/s.

While the system was capable of sorting fruit based on firmness, the error rate was high, 26% for peaches.

For this research, we will concentrate on ultrasonic methods to measure fruit firmness due to our experience in the area.

12

NDT-Examples, AcousticPeleg et al built a fruit firmness sorter based on the principles of

acoustic energy. A small electrodynamic shaker, vibrates the bottom of the fruit The root mean square (RMS) level of the input signal Xi is

measured in the shaker head The output RMS signal level Xo is measured by a miniature

accelerometer attached to the top part of the fruit. A Firmness index PFT is defined by: PFT=X0/(X0-Xi).

Overall, the system performed well with reasonably high confidence and repeatability (>80%).

13

NDT-Examples, AcousticThe picture on the right

shows the ‘sensor wheel’. Fruit moves along the

conveyor Then it’s grabbed by the

acoustic transducers The fruit is held and tested

until it reaches the lower conveyer

The fruit is tested at a rate of 7.5 fruit/s per lane.

14

NDT-Examples, Acoustic The table to the right

shows some values of PFT vs Penetrometer force It shows that the measure

PFT is related to the force measured by the penetrometer

If the fruit is stored in a Controlled Atmosphere, the Penetrometer and PFT show similar increase in reading

15

NDT-Examples, UltrasonicMizrach et al attempted to estimate fruit

qualities from a Ultrasonic measure of fruit firmness

The system used two transducers, one as receiver, the other as a transmitter

The resulting signal was processed The Frequency response Analysed And the speed of sound through the target

measured

The experiment focused on Mangos as the test subject

Representation of the system

16

NDT-Examples, UltrasonicThe graphs on the right show

the received signal and the Fourier transform that of that signal.

The results were compared to known values of firm and soft fruits and a firmness measure made based on the comparison.

The accuracy of this method is reasonably high.

17

NDT-Examples, Ultrasonic The scatter plots here

represent the accuracy of the system

The table below gives a value called the Standard Error of Calibration (SEC)

18

NDT - UltrasoundBasics of Ultrasonic testing Required equipment

Transmitter and Receiver transducers Pulsar/Receiver unit External/internal microcomputer to store

results and control Pulsar/Receiver

Operation Pulsar/receiver applies voltage to the

transmitter Transmitter vibrates and creates high

frequency sound Ultrasound reflects whenever a change in

density occurs. Receiver responds to sound and sends a

voltage based on the amplitude of received signal

19

NDT - UltrasoundHowever, there is a problems with using Ultrasound. The most

common method of ultrasound is called ‘contact using liquid immersion’. This is a problem because…

In an automatic system, contact with the fruit could be awkward and expensive.

Application of conducting liquid could also be awkward.

One possible answer is to use Non-Contact Ultrasound (NCU). The system is very similar to liquid contact except

The Transducers do not contact the target Noise due to lack of contact

large reflections caused by sound waves entering target considered as noise

To reduce reflection from transducer to air, an acoustic lens is used.

20

NDT – Ultrasound (NCU)

The above image shows the behaviour of ultrasonic waves using NCU.

21

Research Method Empirically determine response of the cellular structure of fruit to

ultrasound Possibly use Field 2, which can produce images based on

simulation values or real readings from an ultrasonic system

However, we do not require images, just an overall characterization of fruit firmness

Devise a Neural Network structure or other type of system that is capable of determining fruit firmness (e.g. statistical methods) based on the training data. Early testing of Neural Net to be done in Matlab

This is an example of Field 2 taking a source image and simulating how it would look through ultrasonic testing. The same could be done with a mock up of fruit internals.

22

Proposed system Use Ultrasound on fruit via non-contact transducers to measure

fruit firmness.

Process Ultrasound response via a neural network that will require training for each available fruit type, and evaluate fruit firmness.

Integrate with existing system manufactured by CVS such as a vision system to detect blemishes (Some blemishes are

caused by fruit diseases that would effect firmness also) Weight and volume information (fruit density could prove useful in

determining fruit firmness)

23

Proposed systemThe card to the right is called the

OPCARD. It is a PCI add on card It is an Oscilloscope card designed

for ultrasound It has an 8bit DAC Highly Configurable

12.5MHz … 100MHz SampF High pass and low pass filters

The Transducer shown here is the AT50 from Airmar

Air contact transducer Output signal Frequency of 50MHz

24

Work Completed Research into Non-Contact Ultrasound (NCU)

Based on what I have learned, NCU is a very appropriate technology for this application. However, it is a relatively new method compared to liquid immersion ultrasound, and apparently despite its advantages not widely used so sourcing NCU transducers has been difficult.

Classification system At this stage, a neural network is the most likely system to use for

classification of Fruit Firmness Other systems are possible, such as pattern recognition methods

including statistical analysis.

Physical arrangement of system Some ideas have been discussed, such as the angle between the

emitter and receiver(s) Angles of transducers to fruit surface

25

Future Work Testing of various methods including

Acoustic/ultrasound Determine accuracy of NCU

Machine Vision Laser Air-puff Non-destructive deformation Sensor Fusion

Construction of a system based on results of testing

26

Conclusions Ultrasonic testing can grade firmness with sufficient accuracy.

NCU is applicable in most situations where the more common liquid contact Ultrasonic testing methods are used.

Sensor fusion is a sensible option in fruit firmness testing.

27

ReferencesTexture - http://www.ba.ars.usda.gov/hb66/021texture.pdf

Evolution Of Piezoelectric Transducers To Full Scale Non-Contact Ultrasonic Analysis Mode - http://www.ultrangroup.com/pdfs/WCNDT-NCU-64.pdf

Non-Contact Ultrasound: The Last Frontier In Non-Destructive Testing And Evaluation - http://www.ultrangroup.com/pdfs/esm1.pdf

Field 2 - http://www.es.oersted.dtu.dk/staff/jaj/field/index.html