Artificial Intelligence (AI) is increasingly being used to solve automation issues. Self-learning algorithms are making it easier to eliminate specific but monotonous tasks. Despite the fact that you can use AI technologies for all kinds of tasks, such as speech recognition and text recognition, it is not always the ideal solution for every issue. At Pegamento, for instance, we have dealt with several situations where this was the case. We sat down with Computer Vision & AI Lead Thomas de Wolf to discuss why artificial intelligence is not always the best solution for every challenge.
For this story, we look at the printing industry. An example here is an organisation that prints personalised packaging, such as Tony's Chocolonely, where you can design your own wrapper. Hundreds, maybe thousands, of different types of chocolate wrappers are made, and these need to be transported in boxes to the right address. This type of situation occurs in many organisations within the printing industry. While preparing orders, organisations in the print industry often manually check the products in and the picture (image label) on a box on a conveyor belt and then stick the correct order information on the box. In a given box, for example, there may be chocolate bars with personalised wrappers for a specific customer, and these each need to be marked with the correct stickers. And this needs to happen for thousands of units per day.
Computer vision and Artificial Intelligence behind the scenes
'It's like a baby, who at some point learns that he can grasp something with his hands. He finds out that it is useful and learns from it.'
An automation issue in this context may be that organisations in the printing industry are looking for a way to automate manual checks. One application is a camera system that sees the box on the conveyor belt and recognises the picture on the box. The system compares that picture with the database to then confirm: 'these are Tony's Chocolonely milk chocolate bars and not the dark or hazelnut bars'. ''This is an issue where you don't know in advance what will pass in front of the camera, but you want to recognise it when it does,'' Thomas says. ''For this kind of problem, we show the computer 500 examples of the milk, 500 of the dark, and 500 of the hazelnut wrappers. We then have three types of chocolate bars of which we have 500 images of each wrapper type on a white box in the database.''
With all those examples, the technology is going to train a learning algorithm. ''The software tries to recognise the milk chocolate label on the box, and then it checks whether it has become good at this,'' Thomas explains. ''If it hasn't, it changes all sorts of little numbers and tries again. It's like a baby, who at some point learns that he can grasp something with his hands. He finds out that it is useful and learns from it. That's how a learning algorithm works, too.''
Practically speaking, you give the computer many examples and instruct the software to train based on those examples. After a while, the network will have learned to recognise the Tony's milk chocolate label on the white box. If you then show it a new picture of a box with a label on it, the system will recognise that it is the milk chocolate label and stick the correct order information on it.
The picture here shows an example of image recognition in combination with AI. The camera has learned to recognise a car.
Back to classic image recognition: feature matching
'If new types of labels are constantly being introduced, how can a computer tell that it is a label and not the box?’
Yet AI is not always the best solution to this automation issue. This is because organisations in the printing industry don't have just three categories of labels, but rather many varieties in different sizes, shapes, and colours. This is especially true if you have a business that is similar to Tony's Chocolonely, where you are able to design wrappers, labels, or packaging yourself. Such as a personalised Pegamento chocolate bar wrapper, for example. This means that you may end up with as much as a thousand or more categories. ''This is a typical problem where you don't want to resort to a learning network, but where you have to look specifically at what makes all those labels different,'' Thomas points out. ''Training a network that knows what a label is, but that doesn't know what the milk chocolate label is, doesn't work either. Because when new types of labels come along again, the computer can start to see the label as the box. With such an issue, you have to fall back on classic image recognition: feature matching.''
Feature matching
"First, we write the software so that it recognises that everything that is not background must be the white box. And everything that is not white is then the label. We then extract the outline of the label. We cut that piece out, and then you get to the feature matching part."
In short, feature matching means that an algorithm looks at specific pixels of an image to start comparing them with other images in the database. For example, it grabs a small piece of the wrapper of the Pegamento chocolate bar from the label and compares that pixel group with all the other images in the database. ''Suppose you have a design of a Pegamento wrapper, a Nike wrapper, and an IKEA wrapper. The algorithm will check whether that pixel group appears anywhere. It does that for approximately 1000 pixel groups,'' Thomas says, while showing what this looks like using an image. ''It looks to see which image matches best. That way it can tell, without ever having seen anything beforehand, that something is similar to something else. And that is the trick behind feature matching. That's how the computer knows which order information should be on the box.''
As a result, organisations in the printing industry no longer have to check manually to see what is in and on the box to stick the correct order information on it. This allows an organisation to create a more efficient production process with the same manpower.
For all companies involved in customised printing
Another example Thomas mentions during the conversation is photo gifts on which you can have a photo printed. The types of organisations that print photos on products usually place them in the oven in batches of 400. When they come out of the oven, it is often difficult to trace which product belongs to which order. You only see, for example, a picture of a family on the product. Based on that image, someone has to manually retrieve the corresponding order. ''With something like this, you could also use a camera with feature matching,'' Thomas points out. You place the product with the image in front of the camera and the computer looks for a match based on all those pixel groups. Easy as that! With this technique, you so specifically deduce what something is based on images that it actually becomes as simple as scanning a barcode.''
AI is not the best solution in this case either. Feature matching works better here, because each photo is different. This goes to show that artificial intelligence is not the right solution for all automation issues. Fortunately, there are plenty of other techniques that can provide the ideal solution. And we work with you to look for the technique that fits your problem.
Free: image recognition for beginners
Want to be totally up to date on the possibilities of image recognition in 2023? Download your free white paper here. We will also advise you, without any obligation, whether image recognition can be applied within your organisation.