It’s likely you have unlocked your phone using the built-in facial recognition software, or maybe you have added dog ears or cat whiskers to a favourite headshot in your image gallery. Or better yet, maybe you’ve driven a Tesla and used the autonomous driving feature.
These examples of machine vision and the exciting possibilities they open for us are growing, and the future is now!
Vision data is heralded as the driving force of Industry 4.0, delivering between $1.2 and $3.7 trillion worldwide in value by 2025, according to McKinsey & Company, an advisor and consultant to many global businesses and institutions.
“I see how artificial intelligence (AI) and the fourth industrial revolution will impact every aspect of people’s lives,” says Fei-Fei Li, professor of Computer Science at Stanford University and co-director of Stanford’s Human-Centered AI Institute. Students are teaching robots to be better collaborators with people to motivate patients in recovering from a stroke, for example, and helping Google Health to screen eye images for diabetic retinopathy, a condition where early detection can prevent vision loss.
So, what is machine vision and what exactly can a computer see?
Simply put machine vision is the eye of the machine. It is visual input captured by a camera and converted to data. In manufacturing, machine vision is typically attached to a process system to monitor and improve efficiency, quality and operations on the plant floor.
While it isn’t entirely new, it is experiencing a renaissance as software is rapidly developing along with sophisticated robotic technology that makes full use of visual input. It was back in 1960s, when researchers at Massachusetts Institute of Technology (MIT) first attempted to connect a camera to a computer in the hopes of realizing the full potential of computer intelligence and pattern recognition.
Safe to say, in 40-plus years since the early MIT endeavours, we have had major breakthroughs in the development of less expensive and much more robust hardware, as well as innovative and fast evolving software. This wave of new technology, pairing optical technology with advance algorithms, has gained momentum and is beginning to transform industry. Machine vision and the ability of artificial intelligence to see and do like humans has been the subject of rigorous research and growing interest.
It was in the 1980s and ’90s when machine vision went from the research lab to industry applications. Your favourite bottle of soda or beer was likely scanned for imperfections such as improper fill levels or crooked labels by a machine vision-enabled bottling line.
In the bottling line example, a simple machine vision system could include a light source and sensor at some point along the line to simply detect the presence of the product. A slightly more advanced system might use a camera to take an image of the bottle, and a digitizing device or frame grabber would turn that image into something that could be analyzed automatically. Computer software compares the image to the desired result, like a perfectly filled bottle with a straight label, as well as other prescribed criteria such as a neck label or cap straightness. When a defect is spotted, the product fails and is removed from the line.
That great old “I Love Lucy” episode, you know the one, where Lucy is being overwhelmed by chocolates coming off the conveyor belt, represents a situation that could have been completely avoided by a similar machine vision component that would have stopped or slowed the process when it recognized a back-up of product.
Today, advanced systems using machine vision typically involve high-resolution cameras or other imaging equipment paired with software algorithms capable of recognizing pre-established patterns. More and more frequently, these systems are powered by deep learning algorithms that use “neural networks” (so named as they are modeled on the human brain) to interpret them images. The networks are trained with large sets of sample images with known outcomes, which the algorithm attempts to identify and then learn from.
“I have always been convinced that the only way to get artificial intelligence to work is to do the computation in a way similar to the human brain,” says Geoffrey Hinton, Chief Scientific Advisor at the University of Toronto’s Vector Institute.
Innovation at the Institute has pioneered some very cool systems, like a robotic laboratory where thin-film materials for clean energy applications can be designed, tested and made commercially viable for manufacturing consumer electronics – a process that used to take more than 10 years is now possible in a fraction of the time.
Machine vision is helping manufacturers with everything from quality control, product identification, guidance and other inspection applications to problem assessment, product tracing, product safety, measurement and calibrating, inventory control and farming applications. Product tracing is especially important in industries like pharmaceuticals where it is important to track regulated ingredients, expiration dates and serial numbers.
Inventory control can also be monitored as barcodes and labels are digitally viewed to make certain the correct components are included as a product moves along an assembly line. Similarly, machine vision has tremendous possibilities in food processing where tracking supplies is essential to product safety. Farming, too, is seeing the introduction of harvesting units complemented with machine vision. In this setting, a robotic harvester can locate, identify and gently pick tender fruits without damage.
The oil and gas industry is relying on machine vision to help see and sniff out productivity and safety for offshore drilling platforms, like the four-legged robot Spot that takes on the tasks of data capture, inspection and reporting any anomalies along the sea floor. These insights provide real-time feedback and allow onshore mission control to work virtually with the offshore staff to quickly correct potentially critical problems.
Reliable machine vision has enabled more integrated robotics – systems working closely with human workers on the production floor in many manufacturing applications – allowing robots to take on more roles above and beyond the mundane and repetitive tasks that defined their place in the past.
As this technology continues to evolve, we must ask some pretty serious questions: is the identification and development of patterns or algorithms sufficiently accurate to allow the system to make real-world decisions? Vision and pattern algorithms based on neural networks can be very effective – often faster and more accurate than human eyes and brains, but they’re not infallible. When safety is a factor, or when a decision can affect human lives in some way, do we rely more on the machine, or on the human operator? I suppose it depends on the application.
Privacy has also been a concern with machine vision systems. The amount of data gathered and stored is staggering, and one must at least consider those implications. How should we feel about the camera on our phones guessing at our emotional responses to content presented to us? Our human biases might allow us to applaud a machine vision system for correctly identifying someone having a heart attack in their chair at home alone, saving a life with an alert to emergency response. But then at the same time we might be horrified at the machine watching us sleep.
Cautionary, speculative fiction has so many examples of potential misuse of these technologies, especially when paired with AI and corporate greed. But in reality, what’s next for machine vision?
“I am telling you, the world’s first trillionaires are going to come from somebody who masters AI and all its derivatives, and applies it in ways we never thought of,” say Mark Cuban, billionaire founder of Mark Cuban Companies who owns the NBA’s Dallas Mavericks and has stakes in Magnolia Pictures.
And it’s happening now.
At the start of the new millennium, healthcare began experimenting with machine vision. Early digital microscopes costing in excess of $300,000 were deployed but took over 24 hours to scan a single slide. Only the biggest labs with dedicated technicians could make use of this equipment.
Now two decades later, compact blood analysis equipment is becoming more ubiquitous and affordable for fast, accurate and more accessible diagnostics. Radiology is another application where machine vision, trained to identify patterns is allowing for faster review and quicker detection of diseases or other health anomalies.
No question, AI and machine vision will continue to be a growing part of our lives and workplaces. We’re going to see more complex relationships between machines and humans working side by side in increasingly more scenarios – especially where two sets of eyes, especially two sets with very different processing powers, are better than one.
