What is Vision AI?
Vision AI is the combination of industrial machine vision and artificial intelligence, mainly deep learning, that lets camera systems learn for themselves what good and bad products, objects or situations look like and make a decision about them automatically. Where classic machine vision works with fixed rules set by an engineer, such as a dimension, a contrast threshold or a pattern, Vision AI learns from example images. That allows it to cope with variation that rules can hardly describe: scratches on shiny metal, natural products such as vegetables or meat, or randomly piled parts in a bin. Within industrial automation, Vision AI is a core part of Industrial AI, Industry 4.0 and cyber-physical systems.
Vision AI rarely replaces classic machine vision; it complements it: rule-based where measuring and reading must be exact, learning-based where the appearance of a product varies too much.
🧠 What is the difference between classic machine vision and Vision AI?
Both use the same cameras, lenses and lighting. The difference lies in how the software reaches a verdict.
| Aspect | Classic machine vision | Vision AI (deep learning) |
|---|---|---|
| Principle | Rules and algorithms configured by an engineer | Model that learns patterns from labelled example images |
| Strong at | Measuring, alignment, code reading, presence checks | Cosmetic defects, natural variation, complex classification |
| Data needed | A few reference images | Tens to thousands of images, depending on the task |
| Explainability | High: every rule can be traced | Lower: the model returns a score, not a line of reasoning |
| Computing power | Smart camera or standard IPC | Often a GPU or AI accelerator on an edge device |
| Product change | Adjust rules manually | Add new images and retrain |
In practice many systems combine both: a classic tool locates the product and reads the barcode or QR code, after which an AI model judges the surface.
🔧 Which AI techniques does Vision AI use?
Most models are convolutional neural networks (CNNs); newer systems also use vision transformers. The technique you choose depends on the question the system has to answer.
| Technique | Question the model answers | Typical application |
|---|---|---|
| Classification | What kind of product or condition is this? | Pass/fail, sorting by variant |
| Object detection | Where are which objects? | Pick-and-place, counting, robot guidance |
| Segmentation | Which pixels belong to a defect or object? | Scratches, cracks, measuring weld seams |
| Anomaly detection | Does this deviate from normal? | Defects not known in advance, trained on good products only |
| Deep OCR | What text is printed here? | Date and lot codes on difficult surfaces |
Anomaly detection is often the starting point on the factory floor, because defective products are rare and good products are plentiful. The model learns what “normal” looks like and flags everything that deviates. Supervised classification or segmentation, on the other hand, needs labelled examples of every defect type. All of these techniques are forms of machine learning.
⚙️ What are the components of a Vision AI system?
The architecture closely resembles that of classic vision, with an extra layer for training and model management.
| Layer | Component | Function |
|---|---|---|
| Imaging | Industrial camera, lens, lighting | Consistent, reproducible image; lighting determines most of the quality |
| Trigger and I/O | Sensor, encoder, digital I/O | Capture at the right moment, result back to the line |
| Camera interface | GigE Vision, USB3 Vision, GenICam | Standardised control of cameras from different vendors |
| Inference | Smart camera, IPC or Embedded platform with GPU | The trained model runs locally on every image |
| Integration | Fieldbus, OPC UA, MQTT | Result to PLC, SCADA, MES |
| Training and model management | Workstation or server, labelling tool, model registry | Collect, label, train, validate and version models |
GigE Vision was introduced in 2006 and USB3 Vision in 2013; both are managed by the US trade association A3 (formerly AIA). GenICam, the programming framework both use to address camera settings generically, is hosted by the European Machine Vision Association (EMVA) and was ratified in 2006.
Inference almost always runs through edge computing rather than in the cloud. That gives low latency, reduces network load and keeps the line running if the connection to the data centre drops. An AI model is not strictly deterministic in its computing time, however: processing time varies with GPU load. Hard real-time actions, such as firing a reject mechanism at a fixed position, therefore stay with the PLC, which only uses the result from the vision PC as an input.
🏭 Where is Vision AI used in industry?
| Application | What Vision AI does | Example sector |
|---|---|---|
| Quality control | Cosmetic defects, contamination, incomplete assembly | Automotive, electronics, packaging |
| Sorting | Grading products by quality, size or type | Food, recycling |
| Robot guidance | Determining position and orientation of objects, including bin picking | Assembly, logistics, robotics |
| Mobile robots | Recognising obstacles, people and markers alongside LiDAR | Warehouses with AGVs and AMRs |
| Identification | Reading codes and text for tracking and tracing | Pharma, food, logistics |
| Process monitoring | Assessing flame, foam, fill level or wear | Process industry, predictive maintenance |
| Safety monitoring | Flagging people in a zone or missing PPE | Production halls, loading docks |
In robotics, Vision AI often works together with motion control: the model determines where a product is and the robot or cobot controller calculates the movement. Combinations with 3D cameras, LiDAR and SLAM navigation (Simultaneous Localization and Mapping) are common in mobile robots.
Vision AI also produces data. Defect images and counts flow to the MES, the historian or a Unified Namespace, where they feed SPC, LEAN and Six Sigma improvement work and a digital twin of the line. Inspection thus shifts from merely rejecting products to finding the root cause further upstream.
📡 How do you connect Vision AI to PLC, SCADA and MES?
| Connection | Use | Note |
|---|---|---|
| ProfiNET / Ethernet IP | Trigger and pass/fail result to the PLC | Fast and cyclic; suited to line control |
| Modbus TCP | Simple status values | Mainly in existing installations |
| OPC UA | Recipes, results and status to SCADA and MES | OPC UA for Machine Vision (VDMA 40100) defines a standardised information model |
| MQTT | AI events and statistics to IIoT platforms | Fits well with a Unified Namespace |
The OPC UA companion specification for machine vision was presented as a release candidate at automatica in June 2018 and published in August 2019 as OPC 40100-1, identical to VDMA 40100-1. It describes, among other things, a state machine that lets a controller start and stop a vision system and switch recipes, regardless of vendor. The images themselves rarely travel over these protocols: they are stored locally or sent compressed to an image archive.
🔧 How do you set up a Vision AI project?
- Define the decision - which defects must be rejected, which may pass, and what does a missed defect cost compared with a false reject?
- Optimise the image first - choose camera, lens and lighting so that a person can see the defect on screen. What a human cannot see, a model rarely learns reliably.
- Collect representative data - images from every shift, batch, season and product variant, including edge cases.
- Label consistently - record in a defect catalogue what counts as “bad”; contradictory labels from different operators are the most common cause of a weak model.
- Train and validate separately - hold back a test set the model has never seen and compare against the current (manual) inspection.
- Run in shadow mode first - the model judges alongside without rejecting, so you find errors without production loss.
- Manage the model as software - version control, a change procedure and periodic revalidation, because new raw materials or ageing lighting cause drift.
Indicative values from practice:
| Parameter | Typical order of magnitude |
|---|---|
| Inference time per image on an edge GPU | A few to a few tens of milliseconds |
| Camera resolution for surface inspection | Roughly 2 to 25 megapixels |
| Starting anomaly detection | Tens to a few hundred images of good products |
| Supervised defect classification | Tens to hundreds of labelled examples per defect class |
| Key performance metrics | Escapes (missed defects) and false rejects, measured separately |
🛡️ Can Vision AI perform a safety function?
Vision AI is increasingly used to detect people, but a learning model as the sole safety function is not yet established practice in Europe. Machine safety starts with the risk assessment according to ISO 12100, after which the safety function is designed to ISO 13849 (Performance Level) or IEC 62061 (SIL), derived from IEC 61508. See also functional safety.
For cameras used as protective devices, IEC 61496 has separate parts: IEC TS 61496-4-2 for vision systems using reference patterns and IEC TS 61496-4-3 for stereo vision, both revised in 2022. These specifications assume fixed detection, not the complex classification a deep learning model performs. The informative report ISO/IEC TR 5469:2024 describes how AI relates to functional safety, but it is not a certifiable standard.
In practice, therefore:
- Certified safeguarding stays in charge - a light curtain, laser scanner or safety camera, via a safety PLC or safety relay.
- Vision AI adds a layer - for example by slowing a machine down earlier or issuing a warning, outside the certified safety chain.
- Regulation is shifting - under the Machinery Regulation, which applies from 20 January 2027, safety components with self-evolving behaviour require mandatory assessment by a notified body. Under the AI Act, AI used as a safety component of a machine is high risk; after the digital omnibus those requirements apply from 2 August 2028. Visual quality control without a safety function usually falls into a lower risk class.
🔐 What cybersecurity and privacy risks does Vision AI bring?
A Vision AI system is a PC or embedded device with an operating system, GPU drivers, AI libraries and network connections into the OT environment. It therefore brings cybersecurity risks that a classic smart camera did not have.
| Risk | Possible consequence | Security measure |
|---|---|---|
| Tampered or swapped model | Defects pass or good products are rejected | Signed models, version control, application whitelisting |
| Adversarial input | A sticker or pattern fools the model | Validation with edge cases, classic check as second opinion |
| Vulnerable software and firmware | Takeover of the vision PC | secure boot, patch management |
| Unsecured network connection | Access to the PLC via the vision system | network segmentation, zero trust, IDS |
| Supply chain | Compromised AI library or vendor update | Procurement requirements based on IEC 62443, software bill of materials (SBOM) |
| Image data leak | Trade secrets or personal data exposed | access control, encryption, logging |
Under the Cyber Resilience Act, manufacturers of new smart cameras and vision software must meet cybersecurity requirements from 11 December 2027.
As soon as cameras capture people, the GDPR (in Dutch: AVG) applies too. Images of identifiable employees are personal data; you need a legal basis, must act proportionately and often carry out a DPIA. In the Netherlands, a camera system that can be used to monitor employees requires the consent of the works council. Since 2 February 2025 the AI Act has prohibited emotion recognition in the workplace, except for medical or safety reasons. Design with privacy by design: point the camera at the product, blur faces and keep only what you need.
❓ Frequently asked questions
What is the difference between Vision AI and machine vision?
Machine vision is the broader technique of cameras, lighting and software that judge images automatically, usually with fixed rules. Vision AI is the variant in which a deep learning model learns from example images what good and bad look like. Vision AI is strongest with natural variation and cosmetic defects, classic machine vision with exact measuring and code reading.
How many images does Vision AI need to learn?
For anomaly detection trained on good products only, tens to a few hundred images are often enough. Supervised defect classification typically needs tens to hundreds of labelled examples per defect class. More important than the number is that the images cover all the variation of real production.
Does Vision AI run in the cloud or locally?
On the factory floor Vision AI almost always runs locally on a smart camera, industrial PC or edge device with a GPU, because decisions are needed within milliseconds. Training and model management can take place centrally or in the cloud. That way the production line keeps running if the network connection is lost.
Does Vision AI fall under the AI Act?
Vision AI falls under the AI Act, but visual quality control without a safety function is usually not a high-risk application. If a machine uses Vision AI as a safety component, for example to detect people and stop the machine, it counts as high risk. Emotion recognition of employees with cameras has been prohibited since 2 February 2025.
Can Vision AI replace a light curtain?
Vision AI cannot simply replace a certified light curtain or safety camera at present. The IEC 61496 specifications for vision-based protection assume fixed detection, not learning-based classification. Vision AI is therefore usually used as an additional layer next to safeguarding designed to ISO 13849 or IEC 62061.
How do you secure a Vision AI system in an OT network?
You secure a Vision AI system like any other OT device: in its own network zone, with only the necessary connections to PLC and MES. In addition, the AI models themselves need protection through signing, version control and a controlled change procedure. IEC 62443 provides the framework for zones, conduits and supplier requirements.
📌 In summary
Vision AI brings deep learning to the industrial camera. It learns from example images and can therefore judge variation that fixed rules cannot capture, from cosmetic defects to randomly piled parts for a robot. Success depends less on the model than on good lighting, representative data, consistent labels and managing the model as software. Inference runs locally at the line, the PLC stays in control and the certified safety chain remains in charge.
Vision AI is machine vision that learns from images instead of following fixed rules: powerful for variable inspection and robot guidance, provided you take the image, the data, security and privacy as seriously as the model.