What is Network Congestion?
Network congestion is the condition in which a network link or switch port is offered more traffic than it can forward, so that packets have to wait in buffers and are discarded once those buffers are full. The result is extra delay, varying delay and packet loss. In OT networks this is more serious than in office networks: cyclic control protocols expect their data within a few milliseconds and switch to a safe state when it fails to arrive. Congestion is therefore not a comfort issue but a risk to availability, real-time communication and deterministic behaviour.
🧠 How does network congestion arise?
A switch receives frames on one or more inputs and queues them for the outgoing port. As long as no more bits arrive per unit of time than the output can send, the queue stays empty. If more arrive, the queue grows; once the buffer is full, frames are dropped (tail drop).
The key point is that congestion happens per port and per moment, not per network and per average:
- Many-to-one (incast) - dozens of devices send to one SCADA server or historian at the same time; the port towards that server fills up even though the network as a whole is quiet.
- Speed mismatch - a 1 Gbit/s uplink feeds a 100 Mbit/s device. A burst from the fast side has to fit into the buffer of the slow port.
- Microbursts - peaks lasting a few milliseconds that are invisible in a five-minute average. A port with 5% average utilisation can still lose packets.
- Flooding - broadcast, unknown unicast and unfiltered multicast are copied to every port in a VLAN, loading devices that do not need the traffic at all.
To illustrate the time scale: a maximum-size Ethernet frame of 1,518 bytes takes about 121 µs to transmit at 100 Mbit/s and about 12 µs at 1 Gbit/s. If thirty such frames sit in front of a control frame in the queue of a 100 Mbit/s port, that already adds more than 3.6 ms of delay, longer than a typical PROFINET cycle.
⚠️ Which symptoms do you see in an industrial network?
| Symptom | Effect on industrial systems |
|---|---|
| Increased latency | Cycle times become unpredictable and control loops respond more slowly |
| Jitter | Varying arrival times disturb motion control and time synchronisation |
| Packet loss | Process data or commands are lost; cyclic protocols do not retransmit |
| Watchdog time-outs | The connection between controller and I/O drops and outputs go to their safe value |
| Sluggish HMI screens | Values freeze or show “communication error”, operators lose their overview |
| Gaps in historical data | Logging and reports miss data points |
| High CPU load on devices | Older PLCs and I/O modules are overloaded by traffic not meant for them |
What makes it treacherous is that the symptoms are often sporadic. A fault that only appears during the nightly backup or when someone opens a large trend screen is easily blamed on “bad cables”.
🔍 What are the main causes in OT?
| Cause | Description |
|---|---|
| IT and OT traffic on one segment | Backups, software updates or video streams crowd out control traffic |
| Insufficient network segmentation | No separation between real-time and non-real-time traffic |
| Broadcast storms | A network loop without working RSTP or a faulty network card makes broadcasts circulate endlessly |
| Unfiltered multicast | Multicast I/O (for example with IP) is sent to every port without IGMP snooping |
| Missing or wrong QoS | Control frames get no priority in the queue |
| Unmanaged switches | No priorities, no storm control, no diagnostics |
| Active scans | A network scan or inventory tool floods fragile field devices |
| Attacks | A DDoS attack or malware spreading across the network causes overload, deliberately or as a side effect |
A well-known example is Browns Ferry Unit 3 in the United States. On 19 August 2006 the reactor had to be scrammed manually after both recirculation pumps were lost. The US regulator NRC found that the controllers of the variable frequency drives had locked up because of excessive traffic on the plant’s internal control network. There was no attack; the remedy included firewalls that limit the traffic to susceptible devices.
🔧 Why is congestion worse in OT than in IT?
In IT networks, TCP largely solves congestion by itself. After the congestion collapse of the early Internet in 1986, TCP gained algorithms that reduce the sending rate when packets are lost (slow start and congestion avoidance, now specified in RFC 5681). A web page loads a little slower, but everything arrives.
Industrial real-time protocols behave differently:
- No retransmission - PROFINET RT runs directly on Ethernet and EtherNet/IP implicit I/O runs on UDP. A lost frame is not resent; only the next cycle brings fresh data.
- Watchdogs - with ProfiNET the default watchdog is three times the update time. With a 1 ms update time, the device drops the connection after about 3 ms without data. With EtherNet/IP the time-out follows from the RPI (Requested Packet Interval) times a multiplier.
- Safe reaction - dropping the connection puts outputs in their fail-safe state. With safety protocols this can mean an emergency stop or a machine shutdown.
- Constrained devices - many sensors, I/O stations and older controllers have small network stacks. A broadcast storm that a server shrugs off can block such a device completely.
That is why IEC 62443-3-3 asks under FR 7 (resource availability) for denial-of-service protection (SR 7.1) and resource management (SR 7.2): a system must keep performing its essential functions under heavy network load.
✅ How do you prevent and limit congestion?
| Measure | Explanation |
|---|---|
| Network segmentation | Separate real-time OT traffic from non-critical data flows, for example per zone in line with the Purdue Model; an EtherCAT segment usually runs as its own closed network |
| VLANs | Limit broadcast domains; on a shared trunk, VLANs still share the same bandwidth |
| QoS | IEEE 802.1Q defines eight priority levels; PROFINET RT uses priority 6, and at IP level DSCP marks the traffic |
| Storm control and rate limiting | Cap broadcast, multicast and unknown unicast traffic per port |
| IGMP snooping | Forward multicast only to ports with subscribers, with one querier per VLAN |
| Managed or industrial switches | Adequate buffers, priority queues, port statistics and SNMP diagnostics |
| Network monitoring | Measure load, drops and errors per port; capture traffic via SPAN or a TAP |
| TSN | Time-Sensitive Networking (including IEEE 802.1Qbv) reserves time windows so real-time traffic is guaranteed room |
| Firewall or zone boundary | Limit connections and traffic to vulnerable devices, as was done after Browns Ferry |
More bandwidth only helps when the bottleneck is structural. With microbursts, loops or incast, a faster link often just moves the problem.
🏭 How do you tackle congestion in practice?
A practical approach for an existing installation:
- Map the flows - which controllers talk to which I/O, HMIs and servers, and at what cycle time? Without that overview, QoS is guesswork.
- Measure instead of estimating - read the per-port counters for output drops, CRC errors and broadcast counts. A Wireshark capture through a SPAN port or TAP shows whether there are bursts or loops. Prefer passive monitoring, because active scans can cause congestion themselves.
- Calculate the cyclic load - PROFINET design practice uses a rule of thumb: below 20% cyclic load per link no action is needed, between 20 and 50% you should check the design, and above 50% the load must come down so that room remains for non-real-time traffic.
- Design the topology - avoid long lines with many devices in series, place servers and historians outside the control zone, and use ring redundancy with a configured protocol so that loops do not turn into storms.
- Configure the switches - priorities, storm control, IGMP snooping and disabled unused ports. Record that configuration as a standard.
- Schedule heavy tasks - run backups, firmware roll-outs and historian synchronisation outside production peaks or over a separate segment.
- Monitor continuously - set thresholds for port load and drops, so you see congestion before the installation stops.
For Dutch and EU organisations within the scope of NIS2 and the Dutch Cyberbeveiligingswet, this fits the duty of care to safeguard the availability of network and information systems. Congestion is both a design risk and an attack vector.
🔄 What is the difference between congestion, bandwidth limits and jitter?
| Term | Meaning | Typical remedy |
|---|---|---|
| Congestion | Temporarily more traffic offered than a port can forward | QoS, segmentation, storm control, better traffic distribution |
| Bandwidth limit | Structural capacity of a link, for example 100 Mbit/s | Faster link or less data |
| Latency | The total delay from sender to receiver | Shorter paths, fewer hops, faster links |
| Jitter | Variation in that delay, often a consequence of congestion | Priority, TSN, less queueing |
| Packet loss | Frames that do not arrive, partly because of full buffers | Remove congestion, repair physical faults |
❓ Frequently asked questions
Can a network with low average load still suffer congestion?
Yes, network congestion happens per port and per moment. Microbursts of a few milliseconds, or many devices sending to one server at once, can overflow a buffer while the five-minute average is only a few percent.
Does a faster switch solve network congestion?
A faster switch only helps when capacity is structurally insufficient. With broadcast storms, network loops or unfiltered multicast the traffic grows along with it, and at a transition from 1 Gbit/s to 100 Mbit/s the slow port remains the bottleneck.
Why does a PLC connection drop during congestion?
Industrial protocols such as PROFINET and EtherNet/IP supervise cyclic communication with a watchdog. If no frames arrive for several consecutive cycles, three by default with PROFINET, the device drops the connection and its outputs go to their safe state.
What is the difference between congestion and a DDoS attack?
Network congestion is the effect: more traffic than the network can handle. A DDoS attack is one possible cause, but in OT congestion more often arises unintentionally, from loops, backups, scans or a faulty device.
How do you measure network congestion in an OT network?
You read the per-port counters for load, output drops and broadcasts on each switch, for example via SNMP. A passive capture through a SPAN port or TAP then shows which flows cause the peaks, without adding traffic to the control network.
Does QoS help against network congestion?
QoS does not prevent congestion, but it decides who goes first when a queue fills up. With priority for control traffic, PROFINET and EtherNet/IP frames keep arriving, while a backup or web traffic absorbs the delay.
📌 In summary
Network congestion occurs when a port is temporarily offered more traffic than it can forward; in OT this leads straight to downtime through watchdog time-outs.
In OT networks congestion is an availability risk, because real-time protocols do not retransmit and drop the connection after missed cycles. You manage it with segmentation, QoS, storm control, IGMP snooping, passive monitoring and a well-considered design, not merely with more bandwidth.