Historian

What is a historian?

A historian (or process historian) is a specialised database that efficiently stores time series of industrial process data, such as measurements, states and alarms from PLCs, SCADA and DCS, for many years and makes them quickly available again for trending, analysis and reporting. Every value is recorded with a timestamp and a quality status, so you can see exactly what a process was doing at any given moment. Thanks to clever compression, years of history for tens of thousands of tags fit on a single server. The historian is therefore the memory of the plant and one of the most important bridges between OT and IT.

🧠 What does a historian do?

A historian has four core tasks:

• Collecting — reads data from SCADA, DCS, PLCs and sensors via interfaces and protocols such as OPC DA, OPC UA, Modbus or MQTT Sparkplug B
• Storing as time series — records a timestamp, value and status (good, bad, uncertain) per tag, using compression to limit the volume
• Serving — delivers raw and aggregated data (average, minimum, maximum, interpolated values) to trend displays, reports, MES, ERP and analytics tools
• Retaining — keeps data available for years for monitoring, fault analysis, quality records and compliance

Where a SCADA system shows what is happening now, the historian tells you what has happened. Technically, a historian is a form of time series database, extended with industrial collectors, tag management and ready-made process functions.

🔧 How does a historian store time series?

Not every measured value is stored. A historian applies a storage strategy per tag, so that relevant changes are kept while noise and repetition are not:

In practice historians combine these methods: digital signals on change, analogue values with a deadband or compression, and a maximum interval (for example every 10 minutes) so it stays visible that a tag is still alive.

How does swinging door compression work?

The swinging door trending algorithm was described by E.H. Bristol at an ISA conference in 1990. A variant of it handles compression in historians such as the PI System. From the last stored value, two “doors” are opened with a tolerance (the compression deviation). As long as new values fall within those doors, they lie almost on a straight line and are not stored. Only when a value falls outside the doors is the previous point archived. When data is retrieved, the historian interpolates between stored points, so the trend matches the original within the configured tolerance.

In PI this happens in two steps: the exception test in the interface first filters noise (ExcDev), after which the server applies the compression test (CompDev). Typical defaults:

Match the tolerance to the measurement uncertainty of the instrument: a setting that is too wide throws away process information, one that is too tight stores noise.

📦 What historian software is available?

Classic historians provide collectors, tag management and process functions out of the box. Open-source databases are more flexible and cheaper to license, but you build data collection, contextualisation and administration yourself.

🔄 What is the difference between a historian and an ordinary database?

The boundary is not sharp: some historians, such as the SIMATIC Process Historian, store their data in an SQL database, and TimescaleDB adds time-series functions to PostgreSQL. The difference then lies in the layer on top.

🏗️ Where does the historian sit in the architecture?

In the Purdue Model historians appear at several levels:

Open standards exist for retrieving history. OPC HDA (Historical Data Access) is the older specification based on Microsoft COM/DCOM. Its successor is OPC UA Historical Access, part 11 of the OPC UA specification. Clients use the HistoryRead service to retrieve raw or processed history; the standard aggregates such as average, minimum and maximum are defined in part 13 (Aggregates). In modern architectures, devices also publish their data via MQTT into a Unified Namespace, and the historian becomes one of the consumers. Industrial DataOps and edge computing platforms enrich the data with context before it is sent to the cloud.

🏭 What is a historian used for?

• Process optimisation — comparing temperature, pressure and flow trends between good and bad runs
• Fault analysis — reconstructing afterwards what happened before a trip or incident
• Maintenance — condition monitoring and predictive maintenance based on vibration, current draw and running hours
• Energy — monitoring and reporting consumption per installation or machine
• Batch and quality — linking process data to batches in line with ISA-88
• Compliance — in pharmaceuticals, historians record data for the batch record under GMP and FDA requirements. EU GMP requires batch documentation to be kept for at least one year after the expiry date of the batch or five years after certification by the Qualified Person, whichever is longer.

Industrial AI and Industry 4.0 applications also rely on the historian: without years of reliable history there is little to train on.

🧭 How do you size a historian?

A historian often stays in service for ten years or more. A sound design starts with a few questions:

1. Number of tags — how many signals now, and how many after expansion? Licences are usually priced per tag.
2. Scan rate — 1 second is common for process data; vibration or electrical measurements require much higher rates.
3. Compression settings — define ExcDev, CompDev and maximum intervals per signal type.
4. Retention period — follows from laws and regulations, quality requirements and analysis needs.
5. Availability — with buffering in the collector (store and forward) and a redundant server for high availability, no data is lost during an outage.

Worked example: 5,000 tags that each deliver one value per second produce 432 million values per day. At roughly 15 bytes per value (timestamp, value and status) that is about 6.5 GB per day. With a compression factor of 10, which is not unusual for steady processes, around 650 MB per day remains, roughly 240 GB per year. Include this volume in your backup planning as well.

🔐 How do you secure a historian?

A historian has connections to both the process network and the office network. That makes it an attractive pivot point for attackers: MITRE ATT&CK for ICS describes the Data Historian as a separate target (asset A0006), precisely because it often needs access from the corporate network. A compromised historian also gives an attacker a complete picture of the process. Key measures:

• Network segmentation — place the site historian in the OT network and a replica in the IDMZ or DMZ; IT users only read the replica
• One-way traffic — replicate data from OT to the DMZ through a data diode; vendors such as Sentyron (the Fox DataDiode, formerly Fox Crypto), Waterfall and Owl offer ready-made PI replication across a diode
• No inbound connections — never let the replica write back to the process network
• Authentication and encryption — personal accounts with least privilege and encrypted connections
• Logging and monitoring — record access and configuration changes and forward them to the SOC
• IEC 62443 — treat the historian and its replica as separate zones with controlled conduits

These measures fit within the broader cybersecurity approach for OT and help organisations meet the requirements of NIS2.

❓ Frequently asked questions

Is a historian the same as a time series database?

A historian is a specialised time series database for industrial process data. Beyond storage, a historian provides collectors for OPC and other protocols, tag management, quality statuses and process functions such as interpolation. With a generic time series database such as InfluxDB you have to add these yourself.

Does a historian lose data through compression?

Yes, compression in a historian is usually lossy, but within a tolerance that you configure yourself. Values that lie on the line between stored points are dropped and interpolated on retrieval. Some products, such as Canary, compress losslessly.

How long does a historian keep data?

A historian can keep data online for years and often decades. The retention period follows from legal requirements, quality rules and the need for long-term analysis. In pharmaceuticals, EU GMP sets minimum retention periods for batch documentation.

Why is there a historian in the DMZ?

A historian in the DMZ is a replica of the historian in the process network. IT users and cloud applications only read that replica, so no direct connection from the office network into the OT network is needed. This reduces the chance that an attack from IT reaches the process.

What is the difference between OPC HDA and OPC UA Historical Access?

OPC HDA is the older standard for historical data and runs on Microsoft COM/DCOM, which is hard to secure. OPC UA Historical Access is the platform-independent successor within OPC UA and offers the same functions with built-in security. New historian integrations should preferably use OPC UA.

Can a historian run in the cloud?

A historian can run in the cloud, but the site historian is usually kept on premises so that no data is lost if the internet connection fails. A common pattern is a local historian that forwards data to a cloud platform for analysis across multiple sites.

📌 In summary

A historian is the memory of the plant: a specialised database that efficiently stores process data from SCADA, DCS and PLCs, with timestamp and status, for many years. Clever compression makes that storage affordable, and because the historian connects OT and IT, it should be secured with segmentation, a replica in the DMZ and, where needed, a data diode.