Presentation Details

Session: Emerging Technologies in Process Safety Management

Abstract

Generative AI holds significant promise for transforming process safety by empowering smaller teams to make safer, more informed decisions. A key requirement is providing AI systems with contextual understanding of process flow and instrumentation, most readily conveyed through Piping and Instrumentation Diagrams (P&IDs). This work makes several contributions toward that goal. We introduce the P&ID Question Difficulty Scale (PQDS v1), a two-axis framework that decomposes question difficulty into graph reasoning structure and information complexity, producing a 4×4 matrix mapping to six difficulty levels. We propose a binary evaluation framework with four assessment dimensions: correctness, identification accuracy, connectivity validity, and directional accuracy, designed to produce actionable performance profiles rather than aggregate accuracy scores. We analyze existing public P&ID datasets and identify fundamental limitations in synthetic training data for engineering reasoning tasks and describe an approach to generating engineering-realistic synthetic P&IDs suitable for training and evaluation at the highest difficulty levels. We benchmark several commercial and open-source vision language models against the PQDS framework, demonstrating that current models fail on questions requiring multi-hop graph reasoning (Level 3+), and present early results from PIDgie1, a 10-billion-parameter model that achieves correct answers where models over twenty times larger fail. Finally, we describe a new trimodal architecture fusing vision, graph, and language understanding to address the fundamental limitation that existing models lack structural awareness of connectivity and argue that closing the gap between P&ID digitization and interpretation is essential to enabling reliable AI-assisted process safety.

The Challenge: P&IDs Are Opaque to Machines

Piping and Instrumentation Diagrams encode equipment types, piping connectivity, instrumentation logic, control relationships, safety relief paths, and operational constraints in a standardized visual format — one designed for human interpretation. These diagrams are essential for Process Hazard Analyses (PHAs), Quantitative Risk Assessments (QRAs), siting studies, Management of Change (MOC) reviews, and day-to-day operational decisions. Despite their centrality, P&IDs remain largely opaque to computational tools. While digital interchange formats like DEXPI exist, adoption remains limited — particularly for brownfield facilities — and the vast majority of P&IDs in active use exist as scanned PDFs, CAD exports, or image files.

Forty years of P&ID digitization research has produced impressive symbol-detection accuracy (95–98% precision/recall on industrial diagrams), but compound errors across multi-stage pipelines have prevented any system from reaching the reliability required for unsupervised industrial deployment. More fundamentally, the field has focused on extracting what is on the diagram rather than understanding what it means— the transition from digitization to interpretation requires fundamentally different reasoning capabilities.

Two Primary Contributions

This work makes two primary contributions toward closing that gap:

1. The P&ID Question Difficulty Scale (PQDS v1) — a systematic two-axis framework for rating the difficulty of questions posed to automated P&ID interpretation systems, providing a common language for describing what makes P&ID questions hard and enabling structured benchmarking of model capabilities.
2. Evidence that purpose-trained models of modest size can outperform much larger general-purpose models on P&ID reasoning tasks, alongside the architectural principles — particularly the fusion of graph-structural understanding with visual and linguistic reasoning — that we believe are necessary to achieve reliable P&ID interpretation for safety-critical applications.

The P&ID Question Difficulty Scale (PQDS v1)

Not all P&ID questions are equally hard. Asking “What type of equipment is V-101?” is qualitatively different from asking “Which inflow nozzle of H1007 is on the same flow path as P4712?” PQDS decomposes question difficulty into two independent axes.

The two axes combine via a non-linear mapping into an overall Difficulty Level (0–5). Graph reasoning is weighted more heavily because traversal errors compound: a wrong turn at a junction cascades through all downstream reasoning, while information complexity errors tend to be more isolated.

Binary Evaluation Framework: Four Pass/Fail Dimensions

AI-focused papers typically evaluate models using metrics borrowed from object detection or NLP — metrics that measure whether a system can locate symbols or generate coherent text. These are useful for intermediate development, but they fall short of determining whether a tool is fit for purpose for safety-critical work. A model can achieve high detection accuracy while producing dangerously incorrect engineering conclusions. We propose an evaluation framework with four binary assessment dimensions producing a simple pass/fail vector for each question that can be aggregated into performance profiles across PQDS levels.

All four dimensions are binarybecause partial credit is inappropriate for safety-critical applications. A model that hallucinates one connection has demonstrated it can hallucinate connections; a model that reverses flow direction once has demonstrated it can reverse flow direction. The combination of binary scoring with PQDS difficulty stratification transforms a model evaluation from a research metric into an engineering specification — producing actionable profiles like “passes all four dimensions reliably at Levels 0–2 but fails connectivity validity at Level 3 and above” rather than uninformative aggregate scores like “80% accuracy.”

Critically, three of the four dimensions evaluate the reasoning tracerather than just the final answer. A model that produces only a final answer can only be assessed on Correctness — the least informative dimension. A model that shows its work, documenting which components it identified, which connections it traversed, and in which direction, can be evaluated on whether that work is sound.

Benchmarking Existing Vision Language Models

Multiple commercial and open-source VLMs were evaluated against questions at different PQDS difficulty levels using the DEXPI reference P&ID. Models evaluated included Qwen2-VL-7B, Qwen2.5-VL-7B, Qwen3-VL-235B, MiniCPM-V (2_6, V-4, V-4_5), ChatGPT 4o, ChatGPT 5 (Thinking and Non-Thinking variants), Gemini 2.5 Flash, Gemini 2.5 Pro, and PIDgie1-10B-2508 — a 10-billion-parameter reasoning VLM fine-tuned specifically for P&ID interpretation.

Simple Case (PQDS Level 1): Pump Tag Identification

For the question “What are the tag numbers for the pumps?”, most models correctly identified both pumps (P4711, P4712). Notable failures: MiniCPM-V-2_6 misidentified heat exchanger H1008 as a pump, and Gemini 2.5 Flash denied any pumps existed on the diagram. Even at Level 1, model reliability is not guaranteed — a finding with direct implications for any workflow that would use these models for safety decisions without human verification.

Complex Case (PQDS Level 4): Flow Path Nozzle Identification

For the question “Which inflow nozzle of H1007 is on the same flow path as P4712?”(expected answer: N1), the results were striking. Of 13 model evaluations, only 5 arrived at the correct answer, and of those, 3 required explicit prompting assistance. The PIDgie1-10B-2508 model was the only purpose-trained model to arrive at the correct answer without prompting assistance; among general-purpose models, only the thinking variant of Qwen3-VL-235B succeeded — a model roughly 23× larger and requiring ~470 GB of VRAM versus PIDgie1’s ~20 GB.

These failures highlight a fundamental challenge: P&ID interpretation at PQDS Level 3 and above requires structured graph reasoning that general-purpose VLMs are not trained to perform. The models can see the diagram but cannot reason reliably about the topological relationships it encodes.

The Data Gap: Why Synthetic Data Falls Short

Training data is the critical bottleneck for advancing P&ID interpretation beyond digitization. Only three public P&ID datasets with annotations existed as of early 2026 (SiED, Digitize-PID, PID2Graph). While these are valuable contributions to the digitization problem, examination reveals fundamental limitations for training models on engineering reasoning tasks:

• Closed loops without logical start or end points. Real P&IDs represent processes with defined feed streams, product streams, and utility connections.
• Random pipe and valve size changes.Real engineering practice ties size changes to process requirements; random sizing prevents models from learning the engineering logic that constrains P&ID interpretation.
• No suitable logic for interconnections. Real equipment is connected according to process logic (a pump feeds a heat exchanger feeds a vessel); synthetic datasets generally make random connections.
• No suitable logic for instrumentation. Instrumentation placement in real P&IDs follows control philosophy and safety requirements; synthetic datasets either omit instrumentation or place it without engineering rationale.

These characteristics render the data unsuitable for training models that must reason about flow paths, process logic, or engineering intent. Addressing this gap is a central part of our effort: developing a synthetic P&ID generation capability that produces diagrams with realistic process logic, coherent flow paths, appropriately sized piping and valves, instrumentation placed according to engineering practice, and a representative mix of equipment types including vessels, heat exchangers, pumps, compressors, and relief devices.

New Approach: Trimodal P&ID Understanding

Existing VLMs can extract visual features from P&IDs (identifying symbols, reading tags) but lack the structural understanding of connectivity needed for reliable flow-path reasoning. Our trimodal architecture addresses this gap by fusing three complementary modalities:

The key insight is that inferring connectivity and interpreting visual content place different demands on a vision system. We separate these into two distinct pipelines: a detection-and-segmentation pipeline that extracts graph structure and topology, and a second visual pipeline that provides broader perceptual context. The graph structure becomes a native modality input alongside image data and text — making topological reasoning inherent to the model rather than something it must infer from visual proximity.

A further advantage of the trimodal approach is resilience to single-modality errors. With multiple modalities providing overlapping information, the model can diagnose and reconcile discrepancies: if the graph encoder indicates a connection that visual evidence does not support, the reasoning aspect can weigh the conflicting signals rather than blindly propagating a single modality’s error.

Structured Graph Traversal Reasoning

For questions requiring flow-path analysis (PQDS Level 2 and above), we employ a structured reasoning approach that guides the model through systematic graph traversal. Rather than allowing free-form reasoning — which can result in hallucinated connections, flow direction confusion, or lower efficiency through wasted tokens — the model is allowed to reason but along a disciplined traversal strategy that mirrors how a human engineer reads a P&ID: starting at a reference point, following connections systematically, noting junctions and alternatives, and building up a traced path before drawing conclusions.

Implications for Process Safety

Reliable P&ID interpretation has broad implications for process safety practice:

• Accelerating PHA studies. Pre-populating HAZOP deviation analyses with equipment connectivity data could reduce study duration while improving coverage, particularly for complex systems where manually tracing all flow paths is time-intensive.
• QRA and siting studies. Identifying release scenarios, isolation points, relief device coverage, and flow paths from potential leak sources to ignition sources could be partially automated, accelerating early phases of quantitative risk assessment.
• Day-to-day process safety support.Questions like “What pressure rating is the relief valve on this vessel?” or “What instruments monitor this flow path?” could be answered without manual lookup, reducing the barrier to asking and answering safety-relevant questions.
• Continuous design review.As P&IDs are revised, automated comparison against best practices, inherently safer design principles, and previous PHA findings could enable continuous rather than periodic safety review.

Why Not Public Tools? Why Not RAG Alone?

While commercial models like ChatGPT and Gemini demonstrated some capability on simple P&ID questions, several factors make them unsuitable for process safety applications: confidentiality concerns with cloud-based services, frequent model changes that make regression testing impractical for regulated environments, non-deterministic outputs, and — critically — failure on the connectivity and flow direction reasoning that safety-critical applications require.

Retrieval-augmented generation (RAG) over existing document repositories can surface useful information for certain query types, but is inherently limited to what is explicitly stated in text. A RAG system that retrieves a sentence mentioning a relief valve on a vessel has found a fact; a system that can verify from the P&ID whether that relief valve actually protects the vessel through the claimed flow path has demonstrated understanding. The latter can be audited against the engineering drawings themselves.

Key Takeaways

• Generative AI for process safety is coming fast — and may already be here for the right scope of tasks.
• Summary metrics rarely translate to “fit for purpose” for safety-critical applications. Stratified difficulty + binary dimensions produces actionable specifications.
• Effective models for P&ID interpretation should combine structured graph reasoning with VLMs — vision alone is insufficient for connectivity and flow direction.
• Small, purpose-trained models can outperform much larger general-purpose VLMs — with substantial practical advantages for on-premise deployment, regulatory predictability, and cost.
• The path forward requires better models, better data, and better evaluation — in concert, not isolation.