Combining engineering-grade precision with web-scale knowledge

Visit enough factories and you quickly notice two contradictory truths.

First, no two shops have anything in common. One is machining titanium brackets on five-axis mills. Another is vacuum-forming packaging by the thousand. A third runs a farm of polymer printers overnight without a single operator on the floor. A fourth welds stainless steel assemblies to a standard no one would attempt by eye. They use different processes, materials, and tolerances, operating in conditions far removed from the tidy control of a laboratory. Ambient temperatures drift, batches vary, tools wear out, and keeping output within spec usually relies on an experienced operator noticing a subtle anomaly and adjusting on the fly.

Second, every one of these plants relies on the exact same foundation: thousands of years of documented engineering knowledge. Metallurgy, thermodynamics, polymer science, heat transfer, and fracture mechanics are codified across handbooks, machine manuals, standards, and forum posts written at two in the morning by someone who just solved your exact problem. The welder and the extrusion technician are simply applying different branches of the same unified body of knowledge.

This written record is precisely what modern foundation models have absorbed. They don't know your specific production line, but they know how matter behaves. Our research asks a fundamental question:

Can a model holding web-scale engineering knowledge become precise enough to act on a physical shop floor?

The Unpredictability Challenge

Machine learning has been applied to process monitoring for years, almost exclusively in a narrow format: label hundreds of examples of a defect, train a model, and let it spot that defect on the line.

This works until something unfamiliar happens. The moment an unexpected fault occurs, a traditional vision model breaks down because it never understood the underlying physics—it only learned the visual pattern of specific labeled errors. Yet the defects that cost a factory a week of downtime are rarely the ones anyone thought to label in advance. No plant can afford to label every failure mode into existence, one defect, one machine, and one material at a time.

Human operators do not work this way. Faced with an unfamiliar issue, an engineer reasons from first principles: the melting point of the polymer in the hopper, why a flexible filament buckles when driven too fast, or how fluid viscosity responds to heat.

Large foundation models already hold this first-principles knowledge. The bottleneck is that they cannot produce a physical measurement you can trust.

The Representation Trap

If you ask a standard vision-language model why a semi-crystalline polymer warps as it cools, it will give you a textbook answer. But if you show it a live camera feed of hot material leaving a nozzle and ask for the volumetric flow rate, it will either decline or invent a plausible number.

Neither is usable. A refusal halts the line; a guess risks destroying it. Neither human nor machine can act on ungrounded noise.

This is not a training data deficit; it is a structural flaw. When we fine-tuned a foundation model on our own process imagery, it quickly learned to write fluent, expert-sounding captions describing extrusion quality. Yet, it consistently failed to quantify the material output. Training it longer or unfreezing more parameters did not help. Because language models optimize over discrete tokens, nothing in their internal architecture understands spatial geometry or continuous numerical scales. Fine-tuning an LLM to "see" continuous quantities is like asking someone to estimate millimeter tolerances with their eyes closed using only text descriptions.

The CIPHER Architecture

To solve this, we stopped asking one model to do everything. Our system, CIPHER, separates perception from reasoning through a strategy we call hybrid reasoning: small models where precision matters; large models where knowledge matters.

In-Situ Vision Monitoring: A camera mounted directly on the machine head watches material as it is deposited, synchronized with real-time firmware telemetry. This monitors the process at the exact point of physics creation, rather than inspecting a damaged part after the fact.

The Process Expert: A small, lightweight convolutional neural network is given a single, dedicated job: extract precise, continuous physical metrics (e.g., true volumetric flow rate) from the visual feed.

Direct Token Projection: The Process Expert’s numerical output is projected directly into token space as the leading token of the sequence. Rather than hallucinating a value, the language model is simply handed the true physical measurement as an unquestioned fact.

Knowledge-Grounded Reasoning: Supplied with the measured state alongside retrieved context from a curated manufacturing physics database and machine firmware manuals, the LLM diagnoses the root cause and articulates the issue in plain language.

Actionable Code Generation: The model translates its reasoning into executable machine control code (e.g., G-code) looked up and validated against the machine's specification, closing the control loop at speeds approaching conventional controllers.

Preserving General Knowledge

Adapting foundation models too aggressively causes catastrophic forgetting. In early experiments where we fine-tuned the entire network, the model lost its reasoning breadth and began absurdly describing ordinary everyday photographs using the language of extrusion faults. To protect its built-in engineering inheritance, we kept both the vision encoder and the language model frozen, training only the linear projection layer between them. Measurement errors dropped sharply while the model's vast reasoning capacity remained completely intact.

Empirical Reasoning in Action

What validates hybrid reasoning is not synthetic benchmarks, but emergent physical problem-solving across entirely different manufacturing domains. Without retraining, CIPHER transferred across 3D printing, machining, vacuum forming, and welding, demonstrating reasoning behaviors that mirrored an experienced plant engineer:

Viscosity Management: Raised nozzle temperatures independently to increase flow rate, acting purely on the thermodynamic relationship between temperature and viscosity.

Fault Prevention: Lowered feed rates proactively when handling flexible filaments to prevent mechanical buckling before it occurred.

Process Interruption: Identified a nozzle set-point below the material’s melting threshold and corrected the firmware parameter.

Knowing When to Stop: Assessed a severely compromised build midway through execution and recommended abandoning the print entirely—the judgment of a skilled operator who understands process economics rather than a script blindly adjusting parameters.

When evaluated against domain experts, CIPHER’s corrective commands scored on par with the strongest general-purpose LLMs, but with ground-truth precision backing every instruction.

Prompt-to-Part: Generative Production

We extended this hybrid architecture from correcting parts to generating them from scratch. Given a verbal description or a target image, CIPHER decomposes the geometry into simple spatial primitives, writes parameterized code for each element, and outputs a validated machine instruction file. Successful geometries are stored in an execution library, making the system progressively faster at recurring features.

This highlighted a clear boundary in current AI spatial capabilities: regular geometric primitives succeeded reliably, but organic, freeform shapes exposed severe spatial limits. When relying on pure shape generators, the model produced visually convincing but physically impossible objects—such as a stylized llama with legs too thin to sustain printing, or sunglasses whose temples completely ignored overhang constraints.

Calibrated Confidence and Operational Safety

In manufacturing, a confident wrong number is the most dangerous output a system can generate. To guarantee operational safety, we tracked CIPHER’s token-by-token perplexity during execution.

Without the Process Expert: The standalone LLM was diffuse and poorly calibrated, hedging toward the middle of numerical ranges to play it safe.

With the Process Expert: Token probability tightly concentrated around two or three adjacent physical values. Rather than conceptual confusion, the model exhibited slight hesitation between neighboring physical tolerances.

This distinction dictated our deployment strategy. While a hard-coded perception model offered artificial certainty, we intentionally deployed the variant that retained calibrated hesitation. On an active production line, visible uncertainty is an essential safety feature—it allows the system to defer to a human operator or withhold an action rather than committing to a setting it cannot guarantee.

Furthermore, because CIPHER reasons from structured retrieval and local physical data, manufacturers do not need to transmit proprietary process data offsite, and every automated machine action is accompanied by a human-readable explanation.

Beyond Moravec’s Paradox

There is an old observation in robotics known as Moravec’s paradox: the things machines find easy (arithmetic, high-speed repeatability) and the things humans find easy (perception, contextual understanding) are exact opposites.

A factory floor is the purest illustration of this paradox. It is full of multi-axis machines capable of holding a position to within a single micron all day, yet they remain utterly blind to what is happening right in front of them.

What hybrid reasoning changes is that we can now bridge that gap. By coupling small, hyper-precise perception networks with large, knowledge-rich reasoning engines, we no longer have to choose between a machine that understands the laws of physics and one that can measure them.

‍