What Changes When an AI Model Returns Decisions Instead of Text?

Built for inspection day.
From day one.^™

A typed output removes one class of AI failure. It also creates a much cleaner path from model error to regulated action.

A complaint reaches the quality system.

The AI does not draft a narrative. It receives the available record as state and evaluates one question: does this event meet the threshold for regulatory reportability?

The result is 0.94.

Ordinary software then applies a rule:

if probability >= 0.90 → escalate to regulatory affairs

There is no paragraph to review. No malformed JSON to reject. No hallucinated field name. The model returned exactly the type the software expected.

The workflow changed state anyway.

Once AI produces a choice, score or probability designed to be consumed directly by code, validating the shape of the output stops being the hard part. The harder work moves into the decision policy that turns that output into action.

Jev makes the distinction unusually clear

TypeSafe AI launched Jev in September 2026 as what it calls a System One model. Its interface is deliberately different from a chat model. Software supplies state and typed questions. Jev returns one of three structured forms: Choice, Score or Noul. Choice and Score include probability distributions and a confidence value; Noul returns a probability between zero and one. The intended pattern is for ordinary code to branch, rank, route or escalate from those results.

That is a meaningful architecture change.

A generative model can fail by inventing text, violating a schema or producing an answer that cannot be parsed. Jev constrains the answer space in advance. TypeSafe describes this as eliminating type errors and positions calibrated probabilities as the mechanism software should use when deciding whether to act or seek review.

Type safety is a property of the output format. Decision quality is a separate question entirely, one that type safety alone was never built to answer.

TypeSafe’s own documentation for Jev 1.13 says the model can struggle with literal interpretation, numerical precision, irrelevant context, indirection and adversarial content. It also warns that apparently equivalent formulations do not necessarily produce probabilities with the structural relationships a developer might assume.

That distinction matters in GxP.

A wrong sentence often creates something a person can inspect. A wrong probability can disappear directly into an if statement.

The validated object is the decision contract

For this class of system, I would define the validated object more broadly than the model.

It is the decision contract connecting the regulated process to the action software is permitted to take. Navata’s work on AI architecture and assurance starts from that same definition: what Quality intended, what was built, and what the evidence actually proves.

• State: which data is supplied, what may be missing, and whether the validation set represents the real operating population.
• Decision definition: exactly what each class, score or yes/no question means, including boundary cases.
• Probability: whether observed correctness corresponds sufficiently to the probabilities produced for the intended population and relevant subgroups.
• Threshold: why a particular probability or confidence is sufficient for a particular action, given the consequences of false positives and false negatives.
• Fallback: what happens when confidence is insufficient, inputs are outside the validated boundary or the options do not fit.
• Authority: what the software may actually change, route, block or approve once the threshold is crossed.
• Lifecycle: which model, question definitions, preprocessing and thresholds were validated, and what changes trigger reassessment.

A technically correct API contract does not guarantee the decision contract behind it would survive inspection.

That is where Quality intent, digital architecture and validation evidence have to meet.

Related on Navata Insights: The Most Important AI Control May Be the Action It Cannot Take looks at execution authority for agents that read, interpret and recommend before a person decides. This piece is about a different kind of model: one that never interprets at all, and the authority question it raises starts one step earlier, at the threshold itself.

Accuracy and calibration answer different questions

Suppose a classifier is 97% accurate.

That tells me very little until I know what constitutes the other 3%.

If the system classifies 9,700 routine records correctly and misses 300 of the 300 genuinely critical records, aggregate accuracy is excellent and the system is useless for the intended purpose.

Performance has to be assessed against the consequence structure of the actual decision. Sensitivity, specificity, precision, false-negative rate and subgroup performance can all matter depending on the intended use.

Calibration adds another question.

TypeSafe describes a calibrated probability of 0.8 as meaning that, across comparable predictions assigned that probability, the outcome should be correct approximately 80% of the time. It explicitly notes that this is a property of groups of predictions, not a guarantee for an individual answer.

An 80% calibrated decision is still wrong roughly one time in five across that population. Whether software may act on it depends on what those errors do.

The threshold is an architecture decision

A model can produce 0.91. Whether 0.91 is enough to change a regulated workflow is a separate question for the organisation to answer.

TypeSafe itself makes this distinction in its confidence guidance. It recommends different thresholds for different consequences and says threshold values should be established against the performance of the model for the specific use case.

For regulated use, I would go further. The validation evidence should demonstrate the operating point actually deployed, not merely model performance across the whole test set.

If automatic routing begins at 0.95, measure false positives and false negatives above 0.95. Test the cases around the boundary. Determine whether the result holds for rare but material subgroups. Then measure coverage: what proportion of real records will cross that threshold and therefore be acted on automatically.

Coverage is operationally important. Moving the threshold from 0.90 to 0.98 may reduce model error while sending half the workload back to manual review. That may be the correct control. It may also produce a backlog that defeats the escalation process the model was meant to protect.

The validated state therefore includes both sides of the trade-off: residual decision error and the volume left for humans.

“Undecided” needs to be a real operating state

Confidence only helps when uncertainty changes behaviour.

A low-confidence result that reaches the same downstream process as a high-confidence result only produces observability. Nothing changes for the person or system that has to act on it.

The architecture needs an explicit abstention path: route for human classification, request missing information, apply a deterministic fallback, or prevent the consequential transaction from occurring.

The harder case is the confidently wrong answer.

Calibration cannot tell you which individual 0.99 result belongs to the residual error population. High-confidence automated decisions therefore still need a detection mechanism appropriate to their consequence: sampling, reconciliation, downstream exception monitoring, reviewer overrides or another independent signal capable of exposing systematic error.

Otherwise the cases least likely to receive human attention are also the cases the software treats as safest.

Regulators are already pointing towards this assurance model

The FDA and EMA’s January 2026 Guiding Principles set risk-based validation, oversight and lifecycle management as baseline expectations, without prescribing confidence thresholds. Credibility depends on the particular use and its risk, evaluated against the actual population and consequence at hand.

The European Commission’s draft GMP Annex 22 goes further for models within its scope: intended use, predefined performance metrics, independent test data, confidence logging, threshold setting, an “undecided” result when confidence is too low, and ongoing monitoring. That text remains draft as of September 2026, and its current scope excludes models where identical inputs may produce different outputs. EMA’s 30 June–1 July 2026 workshop asked experts directly how Annex 22 could accommodate adaptive and probabilistic models, and what validation paradigm that would require.

One terminology trap is worth avoiding: a model returning a probability is not automatically a “probabilistic output” in the draft’s sense, since that term turns on repeatability for identical inputs. Before classifying a Jev-based implementation either way, I would want supplier evidence on its actual execution characteristics.

Version control can invalidate the threshold

This becomes especially important when the model is externally supplied.

TypeSafe currently documents both versioned Jev model IDs and movable aliases such as jev-latest. Its documentation states that an alias can move when a release changes, that answers may therefore change without an application change, and that customers who have tuned confidence thresholds against a particular version should pin that version and move deliberately.

That is not a minor MLOps detail in a regulated implementation.

If a 0.96 threshold was justified using labelled evidence from one model version, changing the model can change the relationship between probability and observed error. Changing the question wording, classification criteria, preprocessing or source population can do the same.

The threshold and the evidence supporting it have to move together.

The reliance decision sits with Quality

Quality’s job is to approve reliance on the model. That judgement should rest on more than the probability alone.

Quality needs to define what decision is being influenced, which mistakes matter most and which actions remain subject to accountable human judgement.

Digital needs to translate that into state design, model selection, thresholds, fallback behaviour, permissions, logging and monitoring.

A delivery partner can implement those controls.

Validation has to demonstrate that the implemented operating point produces acceptable performance on appropriate evidence.

Operations then has to detect when the population, model or process moves away from the condition that was validated.

That continuity matters more with decision models because the output can travel from inference to execution without ever becoming something a person naturally stops to read.

The typed response is therefore the easy part.

Before allowing a decision-focused model to route, classify, gate or influence a GxP action, one documented answer is worth having to a harder question: what exact evidence justifies the point at which a probability becomes permission for software to act?

About the author

Rohith Karanam Sreedhar

Founder & Principal

Rohith founded Navata after eighteen years building, validating and operating Veeva and quality systems inside global pharma, including leading Vault QMS architecture at Pfizer. He now helps regulated life-sciences teams find and own the architecture decisions already running inside their platforms.

More from Navata Insights → Explore all insights