Why Are We Using Chat Models to Make Decisions?

For years, the default pattern has been to put an LLM in the middle of almost every AI workflow: need to classify something, ask the LLM; need to route something, ask the LLM; need a risk score, ask the LLM. Then you take the response and run it through text, parse, validate, apply business logic. That's an architectural smell — the model was designed to generate language, and the application needed a decision.

That mismatch gets expensive in production. You end up needing structured-output handling, parsing and validation, retry logic, schema enforcement, guardrails, and a pile of extra failure states — and the business logic starts leaking into prompts, which makes the whole thing harder to reason about.

A more interesting architecture is state, question, decision, code. That's the idea behind TypeSafe's Jev, which evaluates typed questions against application state and returns structured results directly instead of generated text. Instead of asking a model to "analyze this customer and decide what we should do," you separate the decisions — is the customer frustrated, what type of request is this, how severe is it — and let ordinary application code combine those signals. AI makes the judgment; code owns the decision policy.

That same principle generalizes past Jev. LLMs are excellent for generation, reasoning, summarization, and conversation. Decision-oriented models are better suited to classification, routing, scoring, and structured branching. And deterministic code should still own authorization, business rules, state changes, and system-of-record writes. The future of AI architecture probably isn't "put an LLM everywhere" — it's using the right intelligence primitive for each part of the system.

One principle worth keeping: The model should provide judgment; your code should still own the decision.

Where in your current system are you using an LLM to make a decision that could be represented as a structured primitive instead?