The Bellagio job in Ocean's Eleven works because nobody's deliverable is vague.
Basher does not promise to "disrupt the power somehow." He kills the lights across the Las Vegas strip for a specified window, at a specified moment. Livingston delivers a video feed of one specific vault. The acrobat fits inside one specific cart. Eleven specialists, and every handoff between them is defined before anyone walks into the casino.
Now imagine Basher instead saying: "Yeah, the power situation should be mostly handled, give or take, depending on how the grid feels."
That is what you get when you put a language model in the middle of a software pipeline and let it answer in prose.
The Problem Nobody Budgets For
Language models produce text. Software consumes data. Somewhere between those two facts lives an enormous amount of duct tape.
The typical journey goes like this. A team asks the model to extract invoice details and return them "as JSON." It works in testing. In production, the model occasionally wraps the JSON in a friendly sentence. Or adds markdown code fences. Or renames a field. Or returns a number as a string. Each failure gets patched with string cleanup, a regex, a retry loop, a second model call to fix the first one.
Six months later there are 400 lines of parsing code guarding one model call, and it still breaks once a week.
Asking nicely in the prompt does not fix this. "Respond ONLY with valid JSON" is not an engineering control. It is begging.
What Structured Outputs Actually Do
Structured outputs move the requirement out of the prompt and into the API itself. You define a schema: these fields, these types, this shape. The provider enforces it during generation.
The enforcement mechanism is worth understanding, because it explains why the guarantee is real. A language model generates one token at a time, choosing from a set of candidates at each step. With a schema in place, the provider masks that choice: any token that would violate the schema is removed from the candidate list before the model picks. The model literally cannot produce an invalid character.
This is not validation after the fact. It is a constraint during generation. The output matches your schema on the first try, every try.
One distinction trips people up. JSON mode, which most providers shipped first, only guarantees the output is valid JSON. Valid JSON is a low bar. {"lol": "good luck"} is valid JSON. Structured outputs guarantee your JSON: your fields, your types, your enums. If you are choosing between them, this is not really a choice.
Guaranteed Shape Is Not Guaranteed Truth
A schema controls structure. It has no opinion about content.
If your schema requires an invoice_number field and the document has no invoice number, the model still has to put something there. Constrained to produce a value, it will produce one. A plausible one. This is how structured outputs quietly manufacture fabricated data with a perfectly valid shape.
The fix is schema design, not more prompting.
Make fields nullable or optional when the information might legitimately be absent. Give the model a way to say "not found" that is not a lie.
Use enums for closed categories, and include an escape hatch. A category field with five allowed values and no other option forces every edge case into the wrong bucket.
Use field descriptions as instructions. Schema descriptions get read by the model just like prompt text, and they are the most precisely targeted prompting surface you have. A description on the exact field, at the exact decision point, beats a paragraph of rules three thousand tokens earlier.
And for anything that matters, keep semantic validation downstream. Check that the extracted total matches the line items. Check that dates are in plausible ranges. The schema got you a clean handoff. It did not get you truth.
Where This Belongs in Your Stack
The test is one question: who consumes the output?
If a human reads it, prose is the right format. Chat responses, summaries, explanations, drafts. Forcing structure onto writing meant for people usually makes it worse.
If code consumes it, structure is the right format. That covers more of your system than you might think: extraction from documents, classification and tagging, routing decisions in multi-agent pipelines, form filling, report generation from templates, tool call arguments, evaluation scoring. Every one of those is a Basher handoff. The next stage needs specific fields with specific types, and anything less specified is a production incident on a timer.
In our own multi-agent work, every internal handoff is schema-enforced. The orchestrator's routing decision is an enum, not a sentence. The SQL worker returns a defined result object, not a description of what it found. The only prose in the whole pipeline is the final answer, because that is the only part a human reads.
The pattern held up for a simple reason: every place we removed prose from a handoff, a class of intermittent bugs disappeared with it.
The Payoff
Teams adopt structured outputs for reliability and then discover the second-order benefits.
The parsing code gets deleted. All of it. The retry loops, the regex, the cleanup functions.
Failures become visible. When the model cannot fill a field honestly, a nullable schema surfaces that as a null you can count, instead of a fabrication you cannot see.
And the system becomes testable. You can write assertions against fields. You cannot write assertions against vibes.
Look at your codebase this week and find every place a model's text output gets parsed by code. Each one is a schema you have not written yet. Write the schema, delete the duct tape, and let the model answer in prose only where a person is actually listening.
