OpenAI GPT 2026: Function Calling, Structured Outputs, and JSON Schema

Checked on 18 August 2026. Field names and runtime behavior follow the OpenAI Function Calling guide and the Structured Outputs guide. This article does not invent latency, pricing, or success rates. Where public docs leave a gap, it says to confirm against live traffic rather than guessing from a blog post.

If you are pointing an OpenAI-compatible client at another model, structured output and the tool loop need their own acceptance tests. Swapping the Base URL is not enough. Use the in-site OpenAI API → Kimi K3 acceptance checklist as a parallel runbook.

Start here: three contracts changed at once

Plenty of production repos are still running a 2024 mental model: put “return JSON” in the system prompt, then scrape a fenced code block with a regex. On a 2026 agent path that is how you get schema breaks, missing keys, and enum hallucinations straight into the parser. The model can still “sound right” while the object you hand to billing, CRM, or a ticket queue is wrong in a way that only shows up after a human has already acted on it.

The fix is not a larger model alias. You have to rewrite three contracts that used to be one fuzzy blob of “please output JSON”:

Delivery contract

The object you give a user or a downstream service. That is Structured Outputs: text.format on Responses, response_format.json_schema on Chat Completions.

Execution contract

When the model must call your tools, arguments have to match the tool’s own JSON Schema. That is Function Calling. Under the hood it is the same constrained decoding family as Structured Outputs, but the payload lives on the tool loop, not in the user-facing answer.

Compatibility contract

Legacy JSON Mode (json_object) only promises “this looks like JSON.” It does not promise fields, types, or enums that match a schema. Official docs now treat it as the predecessor of Structured Outputs. New projects should not use it as the primary path.

There is a product-line change that is easy to miss if you only upgraded model names: new code should use the Responses API. Chat Completions still works, but default strict policy differs. Responses tries to normalize your schema into strict mode and falls back if that fails. Chat Completions still defaults to non-strict, best-effort arguments unless you set strict yourself.

Teams that treat this as a prompt-engineering problem usually spend a sprint tightening system text and then rediscover the same 400s or silent extra keys. The contract belongs in the request: schema, strict, and a parser that understands refusals. Prompts still matter for which values get chosen. They do not replace the type system.

If you are mid-migration, keep the old JSON Mode path behind a flag for one release. The point of the flag is not nostalgia. It is so you can replay the same traces against both contracts and see whether retries, missing fields, and human rework actually moved. Without that split, “we turned on json_schema” is a deploy note, not an acceptance result.

Models and the main path: gpt-5.6 + Responses

Structured Outputs has been available since the GPT-4o generation. For new projects, the documented recommendation is to use gpt-5.6 directly. Older snapshots such as gpt-4-turbo and earlier still map, in the docs, to JSON Mode rather than full json_schema strict output. Do not assume a gateway alias named “gpt-5” is the same snapshot your SDK helper was tested against. Pin the model string in config and fail closed if the gateway remaps it.

Responses is the path the current examples, parse() helpers, and schema-normalization behavior are written around. Chat Completions remains valid for existing services, especially if you already have middleware that inspects choices[0].message. The migration cost is real: item types, streaming events, and where format lives all change. What you should not do is stay on Chat Completions forever because the old client “already returns JSON.” JSON Mode on Completions and Structured Outputs on Responses are different guarantees.

Pick the entry point before you pick the model

On the SDK side, the habit worth keeping is: do not hand-write a schema that forgets nested additionalProperties. Generate from types with the official helpers. In Python that is client.responses.parse(..., text_format=YourModel). In JavaScript it is zodTextFormat. If you do write Schema by hand, a strict: true request that violates the subset is rejected at request time. You do not get a “model tried its best, retry in the app” loop. That is a feature: broken contracts fail in CI instead of in the ticket queue.

When you compare this path to Gemini, do not read “OpenAI SDK compatible” as “schema behavior is identical.” Google’s capability path is covered separately in Gemini 3.5 Pro: 10 AI capability upgrades. Copying one JSON Schema across vendors often dies first on nested additionalProperties. Keep a single business schema and a per-provider transform, not three slightly different copies of the same ticket object.

Batch, fine-tuning, and regional availability can lag the headline model name. Before you declare gpt-5.6 the default in every environment, run one minimal parse() request against the same project, region, and API key your agents will use. If that request 400s or silently lands on JSON Mode, fix the account and gateway mapping before you rewrite every tool schema. Docs describe the intended default; your tenant is the source of truth for what is actually enabled.

JSON Mode, Structured Outputs, Function Calling

The production mix-up that shows up in incident reviews is simple: the logs contain JSON, so the team assumes Structured Outputs is on. Valid JSON is the floor, not the contract. JSON Mode will happily emit extra keys, omit optional-looking fields, or invent an enum member that is not in your list. Your parser then either drops data or throws after the model already spent tokens. Split the three capabilities the way the docs do:

Use Structured Outputs when the model is assembling a card, a score, a checklist, or any object that is the product. Use Function Calling when the model is requesting a side effect: look up inventory, open a ticket, run a script, charge a wallet. If you stuff function-shaped fields into the user-facing schema, you will either skip the executor (and never actually run the side effect) or run it twice because another layer also parsed “intent.” One object, one job.

JSON Mode still has a narrow role: throwaway prototypes, models that do not support json_schema, and emergency fallbacks when a partner gateway cannot accept your strict subset. Treat those as explicit exceptions in config, not as the default in a shared client. Once two teams share a “return JSON” helper, JSON Mode tends to leak into paths that already have a Pydantic model and a parser that expects required keys.

When Function Calling is the wrong tool

If the model does not need to touch your systems—no inventory lookup, no ticket mutation, no script—and you only need the answer broken into cards, steps, or scores, use Structured Outputs. If the output is “please perform this side effect,” it must go through tools. Do not disguise function arguments as the final answer schema. That pattern looks convenient in a demo and becomes undebuggable once you have both a UI renderer and an executor reading the same blob.

A refusal is not “bad JSON”

On a safety refusal the model will not cram an object into your schema. Responses and Chat Completions expose a separate refusal field. Treat refusal as a first-class outcome: inspect it before output_parsed. An empty object is not success. Streaming UIs should show a refusal state rather than a skeleton of nulls that looks like a half-filled form. If your metrics only count parse failures, refusals will look like quality regressions instead of policy working as designed.

Streaming deserves an extra check. Incremental JSON can look complete in the middle of a chunk and still change before the final parsed object. If you render fields live, keep a “provisional” flag until the helper says parsing finished. Compare the last streamed snapshot to output_parsed in staging; mismatches here are why some teams thought Structured Outputs “flickered” when the bug was the UI committing too early.

Hard rules for strict JSON Schema

With strict on, OpenAI accepts a subset of JSON Schema, not an arbitrary Draft 2020-12 document. The three request-level walls teams hit first:

1. Every key in properties must also appear in required.
2. Every object, including nested ones, must set additionalProperties: false.
3. The root object cannot be anyOf. Optional meaning is “required, but nullable,” for example ["string", "null"].

That last point is the one that surprises people coming from OpenAPI. In strict mode, dropping a field from required to fake optionality is a 400, not a softer schema. Keep the field required and type it as a nullable union. Application code treats null as “not provided.” That is slightly noisier in types and much quieter in production, because missing keys and explicit nulls no longer mean two different things.

Nested objects are where hand-written schemas usually fail. You remember additionalProperties: false on the root and forget it on address, metadata, or a list item. Strict validation is recursive. A playground dump that already has strict enabled is often faster to paste than incrementally “upgrading” an old json_object prompt. When the request is rejected, read which constraint is missing in the error. Downgrading the model almost never fixes a subset violation.

The snippet below is a typical “extract a ticket” object. Nested objects in a larger schema would need the same additionalProperties treatment; the SDK helper generates that from the type:

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class Ticket(BaseModel):
    title: str
    priority: str
    assignee: str | None
    tags: list[str]

response = client.responses.parse(
    model="gpt-5.6",
    input=[
        {"role": "system", "content": "Extract ticket fields from the user description."},
        {"role": "user", "content": "Login page 500, assign to Noah, high priority, tags auth and api."},
    ],
    text_format=Ticket,
)
ticket = response.output_parsed
print(ticket.title, ticket.priority, ticket.assignee)

Enums are the other quiet failure. Strict mode will not invent a value outside the list, which is good. It will still pick a listed value that is wrong for the case. If your enum is a grab-bag of historical statuses, the model will land on the closest label and your workflow engine will take it as gospel. Keep enums small, document them in field descriptions, and validate against live catalogs (assignees, SKUs, queue names) after parse.

Across vendors, run the same check again: a schema that sets additionalProperties: false on every object can 400 on some compatible gateways or other models. Transform per provider instead of maintaining three business schemas. Your domain object should not fork just because one stack is stricter about extra keys.

If you generate schemas from Pydantic or Zod, look at the emitted JSON once in CI. Helpers usually get required keys and nullability right; they can still emit constructs the OpenAI subset rejects (certain anyOf shapes, unconstrained objects, or formats you never meant to send). A snapshot test of the schema is cheaper than discovering the 400 only on the first production parse.

Function Calling in 2026: strict, tool_search, custom tools

Docs now treat Function Calling and tool calling as the same idea: describe callable functions with JSON Schema, then run side effects in your executor. Several 2026 details change the agent loop even if you already had tools in 2024. Parallel calls, deferred tool search, and custom tools with grammars all add item types your old “if function_call then run” switch statement will not recognize.

The executor is still your code. Constrained decoding stops malformed arguments; it does not know your rate limits, idempotency keys, or whether two tools should run in the same turn. If the model asks for refund_order and send_email together, you decide order, transactions, and what to return on partial failure. Log the final strict flag on each tool in the response. On Responses, omitted strict can mean the server rewrote the schema; if that rewrite failed, the tool may come back as non-strict without you noticing unless you record it.

Do not guess the default for strict

• Always set strict: true explicitly.
• Responses: if you omit strict, the server tries to normalize the schema; if normalization fails it falls back to non-strict, and the tool in the response shows strict: false.
• Chat Completions: omitting strict defaults to non-strict.
• Fine-tuned models that call multiple functions in one round may have strict disabled for that round, per the docs.

Tool definitions sit in context and bill as input tokens. Long descriptions and forty tools on every call raise cost and usually hurt tool selection. When the catalog is large, use tool_search to defer uncommon tools. That path exists only on gpt-5.4 and newer. The loop may emit tool_search_call / tool_search_output before a real function_call. An executor that only understands function calls will stall. Replay those two-step traces in staging before you enable search in production.

Trim descriptions before you reach for deferred loading. A one-line “when to call / when not to call” beats a paragraph of edge cases the model will not reliably follow. Group tools by product area if you still inject a subset by hand. tool_search is for catalogs that cannot fit; it is not a substitute for deleting dead tools.

Custom tools: do not jam a DSL into a JSON object

Function tools fit structured arguments. Custom tools fit free-text input and output, with an optional context-free grammar (CFG). SQL fragments, internal query languages, and formats whose tokens must be mutually exclusive are more stable with a CFG than with a string field plus a pile of prompt rules. If the CFG reports unexpected tokens, check overlapping terminals first. That is usually a grammar bug, not a model quality issue.

tools = [{
    "type": "function",
    "name": "get_order",
    "description": "Look up order status by id. Call only when the user provides a concrete order id.",
    "strict": True,
    "parameters": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string"},
            "locale": {"type": ["string", "null"]},
        },
        "required": ["order_id", "locale"],
        "additionalProperties": False,
    },
}]

The execution loop itself is unchanged: see a tool call in finish_reason or item type → run the local function → return the result in the tool role → request again. What changed is that you should not need json.loads as a gamble on arguments. You still must persist the full assistant message, including tool_calls. Dropping call ids is how the second turn loses the thread. Store them even if you only display the final user-facing text.

Timeouts and retries belong in the executor, not in another model round, unless the tool truly needs a different argument. If get_order 504s, retry the HTTP call with the same order_id. Asking the model again often produces a second call with a slightly different id, which is how you double-charge or double-notify. Idempotency keys on mutating tools are still your problem after Structured Outputs landed.

Migration checklist for existing stacks

Split “it runs” from “we can ship this object.” Keep the old backend behind a switch and replay real traffic in an isolated environment. Synthetic prompts will not show you the enum values, nested blanks, and multilingual tickets that already live in production logs.

1. Model: pin new paths to gpt-5.6 (or the equivalent tier your account actually has). Do not let a gateway silently map to an old snapshot.
2. Output: replace json_object with json_schema + strict: true, or Responses text.format.
3. Tools: fill required and nested additionalProperties: false on every function; optional fields become nullable unions that stay required.
4. Parsing: wire parse() and a refusal branch; on streaming, confirm incremental JSON matches the final parsed object.
5. Tool surface: once you have more than a dozen tools, evaluate tool_search (gpt-5.4+). Shorten descriptions first, then defer loading.
6. Compare: same fixed task set against old JSON Mode versus the new schema path—retry rate, missing-field rate, human rework. Measure those on your traces; do not import someone else’s percentages.

Passing is not HTTP 200. Passing is: no regex safety net in the parser, stable tool argument types, refusals in observability, and a rollback switch you have actually flipped. Long-running SDK sessions, replay scripts, and a browser for the streaming UI do not mix well with a laptop that sleeps when the lid closes. That is the practical reason for a always-on Mac in the next section—not a slogan about “AI cloud,” just a box that stays awake while two contracts run overnight.

Roll out by route, not by global client default. A support-ticket extractor can move to Structured Outputs while a legacy batch job stays on JSON Mode until its schema subset is clean. Feature flags per route also give you a place to log which contract produced each object, which is the first question you will ask after a bad write to a downstream system.

FAQ