2026 AI Agent Produces Bad JSON? Trace the Data Flow

AI Agent JSON errors should be traced through the full data flow, not blamed on the model by default: input contract, model generation, API response, parsing, validation, tool execution, and result return must be checked in that order. If the platform supports strict structured output, enable it first, then keep application-level semantic validation.

This guide is for backend developers investigating parser failures, missing Function Calling arguments, and unusable tool results. It also helps engineers maintaining multi-step Agents and operations teams that need correlated production logs.

Start with the failure location, not the prompt

A typical incident looks deceptively simple:

1. The Agent requests a file operation.
2. The model returns a tool call.
3. The application parses the arguments.
4. The parser reports invalid JSON.
5. A retry produces another malformed response.
6. The team changes the prompt, but the real problem is an unsupported schema keyword or a dropped call identifier.

The visible error is often several layers away from the original fault. A useful incident trace therefore separates these payloads:

Data-flow stage	What to capture	Typical failure signal	First action
Input contract	Schema, interface settings, model and SDK versions	Request rejected before generation	Reduce to a minimal accepted schema
Model generation	Completion status, refusal state, stop reason	No usable content or incomplete content	Branch on status before parsing
API response	Raw response envelope and call identifiers	Fields differ from application assumptions	Compare the documented response shape
Parse and validate	Parser result, validator version, exact error path	Invalid syntax or schema violation	Reproduce outside the Agent
Tool execution	Normalized arguments, permissions, resource checks	Valid JSON but failed action	Add semantic and authorization checks
Result return	Tool result, state, call ID, replayed context	Later step loses history	Verify lossless state round-trip

This separation matters because each row needs a different fix. A schema rejection is not repaired by a retry. A lost tool result is not repaired by stricter JSON generation. A permission failure is not a formatting failure.

Diagnostic rule: preserve the raw API response before any JSON parsing, coercion, redaction, or field renaming. Otherwise the application may destroy the evidence needed to identify the failing layer.

First decision: contract, response, or execution?

The fastest way to reduce the search area is to classify the incident using observable evidence.

Symptom	More likely location	What it does not prove	Correct next check
HTTP or API request rejected	Input contract or request configuration	That the model cannot produce the requested shape	Validate the schema and supported feature subset
Refusal or empty usable content	Model response state	That the response is malformed JSON	Read refusal and completion fields before parsing
JSON ends midway through a string or object	Length limit or interrupted transport	That the schema was ignored	Inspect stop reason, output limit, and stream assembly
Parser accepts JSON but validator rejects it	Schema dialect or data mismatch	That the parser is broken	Record validator version and exact path
Validator accepts it but the tool fails	Business, permission, or resource layer	That the Agent generated the wrong syntax	Revalidate arguments against live system state
First call works, later call loses context	State or call-identifier handling	That the model forgot the conversation	Compare serialized request history with the successful trace

For schema behavior, the $schema declaration is not decorative metadata. It identifies the intended JSON Schema dialect, while implementations may support only a subset. The JSON Schema specification overview explains the versioned specification structure, and the JSON Schema basics guide documents how $schema declares the dialect.

The practical implication is simple: record the schema text, dialect, validator library, and validator version as one unit. A development environment that silently defaults to one dialect can disagree with production even when the schema file is unchanged.

Contract check	Minimal test	Restore later
Object root	One object with one scalar property	Nested objects and arrays
Required fields	Only the field needed by the first tool step	Full business-required set
Primitive types	String, number, boolean, or array tested separately	Formats and conditional constraints
References	Inline definitions first	$ref and reusable components
Business constraints	Application validation outside the generation schema	Range, relationship, and authorization rules

Start with the smallest contract that the target interface accepts. Add one constraint at a time. This method identifies the first unsupported feature instead of producing a large request whose error message points nowhere useful.

Step 1: verify the schema before changing the prompt

A Function Calling argument error often starts with a mismatch between the declared tool and the application’s expectation. The following checks should be explicit:

• Is the root type the type required by the interface?
• Does every required property exist in the declared properties map?
• Are property names identical across the schema, executor, and result mapper?
• Does the interface support the chosen formats, unions, references, and nesting?
• Are optional fields handled when they are absent rather than assumed to be present?
• Does the SDK transform the schema before sending it?
• Is the same schema used for generation, validation, and tool execution?

The Gemini Structured Output documentation lists supported schema types and limitations. Those restrictions are operationally important: a schema that is valid under the general specification may still be rejected by a specific model interface.

A good minimal reproduction contains one tool, one input object, one deterministic instruction, and one validation step. It should not include the complete production prompt, every tool, or a live destructive action. The goal is to answer one question: can the interface accept and return this contract at all?

Step 2: branch on refusal, truncation, and API errors

The application must not send every response body directly to a JSON parser. A response can be unavailable for several distinct reasons:

• The request was rejected before model generation.
• The model returned a refusal or safety-related state.
• The output reached a length boundary.
• A streaming connection ended before the final event.
• The API returned an error envelope rather than the expected content.
• The client assembled events in the wrong order.

For streamed responses, the Responses API refusal event reference shows why a refusal delta must be handled as a response state, not parsed as the requested business object. The same principle applies to completion metadata and transport errors: inspect the envelope first, then decide whether parsing is appropriate.

A robust branch looks conceptually like this:

receive response
  if transport or API error:
      record error and stop
  if refusal state:
      record refusal and stop
  if incomplete or length-limited:
      record stop reason and stop
  if no complete tool call or structured payload:
      record missing output and stop
  otherwise:
      parse raw content
      validate syntax
      validate schema
      validate business rules
      execute tool

Retries belong only on a defined retryable branch, such as a transient transport failure or a documented service-unavailable response. Retrying malformed output without preserving the original status can turn one clear failure into a noisy loop, consume budget, and hide the first fault.

Production reminder: a parser exception is not a safe retry signal. The retry policy must know whether the response was truncated, refused, rejected, or corrupted during transport.

Step 3: separate parsing from schema and business validation

There are three different questions:

1. Can the bytes be decoded as JSON?
2. Does the resulting value conform to the expected JSON Schema?
3. Can the business system safely execute the requested action?

Conflating them produces misleading tickets such as “bad JSON” when the real issue is an inaccessible resource.

The OpenAI Structured Outputs guide describes platform-level structured generation, but strict output does not remove the need for semantic checks. A structurally valid object may still contain an account identifier that does not exist, a path outside the allowed workspace, an unsupported operation, or a date relationship that violates policy.

Application validation should therefore include:

• Existence: does the referenced account, order, file, or environment exist?
• Authorization: is this caller allowed to perform the action?
• Scope: does the path, project, or tenant belong to the current request?
• Relationships: are dependent fields mutually consistent?
• Ranges: are quantities, limits, and time windows within policy?
• Idempotency: can the same call safely run again after a timeout?

A useful error returned to the Agent is typed and actionable, such as RESOURCE_NOT_FOUND, PERMISSION_DENIED, or INVALID_STATE. It should not be converted into a generic “invalid JSON” message, because that encourages the model to regenerate syntax instead of correcting the operation.

Step 4: inspect tool-call identity and multi-step history

Multi-step Agents add a state problem that single-response tests often miss. The assistant’s call object, identifier, arguments, tool result, and required conversation context may all be needed for the next turn. If the application stores only the human-readable text, the next request can look valid while lacking the protocol state required to continue.

The Function Calling flow documentation describes the exchange between the model, the application, and the tool. The application should treat that exchange as structured state, not as text to be reconstructed from memory.

Compare a successful and failing trace field by field:

• model and interface name;
• request identifier and conversation identifier;
• ordered message or response items;
• tool-call identifier;
• exact argument object;
• tool result and result type;
• timestamps and retry attempt;
• SDK serialization output;
• state token or continuation data when the interface uses one.

Some interfaces manage conversation state differently from others. The correct procedure is to follow the state rules for the specific API rather than applying one generic replay algorithm. The MCP specification and its tool definition rules also show why tool names, schemas, results, and protocol metadata must remain distinct.

If the tool result is inserted as plain text instead of the expected tool-result structure, the next model turn may lose both the call identity and the result semantics. That can produce repeated calls, fabricated completion claims, or an apparent JSON failure in a later step.

Step 5: build logs that can prove the fault

A production log should support reconstruction without exposing secrets. At minimum, each Agent attempt should have a correlation identifier shared by the request, model response, validator event, tool execution, and final result.

A useful redacted record includes:

• correlation ID and parent workflow ID;
• model, interface, SDK, and validator versions;
• schema hash plus the effective schema or a protected reference;
• sanitized input and prompt version;
• response status, refusal state, stop reason, and output-limit metadata;
• raw response stored under controlled access;
• parser and validator error path;
• normalized arguments after validation;
• tool name, authorization decision, and resource result;
• call identifier and state-replay status;
• retry reason and attempt number.

Do not log credentials, access tokens, private file contents, or unrestricted prompts. Redaction must happen in a controlled logging layer, while the protected raw response remains available to authorized incident responders when policy permits.

The log should also distinguish “not received,” “received but not parsed,” “parsed but schema-invalid,” “schema-valid but semantically invalid,” and “executed but failed.” These labels make dashboards useful because they map to separate owners and remediation paths.

FAQ: targeted answers for recurring failures

The following answers address the long-tail failure patterns that frequently get grouped under AI Agent JSON errors even though they occur at different layers.

Why does an AI Agent keep returning invalid JSON?

The failure may start before generation. An unsupported schema keyword, an invalid required-field definition, a refusal, truncated output, or a parser using a different JSON dialect can all look like a model formatting problem. Inspect the API status and raw response first, then validate the schema independently before changing prompts or increasing retries.

How should missing Function Calling arguments be investigated?

Compare the declared tool schema with the actual call object, including property names, required fields, nesting, and data types. Then check whether the selected interface supports the schema features being used. Log the call identifier and raw arguments before validation, because a later normalization step can hide whether the field was omitted by the model or removed by application code.

Why can Structured Output still fail?

Structured Output can constrain syntax without proving that the request is safe, complete, or meaningful. A refusal, length limit, unsupported schema construct, transport interruption, or client-side parser mismatch can still prevent a usable result. Treat structured generation as one control in the pipeline and retain application-level validation for relationships, permissions, resources, and business limits.

What should be checked when tool results lose context?

Verify that the original tool-call identifier, assistant call object, tool result, and required conversation state are returned in the format expected by that interface. Do not rebuild the exchange from only the visible text. Compare a successful single-step trace with the failing multi-step trace, then confirm that serialization, storage, and replay preserve every required field.

Why is valid JSON still causing a failed business action?

JSON validity only proves that the text can be parsed. It does not prove that an account exists, a path is accessible, an order is within policy, or a requested operation is authorized. Add semantic checks after parsing and before execution. Return a typed business error to the Agent rather than treating every execution failure as a reason to regenerate JSON.

A compact incident checklist

The checklist below is designed for the first investigation pass. It should be completed in order, because later checks depend on evidence from earlier stages.

• [ ] Preserve the original request and response before transformation.
• [ ] Confirm the interface, model, SDK, and validator versions.
• [ ] Validate the schema independently of the model call.
• [ ] Check the declared dialect and supported feature subset.
• [ ] Reduce the contract to one object and one tool.
• [ ] Record refusal, completion, stop, and transport states separately.
• [ ] Confirm that streaming events were assembled completely and in order.
• [ ] Parse only a response branch proven to contain complete content.
• [ ] Run syntax validation before schema validation.
• [ ] Record the exact schema error path and received value type.
• [ ] Run resource, permission, relationship, range, and idempotency checks.
• [ ] Compare call identifiers and tool results across every Agent turn.
• [ ] Reproduce with sanitized input and a fixed schema hash.
• [ ] Retry only when the error classification is explicitly retryable.

Finish with a minimal reproduction, then isolate the environment

A minimal reproduction should be small enough to run outside the production workflow but complete enough to preserve the failing layer. Keep the sanitized input, effective schema, interface configuration, raw response, parser output, validator error, tool arguments, and execution result together. Record the exact change that makes the reproduction pass, rather than only recording that a new prompt “worked.”

For teams reproducing remote build, deployment, or automation failures, the runtime environment can be part of the fault. Differences in shell tools, filesystem permissions, network access, credentials, or installed SDK versions can make a valid Agent payload fail only in production. When the local machine cannot reproduce those conditions, a temporary remote Mac troubleshooting environment can isolate the environment without changing the Agent contract itself.

That environment should not be treated as a substitute for schema validation. It is useful when the remaining uncertainty concerns the executor: operating-system behavior, local dependencies, network reachability, or permission boundaries. The same correlation ID and redacted trace must follow the test so that the team can distinguish an environment failure from an Agent JSON error.

A separate nuvcloud service overview can help teams assess whether a temporary remote machine fits a short-lived reproduction workflow. For region-sensitive access, the available US West remote Mac option is another path to evaluate, but the decision should depend on network location and execution requirements rather than on an assumption that remote hardware will fix malformed data.

The key operational choice is to avoid changing several variables at once. Keep the schema fixed while changing the validator, keep the validator fixed while changing the interface, and keep the payload fixed while testing the executor. That sequence turns a vague “the Agent returned bad JSON” incident into a bounded failure with an owner and a measurable repair.

For a team that currently runs every reproduction on a shared developer laptop, the trade-off is usually clear: local testing can mix credentials, dependencies, and filesystem state; shared machines can make logs and permissions difficult to isolate; and rebuilding the environment for one incident can delay diagnosis. Renting a dedicated Mac through nuvcloud is a better fit when the requirement is a temporary, reproducible remote environment for debugging or validation. It is not the best long-term choice for stable heavy workloads, permanent infrastructure, or tests that require direct physical interfaces, but it can keep a production investigation separate from the main workstation while the actual data-flow fix remains under review.