Why Your LLM Response Stops Early

Before increasing a token limit, check why the API stopped. A blank answer, unfinished JSON, and a tool handoff can look similar in a text-only UI but require different recovery paths. Treat transport completion, model completion, and application success as three separate decisions.

This guide is for developers operating OpenAI Responses API and Claude Messages API integrations. It covers the documentation available on October 31, 2026, not every SDK wrapper, model snapshot, or Chat Completions-compatible service. The diagnostic policy below is our engineering synthesis; its examples are fictional, not API experiments.

Start with the terminal state, not the text length

Preserve the provider’s response status, stop reason, usage counters, request identifier, model identifier, and configured limits in access-controlled operational records. Avoid copying prompts, secrets, or customer documents just to debug a budget problem.

Observed signal	What to investigate	Application decision
OpenAI status: incomplete, reason max_output_tokens	Generated-token budget or remaining context	Keep partial output non-final; inspect both budgets.
Claude stop_reason: max_tokens	Requested output ceiling	Consider a larger bounded budget or a smaller task.
Claude model_context_window_exceeded	Remaining context capacity	Reduce or reorganize context; a larger output ceiling alone is not a fix.
Claude tool_use or pause_turn	Tool workflow state	Follow the appropriate continuation protocol, not a generic text retry.
Stream ends without an observed terminal result	Transport, timeout, cancellation, or missing event handling	Mark the result unresolved; do not invent a token-limit diagnosis.

OpenAI documents the same max_output_tokens incomplete reason for either generated-token or context exhaustion, including cases with no visible answer. It does not, by itself, identify which budget was too small. OpenAI reasoning guide

Claude distinguishes the output cap from context exhaustion. Its tool_use requests a client-tool step; pause_turn pauses a server-tool loop. A custom stop_sequence is another deliberate stopping condition, not evidence that the model ran out of capacity. Claude stop reasons

The output budget is not a visible-text allowance

OpenAI’s reported output total includes generated tokens that do not appear in the answer. Formatting and message-structure tokens can also create a difference even when reported reasoning usage is zero. Do not estimate a fixed hidden-token surcharge or subtract reasoning tokens and label the remainder an exact visible-text count. OpenAI token accounting

Claude likewise counts thinking within max_tokens. Context accounting for earlier thinking blocks varies by model and conversation state, so a rule copied from one model is not a universal capacity formula. Claude context windows

Fictional budget example: a request permits 2,000 generated tokens and your product requires a 1,500-token report. That does not reserve 1,500 tokens for the report. If other generated content consumes 800 tokens, at most 1,200 remain under that ceiling. This is arithmetic illustrating a constraint—not an observed overhead, recommended limit, or forecast of model behavior.

For a bounded extraction job, ask whether all requested records can fit, including output structure. For a long report, consider separately validated sections. Splitting work has its own costs: duplicated context, cross-section inconsistency, and reconciliation effort. It is not automatically cheaper or more accurate.

Raising the cap can miss the actual bottleneck

Check the exact model’s input and output limits separately. An oversized input can fail before generation; an accepted request can still lack enough room to finish. Claude documents a 400 error when input alone exceeds context, while newer-model handling of input plus requested output differs from older models. Claude overflow behavior

Our suggested retry policy is conditional:

1. Output ceiling reached with room available: increase it only within a preapproved cost and latency bound, or reduce the deliverable.
2. Context constrained: remove irrelevant material or split the task while preserving evidence and essential instructions.
3. Unknown cause: collect missing status evidence before changing several settings at once.
4. Repeated failure: stop after a bounded number of attempts and expose a clear incomplete state to the user.

If changing reasoning effort is supported by your model, evaluate the quality trade-off on representative tasks. Do not assume less reasoning preserves correctness. Record model, workload, limits, completion outcome, latency, and cost together; otherwise a cheaper-looking retry can merely produce a less useful answer.

A stream closing is not an application commit

The Responses event reference distinguishes response.completed, response.incomplete, and response.failed. A text-part completion event is not a substitute for the whole response’s terminal state. Handle the final response object and error path, not just text deltas. Responses streaming events

For your own state machine, keep receiving, terminal-success, terminal-incomplete, terminal-failure, and unresolved separate. Only the appropriate terminal-success path should reach final acceptance checks. A disconnect after a tool operation is especially important: lack of a final answer does not prove that no side effect occurred. Reconcile the operation’s durable status before replaying it; see webhook idempotency and crash windows for the same distinction between a reply and a committed action.

Claude says an incomplete tool-use block cut off by max_tokens needs a retry with a higher limit. Do not execute partial arguments. A pause_turn continuation must preserve the paused response; container-based workflows also need the returned container identifier. These are workflow-specific continuations, not interchangeable retry recipes. Claude continuation guidance

Define success before choosing a recovery path

A naturally ended response may still omit required work. Conversely, useful-looking partial prose may be incomplete. For a fictional ten-record extraction, success should require ten accounted-for record identifiers, valid types, and explicit handling of missing evidence—not merely parseable JSON. Our structured-output validation guide covers that next layer.

Before rollout, design fixtures for at least these cases: empty output with a budget stop; partial output with a budget stop; a valid tool handoff; a disconnect without terminal evidence; and a normally ended answer missing a required record. These are proposed tests, not tests we ran against either API. Assert the user-visible state and whether retries or side effects are allowed in each case.

The operational goal is not to make every request retry until something appears. It is to distinguish capacity, workflow, transport uncertainty, and result quality, then spend another request only when it addresses the observed failure.