Designing a clear chat API request.

Recommended structure

Separate behavior instructions, trusted application context, untrusted user input, and the required output format. Roles and field names vary by provider, so map your internal request type into the selected vendor's native format.

{
  "purpose": "answer_question",
  "system_instructions": "Be concise; cite the supplied source.",
  "conversation": [
    {"role": "user", "text": "What changed in the policy?"}
  ],
  "retrieved_context": ["Approved policy excerpt..."],
  "output": {"kind": "text"}
}

Conceptual internal application object—not a request to a real ChatsAPI endpoint.

Use only relevant context

Long histories increase cost and can dilute the information that matters. Keep recent turns, summarize durable state, and retrieve authoritative documents when needed. Make sure external documents are quoted as data rather than followed as instructions.

Record a traceable request

Assign a request ID, selected model, time budget and versioned prompt template. Track latency and error category without unintentionally logging sensitive user content.

Next

See streaming for response delivery or structured output for machine-consumed responses.