1. Define the job
Write down the user task, allowed knowledge sources, expected output and explicit refusal or escalation cases. A narrow assistant is easier to evaluate than a general-purpose agent with unlimited tools.
2. Choose an API and baseline model
Pick one provider, then implement a small server-side adapter. Keep the API key off the client. Record each request's model, duration and error category. Use the model selection guide to compare alternatives after the baseline works.
3. Design the conversation contract
Separate system instructions from user content. Include only relevant conversation turns and retrieved information. Bound the maximum request size and sanitize untrusted input.
4. Implement response behavior
Deliver a complete response first, then add streaming if it improves user experience. Handle cancellation, rate limits, timeouts, empty outputs and retries. Never assume that a partial streamed token sequence is final.
5. Evaluate with real scenarios
Build a small test set drawn from representative tasks. Track correctness, relevance, response format, latency, cost and escalation quality. Include adversarial cases where the model should say it cannot answer.
6. Add tools cautiously
Start with read-only tools. Enforce authorization and explicit approval for changes, purchases and deletions. Follow the agent tool design checklist.
Launch checklist
- No provider keys in browser source or repositories.
- Output escaped before insertion into HTML.
- User data retention policy defined.
- Rate limits and spending limits configured.
- Rollback path documented for model or prompt changes.
Next: RAG for grounded answers and streaming responses.
Browse more guides ↗