Calling a Model API

Introduction

Chat apps are just one way to use a model. Developers add AI to their own software by calling a model API: the app sends text over the internet to a provider and receives the model's reply. This lesson assumes you know basic programming and web requests.

The shape of a request

Most chat APIs work the same way. You send:

• A model name
• A list of messages, each with a role (user or assistant) and content
• Often a system prompt: standing instructions that set the model's behaviour
• Parameters, such as the maximum tokens to generate and the temperature

The model is stateless. It does not remember earlier calls, so your app must send the relevant conversation history each time.

An example using Python and one provider's SDK, shown for reading (model names change, so check the current list in the provider's documentation):

import anthropic

client = anthropic.Anthropic()   # reads the API key from an environment variable

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=500,
    system="You are a friendly tutor. Answer in simple English.",
    messages=[{"role": "user", "content": "What is a variable?"}],
)
print(response.content[0].text)

Other providers use similar fields. The response also reports how many tokens were used, and why generation stopped, for example because it finished or hit the token limit.

Streaming

For long answers, streaming returns the text in pieces as it is generated, so users see words appear quickly instead of waiting for the whole reply. It improves perceived speed, but needs code that handles partial results.

Keep keys safe

An API key is a password that lets others spend your money.

• Never put it in front-end code, mobile apps or public repositories. Anyone can read it.
• Call the API from your server, and have your app call your server. This site's Aira assistant works this way.
• Store keys in environment variables or a secrets manager, and rotate them if exposed.
• Add per-user limits and monitoring, so one person cannot run up your bill.

Cost, speed and limits

• Providers charge by tokens in and out, with different prices per model. Larger, smarter models cost more and respond more slowly.
• Reduce cost with shorter prompts, caching repeated content if the provider supports it, and smaller models for easy tasks.
• Expect rate limits. Handle "too many requests" errors with retries and increasing waits (exponential backoff).
• Set a timeout and a sensible maximum output size.
• Choose the model per task: a fast, cheap model for classification, a stronger model for hard reasoning.

Handling failure

Calls can fail through network problems, overload or content rules. Your app should:

• Catch errors and show a friendly message
• Retry safely for temporary errors
• Never show raw error details or keys to users
• Work, or degrade gracefully, when the AI is unavailable
• Validate outputs before using them, particularly if they are parsed as JSON

Privacy and compliance

Read the provider's terms on data retention and training. Tell users that their input is sent to an AI provider, do not send more personal data than needed, and consider regional data rules.

Try it yourself

Sketch the server code for an endpoint that accepts a question, adds a system prompt, calls a model API, limits the output size and returns the answer. List where the API key lives and how you would limit each user's requests.

Summary

• A chat API takes a model, messages, a system prompt and parameters, and the model is stateless.
• Call the API from a server, keep keys secret, and set limits on usage and cost.
• Handle errors, rate limits and bad output gracefully, and respect privacy.