AI API Cost Control: Why Your AI Bill Tripled and How to Bring It Down

By the Cehpoint teamPublished 6 min read

Short answer: AI API cost is driven by tokens, and most of them are context you choose to send: system prompts, history and retrieved documents. Bills jump because of context creep, long conversations, using large models for simple tasks and agent loops. Measure cost per feature, structure prompts for caching, route easy requests to smaller models, trim context, limit output, cache answers, batch non-urgent work and set hard caps.

The pilot cost almost nothing. Then the AI feature went live, usage grew, someone added a longer system prompt, the support bot started reading full order histories, and an agent began retrying failed calls. Three months later the AI API bill is several times what anyone budgeted, and nobody can say exactly why.

It is a familiar story for Indian businesses that have put AI into production. The good news is that AI API cost is unusually controllable once you understand how it is calculated. This guide explains the token maths in plain terms, then walks through the levers that cut spend without hurting answer quality.

How is AI API cost calculated?

Most language model APIs charge per token, a chunk of text that is usually a short word or part of a longer one. Text in Indian languages can need more tokens for the same meaning, depending on the model's tokenizer, so test with your real content. You pay separately for:

  • Input tokens: everything you send: system prompt, instructions, conversation history, retrieved documents, tool definitions and the user's message.
  • Output tokens: everything the model writes back, usually priced several times higher than input.
  • Extras: some models bill internal "reasoning" tokens, and images, audio, embeddings, web search and tool calls may carry their own charges.

A worked example

The prices below are round numbers chosen for illustration, not any provider's actual rates. Suppose a model costs ₹100 per million input tokens and ₹400 per million output tokens. A support bot sends, on every message:

  • System prompt and policies: 3,000 tokens
  • Conversation history: 2,000 tokens
  • Retrieved help articles: 4,000 tokens
  • User question: 100 tokens
  • Answer: 400 tokens

That is 9,100 input tokens (about ₹0.91) and 400 output tokens (₹0.16): roughly ₹1.07 per message. At 50,000 messages a month, about ₹53,500. Notice that the user's actual question is about 1% of the input. Almost all the cost is context you chose to send. That is where the savings are.

Why AI bills suddenly triple

  1. Context creep. Each new rule added to the system prompt, each extra retrieved document and each longer history is paid for on every single call.
  2. Growing conversations. If you resend the whole chat history every turn, the twentieth message costs far more than the first.
  3. Using the biggest model for everything. Classification, routing and simple extraction rarely need the most capable model.
  4. Agents and loops. One user request can trigger dozens of model calls. Retries on failure multiply that.
  5. Verbose output. No length guidance, so the model writes three paragraphs where one line would do.
  6. No per-user limits. One heavy user, a script or an attacker can consume a large share of the budget. OWASP's LLM10:2025 Unbounded Consumption describes "denial of wallet" attacks that exploit pay-per-use pricing in exactly this way.

Eight levers to cut AI API cost without hurting quality

1. Measure per feature, per call type

You cannot fix a single monthly number. Log input tokens, output tokens, model and cost for every call, tagged by feature, customer and step. Within a week you will see that a few call types drive most of the spend.

2. Use prompt caching

Several providers offer discounts when the beginning of your prompt is identical across calls. OpenAI's prompt caching guide, for example, explains that cache reuse requires the whole prefix to match, that you should put stable content such as instructions, tool definitions and reference material first and variable content last, and that cached input is billed at a reduced rate. Structure your prompts static-first, avoid timestamps or user names at the top, and append to conversations instead of rewriting earlier turns.

3. Route by difficulty

Send each request to the cheapest model that handles it well. A small, fast model can classify intent, extract fields or answer FAQs; a larger model handles the hard cases. A simple router, which can itself be a small model or rules, often moves most traffic to cheaper tiers. Test quality on real samples before switching.

4. Trim the prompt

  • Remove duplicated or outdated instructions; long prompts often contain the same rule three times.
  • Retrieve fewer, better chunks. Improving retrieval quality usually lets you send three passages instead of ten.
  • Summarise older conversation turns instead of resending them in full.
  • Send only the tool definitions relevant to the current step.

5. Control output length

Ask for the format you need: "answer in under 80 words", "return JSON with these fields only". Set maximum output tokens. Since output usually costs more per token than input, this is one of the cheapest fixes.

6. Cache answers, not just prompts

Many business questions repeat: "what are your hours", "how do I reset my password". Cache final answers for identical or near-identical questions, with an expiry, and skip the model call entirely.

7. Batch what is not urgent

Overnight report generation, document classification and bulk enrichment do not need instant answers. OpenAI's Batch API documentation, for instance, offers a 50% discount compared with synchronous calls, with results within 24 hours. Other providers have similar options.

8. Set hard caps

Set limits per user, per feature and per day, alerts at a share of budget, maximum steps for agents and maximum input size. When a cap is hit, degrade gracefully: switch to a smaller model, return a cached answer or hand over to a human.

A quick decision table

Symptom in your logsLikely causeFirst fix
Input tokens far larger than user messagesBloated system prompt or retrievalTrim prompt, retrieve fewer chunks, enable caching
Cost per conversation rises with lengthFull history resent each turnSummarise older turns
Many calls per user requestAgent loops or retriesStep limits, retry caps, better error handling
Output tokens highNo length guidanceFormat instructions and max tokens
A few users drive most spendHeavy users, scripts or abusePer-user quotas and rate limits
Same questions repeated dailyNo answer cacheResponse caching with expiry

Protect quality while cutting cost

Every saving should pass a quality check. Keep a test set of real questions with good answers, and run it before and after each change: new model tier, shorter prompt, fewer retrieved passages. Watch customer-facing signals too, such as escalations to humans and repeat questions. A cheaper answer that sends more customers to your support team is not cheaper.

When self-hosting makes sense

For steady, high-volume workloads, or where data must stay in India or on your own infrastructure, running an open-weight model on your own servers can change the economics from per-token to fixed infrastructure cost. It brings its own costs in GPUs, operations and upgrades, so compare total cost over a year, not just token prices.

How Cehpoint can help

Our cloud and AI cost optimisation service audits your AI usage call by call, then applies caching, routing, prompt trimming and caps with quality tests at each step. For ongoing control, our LLMOps and AI observability service tracks token spend per feature and alerts you before a bill surprises you. If self-hosting may suit your volume, we can assess a private LLM deployment against your current spend.

Frequently asked questions

What is a token and how many tokens is a typical message?

A token is a chunk of text the model processes, usually a short word or part of a longer one, and Indian-language text can need more tokens depending on the model. A short user question may be under 100 tokens, but the full request, including system prompt, history and documents, is often several thousand. That full request is what you pay for.

Does using a cheaper model reduce answer quality?

For some tasks yes, for many no. Classification, extraction, routing and FAQ answers often work well on smaller models, while complex reasoning may need a larger one. Route by difficulty and test on a set of real questions before and after switching. Keep the larger model for the cases where tests show it matters.

What is prompt caching and how much does it save?

Prompt caching lets a provider reuse the processed start of a prompt when it is identical across calls, billing those cached tokens at a reduced rate. Savings depend on the provider, the model and how much of your prompt is stable. Put fixed instructions, tool definitions and reference text first and variable content last to benefit.

How do we stop one user or bot from draining our AI budget?

Set per-user and per-key quotas, rate limits, maximum input sizes and daily spend caps, with alerts before limits are reached. Monitor for unusual patterns such as many long requests from one account. OWASP lists unbounded consumption, including denial-of-wallet attacks, among the top risks for LLM applications.

Need help with this?

Tell us what you are dealing with. We reply with a plain plan, and where it makes sense, a working demo before you commit.