As AI becomes more heavily integrated into legal, budget concerns mean teams must understand how to effectively use their tokens.
What are Tokens?
Tokens are a chunk of text roughly the size of a word or part of a word. There are two kinds of tokens:
- Input tokens: Everything the model including the prompts, attached files, and skills you might invoke.
- Output tokens: Everything the model writes back. Output is priced several times higher than input, but in day-to-day use, input is usually where the budget get consumed.
Why Should You Care?
If you use conversational AI, you have a token budget with that plan. Every message sent or received has a cost. Large Language Models are not incentivised to be efficient. They benefit from verbose models that use up large amounts of tokens that people have to pay for. Employees can combat this by building habits to optimise token usage.
Habit 1: Tailor the Model to the Task
Not every legal task requires the most advanced model. Models will vary in their price per token. Using advanced models with expensive tokens for simple tasks is wasteful. To demonstrate the token price range, here is the Claude lineup as of 4 July 2026:
From Haiku to Fable costs increase tenfold. To choose the right size model for your task use the token-coach skill to get practical advise. This standard skill is available in OnPoint’s AI Skills Library and will recommend a token-efficient setup before you start: which model, what effort/thinking level, whether to setup a project, and whether an existing skill already fits your needs.
Habit 2: Manage the Conversation
Keep conversations short where possible. The model cannot remember anything between prompts, therefore every new message you send, means the model rereads the entire chat. The third message may be cheap, but by the 30th message excessive tokens are being used to reread the conversation and driving up the cost. This compounds the longer the thread runs.
Prolonging conversations can also make answers worse. As context fills up, quality degrades. Recall and accuracy start to worsen even before you hit your full content limit. Keeping your conversations short will save you tokens and budget.
Tips to manage your conversation:
- One job per thread. Start a new conversation for each distinct task. When you switch topics, switch chats. As a rule of thumb, start a fresh chat every 15-20 messages.
- Summarise. When a long thread has done its job but you want to keep going, ask for a short summary, copy into a new chat, and carry on from there.
Habit 3: Use Token Efficient Formats
Format matters. Plain text is usually the most token-efficient format. If document structure is important, Markdown offers a strong balance of token efficiency and readability.
As an indicative example, Markdown can use 30-60% fewer tokens than simple HTML and 50% fewer than XML. This is more efficient compared to PDF, PowerPoint, Word or Excel. These act more as containers than text formats. Before a model can use them, their text, images, tables and structure must be extracted, an overhead process that drives up token use.
- Default to plain text where possible.
- Where structure is important use Markdown.
Habit 4: Build AI Skills
A skill is packaged, reusable instructions for a recurring task. A well-designed skill can save tokens in two ways: you’re not re-explaining the same process every session, and a well-built skill loads only the detail a given step needs thus avoiding context dumping.
Tips for building and using skills:
- Ask for assistance when building a skill. If you explain your intent, the AI can help you draft a reusable prompt or skill.
- Run skill-reviewer (available in OnPoint’s AI Skills Library) on anything you build. This checks a skill is optimally structured and free of duplication which controls how many tokens it consumes when it runs.
Habit 5: Write Effective Prompts
Do not waste tokens correcting a sloppy vague prompt. Be intentional with your prompt wording the first time around:
- Decide the goal, the audience and desired output beforehand.
- Provide an example of what ‘good’ looks like if you have one.
- Follow the CIDI Framework to frame your request.
Element: | What to include |
|---|
Context | Background, role, or persona |
Intent | The outcome you want |
Details | Audience, metrics, deliverables |
Instructions | Output format |
If you have to clarify your prompt, edit the original prompt rather than responding another line. When you find a prompt that works, save it for next time. Build a prompt library.
Why Small Language Models are the Answer?
Small Language Models take a different approach than general purpose frontier models. Rather than being trained to do everything, they’re trained narrowly on a specific domain and task. This focus makes them fast, affordable, and small enough to be deployed within your own infrastructure rather than routed through a third-party provider.
This specialisation is especially important in high-volume legal work. For example, in contract review, if you are running thousands of contracts through an AI model, every document is tokens which are all being billed. A general frontier model is not built to review large volumes of texts in depth, so contract review can get expensive fast. Meanwhile, a SLM specifically trained for contract review can do the same task for lower amounts of tokens because it is specifically designed for your purpose. More accurate, more efficient, and cheaper.
This was seen in research paper, A Few Good Clauses, SLM Olava Extract performed contract review at a rate 8-25 times cheaper when batched, than the large frontier models.
Takeaways:
- Match the model to the task.
- Keep threads short – new chat, new job.
- Default to plain text or Markdown.
- Use skills instead of re-explaining yourself.
- Write the prompt right the first time.
- Look to SLMs as the long-term fix.
Build these habits now, and the bill won’t catch you off guard.
