Nora asks AI to shorten a lunch invitation and gets a speech fit for a company board meeting.
Nora and these scenes are fictional. If you have ever waited for an elaborate answer to a tiny request, you know the feeling.
Give the lunch invitation a quick model
Nora has the time and place in the original note and needs a shorter version, with no grand announcement about the future of sandwiches.
You can start with a fast option that handles simple rewriting well, then check whether it kept the details.
If your app offers free access, you can try it on this job before paying for a more capable model.
Google's Gemini API pricing page lists free access for selected models, with usage limits and data terms you still need to read.
You do not need a long explanation of why the shorter invitation works. Ask for the finished note and read it once.
Keep a clear date in the result and you have something you can send. Lunch can go ahead without a speech.
Turn up the effort when the constraints start fighting
Later, Nora helps with the cafe's staff schedule. Someone needs school pickup time, someone cannot close, and the opening shift still needs cover.
A neat table puts the same person on a late finish and an early start. The table is tidy, the person is tired.
You now have several rules that affect one another. A more capable model, or more reasoning effort on a supported model, may help find a workable schedule.
Model choice and thinking effort are separate controls, and Google's thinking guide lists the effort settings its models support.
Give it the actual staffing rules and ask it to check every shift against them.
You should still inspect the result. More thinking cannot supply a colleague's availability if you never provided it.
Judge the task by its risk as well as its length
A developer at the cafe asks AI to remove duplicate customer records. The request fits on a small sticky note.
The possible mistake does not.
You need to know which record to keep and what happens to the orders linked to it before accepting any generated solution.
A stronger model can help examine those rules, but you still need a safe test and someone to review what changes.
We would give this task careful attention from the start, even though the prompt is short. The lunch invitation can survive a clumsy adjective.
If you already know a job needs difficult checks, sending it through an unsuitable model first can add another call and another draft to review.
For routine work, try a quick option and move up when the result fails your checks. For risky work, choose the checks before choosing the model.
Count the tokens the neat reply does not show
Nora's assistant produces a short answer, so she assumes it used little processing. The reply is small enough to fit on a biscuit wrapper.
Some APIs bill for reasoning tokens you cannot fully see. Check the provider's usage report.
If the report arrives as JSON, our JSON visualizer can help you find its token counts.
In a made-up batch of 20 requests, 300 input tokens and 60 reply tokens per request add up to 7,200 tokens.
If a heavier setting adds 2,000 reasoning tokens to each request, the batch reaches 47,200 tokens. Avoiding that extra work saves 40,000 in this example.
For this example, assume the lighter mode adds no reasoning tokens and neither setting needs retries. Your model must support that lighter mode.
You also need to compare answer quality.
Our token-counting guide covers measuring inputs. A lower price per token and a lower token count are different savings.
Take the model with you when the Wi-Fi leaves
Nora wants to rewrite the cafe's weekly update on a train, where the Wi-Fi gives up before she finishes her coffee.
She has already downloaded the model and all supporting files. Her app runs the model on her own computer using local documents.
You can use a setup like this offline, provided the task and its tools have no network dependency.
The Ollama FAQ explains its local-only mode, which disables cloud models and web search.
Before travelling, disconnect the network and test your actual task. If a login or missing download stops you, reconnect and fix it before you leave.
Nora can edit the update she brought with her. You should not expect an offline model to fetch today's train delays or the cafe's new online orders.
Our local Ollama guide covers getting a model running on your own machine.
Run local inference without a cloud token bill
Back home, Nora sends more drafts through the model on her desktop. No cloud provider is charging for those local inference requests.
Inference means using the trained model to produce an answer, with your computer doing the work instead of a cloud API.
The model still reads and generates tokens on your hardware, even while you have no per-token cloud inference bill.
Keep electricity and hardware costs in the budget. The electricity meter keeps running.
Choose a model that fits the available memory and test its speed on your tasks before buying equipment for it.
A local server address alone does not prove the model runs locally. Check the selected model and disable cloud features if you need an offline setup.
You should also test its answers. Our local model comparison shows why a total score needs the individual results beside it.
Try teaching your style with examples first
Nora wants friendly cafe updates, but the model keeps writing as if the shop is opening a global franchise.
The soup is nice. It does not need an international launch.
You can first give the model approved examples and a short style note. Explain which facts it must keep and how long the update should be.
This supplies context for the current task. It does not change the model's trained weights, the learned values that shape its behavior.
For changing facts such as menu prices, you can provide the current local menu each time. Ask it to use that document instead of guessing.
We would try this before training. You may get the style you need without preparing a dataset or finding extra training memory.
Fine-tune a supported model and keep a fresh test pile
Suppose Nora repeatedly needs the same cafe style across many updates. She collects approved examples and keeps some unseen examples aside for testing.
With data you have permission to use and a model whose licence permits it, you can fine-tune a local model to adapt its behavior.
LoRA is one method. It trains a small adapter, an extra set of learned values, while keeping the original model weights fixed.
Training is separate from running a chat app, and its hardware needs require a separate check. Fitting a model in memory for inference does not establish that it fits for training.
You also need a compatible model and adapter format. The llama.cpp runtime supports loading LoRA adapters.
Compare the adapted model with the original on those unseen examples. Better cafe wording should still preserve the supplied facts.
Pick one routine task today, try a quick model and check whether extra effort earns its place.