Switch Hugging Face Models in Ollama and Check Their Answers

A model that rejected both valid dates still scored six out of eight. You would miss that problem by looking only at the total.

We compared four local models in the same Ollama setup, checking their tool calls and individual test results.

Pull the GGUF files you want to compare

The Hugging Face Ollama guide shows the hf.co syntax for pulling a community GGUF repository.

ollama pull hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF
ollama pull hf.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
ollama pull hf.co/unsloth/Qwen3-14B-GGUF

Without a quantization tag, it chooses Q4_K_M when that file is available. You can select another file with a tag such as :Q8_0.

Use these three downloads alongside the Ollama library's qwen3:8b if you want to repeat our comparison. The saved model metadata identifies each build by digest.

A GGUF file still needs an architecture and template your Ollama version supports. Do not assume the filename guarantees usable tool calling.

You can inspect the downloaded model with ollama show and check its repository card before using the weights.

Step 1, choose the models and record their settings

Llama 3.2 3B was our smaller general model, while Qwen2.5 Coder 7B gave us a code-focused model to test.

Qwen3 14B let us check a larger model against the 8B already used in this series.

All four loaded files were Q4_K_M quantizations. You can check their exact sizes and capabilities in the package's saved show responses.

Check each model card for its license. We did not compare every tensor in these community conversions with the original weights.

We ran this on a 2023 MacBook Pro with an M2 Pro chip and 32 GB, using Ollama 0.40.0.

Step 2, change the model in your config

You can replace the model field in the part eight harness without changing its tools or rules.

"model": "hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF"

For these measurements, we also set num_ctx to 32,768 explicitly. Temperature and seed were zero, and thinking was disabled.

The Ollama context guide explains why a larger allocation needs more memory. Your default can depend on hardware and settings.

We unloaded each model after its tasks so the four allocations did not remain in memory together.

Step 3, keep three tasks and their inputs fixed

For the tool task, ask which laptop our notes describe. Keep the same question and tools for every model you try.

We ran it with and without the rules file. The saved requests and tool messages let you trace both conversations.

The coding task asks for a date parser. You can inspect the full prompt and the eight initial cases in the download.

CODE = "Write a Python function parse_iso(s) that takes a string like 2026-10-09 and returns a tuple of three ints (year, month, day). Raise ValueError for anything that is not exactly four digits, a dash, two digits, a dash, two digits, or for an impossible month or day. Reply with only the code, no explanation and no markdown."

TESTS = [("2026-10-09", (2026, 10, 9)), ("2024-02-29", (2024, 2, 29)),
         ("2023-02-29", None), ("2026-13-01", None), ("26-10-09", None),
         ("2026-1-9", None), ("2026-10-09T10:00", None), ("2026-00-10", None)]

Your scorer needs to distinguish a tuple of three integers from a list or floating-point values that happen to compare equal.

For invalid dates, only ValueError passes. You can check the exception and returned type for every case in scores.json.

In the last task, you compare 20 fixed titles under the uuid, json, ai and site labels. We saved complete responses.

You can open titles.json to see the exact inputs and our editorial labels, which are not equally represented.

Read the measurements beside the task results

The code prompt ran twice per model and produced the same code both times. The speed range uses eval_count divided by eval_duration.

It excludes loading and prompt evaluation. You can compare the raw counts, including each model's output length, in responses.json.

When you compare these speeds, remember each model has its own tokenizer, so the counts represent different amounts of English text.

Loaded sizes come from /api/ps after coding, with 32,768 context reported for each model. GB here means a billion bytes.

The classification column sums 20 API durations, not just generation time. Each model classified the dataset once, so these are small local measurements.

The coding model missed three of our eight cases

Qwen2.5 Coder returned 2 valid dates correctly but passed only 3 of the 6 rejection cases.

You can see it accept 2023-02-29, 26-10-09 and 2026-1-9, violating either calendar validity or the required digit counts.

In our eight cases, Qwen2.5 Coder passed five and Qwen3 14B passed seven. Qwen3 14B also accepted the invalid leap day.

We would inspect this parser before using it, rather than treating eight cases as a verdict on the coding model.

On our laptop question, the coder returned "I do not know" without a tool call, both with and without our rules.

Its metadata declared tool capability, but this run did not demonstrate a working call. The response does not identify why it abstained.

The 3B model scored six by rejecting every test input

Llama's function split the date on dashes, then demanded two parts instead of three. Both valid inputs immediately raised ValueError.

You could call that 75 percent because all six invalid inputs also raised ValueError, even though neither valid date worked.

Separating valid and invalid cases shows the problem immediately. We would reject a parser that fails both examples it should accept.

After Qwen3 8B passed the original eight, we tried 2026-+1-09 and got January 9, exposing another format error.

The prompt required two month digits, and +1 does not meet that requirement. Passing a small test set did not prove the full contract.

All four replies also included Markdown fences despite the prompt asking for plain code. The table grades function behavior after removing those fences.

The rules file changed the 14B laptop response

Qwen3 14B with the part eight rules returned "I do not know" without calling a tool, using 476 prompt tokens.

Without the file, it searched and answered from the notes. Those two turns used 1,358 prompt tokens altogether.

Llama 3.2 3B and Qwen3 8B searched under both settings. You can compare their actual arguments with the 14B refusal.

We kept the notes and enabled tools unchanged. This pair of requests shows a difference for one question, not a general effect of rules files.

After a switch, run questions where a tool is needed as well as ones where it should stay unused.

The shared classification score hides a labeling choice

Every model matched 17 of our 20 stored labels, with the same three mismatches.

We label the JSON visualizer, JSON formatter and UUID generator articles as site because they describe our tools.

The models chose json or uuid from the titles instead. You could reasonably prefer those labels if you wanted subject categories.

The dataset contains 8 uuid titles, 7 json, 4 site and 1 ai. You should not treat it as a balanced classification benchmark.

Llama finished these requests in 1.6 seconds and 14B in 6.0. Both matched the same stored labels, without resolving the label ambiguity.

Before comparing these scores, decide whether the labels describe each article's subject or its role on our website.

Review generated code before you run its tests

The collection script saves model output without executing it. We read all four generated functions before writing them into the reviewed folder.

You can compare each reviewed file with its raw response. Only Markdown fences were removed, and the package records each file's SHA-256.

The scorer imports a reviewed file in a separate Python process with a three-second timeout. That timeout is not a filesystem or network sandbox.

You still need to inspect a new candidate before running it. Do not pass arbitrary model output straight into exec on your workstation.

python3 models.py
python3 score.py reviewed/model-1-repeat-1.py

When you run the first command, it collects responses without replacing the reviewed candidates. The second scores an existing file you have inspected.

Try the smaller model on a question you can check

The complete model comparison package includes the fixed titles and notes, raw responses, reviewed code and case-level scores.

You can inspect a response in the JSON response visualizer before comparing its usage with another model's run.

If you compare local usage with paid APIs, you need the provider's own input and output counts. Our model pricing comparison shows that calculation.

Run one task with a smaller model, inspect its mistakes, and decide whether those mistakes matter for your use.