Results not good for my GPU

[simin75simin]

Description

I have 3060 and 32GB of RAM, but it gives me very old models, as shown below. I think quantized qwen3.6 and Gemma 4 are much better

╭─────────────────────────────────────────────────── Hardware Info ────────────────────────────────────────────────────╮
│ GPU 0: NVIDIA GeForce RTX 3060 — 12.0 GB (CC 8.6, CUDA 12.6) — BW: 360 GB/s │
│ CPU: Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz — 6 cores (AVX2) │
│ RAM: 31.9 GB │
│ Disk free: 100.9 GB │
│ OS: windows │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

                                           Recommended Models

┏━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┓
┃ # ┃ Model ┃ Params ┃ Quant ┃ Published ┃ Downloads ┃ Score ┃ License ┃
┡━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━┩
│ 1 │ openai/gpt-oss-20b │ 21.5B │ Q3_K_M │ 2025-08-04 │ 6.8M │ 74.4 │ apache-… │
│ │ │ (3.6B… │ │ │ │ │ │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 2 │ Qwen/Qwen3-14B │ 14.8B │ Q5_K_M │ 2025-04-27 │ 1.6M │ 74.0 │ apache-… │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 3 │ microsoft/phi-4 │ 14.7B │ Q5_K_M │ 2024-12-11 │ 810.0K │ 71.6 │ mit │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 4 │ Qwen/Qwen3-8B │ 8.2B │ Q5_K_M │ 2025-04-27 │ 11.0M │ 66.2 │ apache-… │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 5 │ deepseek-ai/DeepSeek-R1-Distill-Qwen-14B │ 14.8B │ Q5_K_M │ 2025-01-20 │ 499.5K │ 61.2 │ mit │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 6 │ google/gemma-3-12b-it │ 12.2B │ Q5_K_M │ 2025-03-01 │ 2.8M │ 60.8 │ gemma │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 7 │ deepseek-ai/DeepSeek-R1-Distill-Llama-8B │ 8.0B │ Q5_K_M │ 2025-01-20 │ 443.8K │ 56.1 │ mit │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 8 │ Qwen/Qwen2.5-14B-Instruct │ 14.8B │ Q5_K_M │ 2024-09-16 │ 2.0M │ 56.0 │ apache-… │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 9 │ Qwen/Qwen3.6-27B │ 27.8B │ Q4_K_M │ 2026-04-21 │ 5.3M │ 66.4 │ apache-… │
├─────┼──────────────────────────────────────────┼────────┼────────┼────────────┼───────────┼───────┼──────────┤
│ 10 │ google/gemma-2-9b-it │ 9.2B │ Q5_K_M │ 2024-06-24 │ 394.8K │ 50.6 │ gemma │
└─────┴──────────────────────────────────────────┴────────┴────────┴────────────┴───────────┴───────┴──────────┘
Top pick confidence: Low (direct benchmark but very close (+0.4))
Benchmark reference: 2026-05 curated snapshot; live AA / LiveBench / Aider merged when reachable.
Note: Top candidates are very close (#1 vs #2: 0.4 pts).

Steps to Reproduce

whichllm --gpu "RTX 3060"

Hardware Info

Python Version

No response

Operating System

No response

whichllm Version

No response

[samarthpatel24]

The results look alright for a 3060 12GB. Qwen3.6-27B is in your list at # 9, it just gets pushed down because at Q4_K_M the ranker penalizes the quantization.

For Gemma 4, try running with --refresh — it does show up but with a ~ next to the score, meaning no curated benchmark data yet so the ranker discounts it heavily.

[Andyyyy64]

You're right, and thanks for the detailed report. This is a ranking problem on our end, not your setup.

Qwen3.6-27B has the highest benchmark score of anything in your list (83.5, against 66.7 for the next one). It lands at #9 only because it does not fully fit in 12GB, and the current penalty for partial offload is too aggressive, so it drops below smaller models that fit fully. On a 3060 12GB with 32GB RAM it would run at partial offload, slower but usable, and it should sit near the top with that caveat rather than buried.

@samarthpatel24 thanks for jumping in here. The quant step is part of the picture; the bigger factor in this case is the VRAM fit rather than Q4_K_M itself, since the Q4 penalty is small. And you're right about Gemma 4: it has no curated benchmark yet, so the ranker discounts it, which is why it shows with the ~ marker.

Tracking the fix in a separate issue now, and I'll get the partial-offload calibration sorted out.

[Andyyyy64]

@simin75simin I opened #108 for this.

The fix removes the extra full-GPU sort bonus that was pushing Qwen3.6-27B down after it had already been penalized for partial offload. With the local fix, the 3060 12GB case puts Qwen3.6-27B back in the top group with a clear partial-offload warning instead of burying it under weaker full-GPU models.

I also checked the opposite cases so this does not swing too far the other way: A3000 6GB and RTX 4060 8GB still keep heavier partial-offload picks under control.

[Ramachandrajoshi]

Same for my setup, Suggested top model actually runs, but not usable due to CPU offloading and painfully slow tps