rawmark — real hardware benchmarks

AMD Ryzen 9 3950X + RX 9070 XT GPU AMD Radeon RX 9070 XT 16GB RDNA4 · OS Windows 11 Pro · ROCm 7.1 gfx1201

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
granite4.2:3b	ollama	57	256	125.102	bifrost
mistral:7b-instruct	ollama	66	256	101.867	bifrost
mistral:7b-instruct-v0.3	ollama	65	256	101.735	bifrost
mistral:7b-instruct-v0.3-q4_0	ollama	21	256	101.6	bifrost
qwen2.5-coder:7b	ollama	44	256	97.0	bifrost
bifrost-1a-oss-32k	ollama	120	256	92.24	bifrost
bifrost-1a-oss	ollama	119	256	91.628	bifrost
qwen3-8b-bifrost no answer	ollama	72	256	91.302	bifrost
qwen2.5:7b-instruct	ollama	89	256	89.043	bifrost
bifrost-1a-instruct	ollama	92	256	88.416	bifrost
granite4.2-8b-16k no answer	ollama	30	256	85.3	bifrost
granite-4.1-8b-tekken	ollama	80	256	81.478	bifrost
granite4.2:8b	ollama	52	256	81.352	bifrost
ministral-3:8b-instruct-2512-q4_K_M	ollama	616	256	78.979	bifrost
Granite-4.1-8B (Q4_K_M)	ollama	45	256	76.6	bifrost
qwen3.5-9b-bench	ollama	62	194	70.563	bifrost
gemma4:e4b	ollama	31	256	68.6	bifrost
mistral-nemo:12b	ollama	66	256	68.105	bifrost
bifrost-1a-coder	ollama	64	256	67.793	bifrost
Qwen3.5-9B (Q4_K_M)	ollama	25	256	64.3	bifrost
qwen3:14b	ollama	47	256	55.4	bifrost
gemma3:12b	ollama	70	256	53.056	bifrost
bifrost-interactive no answer	ollama	73	256	52.747	bifrost
phi4:14b	ollama	25	256	52.6	bifrost
bifrost-t-search no answer	ollama	157	256	52.534	bifrost
phi4	ollama	63	256	49.221	bifrost
bifrost-t1b	ollama	43	106	47.1	bifrost
hf.co/bartowski/mistralai_Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q3_K_M	ollama	67	201	36.338	bifrost
devstral:24b	ollama	1290	256	19.398	bifrost
mistral-small3.2:24b retired 2026-08-05	ollama	568	256	12.682	bifrost
mistral-small3.2-bifrost	ollama	571	256	12.646	bifrost
mistral-small3.1:24b	ollama	419	256	12.056	bifrost

AMD Ryzen 7 5700G + RX 5700 XT GPU RX 5700 XT 8GB RDNA1 · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
granite4.2:3b	ollama	53	256	96.222	hearth
ministral-3:3b-instruct-2512-q4_K_M	ollama	616	256	91.145	hearth
bifrost-ha-cpu	ollama	617	256	91.141	hearth
qwen3:4b no answer	ollama	74	256	88.786	hearth
bifrost-1a-hearth-qwen3 no answer	ollama	253	256	87.517	hearth
bifrost-1a-hearth-mistral	ollama	76	256	65.51	hearth
bifrost-1a-hearth-mistral-retired	ollama	77	256	64.507	hearth
mistral	ollama	21	256	62.3	hearth
mimo:7b-rl-q4km	ollama	63	256	61.591	hearth
gemma3:4b	ollama	25	256	60.5	hearth
ministral-8b-tekken	ollama	66	256	59.401	hearth
mistral:7b	ollama	21	196	57.3	hearth
bifrost-1a-hearth-gemma3	ollama	381	256	54.517	hearth
qwen3:8b no answer	ollama	25	256	54.2	hearth
ministral-3:8b-instruct-2512-q4_K_M	ollama	616	256	51.766	hearth
granite-4.1-8b-tekken	ollama	85	256	50.968	hearth
Ministral-8B-2410 (Q4_K_M)	ollama	18	256	50.5	hearth
qwen3.5-4b-32k no answer	ollama	69	256	35.023	hearth
qwen3.5-4b-32k:pp15-20260915 no answer	ollama	69	256	34.931	hearth
qwen3.5:4b	ollama	25	256	32.9	hearth
qwen2.5-coder:7b	ollama	44	256	32.3	hearth
bifrost-1a-hearth-coder	ollama	357	216	32.132	hearth
gemma2:9b	ollama	70	256	29.936	hearth
qwen3.5-9b	ollama	63	256	28.388	hearth
bifrost-1a-hearth	ollama	164	29	23.8	hearth

AMD Ryzen 7 5700G iGPU Vega 8 GPU Vega 8 iGPU 16GB GTT · OS Windows 11 · ROCm N/A

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
qwen3.5:0.8b no answer	ollama	69	256	28.137	hearth-vega8
qwen2.5-coder:3b	ollama	89	256	15.521	hearth-vega8
bifrost-1a-overflow	ollama	137	256	11.0	hearth-vega8
bifrost-ha-agent no answer	ollama	144	256	9.718	hearth-vega8
bifrost-ha-voice no answer	ollama	148	256	9.582	hearth-vega8
bifrost-distill	ollama	847	256	6.1	hearth-vega8

AMD Ryzen AI Max+ 395 (Radeon 8060S) GPU Radeon 8060S 96GB unified · OS Ubuntu 24.04 · ROCm 7.2 gfx1151

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
Qwen3-1.7B-GGUF	lemonade	19	512	131.8	forge
functiongemma retired 2026-08-06	ollama	25	49	100.0	forge
hf.co/LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_M	ollama	25	256	91.4	forge
bifrost-coder-toolmaster-tools	ollama	304	125	68.067	forge
qwen3-coder:30b	ollama	23	256	67.5	forge
qwen3-coder-kilo2	ollama	23	256	65.3	forge
qwen3-coder:30b-a3b-q4_K_M	ollama	23	256	65.1	forge
bifrost-coder-toolmaster	ollama	256	256	64.8	forge
bifrost-t2-qwen3 retired 2026-08-06	ollama	47	2012	63.1	forge
phi4-mini retired 2026-08-06	ollama	18	212	61.3	forge
bifrost-nemotron-toolmaster retired 2026-08-06	ollama	33	256	61.0	forge
bifrost-forge-t1	ollama		102	58.0	forge
kat-coder-v2.5:Q4_K_M	ollama	73	256	56.594	forge
Nemotron-3-Nano-30B-A3B (UD-Q4_K_XL)	ollama	32	256	56.5	forge
hf.co/unsloth/Nemotron-3-Nano-30B-A3B-GGUF:UD-Q4_K_XL retired 2026-08-06	ollama	32	256	56.0	forge
gemma3:4b	ollama	25	256	55.7	forge
frontis-ma1-35b	ollama	25	256	54.2	forge
gemma4:e4b	ollama	31	300	52.6	forge
bifrost-orch-carnice retired 2026-08-06	ollama	25	256	52.2	forge
hf.co/samuelcardillo/Carnice-Qwen3.6-MoE-35B-A3B-GGUF:Q4_K_M retired 2026-08-06	ollama	25	256	51.9	forge
bifrost-kat-long	ollama	25	256	51.8	forge
qwen3.6-35b-a3b:Q4_K_M no answer	ollama	25	256	51.7	forge
bifrost-t2-gemma4	ollama	177	256	49.9	forge
qwen3-coder-next	ollama	69	222	49.391	forge
north-mini-code-1.0:Q4_K_M no answer	ollama	40	256	48.2	forge
bifrost-t2-gemma4-16k	ollama	178	256	47.6	forge
gemma4:26b	ollama	31	256	46.8	forge
bifrost-panel-gpt-oss-20b-16k	ollama	82	256	44.7	forge
hf.co/Andgihat/Nanbeige4.2-3B-GGUF:UD-Q4_K_XL	ollama	96	256	41.704	forge
qwen2.5-coder:7b	ollama	44	252	40.1	forge
qwen3:8b	ollama	25	256	37.3	forge
gpt-oss:120b retired 2026-08-05	ollama	82	256	35.1	forge
laguna-xs.2-8k	ollama	65	256	33.5	forge
laguna-xs.2	ollama	65	256	33.5	forge
hf.co/Andgihat/Nanbeige4.2-3B-GGUF:UD-Q6_K_XL	ollama	97	256	33.377	forge
gpt-oss-igor-65k	ollama	82	256	32.7	forge
orieg/gemma3-tools:12b-ft retired 2026-08-06	ollama	36	153	22.5	forge
Llama-4-Scout-17B-16E (UD-IQ1_M)	ollama	25	256	19.8	forge
Qwen3-VL-235B-A22B (UD-TQ1_0)	lemonade	23	256	17.0	forge
bifrost-coder-kilo2	ollama	283	256	14.4	forge
bifrost-panel-mistral-16k	ollama	373	256	14.4	forge
devstral	ollama	1240	256	14.4	forge
granite4.2:30b	ollama	67	256	11.38	forge
qwen3.8-27b:Q4_K_M	ollama	67	256	11.2	forge
qwen2.5-coder:32b	ollama	44	256	10.6	forge
bifrost-t2p5-r1	ollama	36	961	5.0	forge

AMD Ryzen AI Max+ 395 XDNA2 NPU GPU XDNA2 NPU — 0W GPU draw · OS Ubuntu 24.04 · ROCm FLM NPU

MODEL	BACKEND	PROMPT	OUTPUT	TOK/S	HARDWARE
llama3.2:1b	flm		134	49.0	forge-npu
gpt-oss:20b retired 2026-08-05	flm	82	256	9.1	forge-npu