Skip to content

update(rocm): route top-k with top-p up to vocab 152064 on ROCm #2218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
inureyes merged 6 commits into from
Oct 8, 2026
Merged

update(rocm): route top-k with top-p up to vocab 152064 on ROCm #2218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3a1261f
update(sampling): measure the joint rejection cap on gfx1151 and pin it
inureyes Oct 7, 2026
defe706
update(sampling): add bf16 and per-config runs to the rejection micro…
inureyes Oct 8, 2026
fa87f24
docs(sampling): wip joint-cap doc edits
inureyes Oct 8, 2026
c57f89d
update(rocm): route top-k with top-p up to vocab 152064 on ROCm
inureyes Oct 8, 2026
7d737d4
docs(rocm): record the joint-cap re-measurement on the rebased build
inureyes Oct 8, 2026
822ec21
docs: add technical report for PR #2218
inureyes Oct 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 159 additions & 0 deletions TECHNICAL_REPORTS/2218-rocm-joint-rejection-cap-20261008.en.md
View file
Open in desktop

Large diffs are not rendered by default.

159 changes: 159 additions & 0 deletions TECHNICAL_REPORTS/2218-rocm-joint-rejection-cap-20261008.ko.md
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions ...ixhalo-gfx1151_2026-10-07_joint-cap-measurement_Meta-Llama-3.1-8B-Instruct-4bit_chain.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
model,model_path,prompt_tokens,generated_tokens,prefill_ms,prefill_tok_s,decode_ms,decode_tok_s,date,hardware,mlxcel_version,build_type,max_tokens,prompt,prompt_target_len,mlxcel_commit,mlx_commit,mlx_rocm_overlay_commit,hip_version
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,498.01,1028.10,3771.24,33.94,2026-10-07,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,483.72,1058.47,4045.77,31.64,2026-10-07,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,482.66,1060.78,3918.44,32.67,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
4 changes: 4 additions & 0 deletions ...rixhalo-gfx1151_2026-10-07_joint-cap-measurement_Meta-Llama-3.1-8B-Instruct-4bit_kern.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
model,model_path,prompt_tokens,generated_tokens,prefill_ms,prefill_tok_s,decode_ms,decode_tok_s,date,hardware,mlxcel_version,build_type,max_tokens,prompt,prompt_target_len,mlxcel_commit,mlx_commit,mlx_rocm_overlay_commit,hip_version
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,500.57,1022.84,3394.82,37.70,2026-10-07,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,650.88,786.63,3425.41,37.37,2026-10-07,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Meta-Llama-3.1-8B-Instruct-4bit,models/mlx/Meta-Llama-3.1-8B-Instruct-4bit,512,128,537.25,953.00,3406.74,37.57,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
4 changes: 4 additions & 0 deletions ...ocm_strixhalo-gfx1151_2026-10-07_joint-cap-measurement_Qwen2.5-7B-Instruct-4bit_chain.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
model,model_path,prompt_tokens,generated_tokens,prefill_ms,prefill_tok_s,decode_ms,decode_tok_s,date,hardware,mlxcel_version,build_type,max_tokens,prompt,prompt_target_len,mlxcel_commit,mlx_commit,mlx_rocm_overlay_commit,hip_version
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,337.03,1519.14,2999.31,42.68,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,341.74,1498.22,3053.81,41.91,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,312.67,1637.52,3137.77,40.79,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
4 changes: 4 additions & 0 deletions ...rocm_strixhalo-gfx1151_2026-10-07_joint-cap-measurement_Qwen2.5-7B-Instruct-4bit_kern.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
model,model_path,prompt_tokens,generated_tokens,prefill_ms,prefill_tok_s,decode_ms,decode_tok_s,date,hardware,mlxcel_version,build_type,max_tokens,prompt,prompt_target_len,mlxcel_commit,mlx_commit,mlx_rocm_overlay_commit,hip_version
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,331.26,1545.60,2727.84,46.92,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,311.42,1644.08,2758.81,46.40,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
Qwen2.5-7B-Instruct-4bit,models/mlx/Qwen2.5-7B-Instruct-4bit,512,128,337.93,1515.13,2712.76,47.18,2026-10-08,AMD_Radeon_8060S_Graphics_gfx1151_ROCm10.0.0_96GB,0.7.0,release,128,"Hello, how are you today?",512,unknown,81ba1c6a,75915908,7.15.26333
16 changes: 16 additions & 0 deletions benchmarks/rocm_strixhalo-gfx1151_2026-10-07_rejection-microbench-bf16-run1.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
config,vocab,batch,worst_rounds,routed,iso_baseline_us,iso_rejection_us,iso_speedup,pipe_baseline_us,pipe_rejection_us,pipe_speedup
top-k+top-p,32768,1,3,yes,327.975,142.507,2.3015,2454.112,2458.252,0.9983
top-k+top-p,32768,4,3,yes,682.168,178.404,3.8237,2822.154,2334.184,1.2091
top-k+top-p,32768,8,4,yes,1203.143,207.859,5.7883,3383.396,2395.225,1.4126
top-k+top-p,65536,1,3,yes,422.401,260.298,1.6228,2537.508,2383.962,1.0644
top-k+top-p,65536,4,4,yes,979.855,348.653,2.8104,3117.398,2505.644,1.2442
top-k+top-p,65536,8,4,yes,1753.104,381.064,4.6006,4025.660,2572.882,1.5647
top-k+top-p,128256,1,4,yes,502.521,582.100,0.8633,2640.052,2695.320,0.9795
top-k+top-p,128256,4,5,yes,1244.100,708.777,1.7553,3476.782,2930.128,1.1866
top-k+top-p,128256,8,5,yes,2220.800,1034.898,2.1459,4608.584,3403.746,1.3540
top-k+top-p,151936,1,4,yes,548.878,705.622,0.7779,2682.686,3052.557,0.8788
top-k+top-p,151936,4,4,yes,1386.055,1021.623,1.3567,3662.984,3146.239,1.1642
top-k+top-p,151936,8,6,yes,2494.282,1237.497,2.0156,4861.197,3671.416,1.3241
top-k+top-p,152064,1,4,yes,551.132,682.980,0.8070,2692.299,3134.895,0.8588
top-k+top-p,152064,4,6,yes,1404.741,1036.582,1.3552,3576.230,4609.010,0.7759
top-k+top-p,152064,8,6,yes,2500.754,1247.978,2.0038,5208.904,3524.281,1.4780
16 changes: 16 additions & 0 deletions benchmarks/rocm_strixhalo-gfx1151_2026-10-07_rejection-microbench-bf16-run2.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
config,vocab,batch,worst_rounds,routed,iso_baseline_us,iso_rejection_us,iso_speedup,pipe_baseline_us,pipe_rejection_us,pipe_speedup
top-k+top-p,32768,1,1,yes,328.906,143.569,2.2909,2463.712,2275.236,1.0828
top-k+top-p,32768,4,3,yes,687.177,179.245,3.8337,2846.841,2438.311,1.1675
top-k+top-p,32768,8,4,yes,1212.792,205.365,5.9055,3363.185,2397.552,1.4028
top-k+top-p,65536,1,3,yes,416.901,248.626,1.6768,2555.937,2593.810,0.9854
top-k+top-p,65536,4,4,yes,966.040,342.251,2.8226,3139.172,2651.746,1.1838
top-k+top-p,65536,8,4,yes,1834.488,378.779,4.8432,4001.949,2590.475,1.5449
top-k+top-p,128256,1,4,yes,501.629,563.266,0.8906,2640.637,2706.174,0.9758
top-k+top-p,128256,4,5,yes,1240.464,717.825,1.7281,3439.454,3011.923,1.1419
top-k+top-p,128256,8,5,yes,2220.490,1021.955,2.1728,4751.470,3226.474,1.4727
top-k+top-p,151936,1,4,yes,549.108,722.464,0.7600,2670.391,2909.111,0.9179
top-k+top-p,151936,4,4,yes,1387.250,1036.662,1.3382,3623.478,3320.014,1.0914
top-k+top-p,151936,8,6,yes,2497.680,1235.315,2.0219,5616.204,3430.575,1.6371
top-k+top-p,152064,1,4,yes,549.690,706.734,0.7778,2700.940,2901.890,0.9308
top-k+top-p,152064,4,6,yes,1405.784,1051.781,1.3366,3632.372,3463.004,1.0489
top-k+top-p,152064,8,6,yes,2492.961,1247.878,1.9978,5335.337,3627.819,1.4707
16 changes: 16 additions & 0 deletions benchmarks/rocm_strixhalo-gfx1151_2026-10-07_rejection-microbench-bf16-run3.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
config,vocab,batch,worst_rounds,routed,iso_baseline_us,iso_rejection_us,iso_speedup,pipe_baseline_us,pipe_rejection_us,pipe_speedup
top-k+top-p,32768,1,3,yes,327.924,144.561,2.2684,2465.583,2258.334,1.0918
top-k+top-p,32768,4,3,yes,685.735,179.316,3.8242,2841.379,2332.311,1.2183
top-k+top-p,32768,8,4,yes,1199.698,208.932,5.7421,3376.305,2320.789,1.4548
top-k+top-p,65536,1,3,yes,421.921,252.012,1.6742,2555.728,2513.352,1.0169
top-k+top-p,65536,4,4,yes,965.429,350.016,2.7582,3115.666,2542.227,1.2256
top-k+top-p,65536,8,4,yes,1749.950,374.682,4.6705,4163.998,2764.785,1.5061
top-k+top-p,128256,1,4,yes,501.930,578.854,0.8671,2618.892,2887.600,0.9069
top-k+top-p,128256,4,5,yes,1251.134,709.530,1.7633,3481.594,3136.815,1.1099
top-k+top-p,128256,8,5,yes,2232.122,1027.906,2.1715,4896.524,3273.284,1.4959
top-k+top-p,151936,1,4,yes,548.598,725.941,0.7557,2679.838,3006.282,0.8914
top-k+top-p,151936,4,4,yes,1383.081,1044.898,1.3237,4211.532,3319.862,1.2686
top-k+top-p,151936,8,6,yes,2484.165,1233.601,2.0138,5106.730,3695.511,1.3819
top-k+top-p,152064,1,4,yes,549.871,679.483,0.8092,2683.269,3126.610,0.8582
top-k+top-p,152064,4,6,yes,1382.550,1041.211,1.3278,3624.309,3157.829,1.1477
top-k+top-p,152064,8,6,yes,2496.538,1253.699,1.9913,5073.006,3586.617,1.4144
61 changes: 61 additions & 0 deletions benchmarks/rocm_strixhalo-gfx1151_2026-10-07_rejection-microbench-run1.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
config,vocab,batch,worst_rounds,routed,iso_baseline_us,iso_rejection_us,iso_speedup,pipe_baseline_us,pipe_rejection_us,pipe_speedup
top-k=40,32768,1,3,no,171.311,142.878,1.1990,2297.219,2275.268,1.0096
top-k=40,32768,4,5,no,309.951,172.904,1.7926,2720.903,2603.081,1.0453
top-k=40,32768,8,5,no,528.310,212.468,2.4865,2884.019,2820.762,1.0224
top-k=40,65536,1,4,no,199.824,255.980,0.7806,2348.446,2338.409,1.0043
top-k=40,65536,4,4,no,440.736,357.249,1.2337,2590.720,2930.243,0.8841
top-k=40,65536,8,4,no,796.563,413.184,1.9279,3123.871,3454.500,0.9043
top-k=40,128256,1,4,no,270.587,615.493,0.4396,2386.920,2398.130,0.9953
top-k=40,128256,4,5,no,765.604,807.803,0.9478,2974.457,3301.237,0.9010
top-k=40,128256,8,6,no,1455.808,1117.313,1.3030,4404.197,3815.505,1.1543
top-k=40,151936,1,5,no,298.278,737.772,0.4043,2423.657,2413.410,1.0042
top-k=40,151936,4,5,no,893.464,1106.713,0.8073,3292.167,3231.556,1.0188
top-k=40,151936,8,5,no,1871.235,1381.879,1.3541,4404.679,4148.407,1.0618
top-k=40,152064,1,4,no,298.991,717.253,0.4169,2570.369,2418.310,1.0629
top-k=40,152064,4,6,no,918.351,1158.069,0.7930,3410.429,3095.319,1.1018
top-k=40,152064,8,6,no,1829.677,1685.768,1.0854,4146.764,4427.966,0.9365
top-p=0.9,32768,1,1,yes,261.039,90.089,2.8976,2387.337,2519.492,0.9475
top-p=0.9,32768,4,1,yes,533.970,101.370,5.2675,2671.543,2535.623,1.0536
top-p=0.9,32768,8,1,yes,964.567,112.751,8.5548,3133.880,2576.102,1.2165
top-p=0.9,65536,1,1,yes,328.004,142.558,2.3008,2435.007,2544.924,0.9568
top-p=0.9,65536,4,1,yes,788.898,174.757,4.5143,2960.303,2542.743,1.1642
top-p=0.9,65536,8,1,yes,1490.953,203.221,7.3366,3816.788,2640.611,1.4454
top-p=0.9,128256,1,1,yes,478.106,261.200,1.8304,2617.199,2722.363,0.9614
top-p=0.9,128256,4,1,yes,1425.852,326.351,4.3691,3715.653,2854.136,1.3018
top-p=0.9,128256,8,1,yes,2819.883,433.021,6.5121,5823.425,2760.483,2.1096
top-p=0.9,151936,1,1,yes,539.440,302.607,1.7826,2663.842,2562.448,1.0396
top-p=0.9,151936,4,1,yes,1619.394,413.465,3.9166,4013.248,2916.051,1.3763
top-p=0.9,151936,8,1,yes,3356.067,497.462,6.7464,6333.787,2933.176,2.1594
top-p=0.9,152064,1,1,yes,539.942,307.045,1.7585,2670.848,2708.192,0.9862
top-p=0.9,152064,4,2,yes,1660.701,416.460,3.9877,3965.481,2866.641,1.3833
top-p=0.9,152064,8,2,yes,3285.304,500.919,6.5586,6157.874,2920.476,2.1085
min-p=0.05,32768,1,1,no,129.672,94.016,1.3793,2332.037,2310.464,1.0093
min-p=0.05,32768,4,1,no,145.863,100.318,1.4540,2597.627,2269.229,1.1447
min-p=0.05,32768,8,1,no,207.459,109.626,1.8924,2671.266,2545.728,1.0493
min-p=0.05,65536,1,1,no,153.017,144.571,1.0584,2419.112,2643.729,0.9150
min-p=0.05,65536,4,1,no,220.193,170.920,1.2883,2353.177,2598.079,0.9057
min-p=0.05,65536,8,1,no,323.356,192.140,1.6829,2675.724,2578.940,1.0375
min-p=0.05,128256,1,1,no,200.656,263.474,0.7616,2699.910,2329.791,1.1589
min-p=0.05,128256,4,1,no,345.818,317.345,1.0897,2890.726,2573.234,1.1234
min-p=0.05,128256,8,1,no,548.347,383.559,1.4296,2975.195,2763.692,1.0765
min-p=0.05,151936,1,1,no,218.129,306.284,0.7122,2619.914,2590.178,1.0115
min-p=0.05,151936,4,1,no,391.072,420.157,0.9308,2795.915,2852.701,0.9801
min-p=0.05,151936,8,1,no,623.638,465.422,1.3399,2968.231,2964.502,1.0013
min-p=0.05,152064,1,1,no,247.834,307.056,0.8071,2536.950,2618.273,0.9689
min-p=0.05,152064,4,1,no,390.832,419.826,0.9309,2603.126,2887.939,0.9014
min-p=0.05,152064,8,1,no,619.000,464.170,1.3336,2859.053,3055.026,0.9359
top-k+top-p,32768,1,5,yes,318.316,142.998,2.2260,2446.353,2552.673,0.9583
top-k+top-p,32768,4,5,yes,699.050,173.596,4.0269,2857.738,2596.751,1.1005
top-k+top-p,32768,8,5,yes,1253.578,206.757,6.0630,3452.845,2679.836,1.2885
top-k+top-p,65536,1,4,yes,407.834,257.262,1.5853,2532.517,2673.282,0.9473
top-k+top-p,65536,4,4,yes,1013.118,366.828,2.7618,3158.163,2900.133,1.0890
top-k+top-p,65536,8,4,yes,1865.154,415.719,4.4866,4394.993,2537.038,1.7323
top-k+top-p,128256,1,4,yes,592.099,617.457,0.9589,2730.808,2763.908,0.9880
top-k+top-p,128256,4,5,yes,1794.241,810.629,2.2134,4123.493,3246.141,1.2703
top-k+top-p,128256,8,6,yes,3595.666,1135.728,3.1660,6290.603,3517.863,1.7882
top-k+top-p,151936,1,5,yes,666.468,728.986,0.9142,2808.607,3124.410,0.8989
top-k+top-p,151936,4,5,yes,2077.833,1105.511,1.8795,4666.655,3427.800,1.3614
top-k+top-p,151936,8,5,yes,4019.160,1357.182,2.9614,7083.431,3560.242,1.9896
top-k+top-p,152064,1,4,yes,666.188,721.792,0.9230,2816.758,2909.179,0.9682
top-k+top-p,152064,4,6,yes,2060.220,1136.119,1.8134,4415.835,3420.163,1.2911
top-k+top-p,152064,8,6,yes,4350.641,1344.479,3.2359,7073.620,3505.835,2.0177
61 changes: 61 additions & 0 deletions benchmarks/rocm_strixhalo-gfx1151_2026-10-07_rejection-microbench-run2.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
config,vocab,batch,worst_rounds,routed,iso_baseline_us,iso_rejection_us,iso_speedup,pipe_baseline_us,pipe_rejection_us,pipe_speedup
top-k=40,32768,1,4,no,194.855,155.631,1.2520,2316.586,2282.083,1.0151
top-k=40,32768,4,5,no,313.207,174.076,1.7993,2463.867,2492.067,0.9887
top-k=40,32768,8,5,no,526.407,208.130,2.5292,2894.568,2855.281,1.0138
top-k=40,65536,1,4,no,200.285,255.719,0.7832,2331.357,2333.729,0.9990
top-k=40,65536,4,4,no,440.074,361.156,1.2185,2579.994,2825.502,0.9131
top-k=40,65536,8,4,no,806.661,409.047,1.9720,3125.784,3353.719,0.9320
top-k=40,128256,1,4,no,271.389,608.059,0.4463,2396.637,2409.131,0.9948
top-k=40,128256,4,5,no,763.571,789.148,0.9676,3000.533,3355.683,0.8942
top-k=40,128256,8,6,no,1427.925,1115.350,1.2802,4225.194,3882.334,1.0883
top-k=40,151936,1,5,no,298.689,737.041,0.4053,2454.542,2397.814,1.0237
top-k=40,151936,4,5,no,925.825,1091.686,0.8481,3209.739,5005.202,0.6413
top-k=40,151936,8,5,no,1689.556,1336.183,1.2645,4596.320,4128.620,1.1133
top-k=40,152064,1,4,no,299.001,725.089,0.4124,2432.858,2414.907,1.0074
top-k=40,152064,4,6,no,865.903,1133.504,0.7639,3241.396,3224.991,1.0051
top-k=40,152064,8,6,no,1841.220,1345.842,1.3681,4342.417,4190.104,1.0364
top-p=0.9,32768,1,1,yes,261.119,90.139,2.8968,2387.469,2425.044,0.9845
top-p=0.9,32768,4,1,yes,529.132,101.110,5.2332,2682.513,2437.353,1.1006
top-p=0.9,32768,8,1,yes,962.673,112.761,8.5373,3130.454,2501.676,1.2513
top-p=0.9,65536,1,1,yes,327.874,140.202,2.3386,2449.749,2474.769,0.9899
top-p=0.9,65536,4,1,yes,788.217,175.519,4.4908,2966.360,2508.918,1.1823
top-p=0.9,65536,8,1,yes,1490.153,203.191,7.3338,3790.613,2713.445,1.3970
top-p=0.9,128256,1,1,yes,477.354,263.744,1.8099,2632.350,2639.868,0.9972
top-p=0.9,128256,4,1,yes,1438.706,324.358,4.4355,3728.364,2787.397,1.3376
top-p=0.9,128256,8,1,yes,2780.580,430.957,6.4521,5844.044,2679.123,2.1813
top-p=0.9,151936,1,1,yes,549.410,312.686,1.7571,2687.215,2427.424,1.1070
top-p=0.9,151936,4,1,yes,1616.188,412.853,3.9147,4052.039,2758.947,1.4687
top-p=0.9,151936,8,1,yes,3491.813,484.978,7.1999,6557.189,2753.828,2.3811
top-p=0.9,152064,1,1,yes,540.663,305.162,1.7717,2673.215,2523.989,1.0591
top-p=0.9,152064,4,2,yes,1697.671,418.444,4.0571,4037.144,2905.288,1.3896
top-p=0.9,152064,8,2,yes,3316.194,529.742,6.2600,6595.178,2922.574,2.2566
min-p=0.05,32768,1,1,no,129.032,94.017,1.3724,2424.980,2300.522,1.0541
min-p=0.05,32768,4,1,no,148.598,101.260,1.4675,2599.954,2360.187,1.1016
min-p=0.05,32768,8,1,no,207.679,111.158,1.8683,2535.399,2630.421,0.9639
min-p=0.05,65536,1,1,no,152.646,143.860,1.0611,2274.325,2618.808,0.8685
min-p=0.05,65536,4,1,no,220.403,171.140,1.2879,2385.734,2435.989,0.9794
min-p=0.05,65536,8,1,no,323.967,192.220,1.6854,2720.023,2456.728,1.1072
min-p=0.05,128256,1,1,no,200.246,265.327,0.7547,2578.774,2331.214,1.1062
min-p=0.05,128256,4,1,no,345.868,316.443,1.0930,2694.607,2781.147,0.9689
min-p=0.05,128256,8,1,no,544.360,384.390,1.4162,2807.679,2978.426,0.9427
min-p=0.05,151936,1,1,no,217.107,303.459,0.7154,2369.442,2667.904,0.8881
min-p=0.05,151936,4,1,no,390.542,418.715,0.9327,2532.346,2879.557,0.8794
min-p=0.05,151936,8,1,no,618.890,466.344,1.3271,2774.828,3027.817,0.9164
min-p=0.05,152064,1,1,no,215.193,311.143,0.6916,2460.846,2500.088,0.9843
min-p=0.05,152064,4,1,no,391.183,418.985,0.9336,2772.078,2563.971,1.0812
min-p=0.05,152064,8,1,no,626.434,465.573,1.3455,3001.080,2793.679,1.0742
top-k+top-p,32768,1,5,yes,319.749,143.399,2.2298,2458.518,2358.849,1.0423
top-k+top-p,32768,4,5,yes,701.535,173.625,4.0405,2860.542,2440.091,1.1723
top-k+top-p,32768,8,5,yes,1251.936,204.783,6.1135,3472.702,2403.626,1.4448
top-k+top-p,65536,1,4,yes,402.033,253.355,1.5868,2538.053,2591.252,0.9795
top-k+top-p,65536,4,4,yes,1001.306,361.207,2.7721,3165.131,2675.332,1.1831
top-k+top-p,65536,8,4,yes,1871.717,424.425,4.4100,4401.561,2697.870,1.6315
top-k+top-p,128256,1,4,yes,590.847,619.080,0.9544,2747.959,2848.315,0.9648
top-k+top-p,128256,4,5,yes,1802.527,809.357,2.2271,4122.277,3215.262,1.2821
top-k+top-p,128256,8,6,yes,3511.961,1127.082,3.1160,6426.126,3503.563,1.8342
top-k+top-p,151936,1,5,yes,665.868,724.197,0.9195,2809.915,3054.265,0.9200
top-k+top-p,151936,4,5,yes,2069.397,1100.502,1.8804,4663.215,3552.715,1.3126
top-k+top-p,151936,8,5,yes,4028.228,1345.020,2.9949,6978.657,3728.141,1.8719
top-k+top-p,152064,1,4,yes,667.571,717.525,0.9304,2814.908,3024.097,0.9308
top-k+top-p,152064,4,6,yes,2097.140,1147.320,1.8279,4452.282,3485.748,1.2773
top-k+top-p,152064,8,6,yes,3967.494,1343.798,2.9524,6977.370,3715.382,1.8780
Loading
Loading