InferenceBench

Qwen/Qwen2.5-72B-Instruct

vllm 0.22.1 on 8x NVIDIA H100 80GB HBM3 (bf16) · seed 42 · run 019ebf77-7597-7671-a417-726e28e10417 · signed

Headline metrics

TTFT P50 (ms) 47.83
TTFT P99 (ms) 85.24
Throughput (tok/s) 891
$/M tokens —
J/token 2.56
Power avg (W) 2,287
Power peak (W) 2,312
WER mean —
J / audio s —

All metrics

compliance_rate 1.00
energy_joules_total 210,357
joules_per_token 2.56
ok_rate 1.00
power_avg_w 2,287
power_peak_w 2,312
req_per_s_all 7.49
req_per_s_passing 7.49
slo_hardware_class h100
slo_template_resolved ttft<200ms, tpot<50ms, total<3000ms
throughput_tok_per_s 891
total_p50_ms 2,240
total_p99_ms 2,353
tpot_p50_ms 17.30
tpot_p99_ms 18.76
ttft_p50_ms 47.83
ttft_p99_ms 85.24

Provenance

Model revisionunknown00
Providervllm
Hardware fingerprintc353425077b20b36c71d9c32d9e5d49096478f4a0f7d6078c1dfc83c408ccf50
Datasetbuiltin-chatbot-short · 6f4b1f68fc3a813baa983cbe70cd9ef57f8c86e6b2e6ccc9aaa2a498e588d510
Timestamp2026-06-13T05:32:17.687700+00:00
Envelope JSONc16-b91875fdf860.json

Verify this result

bench verify /bench/leaderboard/envelopes/c16-b91875fdf860.json

bench verify re-downloads this envelope, recomputes the canonical content hash, and validates the Sigstore signature against the embedded certificate + Rekor entry.