InferenceBench

deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

vllm 0.21.0 on 8x NVIDIA H100 80GB HBM3 (fp16) · seed 42 · run 019e3b6f-22ca-7637-9ad5-66763c672117 · signed

Headline metrics

TTFT P50 (ms) 74.43
TTFT P99 (ms) 521
Throughput (tok/s) 134
$/M tokens —
J/token 5.94
Power avg (W) 808
Power peak (W) 855
WER mean —
J / audio s —

All metrics

compliance_rate 0.8800
energy_joules_total 17,249
joules_per_token 5.94
ok_rate 1.00
power_avg_w 808
power_peak_w 855
req_per_s_all 1.16
req_per_s_passing 1.02
slo_hardware_class h100
slo_template_resolved ttft<200ms, tpot<50ms, total<3000ms
throughput_tok_per_s 134
total_p50_ms 2,661
total_p99_ms 3,137
tpot_p50_ms 21.04
tpot_p99_ms 23.46
ttft_p50_ms 74.43
ttft_p99_ms 521

Provenance

Model revisionunknown00
Providervllm
Hardware fingerprint550474fc9132129654f5d20c316eaffec99a1f67c0cc0b4641e60efb46ea79e7
Datasetbuiltin-chatbot-short · 6f4b1f68fc3a813baa983cbe70cd9ef57f8c86e6b2e6ccc9aaa2a498e588d510
Timestamp2026-05-18T14:13:19.690272+00:00
Envelope JSONeb104ca63c1e.json

Verify this result

bench verify /bench/leaderboard/envelopes/eb104ca63c1e.json

bench verify re-downloads this envelope, recomputes the canonical content hash, and validates the Sigstore signature against the embedded certificate + Rekor entry.