Metrum AI Bench Documentation

Metrum AI Bench helps teams run reproducible AI serving benchmarks across models, modalities, serving frameworks, and hardware, then compare and export the results.

Getting started

New to Metrum AI Bench? Start here.

• Quickstart: get your first benchmark running in about 15 minutes.

Learn the product

• User Guide: create projects, configure engine arguments, monitor runs, read logs, and export reports.
• Billing, Plans and Metering: subscription plans, Stripe checkout, usage limits, and invoices.
• Hardware Compatibility: supported accelerators and frameworks per workload type.

Understand the numbers

• Performance Methodology: how every metric is measured.
• KYAI Methodology: how quality evaluation works.
• GenAI-Perf Methodology: how NVIDIA GenAI-Perf is used for LLM serving baselines.
• InferenceX Methodology: the InferenceX measurement model.

Reference

• Feature Reference: per-feature configuration lookup.
• Command Templates: how framework and tool commands are resolved.
• API Reference: the REST and RPC surface for programmatic access.
• Release Notes: what is new and changed.