Skip to main content
KetQat

Appearance

analysis table

Shared-sample decoder comparison: accuracy, latency, throughput, memory

Three decoders (pymatching MWPM, beliefmatching, tesseract) on byte-identical Stim detector samples per configuration, rotated surface code, circuit-level depolarizing noise, 5000 shared shots, seed 7. Measured configurations: d=3 at p=0.008 and p=0.02, and d=5 at p=0.008. d=5 at p=0.02 and all d=7 cells are ABSENT, not zero: the tesseract timing child dies there (suspected OOM in the most-likely-error search; ketqat-benchmarks#8) and an absent measurement is recorded as absence. Accuracy cells are measured counts on the shared sample; latency percentiles (500 shots x 3 randomized repeats), batch throughput and peak RSS are measured per decoder in an isolated child process on the machine named in each row and do not transfer to other machines. Pauli/Stim regime only.

License
Not recorded in the submission
Owner
Unowned (automation import)

Versions

v1
content stored here

Checksum (SHA-256)

0155c900769b96da58f3ec01243469f2c17c3bba81d14976938054430eba0a0c

computed by the platform from the stored content

Size: 5,843 bytes

Rows: 9

Schema

{
  "columns": [
    {
      "name": "distance",
      "type": "integer"
    },
    {
      "name": "physical_error_rate",
      "type": "float64"
    },
    {
      "name": "seed",
      "type": "integer"
    },
    {
      "name": "shots",
      "type": "integer"
    },
    {
      "name": "sample_sha256",
      "type": "string",
      "description": "SHA-256 of the shared detector sample every decoder consumed"
    },
    {
      "name": "machine",
      "type": "string",
      "description": "timing values are machine-dependent"
    },
    {
      "name": "decoder",
      "type": "string"
    },
    {
      "name": "library_version",
      "type": "string"
    },
    {
      "name": "decoded",
      "type": "integer"
    },
    {
      "name": "abstentions",
      "type": "integer"
    },
    {
      "name": "coverage",
      "type": "float64"
    },
    {
      "name": "logical_errors",
      "type": "integer"
    },
    {
      "name": "conditional_error_rate",
      "type": "float64"
    },
    {
      "name": "wilson_low",
      "type": "float64"
    },
    {
      "name": "wilson_high",
      "type": "float64"
    },
    {
      "name": "latency_p50_us",
      "type": "float64"
    },
    {
      "name": "latency_p95_us",
      "type": "float64"
    },
    {
      "name": "latency_p99_us",
      "type": "float64"
    },
    {
      "name": "batch_throughput_shots_per_second",
      "type": "float64"
    },
    {
      "name": "peak_rss_bytes",
      "type": "integer"
    },
    {
      "name": "compile_seconds",
      "type": "float64"
    }
  ],
  "sort_order": [
    "distance",
    "physical_error_rate",
    "decoder"
  ],
  "absent_cells": "d=5 p=0.02 and all d=7 configurations: tesseract timing child death (ketqat-benchmarks#8); absence recorded, never padded",
  "row_granularity": "one row per (distance, physical_error_rate, decoder); all decoders in a row group consumed the same sample_sha256"
}

Lineage

No source runs named.

Generator: python -m ketqat_benchmarks.decoder_comparison --distance D --rounds D --noise P --max-shots 5000 --seed 7 --latency-shots 500 --timing-repeats 3

Content: available via the API