Skip to main content
KetQat

Appearance

Qiskit 2.x · local and customer CI

Catch your first regression.

Start with an intentional circuit change. Then point the same capture command at a function returning your own Qiskit circuit. Free capture and compare require no account or upload.

Requirements · Python 3.11 + GitFirst result · offline HTML report

1. Install and run the sample

Use Python 3.11 and Git. This pre-release source installation pins an actual public commit; it does not depend on an unpublished PyPI package. Review and pin the SDK revision you use in CI.

git clone https://github.com/ketqat/ketqat-sdk.git
cd ketqat-sdk
git checkout e5d37278155058944faf7b32526d8dc787065281
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --require-hashes -r requirements-regression-py311.txt
python -m pip install --require-hashes -r requirements-regression-build-py311.txt
python -m pip install --no-build-isolation --no-deps ./python
ketqat regression sample --output-dir sample

Expected exit: 1 — REGRESSION. The candidate intentionally removes a CX gate from a Bell circuit. This is a real ideal-simulation measurement of a deliberate example, not a reported SDK defect.

Open sample/report/report.html. Total-variation distance is approximately 0.5 against a maximum of 0.05. Portable JSON and Markdown are beside the HTML. Output directories must be new; captures never silently replace a baseline.

See the generated sample report

2. Compare your existing circuit

Expose a function returning your QuantumCircuit. KetQat imports and runs it only on your machine or CI runner. Run code you trust; the local child process is not a security sandbox.

# tests/circuits.py
from qiskit import QuantumCircuit

def prepare_state():
    qc = QuantumCircuit(2)
    qc.h(0)
    qc.cx(0, 1)
    return qc
ketqat regression capture tests/circuits.py:prepare_state \
  --case-id prepare-state --output baseline.json
# Make the intended candidate change; keep baseline.json unchanged.
ketqat regression capture tests/circuits.py:prepare_state \
  --case-id prepare-state --output candidate.json
ketqat regression compare baseline.json candidate.json \
  --policy examples/regression/policy.json --output-dir change-report

Adapt the supplied policy before relying on it. Its relative resource allowance is a fraction: 0.1 means +10%. An upper resource bound is baseline × (1 + relative allowance) + absolute allowance. This resource threshold is separate from the total-variation tolerance.

For a Qiskit-only upgrade, select "changed_axes": ["qiskit"]. Capture in separate environments with identical Python, other dependencies, machine, source, circuit, seed, shots and transpiler settings. Version drift is recorded; only the explicitly selected Qiskit axis is permitted. Do not widen axes just to make a failure disappear.

Python integration is also available: capture(qc, case_id=...) and save_snapshot(snapshot, Path(...)) from ketqat_runner.regression. The full snapshot records source/environment fingerprints, circuit instructions, conditions and selected observations. Inspect it before sharing.

3. Keep the comparison as your CI gate

Use the SDK's pinned Actions example. It captures the candidate and compares a protected baseline. The CLI appends a report when GITHUB_STEP_SUMMARY is set and returns the comparison exit code.

Run untrusted pull requests with read-only permissions and no upload secret. Optional upload belongs in a separate job on a reviewed, protected branch. Do not run fork code with secrets under pull_request_target. A GitHub App and a long-lived PAT are not required. Your CI provider's compute charges are separate.

Preserve the compare exit code. An upload exit of 0 confirms transport only; it never changes a REGRESSION into a pass.

4. Optionally keep private team history

Check current Team availability and limits. Hosted uploads require an enabled service and a server-confirmed subscription in the same test or live environment. The local comparison is complete without this step.

  1. Sign in and create a workspace. The owner manages payment; admins manage repository policies, baseline approvals and tokens; viewers read and export.
  2. Create a repository alias and review its initial policy. Download the active policy JSON from that repository page and use it locally. Adding an alias gives KetQat no access to your GitHub source.
  3. For the first baseline, capture a trusted circuit, compare that capture with itself using the active policy, and upload its summary. An owner or admin reviews the full local report, then approves the candidate on its private report page with a reason. Save the matching capture in protected local/CI storage.
  4. For the next comparison, use that exact approved capture as baseline. The upload checks the baseline fingerprint and active policy. If either changed, reload and rerun; no failed candidate is automatically promoted.
  5. Create an expiring repository upload token. Store it as KETQAT_REGRESSION_TOKEN in protected environment secrets. The value is shown once, stored hashed on the server, and never needed as a command argument.
ketqat regression preview change-report/report.json --output summary.json
# Open and inspect summary.json. Copy the SHA256 printed by preview.
ketqat regression upload summary.json \
  --repository REPOSITORY_ID_FROM_WORKSPACE \
  --confirm-sha256 SHA256_PRINTED_BY_PREVIEW

The preview contains selected metrics, policy, changed field names, verdict and stable hashes. It excludes source code, circuit instructions, raw counts/probabilities, local paths, environment values and raw case or repository names. Hashes and metrics can still be confidential; hashing is not anonymization. No third-party model receives this data.

The client confirms the exact file bytes, requires HTTPS, refuses redirects and makes at most three attempts with 10-second timeouts and 2/4-second delays. Keep the local report if upload fails. Never put private captures or tokens in public Actions artifacts or public issue reports.

Understand the verdict and its scope

Comparison verdicts and exit codes
VerdictExitMeaning / next action
WITHIN_POLICY0All selected observations fit this policy. Review its scope.
REGRESSION1A selected limit was exceeded. Inspect it; do not auto-promote.
INCONCLUSIVE2Insufficient or missing evidence. Revisit the declared sample budget.
INCOMPATIBLE3Fixed conditions changed. Inspect local comparison fields.
ERROR4Capture, process, input or output failed. Investigate locally.
NOT_RUN5Unsupported circuit or unavailable dependency. No successful execution.

Initial support: Qiskit 2.x, 1–12 qubits, up to 100,000 instructions, bound standard gates, no measurements or a final full-register identity-mapped measurement. Decompose custom gates explicitly. Mid-circuit measurements, reset, classical control, delay, noise and hardware execution are unsupported. Captures compile to rx, ry, rz, cx using the recorded seed and optimization level, without a coupling constraint.

Default observations are exact ideal probabilities from Qiskit Statevector, on zero input in the computational basis. No shots are fabricated. Agreement does not prove full circuit equivalence or program correctness and can miss phase differences. Compiled gate counts are not measured device cost, runtime or speed.

With --shots, the report uses simultaneous Hoeffding/union bounds over both samples and every possible output bin, including unseen outcomes. An interval fully within tolerance passes; one fully above it regresses; overlap is INCONCLUSIVE. The conservative bound becomes uninformative as qubit count grows. family_alpha is a per-report error budget, not a lifetime guarantee across repeated reports. Predeclare an error budget across planned comparisons; do not resample until a pass.

For equivalence rather than this measured distribution, consider MQT QCEC. For property-based quantum testing, see QuCheck. KetQat uses existing Qiskit simulation and standard statistical bounds; these are not proprietary scientific inventions.

Troubleshoot without sharing private data

  • ERROR: run your circuit factory locally to inspect its exception. Confirm output paths are new, writable and within size limits. The factory has a 60-second default timeout, configurable from 1–300 seconds.
  • NOT_RUN: install pinned dependencies, bind parameters, decompose custom gates or reduce the qubit count. Unsupported operations do not become successful results.
  • 401 / TOKEN_INVALID: replace an expired or revoked upload token in your protected secret storage. The token must belong to this repository and environment.
  • 403 / TEAM_REQUIRED: check actual server subscription status. The owner can refresh payment status or use the billing portal to resolve payment failure. Reaching a payment return page does not grant access.
  • 409 / POLICY_CHANGED or BASELINE_CHANGED: download the current policy and inspect the approval history. Rerun using the approved baseline capture; never change a failure into a baseline automatically.
  • 413 / 422: regenerate and inspect the allowlisted preview. Keep each upload at or below 100 KiB. No extra scientific or source fields are accepted.
  • 429: check monthly allowance, storage and reset time on the workspace page. Export and delete old reports to free storage; deletion does not reset the UTC monthly upload count. Retry a rate limit later with a bounded delay.
  • Outage or timeout: keep the local result, check the error code and retry later. Do not loop indefinitely. Owners can still manage cancellation while new uploads are paused, when the payment service is reachable.

For a support report, share the diagnostic ID and error code, verdict and failed metric names after review. Do not attach tokens, full snapshots, raw reports, source, repository names, personal information or payment details. A diagnostic ID is designed to locate the server failure without exposing the payload.