Tasks/Math & Scientific Computing/Quantum Computing

Beat MWPM: decode quantum error-correction color codes

Reimplement a colour-code decoder without the usual libraries

qec_decoder_arena Math & Scientific Computing Quantum Computing
instruction.mdthis is what the agent is given

You inherit a plain PyMatching (minimum-weight perfect matching) decoder and a labeled development pack for four superdense color-code memory experiments. Your decoder is then re-run by a sealed grader on four HELD-OUT color settings (same code family, calibrated to the same anchors) that you never see. MWPM is the field standard for surface codes but is structurally wrong for color codes. Each setting is scored on a relative scale whose 1.0 is a reference level recorded in its meta.json; the top of the range is for beating it. You submit a decoder, not answers.

Hard Constraints

  • Edit /app/methods/main/solver.py. Expose decode(setting, dets) -> predictions, a boolean array of shape [n_shots, n_observables] — per shot, the predicted logical-observable flips.
  • stim + pymatching + numpy + scipy + standard library ONLY. No other decoding package is installed, and none can be added — whatever decoder you submit has to be built from these.
  • Predictions are compared bit-for-bit against hidden ground-truth flips: LER = fraction of shots with ANY observable mispredicted. The shape must match the truth exactly or the setting scores 0.
  • Per-setting decode wall-clock budget = setting["meta"]["decode_budget_sec"] (600 s for d5, 900 s for d7/d9). Over budget = 0 for that setting.

What You Have

  • /app/methods/main/solver.py: the inherited weak decoder — plain PyMatching on the published DEM (force-decomposed hyperedges on color codes). This file is graded — improve it in place.
  • /app/arena_harness.py: the verifier's exact scoring code (load_setting, ler_of, setting_score, SCORE_CAP).
  • /app/settings_visible/{color_d5,color_d5_Z,color_d7,color_d9}/: four VISIBLE color-code settings. Each carries meta.json (with the calibrated ler_base / ler_sota anchors), model.dem and model.stim (the published, deliberately miscalibrated noise model), dev.npz (a LABELED dev pack: dets + true obs), and eval.npz (here dets + obs, so you can score locally). The sealed graded settings are siblings of these four.
  • /app/selfcheck.py: free local dry-run using the grader's exact scoring.

What You Submit

Edit /app/methods/main/solver.py, keeping the contract:

def decode(setting, dets):
    # setting: dict from arena_harness.load_setting, keys:
    #   meta       (name, family, style, distance, rounds, n_dev, n_eval,
    #               decode_budget_sec, n_detectors, n_observables, ...)
    #   name, dem (published DEM text), model_stim (published circuit text),
    #   dev_dets [n_dev, n_detectors] bool, dev_obs [n_dev, n_observables] bool,
    #   eval_dets [n_eval, n_detectors] bool  (eval_obs is withheld when graded)
    # dets: the syndromes to decode (== setting["eval_dets"]).
    # Return predictions [n_shots, n_observables] bool.
    ...

Iterate against selfcheck.py, then leave your best solver.py in place.

How It Is Judged

For each sealed color-code setting the grader runs your decode() in an isolated subprocess (never seeing the ground truth), computes ler_agent, and scores

score = clip( (ln ler_base - ln ler_agent) / (ln ler_base - ln ler_sota), 0, CAP )

with LERs floored at 1/(2*shots). ler_base is plain MWPM on the published DEM; ler_sota is the reference level recorded with each setting. metric = mean over the four sealed color settings of that per-setting score. Higher is better; the inherited baseline sits at 0 and the reference level at 1.0. Each per-setting score is capped at 2.5.

Two facts about the setup, stated without a recommended approach: MWPM is the inherited decoder and it is structurally wrong for color codes, and the published DEM is deliberately miscalibrated versus the noise the eval shots were drawn from. The labeled dev pack (dev_dets / dev_obs) is yours to use however you see fit. Only stim, pymatching, numpy and scipy are available.

Metric

mean per-setting score over the four sealed color codes · higher is better

score = clip((ln ler_base - ln ler_agent)/(ln ler_base - ln ler_sota), 0, 2.5), per setting

anchorheld-outreward
Binherited plain PyMatching0.00.00
Spublished open-decoder portfolio1.00.60
Uall four settings at the 2.5 cap2.51.00
normalisation
m <= B0
B < m <= S0.6 * (m - B) / (S - B)
S < m <= U0.6 + 0.4 * (m - S) / (U - S)

m = this run's held-out metric  ·  B = inherited plain PyMatching  ·  S = published open-decoder portfolio  ·  U = all four settings at the 2.5 cap

B=0, S=1.0, U=2.5. Per-setting scores are capped at 2.5 and averaged first; the piecewise-linear map is applied once to that mean.

Rollouts

697 minwall clock
$195.16spend
334.4Mtokens
45versions, 15 kept
0 0.3 0.6 0.9 1.2 1.5 $0 $40 $80 $120 $160 cumulative spend on the run visible 4-setting metric, higher is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44
keptrevertedno scoreturning point
  1. v0Inherited plain PyMatching on the force-decomposed published DEM01 min · $0.16
  2. v1Fit one global noise multiplier from dev marginals, rebuild the DEM, plain MWPM0.00024 min · $0.75
  3. v2PyMatching correlated two-pass decoding after dropping irreducible hyperedgesMatching cannot ingest color-code hyperedges; drop the irreducible ones so the correlated pass can use the rest.0.30586 min · $1.04
  4. v3Three color-pair projections, seven noise scales, logistic vote combinerDecode the color code as three matchable two-color projections and let a dev-fit combiner arbitrate their logical votes.0.634618 min · $3.02
  5. v4Sparse 64-unit detector net on 60k synthetic shots, added to the stack0.756530 min · $4.91
  6. v5Time-integrated inputs; parity-constrained local MILP on the least confident shotsSpend the wall-clock budget where the stack is least confident: exact parity repair on the worst 10-15% of shots.1.2057114 min · $31.86
  7. v6Zero-slack-only repairs, wider coverage, K=400 on the hardest 2% of d91.3235183 min · $55.40
  8. v7Domain-adapt the synthetic net on opposite dev folds with replay and anchoringno full-pack metric189 min · $56.06
  9. v8Rebuild the d9 local hypergraph at the dev-inferred noise scale, gated at 1.25no full-pack metric205 min · $58.13
  10. v9Extend the d9 confidence candidate list from 20% to 30% of shots1.3441214 min · $59.49
  11. v10Cut retained hyperedges from K=300 to 250 past the 20% d9 boundaryno full-pack metric218 min · $60.16
  12. v11Re-tune the d9 artificial detector-slack penalty from cost 26 to 18no full-pack metric222 min · $60.94
  13. v12Raise the hardest-2% d9 MILP basis from K=400 to K=450no full-pack metric226 min · $61.64
  14. v13Second raw-event synthetic net for d9, combined with the integrated oneno full-pack metric255 min · $66.97
  15. v14Diagnostic: supervised MILP schedulers from cross-fit dev error-risk modelsdiagnostic only257 min · $67.31
  16. v15Merge duplicate DEM mechanisms with identical detector and logical supportno full-pack metric261 min · $68.24
  17. v16Lower the d9 syndrome-overlap ranking reward from 2.0 to 1.5no full-pack metric$71.00
  18. v17Overlap reward 2.0 to rank 20%, then 1.5 in the added 20-30% tailno full-pack metric$73.76
  19. v18Diagnostic: exact-syndrome dev lookup as an override on recurrent syndromesdiagnostic only$76.52
  20. v19Widen the time-integrated synthetic net from 64 to 96 hidden unitsno full-pack metric306 min · $79.27
  21. v20Regenerate the d7 local hypergraph at probability scale 0.7no full-pack metric313 min · $81.15
  22. v21Accept incumbents at slack<=2 before rank 20%, slack<=1 in the d9 tail1.3505324 min · $84.05
  23. v22Accept all hardest-2% incumbents, then slack<=1 on every K=300 d9 shotno full-pack metric355 min · $92.79
  24. v23Shorten the d9 per-shot MILP time limit from 0.12s to 0.08sno full-pack metric360 min · $94.24
  25. v24Use K=350 instead of K=300 for d9 confidence ranks 2-20%no full-pack metric368 min · $96.95
  26. v25Diagnostic: syndrome-weight tail scheduler for confidence ranks 20-30%diagnostic only407 min · $108.18
  27. v26Replace the 5%-of-budget local deadline reserve with a fixed 20s reserve1.3515407 min · $108.18
  28. v27Cut the fixed deadline reserve from 20s to 10s1.352456 min · $126.90
  29. v28Cut the reserve again to 5s, about 75 more d9 tail attemptsno full-pack metric472 min · $133.99
  30. v29Cache the dev-fitted global noise scale in the per-call setting dictno full-pack metric493 min · $143.42
  31. v30Swap marginal K=300 shortlist columns for look-ahead ranks 300-349no full-pack metric499 min · $146.12
  32. v31Drop the redundant 0.001 and 2.0 correlated-MWPM scales at distance 9+1.352523 min · $154.31
  33. v32Min-sum BP before local MILP at d<=7, accepting exact-syndrome fixed points1.3553554 min · $158.32
  34. v33Widen the residual d5 MILP shortlist from K=250 to K=300 with BP-saved timeno full-pack metric558 min · $159.03
  35. v34Extend exact-syndrome BP coverage for d7 from rank 35% to 60%no full-pack metric568 min · $160.62
  36. v35Seed 30% of the d5 residual shortlist from unconverged local-DEM BP beliefsno full-pack metric580 min · $162.98
  37. v36Diagnostic: MILP acceptance audit on mechanism count, weight and margindiagnostic only584 min · $163.88
  38. v37Diagnostic: 8- and 12-iteration BP certificates for d5-class settingsdiagnostic only624 min · $173.51
  39. v38Diagnostic: exact-syndrome BP-12 on the d9 local slicediagnostic only$174.85
  40. v39Add all pairwise constituent-vote products to the dev logistic combinerdiagnostic only$176.20
  41. v40Dev-guarded combiner regularization sweep; only d5_Z moves off the default1.3573641 min · $177.54
  42. v41Diagnostic: cross-applied combiner decision-threshold calibration per folddiagnostic only653 min · $181.03
  43. v42Refine the active d5_Z regularization on a finer grid around 1.0diagnostic only654 min · $181.37
  44. v43Dev-select among four combiner feature stacks under a both-fold gateno full-pack metric663 min · $183.94
  45. v44Raise the residual d5-class MILP per-shot cap from 0.08s to 0.12s1.3766674 min · $187.48

45 snapshots, 11.6 h, $195. v5 already held 1.206 of the final 1.377 for $32; the other 39 versions bought +0.17 for $156.

On the hidden set

held-out metricreward
inherited plain PyMatching0.00.00
published open-decoder portfolio1.00.60
all four settings at the 2.5 cap2.51.00
this run1.32960.6879