Your task is to design a reproducible, noise-aware optimizer that minimizes 10-dimensional continuous black-box objectives with smooth periodic, multimodal structure. You inherit a weak uniform-random baseline, and every run uses the box [-5, 5]^10 with a strict budget of 120 objective queries. Your submitted optimizer is re-run on disjoint sealed instances.
Hard Constraints
Edit only /app/methods/main/; solver.py must define Optimizer.
Use only the Python standard library and NumPy 2.2.6.
The required constructor is Optimizer(dim, lower, upper, budget, seed, rng). No task-information argument is supplied.
ask(n) must return a finite NumPy-compatible matrix with 1 through n rows, exactly dim columns, and all coordinates within the supplied bounds.
The verifier owns the objective and query counter. Extra returned points do not increase the 120-query budget.
Import failures, crashes, malformed output, non-finite values, and out-of-bounds proposals invalidate the complete submission.
The submitted process cannot read or modify trusted evaluator assets and has no verifier network access.
Runtime budget.
Submitted optimizer code receives a 120-second aggregate soft budget across the complete sealed suite,
not a separate 120 seconds for every run. The suite contains 400 independent optimizer runs, so
each run must average about 0.3 seconds. There is no five-second scoring cutoff on every individual
ask or tell response; when the aggregate soft budget expires, the verifier stops requesting new
work and evaluates the best completed state. A stalled process can still be terminated at a sealed
safety cap. Use bounded, vectorized per-query work; repeated dense refits or hundreds-wide candidate
scans at every observation are unlikely to fit. This budget applies to sealed execution, not to your
research time.
What You Have
/app/data/visible.json contains twelve public development instances from the same noisy continuous family; sealed instances are distinct.
/app/methods/main/solver.py is a uniform-random baseline.
/app/selfcheck.py evaluates the same higher-is-better normalized anytime/final metric family used by the sealed evaluator on 20 deterministic runs per visible instance. It also reports diagnostic components and latent final-objective summaries.
The supplied rng is np.random.default_rng(seed) and should drive all randomness for deterministic replay.
The verifier repeatedly calls ask(n), evaluates the returned points, and calls tell(X, y) (or tell(X, y, metadata) if accepted). Each value in y is an observed noisy loss, so lower is better even though the aggregate self-check score is higher-is-better.
A positive integer self.batch may request a preferred batch size; the verifier negotiates and caps it to the remaining budget.
What You Submit
Submit optimizer code, not a one-shot point or precomputed answer. The entire submitted optimizer must be self-contained in /app/methods/main/solver.py, which must define the required Optimizer class; sibling modules are not copied to the trusted verifier.
How It Is Judged
The trusted parent evaluates your optimizer on sealed instances and fixed seeds. It records authoritative latent best-so-far traces, aggregates them robustly across seeds, and combines anytime quality with final-query quality under the same metric definition exposed by the visible self-check. Higher normalized quality is better; sealed instances, calibration assets, and evaluator internals remain hidden.
Metric
oracle-normalized anytime/final quality, 20 sealed instances · higher is better
0.70*anytime + 0.30*final quality; uniform-random floor = 0, exact oracle = 1; median of 20 seeds, 120 queries
anchor
visible set
held-out
reward
uniform-random floor (shipped starter)
0.000
0.000
0.00
R
human reference (fixed centered CMA)
0.462
0.504
0.30
exact oracle (planted minimum)
1.000
1.000
1.00
normalisation
q <= 0
0
0 < q <= R
0.3 * q / R
q > R
0.3 + 0.7 * (q - R) / (1 - R)
R = human reference (fixed centered CMA)
R is the instance's own sealed reference quality (mean 0.504). Each of the 20 instances is mapped and clipped, then averaged.
v2Initial normalised CMA scale 0.45 -> 0.350.3087487 min · $0.90
v3Initial normalised scale 0.35 -> 0.300.3748947 min · $1.06
v4Initial normalised scale 0.30 -> 0.250.3801468 min · $1.19
v5Initial normalised scale 0.25 -> 0.200.3954828 min · $1.42
v6Initial normalised scale 0.20 -> 0.150.3914089 min · $1.54
v7Batch size screened 4-30; 10 -> 7 for more adaptation generations0.44506814 min · $2.25
v8Batch 6 confirmation0.43584115 min · $2.36
v9Batch 8 confirmation0.43449415 min · $2.47
v10Initial sigma swept 0.10-0.35; cross-fold-stable 0.175 tried0.43975216 min · $2.71
v11Antithetic sampling: three mirrored pairs plus one free sample per batchMirrored pairs cancel first-order noise that seven independent samples cannot, so each generation adapts on a cleaner signal.0.51310917 min · $2.93
v12Spend the query left by 120 mod 7 on evaluating the learned mean0.51895921 min · $3.63
v13After generation 10, replace each batch's free mutation with a mean evaluation0.53108122 min · $3.98
v14Fixed-radius mutually orthogonal directions for each mirrored trio0.51441331 min · $5.44
v15One random orthonormal frame per run, cycled across generations0.53327433 min · $5.83
v16Sample through a Cholesky covariance root, keeping the frame's orientationEigendecomposition re-rotates the frame each generation; a Cholesky root keeps the planned directions intact as covariance adapts.0.59009134 min · $6.13
v17Switch the odd seventh query to mean evaluation from generation 6, not 100.592437 min · $6.82
v18Three regularised local parabolas per batch, blended 50/50 with rank recombinationMirrored pairs already measure curvature along each direction, so fit it and take a Newton step instead of only ranking.0.61086544 min · $8.35
v19Start centre sampling and quadratic blending at generation 2 instead of 60.6252646 min · $8.89
v20Quadratic curvature regularisation floor 2 -> 50.62231450 min · $10.22
v21Quadratic displacement clip +/-0.75 -> +/-1.00.62942652 min · $10.72
v22Conditional restart at query 49, sigma 0.225, when best loss stays above 400.62921469 min · $15.64
v23Shrink sigma 25% once at query 42 when best noisy loss is already below 150.63088773 min · $16.94
v24Relative restart trigger: loss above 30 and over 70% of its query-14 level0.63155886 min · $21.00
v25Scale the quadratic blend by the positive-curvature fraction for 10 generations0.63637691 min · $23.16
v26Evaluate each model-assisted batch's mean before its six mutations0.63582694 min · $23.93
v27From generation 8, order mutations by evolution-path alignment0.63680797 min · $25.26
v1Own numpy CMA-ES from the origin, sigma0=1.0, lambda=100.410415 min · $2.04
v2GP-LCB with a kernel derived from the family: SE l=3.28, s^2=289, mu0=90Read the kernel off the reverse-engineered generator instead of fitting one: a GP at length-scale 3.28 plus a 90-unit dimple.0.520816 min · $2.10
v4Incremental K^-1 by Sherman-Morrison; analytic gradient refinement of the LCB0.755592 min · $6.08
v5Analytic Gaussian-well prior mean replaces the constant 90; full-box candidatesPut the centre bias in the model rather than in the candidate clipping, so the search can cover the whole box.0.7798119 min · $7.55
v6Objective-scale adaptivity from the initial design; wall-clock guard 0.25 s/runEvery derived constant assumes the family's 90-unit scale, so estimate that scale from the initial design and rescale mu and s2.0.7749161 min · $10.96
v7Hardened rewrite: box- and dim-relative constants, sanitised output, 0.27 s guard0.7731186 min · $14.32
Eight snapshots, 3h39, $18.13, nothing reverted. Every kept decision was re-checked on fresh synthetic instances; an untouched holdout agreed.
v1Mirrored CMA-ES from the origin, lambda=8, sigma=1.1, last 12 queries polish0.446681
v2Clip samples to [-pi,pi] where the family's minimisers live; sigma 1.20The planted minima sit inside [-pi,pi], so spend no queries on the rest of the box.0.515405
v3CMA for 105 queries, then a ridge-quadratic Newton polish0.511947
v4Evaluate the same CMA points nearest-to-best first, for anytime credit0.497502
v5Covariance-shaped (1+1) polish over the last 10 queries, 1/5 success rule0.51909
v6Polish step 0.18 -> 0.220.520645
v7After 4 CMA generations, screen 32 mirrored candidates by a ridge quadraticA generation only ranks what it pays for; score 32 candidates on a local model first so the 8 real queries land on good points.0.550917
v8Quadratic screening starts a generation earlier, warmup_gens 4 -> 30.576564
v9Screen the last partial CMA generation too0.580341
v10Rank candidates by a z-scored blend of ridge quadratic and Gaussian RBFOne surrogate misreads a multimodal landscape; averaging two z-scored models keeps the ranking usable where they disagree.0.592248
v11Blend weight 0.5, leaning on the RBF surrogate0.606318
v12Polish budget to 8, two more filtered CMA queries0.608385
v13Initial sigma 1.20 -> 1.180.610802
v14Candidate pool 32 -> 340.616678
v15RBF fitted on the 80 nearest points; the quadratic stays at 400.624856
v16RBF neighbourhood 80 -> 900.629431
17 snapshots, 1h31; this harness reports no cost or tokens. The verifier's 120 s cap stopped it at 284 of 400 runs, zeroing 6 instances.
v3SPSA with 2-point antithetic updates and Adam smoothingUnder 120 noisy queries in 10 dimensions a two-query antithetic gradient buys more per query than a covariance model.0.159799$0.48
v4Screened nine SPSA perturbation/rate pairs; c=0.50 physical, rate=0.0500.19629$0.60
v516-point uniform basin seeding, then 52 SPSA updatesSPSA is local and the objective multimodal, so spend an eighth of the budget deciding which basin the rest polishes.0.227545$0.72
v6Initial design screened at 8/12/24/32 points; 8 selected0.237962$0.84
v7Randomised rows of a 16x16 Hadamard block replace iid Rademacher directionsIndependent directions interfere across coordinates; an orthogonal block cancels that, so the same queries buy a cleaner gradient.0.269541$0.96
v8Momentum and rate screened; beta=0.70, rate=0.0600.282116$1.08
v9Perturbation decay gamma=0.2 and learning decay alpha=0.10.288508$1.20
v10Reset the SPSA centre to the best symmetric-pair mean every 16 updates0.279126$1.32
v118-point jittered Latin hypercube instead of eight iid global seeds0.269806$1.44
v12Constrain every SPSA centre to [c, 1-c] so paired perturbations stay symmetric0.282575$1.56
v13Spend the last 20 queries on coordinate polishing, radius 0.350.282934$1.68
v1Matern 5/2 GP-BO with LCB, y-normalised, 10 random init, 500 candidates120 queries are too few to search, so fit a GP and let LCB pick where the model is both low and uncertain.0.3399443 min · $0.13
v2Init design 10 -> 5 points; LCB kappa decays linearly 3.0 -> 0.1Tie exploration to the budget: wide kappa early, near-greedy by the last query, and half the init design given back.0.38490931 min · $0.47
v3Latin hypercube for the global half of the candidate pool; init stays random0.39742742 min · $0.71
v4Minimum 0.02 distance to observed points; nested Cholesky fallbacks0.39742769 min · $1.44
Five snapshots, 1h09, $1.48, nothing reverted; 8 unsnapshotted screens all failed. The 120 s cap stopped it at 159 of 400 runs, zeroing 12.