Tasks/Operations Research/Black-Box Optimization

Anonymized Continuous Black-Box Optimizer Design

Ship a self-contained ask/tell optimizer

bbo_synthetic_shifted_v1 Operations Research Black-Box Optimization
instruction.mdthis is what the agent is given

Design a reproducible ask/tell optimizer for 10-dimensional continuous black-box minimization, starting from the inherited uniform-random weak baseline. Minimize the objective under a strict budget of 100 evaluations per instance; for scoring, the submitted optimizer code is re-run on sealed held-out instances drawn from the same anonymized objective family.

Hard Constraints

  • Edit only /app/methods/main/solver.py; it must be a self-contained file defining the submitted Optimizer class. The verifier copies only this file into its runner scratch directory, so sibling modules are unavailable.
  • Use only the Python standard library and NumPy. The runtime provides NumPy 2.2.6, has no verifier network access, and does not provide other third-party optimization packages.
  • Each hidden run has a hard 100-evaluation budget. The verifier owns and enforces the evaluation loop; returning extra points does not grant extra evaluations.
  • ask(n) must return a finite NumPy-compatible matrix with between 1 and n rows, exactly dim columns, and every coordinate within the supplied bounds.
  • The optimizer must answer within the verifier's per-request and total worker time limits. Import failures, timeouts, crashes, malformed output, non-finite values, and out-of-bounds points invalidate the complete submission.
  • The submitted process cannot access the hidden evaluator, hidden instances, frozen normalization data, or objective implementation. It receives only bounds, budget, seed, RNG, requested batch size, evaluated points, and authoritative objective values.

Runtime budget.

Each isolated sealed optimizer run has a 120 seconds total runtime budget for its complete query loop, and each ask or tell response must arrive within 5 seconds. This limit applies to the submitted optimizer, not to your research time. It is shared across 400 independent sealed runs (20 instances × 20 seeds), so each run must average about 0.3 seconds. Use bounded, vectorized per-query work; repeated dense refits or hundreds-wide candidate scans at every observation are unlikely to fit that aggregate budget. The visible self-check is intentionally unlimited and does not relax the sealed runtime budget.

What You Have

  • /app/data/visible.json contains six public development instances from the same anonymized objective family. Hidden instances are distinct and sealed.
  • /app/methods/main/solver.py is a public uniform-random baseline.
  • /app/selfcheck.py is an unlimited visible-data dry run. Execute python /app/selfcheck.py to see the median final visible objective; lower is better.
  • The concrete initialization call is Optimizer(dim=dim, lower=lower, upper=upper, budget=budget, seed=seed, rng=np.random.default_rng(seed)), where lower and upper are NumPy arrays and rng is the harness-owned generator.
  • The verifier calls ask(n). It then calls either tell(X, y) when metadata is null or tell(X, y, metadata) for an optimizer that accepts the metadata argument. This task currently supplies null metadata, so both two-argument and three-argument tell methods are supported.
  • An optional positive integer self.batch requests a preferred batch size, but the verifier always caps requests to the remaining budget.

What You Submit

Submit an optimizer implementation, not a final point or a precomputed answer. Keep the complete implementation in the self-contained /app/methods/main/solver.py; the verifier does not copy sibling modules.

How It Is Judged

For each sealed instance and fixed seed, the verifier repeatedly calls ask, evaluates the returned candidates, and reports lower-is-better values through tell until the query budget is consumed. It builds authoritative best-so-far traces and combines anytime quality with final-query quality under a fixed verifier-only normalization protocol. Higher normalized quality is better; calibration artifacts and the grader mapping remain sealed.

Metric

median final objective over 20 sealed 10-D instances · lower is better

best-so-far value at query 100, median over 20 seeds per instance, then median over the 20 hidden instances

anchorvisible setheld-outreward
uniform random floor116.88118.410.00
reference optimizer (diagnostic)81.360.28
exact oracle (global minimum)01.00
normalisation
m(t) >= F(t)q(t) = 0
F(t) > m(t) > Oq(t) = (F(t) - m(t)) / (F(t) - O)
m(t) <= Oq(t) = 1

m = this run's held-out metric

F(t)=floor median trace, O=exact oracle 0. Per instance a=mean q over t<100, f=q(100); reward=0.7*median(a)+0.3*median(f).

Rollouts

352 minwall clock
$96.95spend
142.1Mtokens
49versions, 45 kept
0 25 50 75 100 125 $0 $20 $40 $60 $80 cumulative spend on the run visible self-check median final objective, lower better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47 v48
keptrevertedno scoreturning point
  1. v0Inherited uniform-random baseline, unchanged116.8812 min · $0.25
  2. v155-point LHS fit of the Rastrigin quadratic envelope, then four gradient stepsStop searching and identify the family: fit the Rastrigin quadratic envelope from a designed sample, then walk downhill on it.4.97954 min · $0.44
  3. v216-run Hadamard corner design; np.block tuple nesting rejected by NumPy 2.xinvalid, score 08 min · $0.84
  4. v3NumPy-safe Hadamard corner regression plus seven full descent steps4.47778 min · $0.94
  5. v4Seven antithetic starts around the shrunk envelope estimate, best one descends4.975314 min · $1.58
  6. v5Descent step raised to 1.7/L, settling step 1/L3.980325 min · $3.00
  7. v6Start axes replaced by seven vertices of a random regular simplex3.482927 min · $3.38
  8. v7Two start probes moved ahead of the corner design; five-vertex simplex afterSpend two start slots before the corner design so a good best-so-far exists early; anytime AUC is 70% of the reward.2.985430 min · $3.82
  9. v8Envelope shrink 0.97 to 0.95; prior-support clip 3.99 to 3.802.985447 min · $6.45
  10. v9Two staged ridge envelope estimates after corners 8 and 122.985448 min · $6.85
  11. v10Best of four row permutations by A-optimal prefix information5.472850 min · $7.30
  12. v11Row-order search moved to a seed-derived RNG, leaving v9's draws intact2.985450 min · $7.60
  13. v12Subtract the one-sided finite-difference bias on the settling gradient2.984956 min · $8.41
  14. v13Exact-budget hybrid: nine 5-D projected cycles, then two full gradientsUpdate inside 5-D subspaces instead of full gradients, so the trace improves nine times within the same 100 queries.2.984964 min · $10.07
  15. v14Three starts at scale 0.12, eleven projected cycles, one settling gradient2.984970 min · $11.55
  16. v15Start-simplex scale 0.12 to 0.163.482471 min · $12.19
  17. v16Projected-step multiplier 1.5 to 1.63.482473 min · $12.94
  18. v17Projected multiplier annealed 1.75 down to 1.403.482475 min · $13.74
  19. v18Slope-norm radius blended 50/50 with the fitted envelope intercept3.482478 min · $14.72
  20. v19Dropped the 0.95 coordinate shrink after radial calibration3.482481 min · $15.80
  21. v20Shift and start clip tightened from 3.8 to 3.7 standard units3.482483 min · $16.62
  22. v21Final-simplex scale 0.16 to 0.183.482486 min · $17.91
  23. v22Second prior probe RMS scale 0.40 to 0.45no self-check logged89 min · $18.32
  24. v23Projected multipliers steepened from 1.75-1.40 to 1.80-1.203.4824106 min · $20.81
  25. v24Projected finite-difference radius 0.001 to 0.02, curvature bias removed3.4824117 min · $22.81
  26. v25Final projected multiplier 1.20 to 1.103.4824120 min · $23.64
  27. v26Projected difference radius annealed 0.030 to 0.010 over eleven cycles3.4824123 min · $24.37
  28. v27Reuse the two prior probes as envelope-intercept observations3.4824129 min · $25.96
  29. v28Midpoint added to the partial 8- and 12-corner ridge fits only3.4824134 min · $27.20
  30. v29Partial estimates moved from corner counts (8,12) to (7,10)3.4824143 min · $29.99
  31. v30First local sweep split into widths 4/3/3; a late 5/5 sweep merged to 103.4824152 min · $33.43
  32. v31Late width-10 repayment replaced by a twelfth projected update3.4824154 min · $34.15
  33. v32Final settle split into two 5-coordinate gradients around a scored midpoint3.4824158 min · $36.89
  34. v33Partial-fit coordinate clip 3.7 to 3.4; full-fit clip unchanged3.4824162 min · $38.46
  35. v34Intercept-radius weight in the radial blend 0.50 to 0.703.4079173 min · $42.84
  36. v35Start subspace keeps 25% radial, 75% tangential component2.9849182 min · $46.56
  37. v36Directional curvature from each projected decrease, blended 25% into the step2.9849187 min · $48.98
  38. v37One projected update on each of the two best starts, then commit to the winner3.9798203 min · $55.52
  39. v38Uniform [-4,4] posterior mean applied to the corner slope, sigma 0.553.9798212 min · $56.80
  40. v39Full-fit and start clip widened 3.7 to 3.8; partial clip stays 3.43.4824270 min · $67.55
  41. v40Projected-gradient momentum, beta 0.053.4824287 min · $72.24
  42. v41Slope-error sigma from the fit residual, shrunk hard toward 0.553.4824297 min · $75.30
  43. v4225% of the final projected velocity carried into both settling gradients3.4824304 min · $77.89
  44. v43Velocity carry decayed 0.30 then 0.20 across the two settles3.4824311 min · $80.97
  45. v44Final coordinates settled in ascending order of absolute velocity3.4824315 min · $82.47
  46. v45Bias-corrected final difference radius 0.001 to 0.0053.4824328 min · $87.10
  47. v46Both final coordinate-settle multipliers 1.0 to 1.33.4824339 min · $92.03
  48. v47Final multipliers specialised: 1.4 on the first settle, 1.2 on the second3.4824344 min · $94.16
  49. v48Second-settle multiplier blends 30% of a curvature estimate from the first3.4824350 min · $96.45

49 snapshots, 5.8 h, $96.95. v0-v7 cost $3.82 in 30 minutes and took the self-check from 116.88 to 2.99; the other 41 versions spent $93 tuning it.

On the hidden set

held-out metricreward
uniform random floor118.410.00
reference optimizer (diagnostic)81.360.28
exact oracle (global minimum)01.00
this run2.9850.8111