Tasks/Earth & Energy/Weather & Climate

Dual-Doppler 3D Wind Retrieval

Recover 3D wind where the vertical component is underdetermined

dual_doppler_wind_retrieval Earth & Energy Weather & Climate
instruction.mdthis is what the agent is given

Design a retrieval that reconstructs the full 3D wind field (u, v, w) from two ground radars that each measure only the line-of-sight (radial) wind, starting from a deliberately weak point-by-point solver. Minimize wind-vector RMSE against the true field; your submitted method is re-run on sealed storms for scoring.

Hard Constraints

  • Modify only /app/methods/main/solver.py as the final submission.
  • Keep the public function retrieve_wind(case) and return one finite numpy array with shape (3, 8, 16, 16) holding (u, v, w) in m/s on the grid.
  • Use only information in case; do not read the sealed truth inside the solver.
  • A solver call must finish within 60 seconds on two CPU cores and use no network.

What You Have

  • /app/problem.py: the grid geometry, radar line-of-sight operator (beam_dirs, project), and case schema.
  • /app/data/visible_cases.npz: six storms, each with two radars' noisy radial-velocity volumes (with coverage gaps), the radar positions, and the full true wind field for self-checking.
  • /app/methods/main/solver.py: a weak baseline that solves each cell independently for (u, v) and leaves w = 0.
  • /app/selfcheck.py: runs the solver on all visible cases and reports wind-vector RMSE.

Two radials per cell leave the wind underdetermined: the vertical velocity w barely projects onto low-elevation beams and must be recovered through mass continuity, and horizontal wind degrades wherever the two beams are near-parallel or a cell is unobserved. Additionally, a fraction of the observed radial-velocity cells carry gross errors (e.g. velocity-dealiasing failures or ground clutter) — which cells are corrupted is not disclosed.

What You Submit

Submit /app/methods/main/solver.py. It should turn the two radial-velocity volumes and geometry into a full (u, v, w) field. You are expected to design the retrieval formalization itself — how to constrain the underdetermined vertical wind, what physical priors and regularization (mass continuity, boundary conditions, smoothness) to impose, and how to solve the resulting system — not merely tune a few given weights.

How It Is Judged

The verifier imports your solver in a restricted child process, gives it new anonymized storms without the truth, and computes wind-vector RMSE over the full sealed field. Lower RMSE is better; the score increases monotonically after clearing the disclosed weak-baseline gate of 12.4, and beating the reference retrieval is the goal.

Metric

mean wind-vector RMSE over the sealed storms · lower is better

RMSE of the retrieved (3,8,16,16) (u,v,w) field vs truth, m/s; 8 sealed storms scored, rewards averaged.

anchorheld-outreward
Bweak baseline (2x2 point solve, w=0)12.3750.00
Rreference retrieval (variational L2)0.6770.30
SSOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
normalisation
m >= B0
B > m >= R0.3 * (u(m) - u(B)) / (u(R) - u(B))
R > m >= S0.3 + 0.3 * (u(m) - u(R)) / (u(S) - u(R))
m < S0.6 + 0.4 * (1 - exp(-(u(m) - u(S)) / 0.772))

m = this run's held-out metric  ·  B = weak baseline (2x2 point solve, w=0)  ·  R = reference retrieval (variational L2)  ·  S = SOTA (physics-basis robust)

u(x) = -log10(x); B=12.375, R=0.677, S=0.0953. No hard bound: past S reward soft-caps below 1. Mapped per storm, then averaged.

Rollouts

151 minwall clock
$17.55spend
24.5Mtokens
12versions, 11 kept
0 4 8 12 16 $0 $4 $8 $12 $16 cumulative spend on the run mean RMSE on visible storms 0-5 (m/s), lower is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11
keptrevertedno scoreturning point
  1. v0Inherited baseline: point-by-point 2x2 horizontal solve with w = 016.27650 min · $0.16
  2. v16144-variable sparse variational retrieval, mass continuity, Huber IRLSaborted on runtime11 min · $0.76
  3. v2Disclosed-family basis: 3 streamfunction vertical modes, potential, continuity wReplace the free 3-D grid state with the storm's own vertical modes, so continuity and both boundary conditions hold.2.125318 min · $1.12
  4. v3Regularization sweep: first-difference 0.003, curvature 0; v2 smoothing was bias0.246220 min · $1.52
  5. v4Retuned on fresh generator seeds: ridge 0.001, Tukey cutoff 1.5, six IRLS steps0.242626 min · $2.06
  6. v5Curvature penalty 0.01 replaces first-difference smoothing0.181937 min · $2.70
  7. v6Grid-point potentials -> 11x11 truncated cosine basis, dense Cholesky IRLSTruncate the potentials spectrally, so the noise-amplifying high modes cannot be fitted at all.0.147969 min · $5.15
  8. v7Fit the disclosed shape family: 3 Gaussian vortices + 2 sources, OMP initStop fitting a generic field and estimate the few shape parameters the generator itself used.0.0281104 min · $9.30
  9. v816-start variable-projection search for rotational shapes in observation space0.02667123 min · $12.04
  10. v9Rotational and divergent amplitudes rescaled to the disclosed 4 and 1.4 m/s RMS0.02566132 min · $13.26
  11. v10RMS-constrained shape update, then three EM/IRLS steps under the 92/8 mixture0.02392140 min · $14.76
  12. v11Fourth EM update, cached XY mesh, research hooks removed; contract checks pass0.02391148 min · $16.63

12 snapshots, 151 min, $17.55; only v1 lost. Tuned on fresh 8/16/32-storm panels from v4 on. Oracle clean-data probe at v11 gained 0.0005.

On the hidden set

held-out metricreward
weak baseline (2x2 point solve, w=0)12.3750.00
reference retrieval (variational L2)0.6770.30
SOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
this run0.026420.806
157 minwall clock
-spend
-tokens
7versions, 7 kept
0 4 8 12 16 0 25 50 75 100 agent step (this harness reports no tokens or timestamps) self-check mean wind-vector RMSE (m/s), lower is better v0 v1 v2 v3 v4 v5 v6
keptrevertedno scoreturning point
  1. v0Inherited baseline: point-by-point 2x2 horizontal solve with w = 016.277
  2. v1Helmholtz basis: mean wind + 3 psi vertical modes + chi mode, w from continuityFit a handful of mode coefficients instead of a 3-D grid, with w diagnosed by the generator's own continuity operator.0.2432
  3. v2Looser median prefilter, 5 Cauchy IRLS passes, hard reject at 2.8, priors retuned0.1965
  4. v3Weaker regularizer on the divergent potential chi than on the streamfunction psi0.195
  5. v4SIG_OBS 0.5 scales every prior up; solver cleaned, w forced to 0 at the lid0.1816
  6. v5Squared-exponential GP prior (L = 10 km) replaces the discrete smoothness and ridgeRegularize each coefficient field by a covariance at the vortices' own length scale, not by finite-difference norms.0.1076
  7. v6GP length 10.5 km, psi weight 5.0 -> 6.0, chi weight 2.5 -> 2.80.1023

7 snapshots, 157 min; this harness reports no token or cost data. All kept; side probes (SVD rows, extra modes, grid 3DVAR) never snapshotted.

On the hidden set

held-out metricreward
weak baseline (2x2 point solve, w=0)12.3750.00
reference retrieval (variational L2)0.6770.30
SOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
this run0.102510.5891
187 minwall clock
$19.86spend
23.9Mtokens
11versions, 11 kept
0 0.5 1.0 1.5 2.0 2.5 $0 $5 $10 $15 $20 cumulative spend on the run tune-set mean RMSE on fresh seeds (m/s), lower is better v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11
keptrevertedno scoreturning point
  1. v1Global variational retrieval: robust IRLS radial fit, continuity + curvature priorsPose one inverse problem over the whole volume instead of a 2x2 solve per gate, solved matrix-free by CG.2.4943 min · $0.68
  2. v2Cosine basis for (u,v); w = -int div dz with an O'Brien top correctionPriors outbid the data ~1000x, and continuity from true (u,v) reproduces true w, so w stopped being a free field.0.246534 min · $3.24
  3. v3Complete cosine basis plus a whitened GP prior at the measured spectrum0.188745 min · $4.17
  4. v4Vertical basis {1, z, cos(pi z/H), sin(pi z/H)}; Gaussian horizontal spectrum prior0.155365 min · $6.49
  5. v5Two-component Gaussian mixture fitted by EM replaces the fixed Tukey cutoff0.151372 min · $7.39
  6. v6Data weights scaled by the EM inlier variance, so prior/data balance self-calibrates0.151974 min · $7.85
  7. v7Tighter damping of vertical modes outside the physical 4-D subspace (0.02 -> 0.002)0.142476 min · $8.06
  8. v8Soft non-divergence penalty outside the one vertical mode carrying the divergenceTruth's divergence sits in one vertical mode while vorticity spreads over three; forced the rest non-divergent.0.118286 min · $9.71
  9. v9Prior amplitude tracks the storm's own wind speed (exponent 0.5)0.118130 min · $13.92
  10. v10Divergent vertical mode measured per storm from an unconstrained first IRLS pass0.1186163 min · $16.82
  11. v11Cleanup only: dead Tukey branch and constants removed, numerics identical0.1192187 min · $19.67

All 11 snapshots kept; failed ideas were reverted before snapshotting. Tune set grew 12 -> 48 storms. Last three versions cost $10, moved 1%.

On the hidden set

held-out metricreward
weak baseline (2x2 point solve, w=0)12.3750.00
reference retrieval (variational L2)0.6770.30
SOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
this run0.116980.5689
35 minwall clock
$6.16spend
6.6Mtokens
33versions, 15 kept
0 4 8 12 16 20 $0 $1.5 $3 $4.5 cumulative spend on the run mean RMSE on visible storms 0-4 (m/s), lower is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31 v32
keptrevertedno scoreturning point
  1. v0Inherited baseline: point-by-point 2x2 horizontal solve with w = 018.28991 min · $0.17
  2. v16144-variable robust sparse retrieval, continuity, w = 0 at ground and topover the 60 s/call limit7 min · $0.61
  3. v2398-coefficient DCT velocity-potential model, exact continuity, Cauchy IRLSSolve for rotational and divergent potentials, so continuity and both vertical boundary conditions hold by construction.0.2939448 min · $0.74
  4. v3Horizontal potential basis 10x10 -> 12x12 (574 coefficients)0.2131218 min · $0.80
  5. v4Horizontal potential basis 12x12 -> 14x14 (782 coefficients)0.1688338 min · $0.87
  6. v5Complete 16x16 horizontal DCT basis (1022 coefficients)0.1680199 min · $0.93
  7. v6Global frequency-ridge multiplier 0.35 -> 1.00.170459 min · $0.99
  8. v7Global frequency-ridge multiplier 0.35 -> 0.100.19889810 min · $1.05
  9. v8Cauchy IRLS scale 0.75 -> 0.50 m/s0.17057510 min · $1.13
  10. v9Cauchy IRLS scale 0.75 -> 1.00 m/s0.17288511 min · $1.20
  11. v10Posterior inlier weights from the disclosed 92/8 noise mixture replace CauchySwapped a hand-picked robust loss for the exact posterior that a gate is clean under the disclosed noise model.0.16326111 min · $1.27
  12. v11Mixture IRLS ridge multiplier 0.35 -> 0.500.15864312 min · $1.35
  13. v12Ridge multiplier 0.50 -> 0.700.15692912 min · $1.43
  14. v13Diagonal empirical-Bayes precision from 2000 draws replaces the scalar ridge0.18096214 min · $1.69
  15. v14Mixture IRLS extended from 4 to 6 solves0.15767315 min · $1.79
  16. v15Mixture IRLS extended from 6 to 8 solves0.15733615 min · $1.89
  17. v16High-frequency penalty coefficient 0.015 -> 0.0250.15523216 min · $2.04
  18. v17High-frequency penalty coefficient 0.015 -> 0.0100.16329917 min · $2.16
  19. v18Post-project the linear solution onto the disclosed 3+2 Gaussian-blob familyworse than linear fit20 min · $2.53
  20. v19Ridge weakened only on the two uniform background-wind columns0.15718920 min · $2.67
  21. v20Rotational and divergent parts rescaled to the disclosed RMS amplitudes0.17831522 min · $2.85
  22. v21Final w rescaled to the disclosed 1.4 m/s RMS, horizontal fit untouched0.1571522 min · $2.99
  23. v22Ridge strengthened 1.3x on the rotational modes onlyFirst version to shrink rotational and divergent modes by different amounts; opened the ridge-factor line.0.15350523 min · $3.44
  24. v23Rotational ridge factor 1.3 -> 1.80.14943324 min · $3.57
  25. v24Rotational ridge factor 1.8 -> 2.50.14604424 min · $3.70
  26. v25Rotational ridge factor 2.5 -> 4.00.14329525 min · $3.92
  27. v26Rotational ridge factor 4.0 -> 6.00.14390525 min · $4.06
  28. v27Divergent-mode ridge factor 1.0 -> 0.7, rotational held at 2.50.14496927 min · $4.35
  29. v28Divergent-mode ridge factor 0.7 -> 0.40.14561128 min · $4.50
  30. v29Divergent-mode ridge factor set to 0.550.14500828 min · $4.65
  31. v30Rotational ridge split per vertical mode: (const, sin, cos) = (2.0, 2.5, 4.0)0.14235630 min · $5.02
  32. v31Horizontal basis cut from complete 16x16 back to 14x14 under the tuned priors0.15010631 min · $5.26
  33. v32Mixture posterior weights conditioned on the disclosed 8% corrupt-gate count0.14460932 min · $5.53

33 snapshots in 33 min for $6.16, one knob each, 18 reverted. v12, v25, v29 and v30 won on tune but lost on 30 fresh storms.

On the hidden set

held-out metricreward
weak baseline (2x2 point solve, w=0)12.3750.00
reference retrieval (variational L2)0.6770.30
SOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
this run0.146920.5345
12 minwall clock
$0.93spend
4.1Mtokens
5versions, 5 kept
0 4 8 12 16 $0 $0.2 $0.4 $0.6 $0.8 cumulative spend on the run self-check mean wind-vector RMSE (m/s), lower is better v0 v1 v2 v3 v4
keptrevertedno scoreturning point
  1. v0Inherited baseline: point-by-point 2x2 horizontal solve with w = 016.27652 min · $0.13
  2. v1Global 3D-Var over (u,v,w): pseudo-Huber data term, continuity and smoothness priorsOptimize the whole volume at once under hand-derived analytic gradients, instead of a 2x2 solve per gate.3.3692 min · $0.17
  3. v2Pure L2 data term; swept lambda_mc = 0.5, lambda_smooth = 1.6, maxiter 1501.79457 min · $0.39
  4. v3Smoothness split into horizontal 5.0 and vertical 0.2, lambda_mc = 1.0, maxiter 2001.399810 min · $0.57
  5. v4Hard continuity: w integrated from divergence; lambda_sh 4.0, lambda_sv 0.2Drop w as a free field and diagnose it from divergence, the same operator the storms were built with.1.249412 min · $0.81

5 snapshots in 12 min for $0.93, all kept. Six offline weight sweeps drove every gain; the run stopped at v4 with 90% of its tokens unused.

On the hidden set

held-out metricreward
weak baseline (2x2 point solve, w=0)12.3750.00
reference retrieval (variational L2)0.6770.30
SOTA (physics-basis robust)0.09530.60
clean-observation oracle (measured)0.00810.90
this run1.307210.2326