Tasks/Computer Vision/Video Understanding

Five-pass latent-video temporal refinement

Tune a five-pass temporal latent refinement policy

video_temporal_denoising_schedule_optimization_vnextComputer VisionVideo Understanding
instruction.mdthis is what the agent is given

You inherit a deterministic policy that leaves a noisy short-sequence latent trajectory unchanged. Improve its temporal refinement schedule while preserving genuine motion and shot boundaries. A trusted evaluator applies the policy to disjoint procedural panels and compares the refined sequence with a clean reference trajectory.

Hard Constraints

  • Edit only /app/methods/main/policy.py; use /app/experiment_log.md to record experiments.
  • Keep build_policy() -> dict as the entry point.
  • Return exactly the temporal-denoise-policy-v1 mapping documented in /app/API.md.
  • The five pass logits must be finite Python floats in [-4, 4].
  • The remaining controls must be finite Python floats within the bounds in /app/API.md.
  • Do not return trajectories, arrays, callbacks, paths, source code, or extra keys.
  • Do not inspect or interfere with another container, Docker, credentials, verifier files, or process controls.
  • The verifier has no network. The submission must be deterministic and use only the Python standard library.

Run the local contract and determinism checks with:

python /app/selfcheck.py

Run a complete visible evaluation with:

python /app/score_visible.py

What You Have

  • Three public panels spanning six balanced families of short latent-video trajectories.
  • /app/API.md, which specifies policy normalization, five-pass accounting, gate behavior, and the visible metric.
  • A weak no-refinement policy in /app/methods/main/policy.py.
  • Deterministic visible feedback. Public panels are development feedback only and are disjoint from verifier panels.

What You Submit

Harbor collects policy.py and the experiment log. The verifier stages policy.py, executes build_policy once in a constrained no-network child, validates the small numeric mapping, terminates that child, and only then evaluates the frozen policy on sealed panels.

How It Is Judged

Lower balanced normalized temporal latent error is better. Each panel-family unit is scored independently and the units are averaged equally, so a schedule must transfer across static content, camera-like drift, acceleration, oscillation, transient corruption, and shot-like changes. The trusted runtime performs exactly five temporal refinement passes per trajectory.

Common Pitfalls

  • Returning integers, booleans, numeric strings, NaN, infinity, or extra keys.
  • Confusing pass logits with already-normalized pass fractions.
  • Applying too much smoothing and erasing real motion or boundaries.
  • Using gates so aggressively that transient corruption is preserved.
  • Overfitting one visible family instead of checking all panel-family units.

Metric

mean normalized temporal latent MSE over 18 sealed units · lower is better

Refined-vs-clean latent error per panel-by-family unit, scaled so the no-refinement starter is 1.0

anchorheld-outreward
Bno-refinement starter (inherited)1.0000000.00
Rreference policy (independent)0.2421160.30
theoretical zero-error bound01.00
normalisation
m >= B0
B > m >= R0.3 * (u(B) - u(m)) / (u(B) - u(R))
m < R0.3 + 0.7 * (u(R) - u(m)) / u(R)

m = this run's held-out metric  ·  B = no-refinement starter (inherited)  ·  R = reference policy (independent)

u(x) = log1p(x), upper = 0. Each of the 18 units maps against its own anchors, clamped to [0,1]; the unit rewards are averaged equally.

Rollouts

47 minwall clock
$12.13spend
14.2Mtokens
10versions, 10 kept
0.2 0.4 0.6 0.8 1.0 $0 $2.5 $5 $7.5 $10 cumulative spend on the run visible-panel mean error, lower is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9
keptrevertedno scoreturning point
  1. v0Inherited no-refinement starter; all five passes leave the trajectory alone11 min · $0.19
  2. v1Coarse 10,800-point sweep: budget 2.5, motion 1, cut 2, robust 0, pull 0Sweep the whole control grid once with uniform pass logits before shaping the gates.0.2432023 min · $0.52
  3. v2Fine gate profiling to motion 1.4, cut 1.8; robust and pull worse away from zero0.2423664 min · $0.76
  4. v3Cross-panel stratified-bootstrap gate consensus, motion 1.43, cut 1.71Pick gates by leave-one-panel-out transfer instead of the lowest all-panel score.0.242368 min · $1.19
  5. v4One-decimal regularization of the all-panel fit: motion 1.4, cut 1.70.24236810 min · $1.67
  6. v5Sign-symmetrized noise-pair risk cancels cross terms; motion 1.45, cut 1.720.24236213 min · $2.37
  7. v6Conservative motion-proxy-integrated ridge point, motion 1.44, cut 1.730.24236129 min · $6.33
  8. v7Oscillation frequencies and amplitude support recovered; motion 1.47, cut 1.71Fit gates to the generator's inferred population rather than to the 72 visible sequences.0.24236838 min · $8.67
  9. v8Shot locations and jump support recovered, degree-1 control variate; 1.47/1.680.24236640 min · $9.58
  10. v9Motion-proxy noise integrated in the population objective; motion 1.46, cut 1.700.24236344 min · $10.94

v1 cost $0.52; the eight versions after it cost $10.42 more and moved the visible mean by 0.0008.

On the hidden set

held-out metricreward
no-refinement starter (inherited)1.0000000.00
reference policy (independent)0.2421160.30
theoretical zero-error bound01.00
this run0.2421310.3005