Tasks/LLM Systems & Training/Agent Systems & Memory

Query-Conditioned Temporal Evidence Selection

Choose which frames to look at under a fixed evidence budget

visualclawarena_system_optimization LLM Systems & Training Agent Systems & Memory
instruction.mdthis is what the agent is given

Improve a deterministic policy that selects a small set of frames from candidate visual tracklets. Each query identifies one source by its first-frame appearance and position and asks for evidence of several directed state changes. Useful selections cover both the beginning and the restoration of multiple changes while avoiding locally convincing distractors.

Hard Constraints

  • Keep both entry points in /app/methods/main/solver.py: propose_config(history=None) -> dict and select_frames(tracklets, query, config) -> list[int].
  • propose_config must return exactly frame_budget and notes, with the declared budget and a string note.
  • select_frames must return exactly the budgeted number of distinct, sorted, in-range Python integers.
  • Use only the candidate tracklets and query passed to the function. Do not hard-code case names, tracklet names, manifest order, source identities, event locations, or stored selections.
  • Submit ordinary Python source under /app/methods; generated outputs, archives, models, binaries, special files, and network dependencies are not accepted.

What You Have

  • /app/methods/main/solver.py: an editable uniform-frame starter.
  • /app/methods/main/v9_core.py: public helpers for appearance similarity, candidate extraction, temporal tracking, and event selection.
  • /app/public/scenarios/: anonymous visible sequences represented as candidate-tracklet observations. Observations contain crop-derived appearance embeddings, position, intensity, contrast, and detector confidence.
  • /app/public/truth/: visible evaluation evidence used by the self-check only.
  • /app/selfcheck.py: the exact visible metric and validation loop.

What You Submit

Edit the starter and, if useful, add small .py modules below /app/methods/main. Your final artifact is the source tree. Run python /app/selfcheck.py after meaningful changes and leave the best general policy in place.

How It Is Judged

The evaluator reruns your source on a separate anonymous collection. For every requested state change, selected frames receive credit only when they provide both onset and restoration evidence for the queried source. Weighted complete-event coverage is normalized within each query, averaged within anonymous strata, and then averaged equally across strata. Higher is better.

The evaluator supplies the same observation schema but different scenes, candidate ambiguities, and event timings. It runs without network access and rejects malformed configurations, selections, artifacts, or executions.

Common Pitfalls

  • Ranking large intensity changes independently and spending the budget on unmatched boundaries.
  • Switching to a same-appearance clone at a crossing or after a short observation gap.
  • Treating confidence as identity rather than combining appearance, anchor, motion, and temporal consistency.
  • Returning NumPy integer scalars, duplicate frames, an unsorted list, or the wrong list length.
  • Tuning to anonymous IDs instead of implementing one policy for every candidate collection.