You inherit a weak inference policy around a frozen local image editor. Improve the reusable policy so each requested localized edit is correct and consistent across semantically equivalent instructions; the submitted code is re-run on disjoint sealed scenes for scoring.
Hard Constraints
- Put the final method in
/app/methods/main/solver.py. It must define:
```python def build_edit_policy(editor, budget, seed): ...
def edit(policy, source_image, instruction): ... ```
editmust return onePIL.Image.Imagewith the same size assource_image. Evaluation converts it to RGB.- Each item is evaluated independently with a fresh policy. The item budget permits one editor call and at most eight total refinement steps.
- Access the frozen editor only through the supplied
editorobject. Its call iseditor.edit(image, prompt, num_inference_steps=..., seed=...). - The policy receives only one source image and one instruction. It must not use item or group identities, sibling instructions, masks, targets, filenames, manifest order, verifier state, process state, credentials, or files outside
/app/methods. - Implement one general method. Do not encode visible answers, exact visible prompt strings, image fingerprints, case counts, or source-specific lookup tables.
- Do not access another container directly, the network, Docker, or the verifier. The submitted method must be deterministic for identical input and seed.
- Invalid images, exceptions, timeouts, resource excess, or budget violations fail the submission.
Run the complete visible evaluation with:
python /app/selfcheck.py
What You Have
/app/methods/main/solver.py: a weak policy that recognizes only a narrow direct instruction form./app/data/visible/groups.jsonl: development groups. Every group contains five semantically equivalent instructions for one source and requested edit./app/data/visible/images/: visible source images, exact targets, and edit-region masks used by the development evaluator./app/selfcheck.py: the complete visible evaluator./app/experiment_log.md: a place to record complete evaluations, hypotheses, and keep/revert decisions.
The visible scenes contain four textured colored shapes. Requests cover localized recoloring, brightening, desaturation, and diagonal striping. The editor accepts a prompt and bounded refinement effort; you may perform deterministic, training-free computation within the item budget.
What You Submit
Harbor collects /app/methods and /app/experiment_log.md. Keep solver.py at the required entry point. Helper Python, JSON, NumPy, or small model files may live beside it under /app/methods/main.
The staged method bundle may contain at most 64 regular files and 64 MiB in total. Every file must be at most 32 MiB and use one of the documented helper formats. Symlinks, hardlinks, device files, writable dependency caches, and runtime installation are rejected.
Keep a concise experiment record. For each complete visible evaluation, record the raw overall and family scores, the hypothesis tested, and whether it was retained or reverted.
How It Is Judged
For each item, the evaluator measures normalized target completion inside the edit mask and preservation outside it. Item quality weights completion more heavily than preservation.
For each five-instruction group, the raw group score combines the worst item quality, mean item quality, and pairwise output consistency. The consistency contribution is gated by actual target completion, so returning the unchanged source cannot earn it. The primary metric is the arithmetic mean of group scores; higher is better.
The separate verifier uses newly generated scenes and independently held instructions from the same four edit families. Submitted code receives each source and instruction separately, without targets, masks, group metadata, or sibling prompts.
Common Pitfalls
- Optimizing average behavior while one equivalent instruction remains unhandled; the worst item receives the largest group weight.
- Changing pixels outside the requested object, which reduces preservation.
- Returning the unchanged source for unfamiliar wording; consistency is gated by target completion.
- Treating visible prompt strings, order, or raster fingerprints as identities; sealed scenes and wording are disjoint.
- Spending more editor calls or refinement steps than an item permits.
- Reporting a quick smoke experiment as if it were a complete visible evaluation.