Tasks/Computer Vision/Generation & Editing

Robust Image Editing Across Nuisance Variants

Keep one semantic edit correct across nuisance variation

nuisance_equivariant_image_editing Computer Vision Generation & Editing
instruction.mdthis is what the agent is given

You inherit a weak, single-call inference policy around a frozen local image editor. Improve it so the same requested semantic edit remains correct, localized, and consistent across nuisance variants; a separate trusted verifier reruns submitted code on disjoint sealed evaluation groups.

Hard Constraints

  • Edit only /app/methods/main/solver.py. It must define build_edit_policy(editor, budget, seed) returning a policy object, and edit(policy, source_image, instruction) returning a PIL.Image with the same size as the input. The verifier imports only this file.
  • The provided editor is frozen. Do not fine-tune it, update parameters, reload model weights, download additional assets, use a network service, or call an online API.
  • All editor access must use the provided editor object. Each evaluated item permits at most 2 editor calls, 8 total inference steps, and 60 seconds of wall time. The wrapper enforces these limits and rejects an item that exceeds them.
  • Evaluation targets, masks, group metadata, and private transform specifications are unavailable while submitted code runs. The verifier executes the policy in a separate unprivileged process.
  • Any local, training-free computation available in the image is allowed if it stays within the item budget and the required API.
  • Results must be deterministic for a fixed input and seed.

What You Have

  • /app/data/visible/groups.jsonl — 40 development groups. Each group has one base item and three nuisance variants of the same source and instruction. Development records include the source, target, edit-region mask, group id, and transform specification required by the self-check. Transform kinds are resize_pad, translate, jpeg, gamma, contrast, color_temp, bg_blur, bg_noise, vignette, and compositions of these.
  • /app/data/visible/images/ — the corresponding 512×512 PNG files.
  • /app/methods/main/solver.py — the inherited weak single-call policy.
  • /app/selfcheck.py — run python /app/selfcheck.py for a full development-set evaluation using the metric below. Evaluation code receives items independently and does not expose group or transform metadata to the submitted policy.
  • The editor API is editor.edit(image, prompt, num_inference_steps=..., seed=...) -> PIL.Image.
  • One NVIDIA GPU and the frozen local analysis models used by the self-check are already installed.

What You Submit

Submit /app/methods/main/solver.py with the required build_edit_policy and edit functions. The file must be self-contained because sibling modules are not copied. During evaluation, each edit call receives only a source image and an instruction string.

How It Is Judged

For an item with target y, edit-region mask m, source x, and output o, all evaluated at 512×512:

  • semantic_gain = clip((semantic_cos(o_region, y_region) − semantic_cos(x_region, y_region)) / (1 − semantic_cos(x_region, y_region)), 0, 1)
  • preserve = clip(1 − perceptual_distance_outside_mask(o, x) / 0.35, 0, 1)
  • q = 0.60·semantic_gain + 0.40·preserve

For each group, with base output o_b, variant outputs o_k, and the recorded nuisance transforms T_k:

  • equivariance = mean_k clip(1 − perceptual_distance(o_k, T_k(o_b)) / 0.25, 0, 1)
  • gate = min(1, 4·mean_items(semantic_gain))
  • group_score = 100·(0.45·min_items(q) + 0.20·mean_items(q) + 0.35·equivariance·gate)

The task metric is the mean group score, and higher is better. Crashes, malformed outputs, non-image returns, and budget violations are rejected.

Common Pitfalls

  • Behavior that varies independently across related inputs can reduce the group consistency term.
  • Unnecessary changes outside the requested region reduce preservation quality.
  • Nuisance transforms preserve the requested semantics; outputs are judged in the coordinate and appearance frame of each input.
  • The call and inference-step limits apply to every item independently.
  • Returning a different size or a nonstandard image mode triggers conversion and usually reduces fidelity.