Tasks/Life Sciences/Protein Structure & Function

Protein-Ligand Co-folding

Co-fold protein and ligand, inference only

protein_ligand_cofolding_posebusters Life Sciences Protein Structure & Function
instruction.mdthis is what the agent is given

You inherit a deliberately weak inference pipeline that predicts a protein-ligand complex from protein sequence, ligand SMILES, and a precomputed MSA. Improve its inference-time success rate under the pinned physical-validity checks and a 2 Å ligand-RMSD threshold; your submitted method is re-run on a sealed post-training-cutoff held-out split for scoring.

Hard Constraints

  • Work only in /app/methods/main/. The graded artifact is that source directory.
  • Keep predict_complex(item: dict) -> dict as the entry point. It must predict each case at grading time and may not return precomputed coordinates.
  • Use only the protein sequences, ligand SMILES, and MSAs in the three-field predict_complex input. Visible identities and crystal truth remain in the development workspace so you can analyze self-check failures, but submitted prediction code must not read or branch on them. The sealed grader stages an anonymous input and enforces that hidden identities, crystal truth, hidden paths, and hidden results are inaccessible to the prediction process.
  • This is co-folding, not docking into a supplied receptor. Do not retrieve experimental structures or use a structure database to identify a case.
  • Inference-time changes are allowed; training or fine-tuning model parameters is not.
  • Stay within one GPU. The autonomous research run and each complete sealed evaluation each have a 12-hour compute window. The verifier fails malformed output, missing output, crashes, and timeouts closed.

What You Have

  • /app/methods/main/solver.py — the deterministic weak baseline you edit.
  • /app/methods/main/cofold_utils.py — one deterministic baseline adapter plus complex-to-PDB/SDF conversion helpers. You may edit it or add source files beside it.
  • /app/data/visible/ — a frozen 20-case development suite with sequences, ligand SMILES, precomputed MSAs, and crystal truth. It contains the accepted author's original visible10 plus 10 cases preregistered from the original held-out set using input-only distribution features; no crystal, model prediction, or score was used to choose them.
  • /app/selfcheck.py — runs your current method and the grading metric on visible data. With no flags it evaluates the full frozen visible suite, starting a fresh prediction process per case under the grading time limits; --case-index and --smoke are diagnostic subsets and are not comparable scores.
  • Pinned local inference assets and all required scientific dependencies are installed in the image. Runtime model downloads are neither needed nor allowed.

What You Submit

Leave your best source implementation in /app/methods/main/. For each call, the input contains exactly:

{
    "protein_chains": [{"chain_id": str, "sequence": str}, ...],
    "ligand_smiles": str,
    "msa_dir": str,
}

Return exactly:

{"protein_pdb": str, "ligand_sdf": str}

The PDB must contain the predicted protein and the SDF must contain one predicted ligand pose. Both structures must come from the same prediction and remain in the same coordinate frame. Helper modules added under /app/methods/main/ are included in the submission; generated poses, model weights, binaries, compressed data blobs, and files outside that directory are not.

The source-only artifact may contain at most 32 UTF-8 Python files, 128 KiB per file and 256 KiB total, with nesting depth at most four. Symlinks, hard links, hidden paths, non-Python files, literals above 16 KiB (or above 64 KiB in aggregate), programs above 20,000 AST nodes, dynamic eval/exec/compile/__import__, and direct imports of base64, binascii, bz2, gzip, lzma, marshal, pickle, or zlib are rejected. Names reserved by the harness (metric.py, source_contract.py, selfcheck.py, evaluate.py, grade.py, child_predict.py, and score_pose_worker.py) may not be added under the artifact. These bounds prevent bundling a public structure lookup table; ordinary inference logic and the installed model libraries remain available. selfcheck.py applies this same byte-identical contract before importing your method, and python /app/source_contract.py /app/methods/main runs it without GPU inference.

The predicted PDB must represent every supplied protein chain and provide sequence-matched C-alpha coordinates for at least 95% of each chain. A few unresolved terminal residues are tolerated; returning only a pocket or one domain is not a valid full-complex co-folding prediction.

How It Is Judged

  • The verifier runs the submitted method independently on the 42 complexes that remain sealed after the preregistered visible20/hidden42 repartition of the accepted author's 10+52 cases. The agent runtime contains no held-out files or repository history and reaches only the model API; the separate verifier is offline and exposes only one anonymous inference input at a time. The repartition preserves every original case exactly once and does not change the input fields, co-folding operation, parameter envelope, case weighting, or metric.
  • For each case, the verifier matches predicted and crystal protein chains by sequence, aligns sequence-matched C-alpha atoms with one rigid transform, and applies that transform to the predicted ligand. Absolute translation and rotation therefore do not affect the score.
  • A case succeeds only if the transformed ligand passes every pinned binary redocking check, including symmetry-aware heavy-atom RMSD at or below 2 Å. Missing or non-boolean checks fail closed.
  • The raw metric is the unweighted fraction of successful complexes. Higher is better. The normalized score is a monotonic function of sealed success rate and is not shown to you; optimize raw success and cross-case generalization.
  • The hidden process exposes only the aggregate result. It does not expose case identities, per-case scores, error traces, or checkpoints, and it is not an optimization oracle.

Metric

success rate over the 42 sealed complexes · higher is better

fraction of cases passing every pinned validity check and 2 A symmetry-aware ligand RMSD

anchorvisible setheld-outreward
Bshipped weak Boltz-1 pipeline0.40000 (8/20)0.40476 (17/42)0.00
Uevery sealed case correct1.00000 (42/42)1.00
normalisation
m <= B0
B < m < U(m - B) / (U - B)
m >= U1

m = this run's held-out metric  ·  B = shipped weak Boltz-1 pipeline  ·  U = every sealed case correct

B = 0.40476, U = 1. Mapped once on the aggregate rate, not per case, then clamped to [0, 1]. One more sealed case is worth 0.04.

Rollouts

1,559 minwall clock
$94.29spend
158.3Mtokens
8versions, 3 kept
0.225 0.300 0.375 0.450 0.525 $0 $20 $40 $60 $80 cumulative spend on the run visible20 success rate, higher is better v0 v1 v2 v3 v4 v5 v6 v7
keptrevertedno scoreturning point
  1. v0Inherited Boltz-1: one diffusion sample, three recycles, supplied MSA0.4$11.79
  2. v1Five Boltz-1 samples ranked by native confidence_score0.35$23.57
  3. v2Keep the one-sample pose unless the batch gains 0.002 confidenceDiversity helped and confidence ranking hurt, so take the batch only when it clears the deterministic pose by a margin.0.45$35.36
  4. v3Chai-1 alone, one sample, A3M converted to sequence-keyed parquet0.2$47.14
  5. v4Boltz-1 one sample with ten recycling steps instead of three0.35$58.93
  6. v5Add one-sample Chai when Boltz confidence >= 0.90 and ligand_iptm < 0.88Call the second model on cases the first is globally confident about yet unsure of at the ligand. Fired twice; submitted.0.5$70.71
  7. v6Five Chai samples when the triggered one-sample Chai scores below 0.500.5$82.50
  8. v7Extra five Boltz samples at step_scale 1.2 beside the two anchors0.5$94.29

No timeline in the bundle; these eight full-suite self-checks come from the in-container ledger. Three sessions, 26 h, $94.29.

On the hidden set

held-out metricreward
shipped weak Boltz-1 pipeline0.40476 (17/42)0.00
every sealed case correct1.00000 (42/42)1.00
this run0.42857 (18/42)0.04