You inherit a protein sequence regression problem: each example is a short amino-acid sequence (an
engineered mini-protein, 43-50 residues) and you must predict its stability score, a real number
measured experimentally. The shipped methods/main/solver.py is a deliberately small linear model over
amino-acid composition. You train a regressor and save it; a sealed verifier then loads your
model and re-runs your predict() on a hidden set of sequences you never see (it does NOT re-train),
scoring the held-out Spearman rank correlation between your predictions and the true scores (HIGHER
is better) — so only a model that genuinely captures sequence-to-stability structure raises the score.
Hard Constraints
- CRITICAL (artifact-eval timeout safety): your
train()MUST persist the model toout_dir(/app/submission/model) before it can time out — write the model as soon as it is fit, and if you add iterative training, checkpoint periodically rather than only at the very end. The grader scores whatever is in/app/submission/modelat the deadline; an empty submission scores 0. - You may only edit code under
/app/methods/main/; you may add sibling.pymodules. The two entrypoints and their signatures must not change — the verifier imports them directly: train(train_data: dict, out_dir: str, device: str = "cpu") -> None— fit a regressor on the labelled sequences and save everything needed to reload it (parameters + any preprocessing config) intoout_dir.predict(model_dir: str, test_ids: list, dirs: dict, device: str = "cpu") -> dict— load the model you saved inmodel_dir, read the sequences named indirs["csv_path"], and return{test_id: predicted_score}(one float per test id).train_datais{"ids": [...], "csv_path": str}, wherecsv_pathis a CSV with columnsid,sequence,protein_length,stability_scorefor every id in"ids".dirsis{"csv_path": str}for the test rows: same columns but WITHOUTstability_score.- Do not attempt to read or reconstruct the sealed test labels; do not hardcode answers. The test
sequences are NOT in your environment — they are supplied only when the verifier re-runs your
predict(), and the true scores live solely in the verifier. - A crash or a missing test id scores the whole submission 0.
What You Have
- The workspace
/app/(/app/README.mdorients you): data/train.csv— 53,614 labelled training sequences (id,sequence,protein_length,stability_score).data/valid.csv— 2,512 labelled validation sequences, same columns. This is a pre-defined validation split (not one you need to carve yourself).methods/main/solver.py— the weak starter regressor, split intotrain()+predict(). This directory is what gets graded, together with the model you save under/app/submission/model/.DATA_CARD.md— what the sequences and labels are and the data layout.python /app/selfcheck.py— a free, unlimited local dry-run: it calls yourtrain()ondata/ train.csv, saves a model to a temp dir, runs yourpredict()ondata/valid.csv(labels withheld frompredict), and reports the validation Spearman correlation. It is a proxy only — the sealed test set is a different, held-out batch of sequences, so keep a margin and do not over-fit the split.
What You Submit
The workflow:
- Edit
/app/methods/main/solver.py, keeping thetrain/predictsignatures above. - Run it to produce the model:
python /app/methods/main/solver.pytrains ondata/train.csvand saves the model to/app/submission/model/. (Equivalently calltrain(train_data, "/app/submission/model")yourself.) - Leave the model in place. The verifier loads
/app/submission/modeland calls yourpredict("/app/submission/model", sealed_test_ids, dirs)on the sealed test sequences.
The budget is 8 CPU cores with no GPU, so think carefully about how to train and save your model efficiently.
There is no submit step and no per-attempt feedback — iterate against the self-check, then leave
your best solver.py and trained model in place; it is graded once at the end.
How It Is Judged
After your run, the verifier copies your methods/main/ and /app/submission/model/ into a sealed
sandbox, loads your model and re-runs your predict() on the hidden test sequences (it does NOT
re-train), and scores your predictions against the SEALED true stability scores:
metric = Spearman rank correlation between your predicted scores and the true scores (HIGHER is better)
gate = score must be strictly above 0.0 (the score of an uninformative, constant prediction) to place
The verifier owns the true scores and recomputes the correlation itself, so the score is honest; it does not read any number your code reports, and you get no feedback from that run.