You are given a collection of real, structure-derived protein surface meshes, each with per-vertex physicochemical properties, and a deliberately weak starting descriptor. Your job is to design a shape-descriptor method that, given one query protein surface, lets a fixed distance-based ranking retrieve the other surfaces belonging to the same real structural class; your method is re-run unchanged on a sealed held-out set of query surfaces you never see, ranked against the same visible collection, to compute your score.
Hard Constraints
- Submit a method:
methods/main/solver.pydefining exactlydescribe(shape) -> 1-D numpy array(a fixed-length vector; the length may be anything but must be the same on every call within a run). The judge re-runs THIS file, unmodified, on the sealed held-out queries. shapeis{"vertices": (P, 3) float64, "properties": (P, 3) float64}-- a real point sample of one protein surface (Preal mesh vertices and their real physicochemical properties, same row order). Seeenvironment/shape_io.py.- The judge, not you, ranks collection items for a query by ascending
Euclidean (L2) distance between descriptor vectors --
describeis the only thing you write. - Each
describecall runs in its own process under a 10-second wall-clock budget (measured outside your process) for a query call, and the whole-collection batch call has a generous shared budget; crashed, over-budget, or inconsistent-length outputs score as the worst case for that call. - CPU only, no network; runtime provides Python 3.11 + numpy.
- Only
methods/main/solver.pyis graded; do not rely on modifyingshape_io.py,retrieval.py,selfcheck.py, ordata/.
What You Have
data/collection/*.npz-- the visible collection (real protein surfaces) every query is ranked against, plusdata/collection_labels.json(each collection item's real structural class). This is the intended primary source of labeled supervision if your method wants any (e.g. fitting a projection) -- 1059 real items across all 65 real structural classes of the evaluation pool. Class sizes are not uniform (this mirrors the real, imbalanced source distribution): from 1 to 109 items per class, averaging about 16.data/visible_queries/*.npz-- a visible slice of real query surfaces, drawn from the same class distribution as the judged queries, split into two label files:data/visible_query_labels_train.json(66 ids) -- labels you may freely use as extra fitting signal. Because classes are imbalanced, not every class has a train example (some small classes contribute 0).data/visible_query_labels_dev.json(112 ids) -- labels used only byselfcheck.pyto report an out-of-sample proxy score (again, not every class necessarily has a dev example). Do not fit anything (train a classifier, tune a projection, hard-code per-id logic, etc.) against these ids or their labels -- doing so makesselfcheck.py's number meaningless for predicting your judged score, the same way it would in any real train/validation split.methods/main/solver.py-- a deliberately naive starting descriptor. Replace it freely.python3 selfcheck.py-- free and unlimited: computes mean Average Precision (mAP, higher is better) of your currentdescribeon the visible dev queries against the collection. Because the dev ids are never used above, this is a genuine out-of-sample estimate of how your method generalizes -- unlike training and evaluating on the exact same queries, which would only measure memorization. This mirrors the only feedback you get in practice; the judged queries are held back.
What You Submit
Leave your best methods/main/solver.py in place. There is no submit step
and no per-attempt feedback on the judged queries: whatever sits in
methods/main/ at the end is what the judge re-runs and scores.
How It Is Judged
The judge draws queries from a sealed held-out split of the same real query
pool (same collection, same labeling protocol, never seen during
development), runs your describe once per collection item and once per
held-out query, ranks each query's retrieved list by ascending descriptor
distance, and scores mean Average Precision (mAP) against the real
structural-class ground truth -- higher is better. The judge recomputes
mAP itself from your descriptors; it does not read any number you report, and
you receive no feedback from that run.