Predict gene expression from scATAC-seq peaks (PBMC)

Single-cell ATAC-seq measures chromatin accessibility as a sparse, high-dimensional set of binary peaks per cell. You inherit the classical Gene Activity baseline — it merely sums accessibility near each gene's transcription start site, capturing only local proximity — and your job is to predict, for each cell, the discretized expression (bins 0–9) of each target gene from that cell's binary peak set. Improve the method in place; it is re-run by a sealed verifier on a hidden test split and scored by the mean per-gene rank correlation, so only a method that generalizes the global peak-to-gene program counts.

Hard Constraints

• Submit an algorithm (solve), not precomputed numbers — the verifier re-runs your code on hidden cells.
• Keep the exact signature solve(atac_train, expr_train, atac_test, peak_names, gene_names) -> np.ndarray (n_test_cells, n_genes).
• Input peaks are binary (accessible / not) — do not assume read counts. Predict for the fixed target gene list (gene_names); predictions may be continuous floats (the metric is rank-based).
• Output must be finite and shaped (n_test_cells, n_genes), or the submission scores 0.
• Train only on the provided fine-tuning split; the test cells are sealed in the verifier and there is no network at grade time. Do not hardcode or look up answers.

What You Have

• The workspace /app/:
• Visible data (/app/data/): the fine-tuning split — paired ATAC peaks (pbmc_atlas_atac.h5ad, binary; var_names="chr-start-end") + binned RNA (pbmc_atlas_rna_binning_10.h5ad, bins 0–9, cells paired with ATAC) + the target gene list (pbmc_atlas_variable_genes.csv). Ground truth is included here for local debugging only.
• The editable baseline /app/methods/main/ — this directory is what gets graded: the weak Gene Activity baseline, which sums peak accessibility near each gene's TSS. It is the only method provided. Improve main in place or rewrite it entirely.
• Your self-check surface (free): python /app/selfcheck.py cuts the visible split into an internal train/val and reports SRCC / PRCC / MSE on the val cells. This is a proxy — the sealed test cells differ, so a good self-check is necessary, not sufficient.

What You Submit

Edit /app/methods/main/solver.py to expose:

def solve(atac_train, expr_train, atac_test, peak_names, gene_names):
    # atac_train : scipy.sparse CSR (n_train_cells, n_peaks), binary {0,1}
    # expr_train : np.ndarray int   (n_train_cells, n_genes), binned 0..9
    # atac_test  : scipy.sparse CSR (n_test_cells,  n_peaks), binary {0,1}
    # peak_names : list[str] length n_peaks, "chr-start-end"
    # gene_names : list[str] length n_genes
    # returns    : np.ndarray float (n_test_cells, n_genes) predicted expression

You may add helper modules next to solver.py. The runtime provides numpy / scipy / scikit-learn / torch (and anndata for the self-check). There is no submit step and no per-attempt feedback — self-check for as long as your run window allows, then leave your best solver.py in place; it is graded once at the end on the hidden cells.

How It Is Judged

After your run, the verifier fits your solve on the hidden fine-tuning split and predicts on the hidden test split (your code runs in an isolated subprocess that receives only feature arrays, so it cannot read the answer). For each target gene it computes the Spearman correlation between predicted and true expression across test cells:

metric = mean over genes of  Spearman(pred[:, g], true[:, g])     (genes with zero true variance excluded)

The metric is mean per-gene SRCC (higher is better). Per-gene rank correlation is invariant to per-cell constant shifts, so emitting a constant or copying the input cannot score.