#7 · Genomics

Genomic language models are being interrogated for regulatory and biological structure

The common thread is the use of learned representations from DNA sequence and single-cell data to infer regulatory function, with interpretability and transfer as the main axes. Two preprints apply sparse autoencoders to Geneformer V2-316M and scGPT. One reports inhibitory dominance, biological coherence, and cross-model conserved features under causal circuit tracing; the other, with feature atlases of 82525 and 24527 features, finds organized biological knowledge but minimal regulatory logic. Genomic language models extend to cross-species expression prediction across 17 plant species (PlantCaduceus), circRNA identification from few authenticated examples (circFormer, Nucleotide Transformer), and missense variant prediction combining AlphaFold structure voxels with MSA embeddings (EvoStructCLIP). Published work includes Evo 2 trained on 9 trillion base pairs, ConvNeXt V2 early feature extractors for 4-bp chromatin accessibility prediction, and deep learning models of low-affinity Exd-Hox TF-DNA binding. The most distinctive contribution in this set is mechanistic probing of single-cell foundation models, which the other items do not address directly.