#23 · ML / AI methods

Protein language models encode nativeness, taxonomy, and memorization beyond training

The dominant thread is diagnosing what protein language models encode beyond their training objective and benchmark accuracy. The viral proteins study shows embeddings carry a dominant nativeness axis separating cellular, viral, and shuffled sequences. TaxoFormer pairs ESM-2 with an autoregressive decoder to predict full taxonomic lineages, and the multi-species PPI study shows classifiers can exploit taxonomic shared origin as a shortcut. Pseudoperplexity tests whether ProtT5 memorizes training proteins relative to matched novel holdouts. The DNA compression study addresses a different question, evaluating genomic language models as arithmetic-coding compressors, so it is adjacent rather than protein-centric. What is new is treating learned representations as objects of diagnosis rather than relying on accuracy alone.