#2 · Tools & infrastructure

Graph storage and streaming parsers reshape population-scale variant computation

The dominant thread is software infrastructure for population-scale variant data, where the storage model and computational representation determine what scales. GraphMana stores variant data in a graph database with packed genotype arrays, supporting incremental sample addition and provenance tracking. GraphPop computes population genomic summary statistics from pre-aggregated allele counts held as node properties, decoupling computation from sample count. On the file-processing side, vcfilt, a Go streaming filter using zero-allocation byte-scan parsing, reports a 12.2-fold single-thread speedup over bcftools 1.18 on 1000 Genomes chr20, and fastVEP, a Rust reimplementation of the Ensembl VEP annotation engine, annotates GIAB HG002 whole-genome data in 86 s, running 22 to 24-fold faster than VEP. The 6-bit encoding preprint is evaluated with null models and DepMap/STRING rather than engineering benchmarks and is not part of this engineering thread. What is new is the emphasis on pre-aggregated and streaming representations as the means of scaling.