Nucleotide Transformer: building and evaluating robust foundation models for human genomics

Topics

Nucleotide Transformer · foundation models · Biological sequence prediction · molecular phenotype prediction · Transformer models · context-specific nucleotide representations · 3,202 human genomes · 850 diverse species genomes · fine-tuning · low-data settings · transfer learning · genetic variant prioritization · attention mechanisms · genomic elements · 2.5 billion parameters · pre-training on DNA sequences · multi-species genomics · Zero-shot learning