Transformer architecture research papers. Locates seminal and current papers detailing transformer variants, theory, and benchmarks. Supports academic work.
Transformer Architecture Research Papers
Curated academic reading list of 19 sources covering foundational architectures, efficient and long-context variants, theoretical analyses, and empirical benchmarks. Duplicate URLs and lower-relevance application papers were excluded.
Seminal architectures
Foundational Transformer papers and surveys that organize the major architectural families, including the original Transformer and vision Transformers.
Attention Is All You Need
Architectural variants and efficiency
Papers on sparse, efficient, long-context, and alternative sequence architectures, including canonical sparse-attention work and newer surveys or preprints.
Theory and mechanisms
Theoretical and mechanistic studies of Transformer expressivity, induction heads, linear attention, and length generalization.
Benchmarks and evaluation
Benchmark papers assessing attention efficiency, long-context modeling, and comparisons among Transformer and alternative sequence architectures.
Create your account to keep this answer and continue from it later.
Let's look at alternatives:
- Modify the query.
- Start a new thread.
- Remove sources (if manually added).
