Sustainability & AI
Transformer model
The neural-network architecture behind modern language and multimodal AI, built on attention mechanisms.
Definition
A neural-network architecture built on attention mechanisms that weigh the relevance of different parts of the input, enabling effective learning from large sequential datasets.
Quick reference
At a glance
- Subject
- Sustainability & AI
- Editorial status
- Editorial draft
- Definition status
- Established
- Last updated
- 21 August 2026
References
This source provides part of the technical or institutional basis for the definition.
This source supports the explanation of how the term is applied, measured or governed in practice.
This source supports the wider sustainability significance and context described in the entry.
Overview
What it means
Introduced in the 2017 paper 'Attention Is All You Need', the transformer displaced recurrent architectures and became the basis of modern language models, and increasingly of vision and scientific models. Its scalability drives both capability growth and training energy demand.
How it is used
Transformers power the generative AI tools now used in sustainability analysis and reporting, and also scientific models such as weather forecasters — ECMWF's AIFS and Google's GraphCast both build on transformer-style designs.
Why it matters
One architecture now underlies most frontier AI, from chatbots to climate models. Understanding its attention mechanism is less important than grasping the consequence: capability concentrated in very large models trained by very few organisations.
Review
Help keep this definition useful and accurate.
Submitted reviews stay private until an editor decides whether to accept and attribute them.
Endorse this definition
Confirm what works
Endorse the current wording when it is accurate and useful. Suggested changes use the separate editorial form below.