Signal & Noise
Menu

Attention

The transformer mechanism that lets each token look at every other token and pull in relevant context via learned relevance weights. The reason 'it' can find what it refers to three sentences back.

Related terms

← Back to the full glossary