Part 9 Appendix I

Attention as a Soft Lookup

Mehmet Kerem Turkcan
Associate Research Scientist
Center for Smart Streetscapes, Columbia University
New York, USA
keremturkcan.com; mkt2126@columbia.edu

Move through the deck with (i) the arrow keys, (ii) a presentation clicker, or (iii) the buttons at the bottom left. The gear at the bottom right opens the slide settings.

Part 9 Overview

  1. The mechanism: Part 9 built attention out of matrices, softmax rows, and a mask, and computed every number by hand.
  2. The same mechanism again: the matrices become a directory of words, and the mask becomes a cover over the future.
  3. The destination: the shares the model assigns to the next word, sharpened block by block.

Read each claim twice: first in plain words, then in the standard vocabulary on the smaller line beneath.

Shares over the next word

The bread rolls were on the table because they ____

The model has one job: give every word it knows a share of being next.

CandidateShare
smelled0.31
were0.24
looked0.12
every other word it knows0.33

Everything between the sentence and these shares is two moves, a lookup and a rewrite, repeated, and one final scoring.

Standard terms Next token prediction: a probability distribution over the vocabulary. Real models read tokens, which are pieces of words; the example sentence splits into whole words, so each token is a word. The probabilities in the table are illustrative.

Which word decides the blank?

The word before the blank is they, and they is only usable once the machine knows what it stands for. The word that settles it, rolls, sits six places earlier, and the word that settles what rolls means, bread, sits beside rolls.

The bread rolls were on the table because they ____
The deciding word can sit anywhere in the sentence. How can every word consult every other word, whatever the distance?

Two machines, (i) an older one that reads in order and (ii) the one inside every current assistant, answer that question.

Standard terms The dependency between they and rolls is a long range dependency, the case that separates the two designs.

How a recurrent network reads

The older machine reads the way a line of people passes a whispered message: each person hears only their neighbor, adds their own word, and whispers onward.

The bread rolls were on they the message grows fainter along the line step 6 cannot begin until step 5 has finished
Six of the nine words along the line (illustrative).

By the time they is reached, everything known about rolls has survived six retellings.

Standard terms A recurrent neural network carries a hidden state from each position to the next, so information between two positions travels a path as long as the distance between them.

Three costs of the chain

CostIn the line of whisperersThe standard account
FadingWhat bread contributed reaches they only through retellingsGradients travel the same path backward and fade the same way (Hochreiter and Schmidhuber, 1997); the path length is O(n), which grows in step with the distance
WaitingNobody can whisper until their neighbor has finishedThe 2017 paper: the "inherently sequential nature precludes parallelization within training examples"
The bottleneckEverything said so far shares one person's memoryBahdanau, Cho, and Bengio (2014) identified one fixed length vector carrying a whole sentence as a bottleneck

The first cost limits how far back the machine can usefully look, the second limits how fast it can be trained, and the third limits how much of the sentence survives the trip.

Standard terms Vanishing gradients over long paths, sequential training, and a fixed size hidden state. Attention removes all three at once.

A directory of words

The current machine reads with a directory. Every word publishes one line in it, a heading saying what it is and an entry saying what it can tell, and every word arrives with a description it wants to look up.

word heading: what I am entry: what I can tell bread a food noun baking and breakfast table a furniture noun kitchens and rooms were a past tense verb something already happened what rolls looks up "the word I am attached to"
Three directory lines beside the description one word brings (illustrative).

In a phone book you look up a name you already know. Here a word looks up a description, and every heading is a description, so the lookup is a matter of fit.

Standard terms The heading is the key, the entry is the value, and the description a word looks up is the query. The three names are the ordinary vocabulary of a lookup table: a query is brought to a table of keys, and values are taken away.

A soft lookup

A phone book lookup returns one entry. This lookup returns every entry at once, each poured in according to how well its heading fits the description.

0.7 the entry of bread 0.2 the entry of table 0.1 the entry of were what rolls takes away mostly the entry of bread
Weights of 0.7, 0.2, and 0.1, which sum to 1 (illustrative).

The weights always sum to one, so the result is an average of entries and never runs away.

Standard terms The fit between a query and a key is a dot product; a softmax maps the fits to positive weights that sum to 1; the output is the weighted sum of the value vectors. Part 9 computes one such row by hand.

One lookup for every word

bread table were rolls 0.7 0.2 0.1
Thicker arrows carry more weight (illustrative).

Every word runs its own lookup in the same way, and distance never lengthens the path: they reaches rolls in one step, six words back or sixty.

Standard terms Each position issues its own query over every position it may read, so the maximum path between any two tokens is O(1), one step whatever the distance; Table 1 of the 2017 paper sets that against O(n) for recurrence. Reaching a distant token is a single step of the mechanism.

Every word at once

Because every lookup is independent of every other, nobody waits for a neighbor.

The bread rolls they one shared directory all the lookups happen in the same step
Four words, one directory, one step (illustrative).

A room full of processors can each take some of the lookups, so a whole sentence is processed in one go. The waiting cost of the chain is gone, one of the changes that let the models grow.

Standard terms Attention over a sequence is two matrix products with a softmax between them, and the whole computation parallelizes across positions, removing the sequential training bottleneck.

The mask over the future

Training covers a word and scores the model on producing it. One sentence of ten words carries nine covered blanks, and the parallel step scores all nine in a single pass.

That is only honest under one rule: a word may consult only itself and the words before it, because the word one place ahead is the answer to the blank this word must set up.

Standard terms The causal mask sets every forbidden fit to -\infty before the softmax, so the forbidden share comes out exactly zero. Part 9 shows the matrix.

At writing time the rule costs nothing: when the model generates, the future words do not exist yet, so the cover matches reality exactly.
the word consulted the word looking The bread rolls were on The bread rolls were on not yet written may be consulted
Open cells may be consulted; each word may always consult itself (illustrative for five words).

Several lookups per word

One description finds one kind of thing, and a word usually needs several kinds at once, so each word runs several lookups side by side.

What they looks upWhere the weight lands
the thing I stand forrolls
one thing or many?rolls, were
the topic of the sentencebread, table

Each lookup brings back its own blend, and the blends are laid side by side before the word moves on.

Standard terms Multi head attention: each head has its own W_Q, W_K, and W_V, and the head outputs are concatenated and passed through an output projection.

Nobody writes the descriptions: each word derives its descriptions, headings, and entries from its own numbers, through learned weights shared across positions, and what a head ends up looking for is decided by training.

The rewrite after the lookup

After the lookup, each word works alone. It takes its own numbers and everything it just gathered, and rewrites its numbers, consulting nobody.

This private step is the classic hidden layer, which reads numbers in, passes weighted sums through an activation function, and writes numbers out.

Standard terms A position wise feedforward network, the hidden layer of Part 3, applied at every position with the same weights. Most of the model's parameters live here.

Why is a private step needed when the lookup already gathered everything?

The lookup decides what a word reads. The rewrite decides what the word makes of what it read.

A fresh directory at every block

A block is the lookup followed by the rewrite. The model stacks dozens of blocks, and here is the move the whole design turns on: because every word rewrote its numbers, the next block prints a fresh directory, with new headings, new entries, and new descriptions to look up, all derived from the rewritten numbers.

So the second block asks better questions of a better informed directory, and the third improves on the second.

Standard terms Every block computes its queries, keys, and values from the previous block's output. Real blocks also carry residual connections and layer normalization, which keep training stable and leave the lookup and the rewrite unchanged.

The one sentence version a teacher can carry into class: the whole model is a directory, consulted and reprinted dozens of times.

Block 1: bread and rolls

Watch one line of the directory across one block. In block 1, rolls looks up "the word I am attached to", and the heading of bread fits best.

rolls, as printed for block 1 heading: a noun, or perhaps a verb entry: rolling, or bread rolls, as reprinted for block 2 heading: a plural food noun entry: bread rolls, food on a table lookup, then rewrite
One directory line before and after one block (illustrative).

The ambiguity of rolls is resolved by its nearest neighbor, and the resolved reading is what the word now advertises.

Standard terms After block 1 the vector of rolls encodes the compound bread rolls, and block 2 computes its query, key, and value from that vector.

Block 2: they and the rolls

In block 2, they looks up "the plural thing I stand for", and the fit it finds exists only because block 1 reprinted the directory.

the reprinted heading of rolls a plural food noun what they looks up the plural thing I stand for they, as reprinted for block 3 stands for the rolls; plural; food
A lookup succeeding against a reprinted heading (illustrative).

On the directory as first printed, the heading of rolls said a noun, or perhaps a verb, and such a heading fits the description of they no better than the heading of table does. Depth is lookups building on lookups.

Standard terms Layers compose. Tenney, Das, and Pavlick (2019) locate word level grammar in earlier layers and coreference in later layers of a deep language model, and block 1 resolved grammar while block 2 resolved reference, in the same order.

The evolution of the last word

The word that must set up the blank is the last one written, they. Its numbers are rewritten at every block, and each rewrite carries more of the sentence.

Read atWhat the numbers of they say
block 1a pronoun, just after because
midwaystands for the rolls; the subject of a coming verb
top blocka past tense verb about food on a table is due next

By the top block, the numbers of they have stopped describing a pronoun and started describing the word that should come next.

Standard terms During generation only the last position's final vector reaches the output layer; during training every position is scored, one prediction for each of the nine targets the causal mask keeps hidden. The table lines are illustrative.

Sharper shares at every block

Hand the numbers of they to the scorer at different heights of the stack, and the shares over the next word sharpen.

after block 1 .09 .08 .07 smelled were looked midway up the stack .18 .15 .09 smelled were looked after the top block .31 .24 .12 smelled were looked
The middle panel shows the scorer applied early (illustrative values).

This is what the blocks are for. Every lookup and rewrite exists to move these shares toward the right word.

Standard terms The output projection maps the final vector to one logit per vocabulary word, and a softmax maps the logits to the distribution over the vocabulary. Training adjusts every block to sharpen it.

The whole model

consult the directory every word runs its lookup rewrite every line each word alone score the last word shares over the next word dozens of blocks, each printing a fresh directory
Two moves repeated through the stack, then the scoring of the last word (illustrative).

A language model is a directory of its words, consulted and reprinted dozens of times, with the last word, as finally rewritten, scored against every word it knows.

Standard terms Once token embeddings have entered the stack, each Transformer block, attention then feedforward, revises them, until an output projection and a softmax produce the next token distribution.

Recurrence and attention

The line of whisperersThe directory
Reaching a word six backsix retellingsone lookup
Training on a sentenceone word after anotherevery position in one pass
Carrying the sentenceone shared memoryone line per word
Keeping the future hiddenreading order does itthe mask does it

The fading and the waiting disappear, and reaching a distant word, the task the chain struggled with, is one lookup.

Standard terms Path length O(1) against O(n), parallel training against sequential, per token state against a single hidden state, and an explicit causal mask.

Two misconceptions

Misconception

"The attention weights show what the model understands." The weights say where the numbers came from. Whether that amounts to an explanation is contested in print: Jain and Wallace argued against reading them so, and Wiegreffe and Pinter argued the tests themselves need a definition of explanation.

Misconception

"Each lookup was designed to ask its question." Nobody assigns the questions. Training decides what every head looks for, and working out what a trained head does afterwards is a research field of its own.

Check for understanding

1. Which column of the directory does a word use to advertise itself, and which standard name does that column carry?
2. During training, why may word five consult word four but never word six?
3. The lookup of they succeeds in block 2 and would fail in block 1. What changed between the blocks?

1. The heading, which is the key.   2. Because word six is the answer to one of the covered blanks, and the mask keeps it covered.   3. Block 1 rewrote the line of rolls to say a plural food noun, and the description they looks up fits that reprinted heading.

Sources and further reading

  • A. Vaswani et al., 2017, "Attention Is All You Need," Advances in Neural Information Processing Systems 30. The quoted words on the sequential nature of recurrence come from the introduction; Table 1 gives the maximum path length as O(1) for self attention and O(n) for a recurrent layer; Section 3.2.3 describes masking out illegal connections by setting them to -\infty before the softmax.
  • D. Bahdanau, K. Cho, Y. Bengio, 2014 manuscript, "Neural Machine Translation by Jointly Learning to Align and Translate," arXiv:1409.0473; published at ICLR 2015. The paper identifies the single fixed length vector, carrying a whole source sentence between two networks, as a bottleneck, and introduces the learned soft alignment retold here as the soft lookup.
  • S. Hochreiter, J. Schmidhuber, 1997, "Long Short-Term Memory," Neural Computation 9(8). The paper analyses how error signals fade across many recurrent steps and builds a memory cell against the fading.
  • I. Tenney, D. Das, E. Pavlick, 2019, "BERT Rediscovers the Classical NLP Pipeline," ACL. The paper locates part of speech work in earlier layers and coreference in later layers of a deep language model.
  • A. Radford et al., 2019, "Language Models are Unsupervised Multitask Learners" (GPT-2). The four released models stack 12 to 48 Transformer blocks.
  • S. Jain, B. C. Wallace, 2019, "Attention is not Explanation," NAACL-HLT. S. Wiegreffe, Y. Pinter, 2019, "Attention is not not Explanation," EMNLP-IJCNLP. The two positions summarized on the misconception slide.
  • Part 9 of this series carries the same mechanism with the matrices, the scaling, the mask matrix, and worked softmax rows; Part 2 carries recurrent reading; Parts 3 and 4 carry the hidden layer and the training step.
  • The following are illustrative: the directory lines, the headings, the entries, the looked up descriptions, the weights 0.7, 0.2, and 0.1, the share tables and bars, and the five word mask grid. In a trained model every weight and every share is learned, and heads are read only after training.
  • Owner: Mehmet Kerem Turkcan; Associate Research Scientist; Center for Smart Streetscapes, Columbia University; New York, USA; keremturkcan.com; mkt2126@columbia.edu.
Intro to AI, Part 9 Appendix I: Attention as a Soft LookupM. K. Turkcan, Columbia University