Learn Transformers from Attention Is All You Need on BaseHalf

Transformer diagrams get easier when the paper itself sits on the canvas and each hard sentence becomes its own note. We put Vaswani et al., Attention Is All You Need, in an ordinary Desktop folder, opened that folder in BaseHalf, and grew five branch notes from selected passages in the PDF reader - not by typing summaries into empty files first. After that walk we wrote a short Q / K / V practice note, answered it ourselves, and asked Codex in the Agent Area to grade the four one-liners.

The folder on disk was Desktop/basehalf-learn-sessions/attention-is-all-you-need/. After the grows, a rename pass, and the practice card, the canvas held:

attention-is-all-you-need/
  attention-is-all-you-need.pdf
  01-transformer-claim.md
  02-why-not-only-recurrence.md
  03-scaled-dot-product.md
  04-three-attention-uses.md
  05-positional-encoding.md
  06-qkv-practice.md

Grow a note from a passage, not from a blank card

Double-click the PDF card to read it in Card Detail. Pick the selection tool, drag across a passage, and choose Grow branch. BaseHalf creates a Markdown note that already lists the PDF under upstream:. That is the durable link - the same storage model as any other connection, written into the downstream file so git and later sessions can see what the note builds on.

We did that five times, once per idea we wanted to keep. Canvas connections you draw by hand are a different move; Grow branch only runs inside the PDF or book reader on a selection. Grow a note from a passage in a PDF or book is the product write-up for this path.

Claim first: attention alone

01-transformer-claim.md grew from the abstract. The quote on the card is the paper's punchline: sequence transduction had been recurrent or convolutional stacks with attention bolted on; the Transformer keeps attention and drops recurrence and convolutions for the architecture itself.

That is the north star for the rest of the walk. Everything below is answering why that swap is worth it and how the pieces fit.

Why recurrence was the bottleneck

02-why-not-only-recurrence.md grew from the introduction. Sequential computation blocks parallelization inside a training example. Long sequences make that expensive. Attention was already popular for linking distant positions; this paper makes that linking the whole model instead of a helper.

Plain takeaway we left on the note: RNNs wait step by step; attention can relate positions in one shot and train across the sequence length more easily.

Scaled Dot-Product Attention

03-scaled-dot-product.md grew next to Figure 2 on page 4 - the diagram almost everyone redraws. Query asks. Key is what each position advertises. Value is the payload you mix. Scores are a scaled dot product of Q and K; softmax turns scores into weights; the output is the weighted sum of V. Multi-head is several of those attention layers in parallel, then concatenated and projected.

We did not need every matrix shape on day one. The card holds the figure caption plus the section open, and a short mental model under the quote.

Three wirings of the same mechanism

04-three-attention-uses.md grew from §3.2.3. Same multi-head block, three jobs in the original encoder-decoder Transformer:

• Encoder self-attention - positions in the source attend to the source
• Decoder masked self-attention - positions in the target attend leftward only
• Encoder-decoder attention - decoder queries attend over encoder memory

Once those three wirings click, "attention is all you need" stops sounding like a slogan and starts sounding like a parts list.

Positional encodings keep order

05-positional-encoding.md grew from the sine/cosine section on page 6. Attention alone does not know token order. The paper sums fixed sinusoidal positional encodings with the embeddings so position rides with content. Later models swap the recipe; the need for some order signal stays.

Practice: Q, K, V in one sentence each

After the five grown notes, we still wanted a check that the Q / K / V story would stick without looking at the figure. We added 06-qkv-practice.md on the canvas, connected it upstream to the scaled-dot-product note, the three-wirings note, and the recurrence note, and answered four prompts in one sentence each:

1. In scaled dot-product attention, what does the Query represent?
2. What does the Key represent, and how does it meet the Query?
3. Once the weights are set, what is the Value for?
4. Why does the paper run multi-head attention instead of one wide head?

Our answers on the card:

1. Query is what this position is looking for in the rest of the sequence.
2. Key is what each position advertises; the scaled dot product of Q and K scores how well each Key matches the Query.
3. Value is the payload you actually mix - the weighted sum of Values becomes the attention output.
4. Several heads in parallel let the model keep different viewpoints (for example different dependency patterns) instead of forcing one averaging into a single head.

Then we opened the Agent Area on the same folder and asked Codex to read 06-qkv-practice.md and grade each line Correct / Partial / Wrong with one short reason. It marked all four Correct: Q as what the current position seeks, K as each position's matching features, V as the content those weights retrieve, and multi-head as separate learned Q / K / V projections that keep different relationships instead of one averaged head.

What each note carries

Every branch file starts like this (paths relative to the folder):

upstream:
  - attention-is-all-you-need.pdf

The practice note lists the PDF plus the mechanism cards it depends on. Under that, the grown quote (or the exercise), then a few sentences we wrote in our own words. The PDF stays the source of truth. The notes are the files we will reopen - not a chat transcript. Selecting text without Growing still feeds a one-off ask while the window is open; Growing is for excerpts that should keep feeding later work. Connect what should stick; select what is for this ask is the same split.

A shallow-to-deep order that matched the paper

1. Abstract claim - attention as the architecture
2. Pain - recurrence blocks parallel training
3. Mechanism - scaled dot-product and multi-head
4. Layout - three places attention runs in the model
5. Order - positional encodings
6. Check - write Q / K / V in one sentence each and get the Agent Area to grade

That order is how we actually grew the branches walking the PDF, then closed the loop with a practice card - not a syllabus invented after the fact. If you want the illustrated tour afterward, Jay Alammar's The Illustrated Transformer and Harvard NLP's The Annotated Transformer still pair well with these notes - as reading, not as a substitute for the paper card on the canvas.