How a Network Learns

The brief

Simplify a complex concept with an interactive learning experience, a quiz and a roadmap.

The idea

You understand a network the moment you watch one fail. So the lesson is a live experiment: pick a dataset, train, and then deliberately give it too few neurons and watch the loss stop falling.

Decisions

• Real backpropagation written out in plain loops, so the source is part of the lesson.
• Each point shows the network's current opinion as colour behind the data.
• Try to break it is the best section, because failure teaches.
• Atkinson Hyperlegible was designed for readability, which suits a teaching page.

How it came together

1. Wrote the network and derived the gradients directly.
2. Tested that it reaches 98 per cent on the spiral.
3. Wrote the lesson around the ways it can be made to fail.

How it works

A plain multilayer perceptron with tanh hidden units and a sigmoid output, trained by full-batch gradient descent on cross-entropy.