← All topics

Guide · Foundations · Topic 1 of 32

Things I know

Six habits that outlast any model, and how silent failures are caught.

7 minute read

Six principles that outlast any model

Model names change every few months. These habits do not, and they decide whether a project works more often than the choice of architecture does.

  • Data beats clever algorithms. The scale, variety and quality of the data set the ceiling. A better dataset usually helps more than a new architecture, and the largest recent gains in robotics came from more varied data.
  • General methods that scale win. This is Rich Sutton’s “bitter lesson”: methods that improve with more data and compute keep overtaking methods that encode human knowledge by hand. Prefer designs that get better as the data grows.
  • All models are wrong. A good score on held-out data is not a working robot. Judge a model in the loop it will actually run in.
  • Failing models don’t throw exceptions. A broken pipeline still produces a loss curve. The only way to know is to look at the inputs, labels and outputs yourself.
  • Evaluation leaks. Every choice made against the test set moves a little of it into the model. Public benchmarks may already be in the pretraining data.
  • One variable per experiment. Change one thing, keep a matched control, and know what a trivial baseline scores before you start.
Three horizontal bars: a trivial baseline at 22 percent, a control at 61 percent and an experiment at 70 percent, with the gap between control and experiment marked
The last principle as a picture. Three numbers make a result readable: the trivial baseline, a matched control, and the experiment that differs from the control in one way.

Silent failure: what it looks like

Deep learning code fails quietly. Nothing crashes, the loss goes down, and the model has still learned the wrong thing. Four failures account for most of these cases.

Left: two training-loss curves that fall almost identically. Right: the model that learned the task succeeds 78 percent of the time on the robot and the one that learned a shortcut 6 percent
An illustrative pair of runs. The training logs are nearly the same; only running the policy tells them apart.
FailureWhat happenedSymptom
Label or input bugImages and labels are misaligned, normalisation is wrong, or two cameras are swappedThe loss falls slowly and levels off high
Data leakTest samples, or near-copies of them, are in the training setThe validation score looks too good
Shortcut learningThe model uses a cue that happens to correlate with the answerGreat on the benchmark, fails in new scenes
Evaluation bugThe metric is computed on the wrong split or the wrong outputThe numbers do not match what you see

Shortcut learning deserves a closer look because it is the hardest to spot. A classifier can learn the background in place of the object. A robot policy can learn to copy its previous action from its own joint readings and look fine on recorded data, then freeze on the real robot.

Recorded z-motion and gripper actions from one robot episode, with a copy of the previous action drawn on top and almost indistinguishable from it
Real data from one LIBERO demonstration (214 steps). Repeating the previous action gives a mean absolute error of 0.020, about 13 times lower than predicting the average action (0.253). It follows the recording closely and still cannot do the task, because it never starts a motion and is late at all four gripper changes.

That is why the dumb baseline matters. On recorded data, “repeat the last action” is a strong score to beat, and a model that only matches it has learned nothing about the task.

How much of a result is noise

Before asking whether a change helped, ask how much the number moves when nothing changes. Training is random in its initial weights, in the order of the data and in dropout, and that randomness shows up in the final score.

A histogram of validation accuracy for 30 training runs that differ only in their random seed. The runs range from 90.3 to 94.2 percent with a mean of 92.3.
Measured for this note: one small classifier, one dataset, one configuration, 30 random seeds. The runs span 3.8 points, with a standard deviation of 0.8.

Here the same code on the same data gives anything from 90.3% to 94.2%. A “1.5-point improvement” from a single run of each configuration would be well inside that spread, and would mean nothing. Three practices follow.

  • Run several seeds. Report the mean and the spread. Three is a minimum; five or more for a claim that matters.
  • Compare the gap with the spread. A difference between two means is only interesting when it is large compared with σ/n\sigma / \sqrt{n}, the standard deviation of the runs divided by the square root of how many there were.
  • Fix what can be fixed. Use the same seeds, data order and evaluation set for both arms, so that the remaining difference is the change being tested.

On a robot the same problem is larger, because each evaluation is a few dozen physical trials on top of the training noise. The note on evaluation works out how many trials a comparison needs.

Numbers you can check before trusting a run

Several quantities have a known correct value. Checking them costs minutes and catches most broken pipelines.

CheckWhat it should beWhat a wrong value means
Loss at the first stepFor a classifier with CC balanced classes, ln⁡C\ln C. The small network above starts at 2.308, and ln⁡10=2.303\ln 10 = 2.303.A much larger value points to bad initialisation or unnormalised inputs; a much smaller one to a label leaking into the input
Overfitting a single batchLoss goes to zero and the model reproduces those samples exactlyIf it cannot, there is a bug in the model, the loss or the labels
A shuffled-label runValidation performance at chanceAnything better means information is leaking from the evaluation set
Input ablationBlanking an input the task needs makes performance collapseIf nothing changes, the model is not using that input
Predicted against recorded, by eyePlots of predicted and recorded actions for whole episodes overlapA constant offset or a lag points to normalisation or timing

Leakage in robot data

Leakage is information about the evaluation set reaching the model. Robot datasets make it easy, because consecutive frames are nearly identical and episodes from one session share almost everything.

  • Split by episode, never by frame. Frames 100 and 101 of one episode are the same image. A random split by frame puts near-copies of every validation frame in the training set, and validation loss then measures memory.
  • Then ask what else is shared. Episodes from the same scene, object set or operator share more than the split suggests. To claim generalisation to new scenes, hold out whole scenes.
  • Compute statistics on the training split only. Normalisation means and ranges computed over all the data carry information from the validation set.
  • Count the choices made on the test set. Every checkpoint, threshold or hyperparameter chosen by looking at test results fits the test set a little. Keep a final set that is looked at once.

Habits that catch it

  • Look at the data. View raw samples after every transform, not before.
  • Overfit one batch. If the model cannot memorise ten samples, there is a bug.
  • Beat a dumb baseline. Compare with “predict the mean” or “repeat the last action”. If you cannot beat it, stop and find out why.
  • Ask what shortcut exists. Could the model score well without solving the task? Design a test that separates the two.
  • Test in the loop. Run the policy. Do not only score it.

A falling loss proves that the code runs. It does not prove the model learned what you intended.

Common mistakes

  • Changing the model first. When a result surprises you, check the data and the evaluation before touching the architecture.
  • Trusting an aggregate. One average hides which cases fail. Break results down before drawing a conclusion.
  • Skipping the control. Without a matched run that differs in one way, an improvement cannot be attributed to anything.

Terms

Baseline
The simplest method that could plausibly work; the number to beat.
Control
A run identical to the experiment except for the one thing being tested.
Shortcut
A cue that predicts the answer in the training data without the task being solved.
Leak
Information from the test set reaching the model during training or tuning.
Seed
The starting value of the random number generator; it fixes the initial weights and data order.
Standard error
The standard deviation of a mean: the spread of individual runs divided by n\sqrt{n}.

Further reading