← All topics

Guide

Glossary

Every term defined in the notes, in one place. Each entry links to the note that explains it.

244 terms

6

6D pose
An object’s 3D position and 3D orientation. Perception and 3D

A

Ablation
Removing one thing and measuring the difference, to learn what it contributed. Data for robots
Action chunk
A short sequence of future actions produced in one call. Robot policies today
Action expert
A small network attached to a vision-language backbone that generates continuous actions. Robot policies today
Action head
The part of the model that outputs actions. Robot policies today
Action-conditioned
Predictions that depend on the action given. World models
Active learning
Choosing which examples to label so that each label is as informative as possible. The data flywheel
Adapter
A small trained module added to a frozen pretrained model. Foundation models and adaptation
Advantage
How much better an action is than the average action in that state. Reinforcement learning
Alignment
The training stage that teaches a language model to use image tokens. Vision-language models
Alignment and uniformity
The two things a contrastive loss optimises: matched pairs close together, and all points spread evenly. Contrastive and multimodal learning
Approximate nearest neighbour
Search that examines part of a collection and accepts a small chance of missing a true neighbour. Retrieval and multimodal retrieval
Asynchronous
Running in separate loops, with neither waiting for the other. System 0, 1 and 2 architectures
Attention entropy
A measure of how spread out a head’s weights are; its exponential is the effective number of tokens attended to. Attention in depth
Autoencoder
An encoder and decoder trained to compress an input and rebuild it. Generative modeling

B

Back-projection
Turning a pixel and its depth into a 3D point using the camera’s intrinsics. Perception and 3D
Backdrivable
Able to be moved by an external force acting on the joint. Embodiments
Baseline
The simplest method that could plausibly work; the number to beat. Things I know
Batch
The handful of samples used for one update. Foundations that survived
Decoding that keeps several candidate outputs and extends the best of them. Sequence-to-sequence models
Behaviour cloning
Supervised learning of actions from demonstrations. Robot learning basics
Bellman equation
The recursive relation between the value of a state and the values of the states that follow it. Reinforcement learning

C

Calibration
Whether “90% confident” is right 90% of the time. Evaluation
Calibration data
A small representative dataset used to choose quantisation ranges. Deployment on the robot
Cartesian space
Describing the hand by its position and orientation, as opposed to by joint angles. Robot learning basics
Catastrophic forgetting
Losing earlier abilities when a network is trained further on new data alone. The data flywheel
Chain of thought
Writing intermediate reasoning steps before an answer or action. Long-horizon tasks
Checkpoint
A saved copy of the model partway through training. Training at scale
Classifier-free guidance
Exaggerating the difference between prompted and unprompted predictions to follow the prompt more closely. Stable Diffusion and latent diffusion
Closed loop
Actions affect the next observation. Robot learning basics
Closed-loop evaluation
Running the policy so its actions affect what it sees next. Evaluation
Co-training
Training on robot data and web data together. Vision-language models
Codebook
A learned list of vectors; a discrete token is the number of the nearest one. Generative modeling
Collaborative robot
An arm designed to work safely beside people. Embodiments
Collapse
When every input maps to nearly the same embedding. Representation learning
Completion detection
Deciding that a sub-task has finished, successfully or not. Long-horizon tasks
Confidence interval
The range the true rate plausibly lies in, given the trials run. Evaluation
Consistency
Whether demonstrations of the same situation agree with each other. Failure triage
Context length
The most tokens the model can read at once. The transformer
Context window
The amount of input the model can read at once. Foundation models and adaptation
Control
A run identical to the experiment except for the one thing being tested. Things I know
Convergence
The point where more training no longer improves the validation metric. Training at scale
Convolution
A small grid of weights applied at every position of an image. Foundations that survived
Covariate shift
The difference between the states a policy was trained on and the states it visits when run. Robot learning basics
Coverage
Whether the training data includes situations like the one being tested. Failure triage
Cross-attention
Attention in which queries come from one sequence and keys and values from another. Attention in depth
Cross-embodiment training
Training one policy on data from several different robot bodies. Robot policies today
Curation
Checking, scoring, filtering and labelling a dataset before training on it. Data for robots

D

DAgger
Collecting expert labels in the states the learned policy reaches, and retraining on them. Robot learning basics
Data efficiency
How much task success a method gets from a given amount of data. Data efficiency and generalisation
Data flywheel
The loop in which a deployed model’s failures are captured and used to train its successor. The data flywheel
Decoder
The part that writes the output sequence, one token at a time. Sequence-to-sequence models
Degrees of freedom
The number of independent ways a robot can move. Embodiments
Delta action
An action given as a change from a reference pose, not as a destination. Robot learning basics
Delta rule
A memory update that removes what is stored under a key before writing the new value. Attention in depth
Denoiser
The network that predicts the noise in a noisy latent. Stable Diffusion and latent diffusion
Denoising step
One refinement of a noisy sample toward clean data. Generative modeling
Diffusion transformer
A denoiser built from transformer blocks over latent patches. Stable Diffusion and latent diffusion
Digital twin
A simulated copy of one specific real robot and workspace, validated against it. Data for robots
Discrete cosine transform
A change of basis that writes a signal as a sum of cosines of increasing frequency. Robot policies today
Disparity
The shift of a point between the two images of a stereo camera; inversely proportional to its depth. Senses beyond vision
Distillation
Training a smaller model to reproduce the outputs of a larger one. Deployment on the robot
Drift
Deployment inputs slowly departing from the training data. Deployment on the robot
Dual-system architecture
A policy split into a slow reasoning model and a fast acting model. System 0, 1 and 2 architectures

E

Effective success rate
The success rate of a step once retries are allowed. Long-horizon tasks
Egocentric video
Video recorded from the wearer’s own point of view. Data for robots
ELBO
A lower bound on the log-likelihood that can be optimised when the likelihood itself cannot. Generative modeling
EMA weights
A slow running average of the model’s weights, used for evaluation and deployment. Training at scale
Embedding
The vector a token is mapped to. The transformer
Embodiment
A particular robot body: its joints, reach, hand and sensors. Robot policies today
Emergency stop
A hardware control that cuts power to motion. Understanding and safety
Encoder
The part that reads the input sequence. Sequence-to-sequence models
End effector
The tool at the end of the arm, usually a gripper or hand. Embodiments
Episode
One recorded attempt at a task. Data for robots
Epoch
One full pass over the training data. Foundations that survived
Expected calibration error
The average gap between a model’s confidence and its accuracy, over bins of confidence. Understanding and safety
Exploration
Trying actions whose outcome is unknown in order to learn about them. Reinforcement learning
Exposure bias
Errors that arise because the model never saw its own mistakes during training. Sequence-to-sequence models

F

Failure taxonomy
A fixed list of failure stages and causes used to label and count failures. Failure triage
False-alarm rate
How often a monitor fires on inputs that are in fact normal. Understanding and safety
Few-shot
Adapting to something new from a handful of examples. Data efficiency and generalisation
Fine-tuning
Continuing training on task data. Foundation models and adaptation
Force closure
A grasp that can resist a force or twist in any direction. Dexterous hands
Force-torque sensor
A sensor, usually at the wrist, that measures the total push and twist applied to the hand. Senses beyond vision
Forward kinematics
Computing the gripper pose from the joint angles. Robotics fundamentals
Foundation model
A large model pretrained on broad data and reused for many tasks. Foundation models and adaptation
Frame
A point with axes attached, in which positions and orientations are expressed. Robotics fundamentals
Friction cone
The set of contact forces a fingertip can apply without slipping. Dexterous hands

G

Gear ratio
How many turns of the motor make one turn of the joint. Embodiments
Gradient
How the loss changes as each weight changes. Foundations that survived
Gradient accumulation
Summing gradients over several small batches before one update. Training at scale
Gradient checkpointing
Recomputing activations during the backward pass instead of storing them, to save memory. Training at scale
Gradient clipping
Scaling down a gradient that exceeds a set size. Training at scale
Gravity compensation
Adding the torque needed to hold the arm’s own weight to every command. Robotics fundamentals
Guidance
A knob that trades variety for closeness to the condition. Generative modeling

H

Hallucination
Fluent output that is not supported by the input or by fact. Foundation models and adaptation
Hand synergy
A coordinated whole-hand posture; a few of them, blended, reproduce most grasps. Dexterous hands
Hard negative
A negative that the model currently places close to the anchor. Contrastive and multimodal learning
One of several attention computations run in parallel. The transformer
Held-out
Data or tasks the model never saw in training. Evaluation
Homogeneous transform
A 4 × 4 matrix that holds a rotation and a translation, so that poses compose by multiplication. Robotics fundamentals
Horizon
How many steps or how much time a task takes. Long-horizon tasks
Hyperparameter
A setting chosen by you, not learned by the model. Training at scale

I

Impedance control
A controller that makes the robot act like a spring and damper around its target. Robotics fundamentals
In-distribution
Test conditions that resemble the training data. Data efficiency and generalisation
In-hand manipulation
Moving an object within the hand without putting it down. Dexterous hands
Index
A stored structure that makes nearest-vector search fast. Retrieval and multimodal retrieval
Influence
An estimate of how much one training example changed a model’s behaviour on a given test case. Data quality and curation
Inpainting
Regenerating only a masked region of an image. Stable Diffusion and latent diffusion
Instruction tuning
Training on many tasks phrased as instructions. Vision-language models
Intersection over union
The area two regions share divided by the area they cover together. Perception and 3D
Intervention
A person stepping in to correct or rescue the robot. Evaluation
Intervention weighting
Giving human corrections more weight than other data during training. The data flywheel
Intrinsics and extrinsics
A camera’s internal geometry, and its pose relative to the robot. Robotics fundamentals
Inverse dynamics model
A model that infers the action taken between two observations. World models
Inverse kinematics
Finding joint angles that give a wanted gripper pose. Robotics fundamentals

J

Jacobian
The matrix relating joint velocities to hand velocity, and hand forces to joint torques. Robotics fundamentals
Jerk
The rate of change of acceleration; a measure of how smooth a motion is. Data quality and curation

K

Key-value cache
Stored keys and values of earlier tokens, reused while generating. Attention in depth

L

Latency
The delay between sensing and acting. Deployment on the robot
Latent
A compressed representation that a generator works in. Generative modeling
Latent action
A learned code that stands for the change between two frames, with no robot action label. World models
Latent diffusion
Running the diffusion process in an autoencoder’s compressed space instead of on pixels. Stable Diffusion and latent diffusion
Latent interface
A learned vector passed from the slow model to the fast one. System 0, 1 and 2 architectures
Leak
Information from the test set reaching the model during training or tuning. Things I know
Learning curve
Performance plotted against the amount of training data. Data efficiency and generalisation
Linear attention
Attention without the softmax, computable in time proportional to the sequence length. Attention in depth
Linear probe
A linear classifier trained on frozen features, used to measure what the features encode. Representation learning
Location token
A vocabulary entry that stands for a coordinate, used to write boxes and points. Vision-language models
Logit
A raw score output by a network before it is turned into a probability. Foundations that survived
Loop time
The time from seeing a failure in the field to deploying a release that handles it. The data flywheel
LoRA
Adapting a model by training small added matrices. Foundation models and adaptation

M

Manipulability
A measure of how freely the hand can move in every direction at a given configuration. Robotics fundamentals
Markov decision process
The formal setting of reinforcement learning: states, actions, transitions, rewards and a discount. Reinforcement learning
Mask
A rule fixing which tokens may attend to which. Attention in depth
McNemar’s test
A test for paired yes-or-no outcomes that uses only the pairs on which the two methods disagree. Evaluation
Mean average precision
A score that rewards ranking all relevant items near the top. Retrieval and multimodal retrieval
Mel scale
A frequency scale spaced the way human hearing resolves pitch. Senses beyond vision
Mining
Selecting informative pairs or triplets to train on. Contrastive and multimodal learning
Mixed precision
Computing mostly in 16-bit numbers while keeping sensitive values in 32 bits. Training at scale
Modality dropout
Randomly removing one input type during training so the policy cannot depend on a single one. Senses beyond vision
Modality gap
The tendency of different data types to occupy separate regions of a shared embedding space. Contrastive and multimodal learning
Model-predictive control
Planning a short sequence, executing its first action, and planning again. World models
Momentum
A running average of past gradients used as the update direction. Foundations that survived
Momentum encoder
A copy of a network whose weights are a slow moving average of another’s. Representation learning
Multiple comparisons
The inflation of false positives that comes from running many tests. Evaluation

N

Nearest-neighbour distance
The distance from a sample to the closest example in a reference set; a simple measure of familiarity. Failure triage
Noise schedule
The rule that sets how much noise is mixed in at each step. Stable Diffusion and latent diffusion
Non-autoregressive
Producing the whole output at once instead of token by token. Sequence-to-sequence models
Normalisation statistics
The per-value mean and spread, or percentiles, used to rescale actions and state. Robot learning basics

O

Occupancy grid
A map made of cells marked free or occupied. Perception and 3D
ODE sampler
A sampler that follows a deterministic differential equation from noise to data. Generative modeling
Offline RL
Reinforcement learning from a fixed dataset with no new trials. Reinforcement learning
On-policy and off-policy
Whether a method needs fresh data from the current policy or can reuse old data. Reinforcement learning
Open vocabulary
Able to find or name things described in text, beyond a fixed list of classes. Perception and 3D
Out of distribution
Unlike the data the model was trained on. Understanding and safety
Out-of-distribution
Test conditions that differ from the training data along some axis. Data efficiency and generalisation

P

Paired comparison
Testing two policies under the same conditions, trial by trial. Evaluation
Perplexity
The exponential of the average per-token loss; the effective number of choices the model is uncertain between. Sequence-to-sequence models
Persistence baseline
Predicting that the future frame equals the current one; the score any world model must beat. World models
Point cloud
A set of 3D points, usually one per pixel with a depth value. Perception and 3D
Policy
The function from observations to actions. Robot learning basics
Pose
The position and orientation of one frame relative to another. Robotics fundamentals
Positive and negative
Items that should and should not be close to the anchor. Contrastive and multimodal learning
Power and force limiting
Designing and controlling a robot so that contact with a person stays below harmful levels. Understanding and safety
Power law
A relationship in which one quantity is proportional to a fixed power of another; a straight line on log-log axes. Data efficiency and generalisation
Pre-normalisation
Applying normalisation at the input of each sub-layer, leaving the residual path untouched. The transformer
Prefix
The part of the input given to the model, as opposed to the part it must generate. Vision-language models
Pretext task
The invented prediction problem used for self-supervision. Representation learning
Product quantisation
Compressing a vector by replacing each of its sub-vectors with a codebook index. Retrieval and multimodal retrieval
Projector
The small layer that maps image features into a language model’s token space. Foundation models and adaptation
Proprioception
The robot’s sense of its own joints. Robot learning basics
PSNR
Peak signal-to-noise ratio: a logarithmic measure of pixel error between two images, in decibels. World models

Q

Quantisation
Representing numbers with fewer bits. Deployment on the robot
Quantisation-aware training
Training with rounding simulated in the forward pass, so the model adapts to it. Deployment on the robot
Query, key, value
The three vectors each token produces: what it seeks, what it offers to be matched on, and what it passes on. Attention in depth

R

Rank
The number of independent directions in a matrix; a low-rank matrix can be written as a product of two thin ones. Foundation models and adaptation
Re-planning
Choosing a new next step after observing the result of the last one. Long-horizon tasks
Re-ranking
Re-ordering a retrieved short list with a slower, more accurate model. Retrieval and multimodal retrieval
Recall at k
The share of queries with at least one relevant item in the top k. Retrieval and multimodal retrieval
Reflected inertia
The motor’s own inertia as felt at the joint, multiplied by the square of the gear ratio. Embodiments
Registration
Finding the rigid transform that aligns two sets of 3D points. Perception and 3D
Regression suite
A fixed set of tests for failures that were fixed before, run on every release. The data flywheel
Release
The weights plus every setting needed to run them, versioned together. Deployment on the robot
Repeatability
How closely a robot returns to a position it has visited before. Embodiments
Replay
Sending a recorded episode’s actions to the robot with no policy in the loop. Failure triage
Retargeting
Converting a human hand’s motion into the motion of a robot hand. Dexterous hands
Retrieval
Finding the items in a collection that best match a query. Retrieval and multimodal retrieval
Return
The sum of rewards over an episode, usually discounted. Reinforcement learning
Reward
A number the environment gives the agent after each step. Reinforcement learning
Rollout
A sequence of predictions, each built on the last. World models
Runtime
The software that executes an exported model on a specific chip. Deployment on the robot

S

Sampler
The rule that turns the denoiser’s prediction into the next, less noisy latent. Stable Diffusion and latent diffusion
Sampling weight
How often a dataset is drawn from when several are mixed. Data for robots
Scaling law
An empirical power-law relation between model performance and parameters, data or compute. Foundation models and adaptation
Score
The gradient of the log-density of the data; the direction in which samples become more probable. Generative modeling
Seed
The starting value of the random number generator; it fixes the initial weights and data order. Things I know
Segmentation mask
The set of pixels that belong to one object. Perception and 3D
Sequence length
The number of tokens the model processes in one pass. Deployment on the robot
Shared attention
Two token streams with separate weights that attend over their combined tokens. Attention in depth
Short-time Fourier transform
The Fourier transform applied to successive short windows of a signal, giving frequency content over time. Senses beyond vision
Shortcut
A cue that predicts the answer in the training data without the task being solved. Things I know
Signal-to-noise ratio
The ratio of image to noise power at a given step. Stable Diffusion and latent diffusion
Sim-to-real
Moving a policy from simulation to hardware. Data for robots
Singularity
An arm configuration where some hand motions need very large joint motions. Robotics fundamentals
SLAM
Building a map while tracking the robot’s position in it. Perception and 3D
Spectrogram
A picture of a sound: time across, pitch up, loudness as brightness. Senses beyond vision
Speed and separation monitoring
Reducing a robot’s speed as a person gets closer, and stopping it at a set distance. Understanding and safety
Staleness
The age of the information an action was computed from at the moment it takes effect. System 0, 1 and 2 architectures
Standard error
The standard deviation of a mean: the spread of individual runs divided by n\sqrt{n}. Things I know
Statistical power
The chance that a test detects a difference that is really there. Evaluation
Stop-gradient
Blocking a loss from updating part of a network. Robot policies today
Sub-task
One step of a longer task, small enough for the low-level policy to carry out. Robot policies today
Sub-word tokenisation
Splitting text into frequent character sequences, so that any string can be written with a fixed vocabulary. Sequence-to-sequence models
System 0
The fastest layer, handling balance, contact and joint tracking. System 0, 1 and 2 architectures
System 1
The fast model that turns intent and vision into motion. System 0, 1 and 2 architectures
System 2
The slow model that understands the scene and decides what to do. System 0, 1 and 2 architectures

T

Tactile sensor
A sensor on the fingers or palm that measures contact over an area. Senses beyond vision
Take-over
A person assuming control from the policy; also called an intervention. The data flywheel
Teacher forcing
Training the decoder on the correct earlier tokens instead of its own predictions. Sequence-to-sequence models
Teleoperation
A person controlling the robot to record demonstrations. Data for robots
Temperature
A scale applied to similarities before the softmax; lower values sharpen the focus on the hardest negatives. Contrastive and multimodal learning
Temperature scaling
Dividing a model’s logits by a constant fitted on held-out data, to correct its confidence. Understanding and safety
Temporal ensembling
Averaging the overlapping predictions that successive action chunks make for the same moment. Robot learning basics
Temporal-difference error
The gap between a value estimate and the reward plus the discounted estimate of the next state. Reinforcement learning
Tendon drive
Moving a joint with a cable pulled by a motor located elsewhere, usually in the forearm. Dexterous hands
Throughput
The amount of completed work per unit of time. Evaluation
Tiling
Splitting a large image into pieces that are each encoded at the encoder’s native size. Vision-language models
Token
One unit of input, represented as a vector. The transformer
Trajectory
The sequence of observations and actions in an episode. Data for robots

U

Under-actuated
Having fewer motors than joints, so some joints move together. Dexterous hands

V

VAE
An autoencoder that maps data to a smooth latent space and back. Generative modeling
Value
The return expected from a state, or from an action in a state. Reinforcement learning
Vanishing gradient
A gradient that shrinks toward zero as it is propagated back through many layers. Foundations that survived
Vision-language model
A language model that takes images as well as text as input. Vision-language models
VLA
A vision-language model trained to output robot actions. Robot policies today

W

Weight decay
A small pull of every weight toward zero at each step. Foundations that survived
Weight tying
Using the same matrix for the input embedding and the output layer. The transformer
Whole-body control
Coordinating legs, torso and arms together. Embodiments
Wilson interval
A confidence interval for a proportion that stays accurate for small samples and rates near 0 or 1. Evaluation
World model
A model that predicts future observations given actions. World models
World-action model
One model that outputs both future observations and actions. World models

Y

Yield
The share of captured episodes that end up in the training set. The data flywheel

Z

Zero-shot
Doing a task with no training examples for it. Representation learning