← All topics
Guide
Glossary
Every term defined in the notes, in one place. Each entry links to the note that explains it.
6
- 6D pose
- An object’s 3D position and 3D orientation. Perception and 3D
A
- Ablation
- Removing one thing and measuring the difference, to learn what it contributed. Data for robots
- Action chunk
- A short sequence of future actions produced in one call. Robot policies today
- Action expert
- A small network attached to a vision-language backbone that generates continuous actions. Robot policies today
- Action head
- The part of the model that outputs actions. Robot policies today
- Action-conditioned
- Predictions that depend on the action given. World models
- Active learning
- Choosing which examples to label so that each label is as informative as possible. The data flywheel
- Adapter
- A small trained module added to a frozen pretrained model. Foundation models and adaptation
- Advantage
- How much better an action is than the average action in that state. Reinforcement learning
- Alignment
- The training stage that teaches a language model to use image tokens. Vision-language models
- Alignment and uniformity
- The two things a contrastive loss optimises: matched pairs close together, and all points spread evenly. Contrastive and multimodal learning
- Approximate nearest neighbour
- Search that examines part of a collection and accepts a small chance of missing a true neighbour. Retrieval and multimodal retrieval
- Asynchronous
- Running in separate loops, with neither waiting for the other. System 0, 1 and 2 architectures
- Attention entropy
- A measure of how spread out a head’s weights are; its exponential is the effective number of tokens attended to. Attention in depth
- Autoencoder
- An encoder and decoder trained to compress an input and rebuild it. Generative modeling
B
- Back-projection
- Turning a pixel and its depth into a 3D point using the camera’s intrinsics. Perception and 3D
- Backdrivable
- Able to be moved by an external force acting on the joint. Embodiments
- Baseline
- The simplest method that could plausibly work; the number to beat. Things I know
- Batch
- The handful of samples used for one update. Foundations that survived
- Beam search
- Decoding that keeps several candidate outputs and extends the best of them. Sequence-to-sequence models
- Behaviour cloning
- Supervised learning of actions from demonstrations. Robot learning basics
- Bellman equation
- The recursive relation between the value of a state and the values of the states that follow it. Reinforcement learning
C
- Calibration
- Whether “90% confident” is right 90% of the time. Evaluation
- Calibration data
- A small representative dataset used to choose quantisation ranges. Deployment on the robot
- Cartesian space
- Describing the hand by its position and orientation, as opposed to by joint angles. Robot learning basics
- Catastrophic forgetting
- Losing earlier abilities when a network is trained further on new data alone. The data flywheel
- Chain of thought
- Writing intermediate reasoning steps before an answer or action. Long-horizon tasks
- Checkpoint
- A saved copy of the model partway through training. Training at scale
- Classifier-free guidance
- Exaggerating the difference between prompted and unprompted predictions to follow the prompt more closely. Stable Diffusion and latent diffusion
- Closed loop
- Actions affect the next observation. Robot learning basics
- Closed-loop evaluation
- Running the policy so its actions affect what it sees next. Evaluation
- Co-training
- Training on robot data and web data together. Vision-language models
- Codebook
- A learned list of vectors; a discrete token is the number of the nearest one. Generative modeling
- Collaborative robot
- An arm designed to work safely beside people. Embodiments
- Collapse
- When every input maps to nearly the same embedding. Representation learning
- Completion detection
- Deciding that a sub-task has finished, successfully or not. Long-horizon tasks
- Confidence interval
- The range the true rate plausibly lies in, given the trials run. Evaluation
- Consistency
- Whether demonstrations of the same situation agree with each other. Failure triage
- Context length
- The most tokens the model can read at once. The transformer
- Context window
- The amount of input the model can read at once. Foundation models and adaptation
- Control
- A run identical to the experiment except for the one thing being tested. Things I know
- Convergence
- The point where more training no longer improves the validation metric. Training at scale
- Convolution
- A small grid of weights applied at every position of an image. Foundations that survived
- Covariate shift
- The difference between the states a policy was trained on and the states it visits when run. Robot learning basics
- Coverage
- Whether the training data includes situations like the one being tested. Failure triage
- Cross-attention
- Attention in which queries come from one sequence and keys and values from another. Attention in depth
- Cross-embodiment training
- Training one policy on data from several different robot bodies. Robot policies today
- Curation
- Checking, scoring, filtering and labelling a dataset before training on it. Data for robots
D
- DAgger
- Collecting expert labels in the states the learned policy reaches, and retraining on them. Robot learning basics
- Data efficiency
- How much task success a method gets from a given amount of data. Data efficiency and generalisation
- Data flywheel
- The loop in which a deployed model’s failures are captured and used to train its successor. The data flywheel
- Decoder
- The part that writes the output sequence, one token at a time. Sequence-to-sequence models
- Degrees of freedom
- The number of independent ways a robot can move. Embodiments
- Delta action
- An action given as a change from a reference pose, not as a destination. Robot learning basics
- Delta rule
- A memory update that removes what is stored under a key before writing the new value. Attention in depth
- Denoiser
- The network that predicts the noise in a noisy latent. Stable Diffusion and latent diffusion
- Denoising step
- One refinement of a noisy sample toward clean data. Generative modeling
- Diffusion transformer
- A denoiser built from transformer blocks over latent patches. Stable Diffusion and latent diffusion
- Digital twin
- A simulated copy of one specific real robot and workspace, validated against it. Data for robots
- Discrete cosine transform
- A change of basis that writes a signal as a sum of cosines of increasing frequency. Robot policies today
- Disparity
- The shift of a point between the two images of a stereo camera; inversely proportional to its depth. Senses beyond vision
- Distillation
- Training a smaller model to reproduce the outputs of a larger one. Deployment on the robot
- Drift
- Deployment inputs slowly departing from the training data. Deployment on the robot
- Dual-system architecture
- A policy split into a slow reasoning model and a fast acting model. System 0, 1 and 2 architectures
E
- Effective success rate
- The success rate of a step once retries are allowed. Long-horizon tasks
- Egocentric video
- Video recorded from the wearer’s own point of view. Data for robots
- ELBO
- A lower bound on the log-likelihood that can be optimised when the likelihood itself cannot. Generative modeling
- EMA weights
- A slow running average of the model’s weights, used for evaluation and deployment. Training at scale
- Embedding
- The vector a token is mapped to. The transformer
- Embodiment
- A particular robot body: its joints, reach, hand and sensors. Robot policies today
- Emergency stop
- A hardware control that cuts power to motion. Understanding and safety
- Encoder
- The part that reads the input sequence. Sequence-to-sequence models
- End effector
- The tool at the end of the arm, usually a gripper or hand. Embodiments
- Episode
- One recorded attempt at a task. Data for robots
- Epoch
- One full pass over the training data. Foundations that survived
- Expected calibration error
- The average gap between a model’s confidence and its accuracy, over bins of confidence. Understanding and safety
- Exploration
- Trying actions whose outcome is unknown in order to learn about them. Reinforcement learning
- Exposure bias
- Errors that arise because the model never saw its own mistakes during training. Sequence-to-sequence models
F
- Failure taxonomy
- A fixed list of failure stages and causes used to label and count failures. Failure triage
- False-alarm rate
- How often a monitor fires on inputs that are in fact normal. Understanding and safety
- Few-shot
- Adapting to something new from a handful of examples. Data efficiency and generalisation
- Fine-tuning
- Continuing training on task data. Foundation models and adaptation
- Force closure
- A grasp that can resist a force or twist in any direction. Dexterous hands
- Force-torque sensor
- A sensor, usually at the wrist, that measures the total push and twist applied to the hand. Senses beyond vision
- Forward kinematics
- Computing the gripper pose from the joint angles. Robotics fundamentals
- Foundation model
- A large model pretrained on broad data and reused for many tasks. Foundation models and adaptation
- Frame
- A point with axes attached, in which positions and orientations are expressed. Robotics fundamentals
- Friction cone
- The set of contact forces a fingertip can apply without slipping. Dexterous hands
G
- Gear ratio
- How many turns of the motor make one turn of the joint. Embodiments
- Gradient
- How the loss changes as each weight changes. Foundations that survived
- Gradient accumulation
- Summing gradients over several small batches before one update. Training at scale
- Gradient checkpointing
- Recomputing activations during the backward pass instead of storing them, to save memory. Training at scale
- Gradient clipping
- Scaling down a gradient that exceeds a set size. Training at scale
- Gravity compensation
- Adding the torque needed to hold the arm’s own weight to every command. Robotics fundamentals
- Guidance
- A knob that trades variety for closeness to the condition. Generative modeling
H
- Hallucination
- Fluent output that is not supported by the input or by fact. Foundation models and adaptation
- Hand synergy
- A coordinated whole-hand posture; a few of them, blended, reproduce most grasps. Dexterous hands
- Hard negative
- A negative that the model currently places close to the anchor. Contrastive and multimodal learning
- Head
- One of several attention computations run in parallel. The transformer
- Held-out
- Data or tasks the model never saw in training. Evaluation
- Homogeneous transform
- A 4 × 4 matrix that holds a rotation and a translation, so that poses compose by multiplication. Robotics fundamentals
- Horizon
- How many steps or how much time a task takes. Long-horizon tasks
- Hyperparameter
- A setting chosen by you, not learned by the model. Training at scale
I
- Impedance control
- A controller that makes the robot act like a spring and damper around its target. Robotics fundamentals
- In-distribution
- Test conditions that resemble the training data. Data efficiency and generalisation
- In-hand manipulation
- Moving an object within the hand without putting it down. Dexterous hands
- Index
- A stored structure that makes nearest-vector search fast. Retrieval and multimodal retrieval
- Influence
- An estimate of how much one training example changed a model’s behaviour on a given test case. Data quality and curation
- Inpainting
- Regenerating only a masked region of an image. Stable Diffusion and latent diffusion
- Instruction tuning
- Training on many tasks phrased as instructions. Vision-language models
- Intersection over union
- The area two regions share divided by the area they cover together. Perception and 3D
- Intervention
- A person stepping in to correct or rescue the robot. Evaluation
- Intervention weighting
- Giving human corrections more weight than other data during training. The data flywheel
- Intrinsics and extrinsics
- A camera’s internal geometry, and its pose relative to the robot. Robotics fundamentals
- Inverse dynamics model
- A model that infers the action taken between two observations. World models
- Inverse kinematics
- Finding joint angles that give a wanted gripper pose. Robotics fundamentals
J
- Jacobian
- The matrix relating joint velocities to hand velocity, and hand forces to joint torques. Robotics fundamentals
- Jerk
- The rate of change of acceleration; a measure of how smooth a motion is. Data quality and curation
K
- Key-value cache
- Stored keys and values of earlier tokens, reused while generating. Attention in depth
L
- Latency
- The delay between sensing and acting. Deployment on the robot
- Latent
- A compressed representation that a generator works in. Generative modeling
- Latent action
- A learned code that stands for the change between two frames, with no robot action label. World models
- Latent diffusion
- Running the diffusion process in an autoencoder’s compressed space instead of on pixels. Stable Diffusion and latent diffusion
- Latent interface
- A learned vector passed from the slow model to the fast one. System 0, 1 and 2 architectures
- Leak
- Information from the test set reaching the model during training or tuning. Things I know
- Learning curve
- Performance plotted against the amount of training data. Data efficiency and generalisation
- Linear attention
- Attention without the softmax, computable in time proportional to the sequence length. Attention in depth
- Linear probe
- A linear classifier trained on frozen features, used to measure what the features encode. Representation learning
- Location token
- A vocabulary entry that stands for a coordinate, used to write boxes and points. Vision-language models
- Logit
- A raw score output by a network before it is turned into a probability. Foundations that survived
- Loop time
- The time from seeing a failure in the field to deploying a release that handles it. The data flywheel
- LoRA
- Adapting a model by training small added matrices. Foundation models and adaptation
M
- Manipulability
- A measure of how freely the hand can move in every direction at a given configuration. Robotics fundamentals
- Markov decision process
- The formal setting of reinforcement learning: states, actions, transitions, rewards and a discount. Reinforcement learning
- Mask
- A rule fixing which tokens may attend to which. Attention in depth
- McNemar’s test
- A test for paired yes-or-no outcomes that uses only the pairs on which the two methods disagree. Evaluation
- Mean average precision
- A score that rewards ranking all relevant items near the top. Retrieval and multimodal retrieval
- Mel scale
- A frequency scale spaced the way human hearing resolves pitch. Senses beyond vision
- Mining
- Selecting informative pairs or triplets to train on. Contrastive and multimodal learning
- Mixed precision
- Computing mostly in 16-bit numbers while keeping sensitive values in 32 bits. Training at scale
- Modality dropout
- Randomly removing one input type during training so the policy cannot depend on a single one. Senses beyond vision
- Modality gap
- The tendency of different data types to occupy separate regions of a shared embedding space. Contrastive and multimodal learning
- Model-predictive control
- Planning a short sequence, executing its first action, and planning again. World models
- Momentum
- A running average of past gradients used as the update direction. Foundations that survived
- Momentum encoder
- A copy of a network whose weights are a slow moving average of another’s. Representation learning
- Multiple comparisons
- The inflation of false positives that comes from running many tests. Evaluation
N
- Nearest-neighbour distance
- The distance from a sample to the closest example in a reference set; a simple measure of familiarity. Failure triage
- Noise schedule
- The rule that sets how much noise is mixed in at each step. Stable Diffusion and latent diffusion
- Non-autoregressive
- Producing the whole output at once instead of token by token. Sequence-to-sequence models
- Normalisation statistics
- The per-value mean and spread, or percentiles, used to rescale actions and state. Robot learning basics
O
- Occupancy grid
- A map made of cells marked free or occupied. Perception and 3D
- ODE sampler
- A sampler that follows a deterministic differential equation from noise to data. Generative modeling
- Offline RL
- Reinforcement learning from a fixed dataset with no new trials. Reinforcement learning
- On-policy and off-policy
- Whether a method needs fresh data from the current policy or can reuse old data. Reinforcement learning
- Open vocabulary
- Able to find or name things described in text, beyond a fixed list of classes. Perception and 3D
- Out of distribution
- Unlike the data the model was trained on. Understanding and safety
- Out-of-distribution
- Test conditions that differ from the training data along some axis. Data efficiency and generalisation
P
- Paired comparison
- Testing two policies under the same conditions, trial by trial. Evaluation
- Perplexity
- The exponential of the average per-token loss; the effective number of choices the model is uncertain between. Sequence-to-sequence models
- Persistence baseline
- Predicting that the future frame equals the current one; the score any world model must beat. World models
- Point cloud
- A set of 3D points, usually one per pixel with a depth value. Perception and 3D
- Policy
- The function from observations to actions. Robot learning basics
- Pose
- The position and orientation of one frame relative to another. Robotics fundamentals
- Positive and negative
- Items that should and should not be close to the anchor. Contrastive and multimodal learning
- Power and force limiting
- Designing and controlling a robot so that contact with a person stays below harmful levels. Understanding and safety
- Power law
- A relationship in which one quantity is proportional to a fixed power of another; a straight line on log-log axes. Data efficiency and generalisation
- Pre-normalisation
- Applying normalisation at the input of each sub-layer, leaving the residual path untouched. The transformer
- Prefix
- The part of the input given to the model, as opposed to the part it must generate. Vision-language models
- Pretext task
- The invented prediction problem used for self-supervision. Representation learning
- Product quantisation
- Compressing a vector by replacing each of its sub-vectors with a codebook index. Retrieval and multimodal retrieval
- Projector
- The small layer that maps image features into a language model’s token space. Foundation models and adaptation
- Proprioception
- The robot’s sense of its own joints. Robot learning basics
- PSNR
- Peak signal-to-noise ratio: a logarithmic measure of pixel error between two images, in decibels. World models
Q
- Quantisation
- Representing numbers with fewer bits. Deployment on the robot
- Quantisation-aware training
- Training with rounding simulated in the forward pass, so the model adapts to it. Deployment on the robot
- Query, key, value
- The three vectors each token produces: what it seeks, what it offers to be matched on, and what it passes on. Attention in depth
R
- Rank
- The number of independent directions in a matrix; a low-rank matrix can be written as a product of two thin ones. Foundation models and adaptation
- Re-planning
- Choosing a new next step after observing the result of the last one. Long-horizon tasks
- Re-ranking
- Re-ordering a retrieved short list with a slower, more accurate model. Retrieval and multimodal retrieval
- Recall at k
- The share of queries with at least one relevant item in the top k. Retrieval and multimodal retrieval
- Reflected inertia
- The motor’s own inertia as felt at the joint, multiplied by the square of the gear ratio. Embodiments
- Registration
- Finding the rigid transform that aligns two sets of 3D points. Perception and 3D
- Regression suite
- A fixed set of tests for failures that were fixed before, run on every release. The data flywheel
- Release
- The weights plus every setting needed to run them, versioned together. Deployment on the robot
- Repeatability
- How closely a robot returns to a position it has visited before. Embodiments
- Replay
- Sending a recorded episode’s actions to the robot with no policy in the loop. Failure triage
- Retargeting
- Converting a human hand’s motion into the motion of a robot hand. Dexterous hands
- Retrieval
- Finding the items in a collection that best match a query. Retrieval and multimodal retrieval
- Return
- The sum of rewards over an episode, usually discounted. Reinforcement learning
- Reward
- A number the environment gives the agent after each step. Reinforcement learning
- Rollout
- A sequence of predictions, each built on the last. World models
- Runtime
- The software that executes an exported model on a specific chip. Deployment on the robot
S
- Sampler
- The rule that turns the denoiser’s prediction into the next, less noisy latent. Stable Diffusion and latent diffusion
- Sampling weight
- How often a dataset is drawn from when several are mixed. Data for robots
- Scaling law
- An empirical power-law relation between model performance and parameters, data or compute. Foundation models and adaptation
- Score
- The gradient of the log-density of the data; the direction in which samples become more probable. Generative modeling
- Seed
- The starting value of the random number generator; it fixes the initial weights and data order. Things I know
- Segmentation mask
- The set of pixels that belong to one object. Perception and 3D
- Sequence length
- The number of tokens the model processes in one pass. Deployment on the robot
- Two token streams with separate weights that attend over their combined tokens. Attention in depth
- Short-time Fourier transform
- The Fourier transform applied to successive short windows of a signal, giving frequency content over time. Senses beyond vision
- Shortcut
- A cue that predicts the answer in the training data without the task being solved. Things I know
- Signal-to-noise ratio
- The ratio of image to noise power at a given step. Stable Diffusion and latent diffusion
- Sim-to-real
- Moving a policy from simulation to hardware. Data for robots
- Singularity
- An arm configuration where some hand motions need very large joint motions. Robotics fundamentals
- SLAM
- Building a map while tracking the robot’s position in it. Perception and 3D
- Spectrogram
- A picture of a sound: time across, pitch up, loudness as brightness. Senses beyond vision
- Speed and separation monitoring
- Reducing a robot’s speed as a person gets closer, and stopping it at a set distance. Understanding and safety
- Staleness
- The age of the information an action was computed from at the moment it takes effect. System 0, 1 and 2 architectures
- Standard error
- The standard deviation of a mean: the spread of individual runs divided by . Things I know
- Statistical power
- The chance that a test detects a difference that is really there. Evaluation
- Stop-gradient
- Blocking a loss from updating part of a network. Robot policies today
- Sub-task
- One step of a longer task, small enough for the low-level policy to carry out. Robot policies today
- Sub-word tokenisation
- Splitting text into frequent character sequences, so that any string can be written with a fixed vocabulary. Sequence-to-sequence models
- System 0
- The fastest layer, handling balance, contact and joint tracking. System 0, 1 and 2 architectures
- System 1
- The fast model that turns intent and vision into motion. System 0, 1 and 2 architectures
- System 2
- The slow model that understands the scene and decides what to do. System 0, 1 and 2 architectures
T
- Tactile sensor
- A sensor on the fingers or palm that measures contact over an area. Senses beyond vision
- Take-over
- A person assuming control from the policy; also called an intervention. The data flywheel
- Teacher forcing
- Training the decoder on the correct earlier tokens instead of its own predictions. Sequence-to-sequence models
- Teleoperation
- A person controlling the robot to record demonstrations. Data for robots
- Temperature
- A scale applied to similarities before the softmax; lower values sharpen the focus on the hardest negatives. Contrastive and multimodal learning
- Temperature scaling
- Dividing a model’s logits by a constant fitted on held-out data, to correct its confidence. Understanding and safety
- Temporal ensembling
- Averaging the overlapping predictions that successive action chunks make for the same moment. Robot learning basics
- Temporal-difference error
- The gap between a value estimate and the reward plus the discounted estimate of the next state. Reinforcement learning
- Tendon drive
- Moving a joint with a cable pulled by a motor located elsewhere, usually in the forearm. Dexterous hands
- Throughput
- The amount of completed work per unit of time. Evaluation
- Tiling
- Splitting a large image into pieces that are each encoded at the encoder’s native size. Vision-language models
- Token
- One unit of input, represented as a vector. The transformer
- Trajectory
- The sequence of observations and actions in an episode. Data for robots
U
- Under-actuated
- Having fewer motors than joints, so some joints move together. Dexterous hands
V
- VAE
- An autoencoder that maps data to a smooth latent space and back. Generative modeling
- Value
- The return expected from a state, or from an action in a state. Reinforcement learning
- Vanishing gradient
- A gradient that shrinks toward zero as it is propagated back through many layers. Foundations that survived
- Vision-language model
- A language model that takes images as well as text as input. Vision-language models
- VLA
- A vision-language model trained to output robot actions. Robot policies today
W
- Weight decay
- A small pull of every weight toward zero at each step. Foundations that survived
- Weight tying
- Using the same matrix for the input embedding and the output layer. The transformer
- Whole-body control
- Coordinating legs, torso and arms together. Embodiments
- Wilson interval
- A confidence interval for a proportion that stays accurate for small samples and rates near 0 or 1. Evaluation
- World model
- A model that predicts future observations given actions. World models
- World-action model
- One model that outputs both future observations and actions. World models
Y
- Yield
- The share of captured episodes that end up in the training set. The data flywheel
Z
- Zero-shot
- Doing a task with no training examples for it. Representation learning