Visual & Physical AI glossary
A practical reference for the terms behind computer vision, visual AI, and physical AI, from images and video to LiDAR and multimodal data. Each entry gives a clear definition, real examples, and how the concept fits a modern data pipeline, from labeling raw data to evaluating models.
Data & Curation
Most model problems are data problems. Definitions for the datasets, embeddings, quality signals, and curation methods that decide what is worth labeling and training on in the first place.
- Class imbalanceWhen some classes are far more frequent than others in a dataset.
- ClusteringGrouping unlabeled samples by similarity to reveal structure in a dataset.
- Coreset selectionChoosing a small representative subset that stands in for the full dataset.
- Data augmentationExpanding training data by transforming existing examples into new variants.
- Data curationDeliberately choosing which data to keep, label, and train on, so better data drives model gains.
- Data flywheelA data flywheel is a self-reinforcing loop where deploying a model generates new data that improves the model, which drives more deployment and more data.
- Data lineageA record of data's origin and the transformations it passed through.
- Data samplingSelecting a subset of data from a larger pool for training or evaluation.
- Data sliceA subset of a dataset defined by a shared attribute, used for targeted evaluation.
- Data triageData triage decides which recorded robot or vehicle data to keep or upload and which to discard, prioritizing the most valuable segments.
- Data versioningTracking changes to a dataset over time so any version can be reproduced.
- Data-centric AIAn approach that improves models mainly by improving their data.
- DatasetAn organized collection of examples, labels, and metadata used to train and evaluate a model.
- Dimensionality reductionProjecting high-dimensional data into few dimensions while preserving structure.
- Edge caseAn edge case is a rare, unusual situation outside a system's common operating conditions, where it is more likely to fail despite being infrequent.
- Embedding and semantic searchAn embedding represents data as points in a vector space where similar items sit close together, and semantic search uses this to find data by meaning.
- Failure miningFailure mining systematically searches recorded data to find where a model performed poorly, then uses those cases to guide targeted improvement.
- Image blurrinessA per-image score for how out-of-focus or motion-blurred an image is, used as a curation signal.
- Image embeddingsA vector representation of an image's content, used to search, cluster, and curate data.
- LeRobotLeRobot is an open source robot learning library and dataset format that pairs video-compressed camera data with tabular state and action data.
- LeRobotDatasetLeRobotDataset is the LeRobot format for robot demonstrations: episodes with Parquet state and action data, MP4 camera video, and metadata files.
- Longtail investigationLongtail investigation is the practice of finding and studying rare, underrepresented cases in a dataset, where models are most likely to fail.
- MCAPAn open-source, serialization-agnostic container file format for storing timestamped, multimodal robotics data, and the default recording format for ROS 2.
- MetadataDescriptive attributes attached to each sample, used to filter and understand data.
- Model Context Protocol (MCP)An open protocol that lets AI agents connect to external tools and data through a standard client and server interface.
- Multimodal dataMultimodal data combines two or more distinct data types, such as images, text, audio, and sensor signals, that describe the same subject or moment.
- Near-duplicate detectionFinding samples that are identical or nearly identical within a dataset.
- Open X-EmbodimentOpen X-Embodiment is a large public dataset unifying over a million real robot trajectories from many robot bodies and labs into a common format.
- Outlier detectionIdentifying samples that deviate sharply from the rest of a dataset.
- Protocol Buffers (protobuf)Google's language-neutral format for serializing structured data into compact binary messages from a .proto schema.
- RLDSRLDS is a data format that stores sequential decision-making data as episodes of ordered steps, each with an observation, action, reward, and related fields.
- Scenario miningScenario mining searches large volumes of recorded data to find specific, meaningful driving or robot situations for testing, evaluation, and targeted collection.
- Synthetic dataArtificially generated data used to supplement or replace real data.
- Time-series sensor dataTime-series sensor data is a sequence of sensor readings recorded over time, where the order and timing of measurements carry meaning.
- Uncertainty samplingAn active learning strategy that labels the samples a model is least sure about.
Annotation & labeling
Annotation is where model quality is won or lost. Definitions for the label types, formats, and quality measures that turn raw data into training data, from bounding boxes to ground truth.
- 3D annotationLabeling objects and regions in three-dimensional data like point clouds and 3D scenes.
- ADAS annotationLabeling sensor data for driver-assistance features like lane keeping and automatic braking.
- Active learningA strategy where the model picks the most informative samples to label next, so you label less.
- Agentic labelingTraining a labeling agent in natural language, tuned to your data, then applying it to label at scale.
- AnnotationAdding structured labels to raw data so a model can learn from it.
- Annotation QAThe process of reviewing and correcting labels before they reach training.
- Annotation guidelinesThe documented rules and examples annotators follow to label data consistently.
- Annotation ontologyThe structured set of labeled concepts and the relationships between them.
- Annotation qualityHow accurate, consistent, and complete a dataset's labels are.
- Annotation taxonomyThe hierarchical organization of a labeling task's categories, from broad to specific.
- Annotation workflowThe end-to-end process from raw data to trusted labels.
- Attribute annotationLabeling descriptive properties on an object, like color or state, beyond its class.
- Audio annotationLabeling sound recordings, transcription, sound events, and speakers, over the audio timeline.
- Auto-labelingUsing a model to generate labels automatically, with humans reviewing rather than drawing from scratch.
- Autonomous driving annotationLabeling fused camera, LiDAR, and radar data for self-driving perception.
- Bounding boxA rectangle that marks an object's location in an image.
- Click-to-segmentClicking an object to generate a full segmentation mask from a model, refined with more clicks.
- Computer vision annotationLabeling images and video so computer vision models can learn to see.
- Cuboid (3D bounding box)A 3D box capturing an object's position, size, and orientation in space.
- DICOM annotationLabeling medical images in the DICOM clinical format, across volumetric, multi-planar views.
- Data labelingAssigning tags or classes to raw data so a model can learn from it.
- Event annotationLabeling actions and events in time, with start and end boundaries.
- Foundation model labelingUsing large pretrained models like SAM or CLIP to label data, often from a prompt and without task-specific training.
- Ground truthThe labels accepted as correct, the reference a model is trained and measured against.
- Human in the loop (HITL)A pattern where people review and correct a model's outputs instead of leaving it fully automated.
- Image annotationLabeling images with tags, boxes, masks, or keypoints so a model can learn from them.
- Image classificationAssigning one or more class labels to a whole image, with no location.
- Instance segmentationGiving each individual object in an image its own pixel mask and identity.
- Keypoint annotationMarking specific labeled points on an object, like joints or facial landmarks.
- Label propagationLabel propagation spreads a few human annotations to many unlabeled examples automatically, most often across the frames of a video or sequence.
- Label schemaThe defined structure of a labeling task, its classes, label types, and attributes.
- Label trackA label track is the set of annotations following one object across the frames of a sequence, tied together under a single consistent identity.
- LiDAR annotationLabeling LiDAR point cloud data with 3D cuboids or per-point classes.
- MCAP annotationLabeling robotics and sensor log data stored in the MCAP container format.
- Medical image annotationLabeling clinical images (X-ray, CT, MRI, pathology) to train medical AI.
- Model-assisted labelingA model suggests labels in real time while a human accepts or corrects them.
- Multimodal annotationLabeling data that combines multiple modalities, and aligning them across streams.
- OCR annotationLabeling text in images, where it is and what it says, to train OCR.
- Object detectionLocating and classifying each object in an image with a bounding box.
- Object trackingFollowing the same object across video frames with a consistent identity.
- OpenLABELOpenLABEL is an ASAM open standard for annotating multi-sensor data and tagging scenarios, defining a portable JSON format for labels.
- Panoptic segmentationLabeling every pixel with a class and giving countable objects instance IDs, semantic plus instance combined.
- Point cloud annotationLabeling 3D point cloud data with cuboids or per-point classes.
- Polygon annotationOutlining an object with connected boundary points to capture its shape.
- Polyline annotationTracing a linear feature, like a lane line, with connected points in an open line.
- Pose estimationPredicting an object's keypoints and skeleton to capture posture and movement.
- Pre-labelingRunning a model to draft labels in batch before humans correct them.
- Pseudo-labelingUsing a model's confident predictions on unlabeled data as training targets, semi-supervised.
- Relationship annotationLabeling how objects relate (subject-predicate-object), building up a scene graph.
- Segmentation maskA pixel-level label marking exactly which pixels belong to an object.
- Semantic segmentationLabeling every pixel in an image with a class, without separating individual objects.
- Task annotationTask annotation attaches natural-language descriptions to robot episodes and segments, the instructions that language-conditioned policies and VLAs learn from.
- Temporal detectionTemporal detection locates when an event begins and ends within a sequence over time, extending detection from a single frame into an interval.
- Text annotationLabeling text for NLP, classification, entity tagging, sentiment, and more.
- Video annotationLabeling objects and events across video frames, tracking them over time.
- Video annotation interpolationFilling in an object's labels on frames between keyframes by estimating its position.
- Weak supervisionGenerating labels from cheap, noisy sources like rules and heuristics instead of manual annotation.
- Zero-shot labelingLabeling classes a model was never trained on, specified at inference, usually through a text prompt.
Evaluation & Metrics
A model is only as trustworthy as the way you measure it. Definitions for the metrics and methods that tell you whether a model works, and where a strong headline number can still hide failure.
- AccuracyThe fraction of predictions a model gets right, intuitive but misleading on imbalanced data.
- Average precision (AP)A per-class detection score that summarizes the precision-recall curve as the area under it, in practice computed with an interpolated approximation.
- BenchmarkA standard dataset and protocol for comparing models fairly.
- Confidence scoreA per-prediction certainty value used to accept or reject model outputs.
- Confusion matrixA table of true versus predicted classes that shows exactly which classes a model confuses.
- Dice coefficientAn overlap score equal to twice the intersection over the sum of two regions.
- DisengagementA disengagement is when an autonomous system's automated mode is deactivated and a human takes control, due to a detected failure or a safety operator's judgment.
- Distribution shiftDistribution shift, or domain shift, is when the data a model sees in deployment differs from its training data, degrading performance.
- Error analysisExamining a model's mistakes to find patterns and guide improvement.
- F1 scoreThe harmonic mean of precision and recall, expressed as a single score.
- Gold setA small, expert-verified subset of data used as a trusted benchmark for quality.
- Inter-annotator agreementHow consistently different annotators assign the same labels to the same data.
- Intersection over union (IoU)Overlap divided by combined area, the standard 0-to-1 score for boxes and masks.
- Label noiseIncorrect, imprecise, or inconsistent labels mixed into a dataset's ground truth.
- Mean IoU (mIoU)The class-averaged overlap between predicted and true segmentation masks.
- Mean average precision (mAP)The average of per-class average precision, the standard detection metric.
- Mean squared errorThe average of squared prediction errors, a regression loss and metric where lower is better.
- Model calibrationHow well a model's confidence scores match its true accuracy.
- Model evaluationThe process of measuring a model's performance and understanding its errors.
- Panoptic qualityThe standard metric for panoptic segmentation, combining segmentation and recognition quality.
- Policy evaluationPolicy evaluation measures how well a learned control policy performs, usually by running it many times, with success rate as the standard headline metric.
- PrecisionThe share of a model's positive predictions that turn out to be correct.
- Precision-recall curveA plot of precision versus recall across all decision thresholds.
- ROC curve and AUCA threshold sweep of true positive versus false positive rate, summarized by area under the curve.
- RecallThe share of real positive cases that a model correctly identifies.
Models & Architectures
The building blocks that learn from your data. Definitions for the networks, foundation models, and algorithms behind modern visual AI, and what each one is actually good for.
- Approximate nearest neighbor (ANN)Fast similarity search that trades a little exactness for enormous speed.
- Attention mechanismA method that lets a model weigh which parts of its input are most relevant.
- AutoencoderA network that compresses input to a compact code and reconstructs it.
- BackboneThe core feature-extracting network shared across model tasks.
- Contrastive learningA representation-learning method that learns embeddings by pulling similar pairs together and pushing dissimilar pairs apart, usually self-supervised but also used with labels or paired data like image and text.
- Convolutional neural network (CNN)A neural network that learns spatial features from images with convolutional filters, the workhorse of computer vision.
- Cosine similarityA similarity score based on the angle between two vectors, standard for embeddings.
- Diffusion modelA generative model that creates data by iteratively reversing a noising process.
- EmbeddingA learned vector representation of data in which similarity becomes measurable distance.
- Embedding driftA shift in embedding distributions over time, an early signal of data change.
- Embedding modelA model that converts inputs into vectors whose distances reflect similarity.
- Embedding poolingAggregating many embeddings into a single summary vector.
- Embedding spaceThe high-dimensional space where embeddings live and distance encodes similarity.
- Fine-tuningContinuing to train a pretrained model on task-specific data to specialize it.
- FoundationPoseNVIDIA's foundation model for 6D pose estimation and tracking of new objects, from a CAD model or a few reference images.
- Frame embeddingAn embedding of one video frame, the finest temporal granularity for video search.
- Generative adversarial network (GAN)A generative model where a generator and discriminator compete to produce realistic data.
- Generative pre-trained transformerA transformer pretrained on large text corpora to predict the next token, producing fluent language and serving as the foundation for many modern large language and vision-language models.
- Knowledge distillationTraining a small student model to imitate a larger teacher model.
- Linear probingEvaluating embeddings by training a simple linear classifier on top of them.
- Logistic regressionA linear classifier that outputs class probabilities, and the softmax head on top of most image classifiers.
- Multimodal AIMultimodal AI refers to models that take in, combine, and reason across multiple data types, such as images, text, audio, and sensor signals, in one system.
- Multimodal embeddingAn embedding in a space shared across data types, aligning images, text, and more.
- Object-level embeddingAn embedding of one object in a scene, enabling object-level search and matching.
- Patch embeddingThe vector representation of a small image patch, the token unit of Vision Transformers.
- Positional encodingPosition information added to token embeddings so transformers know order and layout.
- QuantizationReducing the numerical precision of a model to make it smaller and faster.
- Reinforcement learningTraining an agent to act by rewarding good outcomes over time.
- SAM 2 (Segment Anything Model 2)Meta's SAM successor that extends promptable segmentation to video, tracking objects across frames.
- SAM 3 (Segment Anything Model 3)Meta's 2025 model that segments every instance of a concept from a text phrase or example, across images and video.
- Scene embeddingAn embedding of a scene's overall setting and conditions rather than its objects.
- Segment Anything Model (SAM)Meta's foundation model that segments any object from a click or box, zero-shot.
- Segment-level embeddingAn embedding of a segment, a time slice, clip, or region, rather than a whole item.
- Self-supervised learningTraining on unlabeled data using supervision signals derived from the data itself.
- Text embeddingA vector representation of text where similar meanings land close together.
- Token embeddingThe vector assigned to each input token, where discrete data becomes computable.
- Transfer learningReusing a model trained on one task as the starting point for another.
- TransformerA neural network architecture built on self-attention, the basis of most modern AI models.
- Vector databaseA database purpose-built to index and search embedding vectors at scale.
- Video embeddingA vector representing a video or clip, capturing content and motion over time.
- Vision transformer (ViT)A transformer applied to images by treating patches as tokens.
- Vision-Language Model (VLM)A vision-language model understands images and text together, connecting visual content to language to describe scenes, answer questions, and follow instructions.
Physical AI
Physical AI is AI that perceives and acts in the real world, in robots, autonomous vehicles, and drones. Definitions for the multimodal and sensor concepts behind it, from LiDAR, point clouds, and sensor fusion to world models and sim-to-real.
- 6D object pose estimationEstimating an object's 3D position and 3D orientation relative to the camera, so a robot knows where it is and how it is turned.
- ADAS (advanced driver-assistance systems)ADAS are electronic systems that assist a human driver by sensing the environment and providing warnings or partial control, without replacing the driver.
- Action Chunking with Transformers (ACT)Action Chunking with Transformers (ACT) is an imitation learning policy that predicts short sequences of robot actions at once from images and joint states.
- Action chunkingAction chunking has a policy predict a short sequence of future actions at once instead of one step at a time, improving smoothness and reducing compounding error.
- Action embeddingA vector representation of robot actions, inside policies and for organizing data.
- Action headAn action head, or action decoder, is the model component that turns an internal representation into an executable motor command such as a joint movement.
- Action spaceAn action space defines what a robot policy's outputs mean: which joints or tool coordinates they command, in what units, and whether as targets or changes.
- AffordanceAn action that an object or environment makes possible for an agent.
- Autonomous driving data curationSelecting the most valuable driving scenarios, rare events, and long-tail cases from petabyte-scale fleet logs for training and evaluation.
- Autonomous mobile robot (AMR)An autonomous mobile robot (AMR) navigates flexibly using onboard sensors and software, choosing its own paths and reacting to obstacles in real time.
- Bimanual manipulationManipulation that uses two arms or hands together, such as holding a jar with one and turning the lid with the other.
- Bin pickingA robot picking individual parts out of a bin where they lie jumbled, overlapping, and in random orientations.
- Bird's-eye view (BEV)A top-down scene representation that fuses sensors into a common overhead map.
- Collaborative robot (cobot)An industrial robot arm designed to work near people, with sensing and limits on force and speed built in.
- Cross-embodimentCross-embodiment is training or transferring one control policy across multiple robot bodies with different sensors, joints, and action spaces.
- DROID (Distributed Robot Interaction Dataset)DROID is a large real-world robot manipulation dataset of about 76,000 teleoperated demonstrations across 564 scenes, collected on one standard Franka arm setup.
- Data engineA closed-loop process that mines model failures from production data, curates and labels them, retrains, and redeploys.
- Data mixtureA data mixture is the set of proportions a training run uses to sample from its source datasets, such as different robots, tasks, or collection sites.
- Degrees of freedom (DoF)Degrees of freedom (DoF) is the number of independent ways a robot can move, usually counted as its independently controllable joints or axes.
- Diffusion PolicyDiffusion Policy is an imitation learning method that generates robot action sequences by iteratively denoising random noise, conditioned on observations.
- Digital twinA virtual replica of a physical system kept in sync with the real one.
- Domain randomizationRandomizing a simulator's appearance and physics so models transfer to reality.
- Dual-system architectureA dual-system architecture pairs a slow, deliberative reasoning model with a fast, reactive control policy, letting each run at the frequency it needs.
- Ego vehicleThe ego vehicle is the vehicle from whose perspective an autonomous or driver-assistance system perceives and reasons, the reference point for the scene.
- Embodied AIEmbodied AI is the field of AI concerned with systems that learn and act through physical interaction with an environment, using a real or simulated body.
- EmbodimentEmbodiment is the specific physical body of a robot, including its kinematics, degrees of freedom, sensors, and available actions.
- Emergency stop (E-stop)A button or switch that stops a robot or machine as quickly as safely possible, overriding every other command.
- End-effectorAn end-effector is the device at the end of a robot arm, such as a gripper, hand, or tool, that interacts with the world.
- EpisodeAn episode, trajectory, or rollout is one complete attempt at a task, recorded as a time-ordered sequence of observation and action pairs.
- Episode embeddingAn embedding of a whole robot episode, making recorded behavior searchable.
- Forward kinematicsThe calculation of where a robot's end-effector is, and which way it points, from the angles of its joints.
- GazeboAn open source 3D robot simulator, maintained alongside ROS, for testing robots and generating sensor data before using real hardware.
- GraspingA robot securely picking up and holding objects with a gripper or hand.
- Human-in-the-loop data collectionHuman-in-the-loop data collection records human corrections during robot policy rollouts, turning the situations where a robot struggles into training data.
- Humanoid robotA humanoid robot has a human-like body plan, with a torso, arms, and legs or a mobile base, so it can operate in spaces and with tools designed for people.
- Imitation learningImitation learning trains a policy to mimic expert demonstrations, and behavior cloning is its direct form that reproduces the expert's action for each observation.
- Inertial measurement unitA sensor measuring acceleration and rotation to track motion and orientation.
- Inverse kinematicsComputing the joint angles needed to place an end-effector at a target pose.
- LIBERO (lifelong robot learning benchmark)LIBERO is a simulated robot manipulation benchmark of 130 language-specified tasks in four suites, widely used to report imitation learning and VLA success rates.
- Leader-follower teleoperationLeader-follower teleoperation records robot demonstrations by having a follower arm mirror the joint positions of a leader arm that a person moves by hand.
- LiDARLiDAR measures distance by timing reflected laser pulses, producing a dense 3D point cloud of the surrounding surfaces.
- Long-horizon taskA long-horizon task is a multi-step task that requires stringing many actions together and recovering gracefully from mistakes made partway through.
- Machine tendingA robot loading raw parts into a machine, such as a CNC mill, and unloading finished parts so the machine keeps running.
- Motion planningComputing a collision-free, feasible path from a start to a goal state.
- Motion smoothness metricsScores such as jerk, LDLJ, and spectral arc length that rate how smoothly a robot or person moved in a recording.
- Multimodal data curationSelecting what to keep, label, and train on across modalities using deduplication, quality signals, embeddings, and cross-modal alignment checks.
- Multimodal data managementThe ongoing discipline of organizing, versioning, tracking, and governing multimodal datasets so they stay reproducible and trustworthy over time.
- Multimodal data platformA unified system for representing, querying, and managing datasets that span many modalities across the full data lifecycle.
- Multimodal data toolsThe categories of software used to store, annotate, curate, visualize, evaluate, and version datasets that span multiple modalities.
- Multimodal data visualizationViewing images, video, text, audio, sensor data, and embeddings together to understand the structure of a whole dataset.
- Multimodal sensor dataMultimodal sensor data is data captured at once from several sensor types, such as cameras, lidar, radar, and inertial units, describing the same scene.
- Multimodal sensor embeddingA joint embedding of a moment across multiple synchronized sensor streams.
- Neural reconstructionNeural reconstruction uses neural representations to build a detailed 3D scene model from images or sensor data, renderable from new viewpoints.
- Occupancy gridA grid map storing the probability that each cell of space is occupied.
- Occupancy networkA model predicting which points in 3D space are occupied by matter.
- OdometryEstimating a robot's movement over time from its own motion sensors.
- Operational design domain (ODD)An operational design domain (ODD) is the specific set of conditions, such as roads, weather, and areas, under which an automated driving system is designed to operate safely.
- Physical AIPhysical AI is artificial intelligence embodied in machines that perceive their surroundings and take physical action in the real world.
- Physical AI data collectionGathering real-world and simulated sensor and action data, through teleoperation, fleet logging, sensor rigs, and simulation, to train physical AI systems.
- Physical AI data curationSelecting which episodes and samples from collected sensor data are worth keeping, labeling, and training on for physical AI systems.
- Physical AI data platformA platform for managing, curating, visualizing, and evaluating the multimodal sensor data that powers physical AI systems.
- Physical AI model evaluationMeasuring how well perception models and robot policies perform using recorded logs, simulation benchmarks, scenario-based tests, and real-world trials.
- Physical AI stackThe layered architecture of a physical AI system, from data collection and curation through training to real-world deployment.
- Physical AI training dataTime-synchronized sensor observations and actions used to train models that perceive and act in the physical world.
- Point cloudA point cloud is a set of 3D points, each with coordinates and sometimes attributes like intensity or color, that together represent the surfaces of a scene or object.
- Point cloud embeddingA vector representation of a 3D point cloud, making geometry searchable by similarity.
- Point cloud visualizationRendering and interactively exploring 3D point data from LiDAR, depth cameras, or photogrammetry to inspect scene geometry.
- PolicyA policy is the learned function that maps an observation to an action, and it is what "the model" usually refers to in a robotics control setting.
- ProprioceptionProprioception is a robot's internal sense of its own body, such as joint positions, velocities, torques, end-effector pose, and gripper state.
- ROS 2An open-source framework and middleware for building robot software.
- ROS nodeA single program in a ROS system that does one job and talks to other nodes through topics, services, and actions.
- RVizRViz is a 3D visualization tool for ROS that displays a robot's model, sensor data, and state in an interactive 3D view for debugging and understanding.
- RadarRadar emits radio waves and analyzes their reflections to measure objects' distance, direction, and velocity, working well in poor visibility.
- Reward functionThe signal that defines success and guides a reinforcement learning agent.
- Robot brainIndustry shorthand for the AI model, or stack of models, that turns what a robot senses into what it does.
- Robot demonstrationA robot demonstration is a recorded example of a task, usually performed by a person teleoperating the robot, that a policy learns to imitate.
- Robot manipulationA robot changing its environment through physical contact, from picking and placing to pushing, pouring, and assembling.
- Robotic work cellThe complete setup a robot needs for one job: the robot, its tooling, fixtures, sensors, controller, and safeguarding.
- Robotics data platformInfrastructure for ingesting, managing, curating, and reviewing robot log data like MCAP files and rosbags at fleet scale.
- Robotics observabilityRobotics observability collects and analyzes a robot fleet's data and logs to understand why the robots behave as they do, for monitoring and debugging.
- SLAMBuilding a map of an unknown environment while tracking your position in it.
- SO-101 robot armThe SO-101 is a low-cost, open source robot arm used with LeRobot, built as a leader and follower pair for recording teleoperated demonstrations.
- Sensor calibrationSensor calibration determines a sensor's internal parameters (intrinsics) and its position relative to other sensors (extrinsics and transforms) for consistent measurement.
- Sensor data visualizationInspecting time-aligned multi-sensor streams from a robot or vehicle log together on a shared timeline for debugging and review.
- Sensor fusionSensor fusion combines data from multiple sensors into one more accurate and reliable estimate of the environment than any single sensor could give.
- Sensor synchronizationSensor synchronization aligns the timing of multiple sensors so their readings can be matched to the same moment in time.
- Sim-to-real transferSim-to-real transfer trains a policy in simulation and deploys it on a real robot, bridging the reality gap between simulated and real physics and sensing.
- SimulationUsing virtual environments to model and test robot behavior before real deployment.
- SmolVLASmolVLA is a compact, open source vision-language-action model from Hugging Face, pretrained on community LeRobot datasets and small enough for consumer GPUs.
- Spatiotemporal embeddingAn embedding of content across space and time, capturing how scenes and motion unfold.
- Success detectionSuccess detection decides whether a recorded robot episode actually completed its task, producing the outcome labels that training and evaluation rely on.
- TeleoperationTeleoperation is a human remotely piloting a robot, often via VR controllers or a leader-follower rig, to generate demonstration data.
- URDFURDF is an XML format describing a robot as a tree of links and joints, capturing its kinematics, geometry, and physical properties.
- Vision-Language-Action model (VLA)A Vision-Language-Action model maps camera input and a language instruction directly to robot actions, extending a vision-language model with an action output.
- World modelA world model learns to predict how an environment will change over time and in response to actions, letting an agent imagine consequences before acting.
- Wrist cameraA camera mounted on a robot's wrist or gripper that moves with the hand, giving a close-up view of what it is grasping.
- rosbagThe original ROS 1 tool and .bag format for recording a robot's message streams, predecessor to rosbag2.
- rosbag2The ROS 2 tool for recording robot message streams to bag files and replaying them.
Visual AI
Visual AI is AI that interprets images and video to detect, segment, track, and make sense of what appears in a scene. Definitions for the core tasks and techniques behind it, from object detection, segmentation, and tracking to depth estimation, optical flow, and 3D scene reconstruction.
- Depth estimationPredicting the distance of each pixel from the camera to recover 3D structure.
- Feature extractionTurning raw data into compact, informative values that capture its structure.
- Gaussian splattingRepresenting a 3D scene as many colored Gaussians for fast, photorealistic rendering.
- Image captioningGenerating a natural-language description of what an image shows.
- Image segmentationDividing an image into meaningful regions at the pixel level, the umbrella task that spans semantic, instance, and panoptic segmentation.
- Multi-object trackingDetecting many objects in video and keeping a consistent ID for each over time.
- Neural radiance field (NeRF)A neural representation of a 3D scene learned from images for novel-view rendering.
- Non-maximum suppression (NMS)A step that removes overlapping duplicate detections, keeping the best box per object.
- Open-vocabulary detectionDetecting object categories specified at inference, not fixed at training time.
- Optical character recognition (OCR)Detecting and converting text in images into machine-readable characters.
- Optical flowThe per-pixel apparent motion between consecutive video frames.
- Re-identificationRecognizing the same object or person across different cameras or times.
- Visual inspectionUsing cameras and AI models to find defects, such as cracks, scratches, or missing parts, on products, parts, or structures.
- Visual question answeringAnswering natural-language questions about the content of an image.
A
ADAS (advanced driver-assistance systems)
ADAS annotation
Accuracy
Action Chunking with Transformers (ACT)
Action chunking
Action embedding
Action head
Action space
Active learning
Affordance
Agentic labeling
Annotation
Annotation QA
Annotation guidelines
Annotation ontology
Annotation quality
Annotation taxonomy
Annotation workflow
Approximate nearest neighbor (ANN)
Attention mechanism
Attribute annotation
Audio annotation
Auto-labeling
Autoencoder
Autonomous driving annotation
Autonomous driving data curation
Autonomous mobile robot (AMR)
Average precision (AP)
C
D
DICOM annotation
DROID (Distributed Robot Interaction Dataset)
Data augmentation
Data curation
Data engine
Data flywheel
Data labeling
Data lineage
Data mixture
Data sampling
Data slice
Data triage
Data versioning
Data-centric AI
Dataset
Degrees of freedom (DoF)
Depth estimation
Dice coefficient
Diffusion Policy
Diffusion model
Digital twin
Dimensionality reduction
Disengagement
Distribution shift
Domain randomization
Dual-system architecture
E
F
G
I
L
M
MCAP
MCAP annotation
Machine tending
Mean IoU (mIoU)
Mean average precision (mAP)
Mean squared error
Medical image annotation
Metadata
Model Context Protocol (MCP)
Model calibration
Model evaluation
Model-assisted labeling
Motion planning
Motion smoothness metrics
Multi-object tracking
Multimodal AI
Multimodal annotation
Multimodal data
Multimodal data curation
Multimodal data management
Multimodal data platform
Multimodal data tools
Multimodal data visualization
Multimodal embedding
Multimodal sensor data
Multimodal sensor embedding
N
O
P
Panoptic quality
Panoptic segmentation
Patch embedding
Physical AI
Physical AI data collection
Physical AI data curation
Physical AI data platform
Physical AI model evaluation
Physical AI stack
Physical AI training data
Point cloud
Point cloud annotation
Point cloud embedding
Point cloud visualization
Policy
Policy evaluation
Polygon annotation
Polyline annotation
Pose estimation
Positional encoding
Pre-labeling
Precision
Precision-recall curve
Proprioception
Protocol Buffers (protobuf)
Pseudo-labeling
R
S
SAM 2 (Segment Anything Model 2)
SAM 3 (Segment Anything Model 3)
SLAM
SO-101 robot arm
Scenario mining
Scene embedding
Segment Anything Model (SAM)
Segment-level embedding
Segmentation mask
Self-supervised learning
Semantic segmentation
Sensor calibration
Sensor data visualization
Sensor fusion
Sensor synchronization
Sim-to-real transfer
Simulation
SmolVLA
Spatiotemporal embedding
Success detection
Synthetic data
T
V