Annotation & Labeling

Episode-level labels

• success and failure
• quality tier
• task variant
• environment notes
• operator or collection flags

Temporal annotations

• task phases
• failure windows
• recovery segments
• intervention periods
• event-aligned markers on synchronized timelines

Frame and spatial annotations

• 2D boxes
• 3D boxes
• segmentation masks
• grasp points
• contact points
• target regions

Language and semantic labels

The platform should support whichever annotation type best reflects the learning problem, not force all tasks into one visual ontology.

• instruction labels
• task descriptions
• semantic tags
• multi-language or synonym-aware label layers when needed

Schema definition

• Define allowed labels, enums, relationships, and required fields per task or program.
• Separate global taxonomy from task-specific annotation rules.
• Support evolution of schemas without detaching older annotated data.

Validation

• Prevent invalid labels, missing required fields, and incompatible annotation shapes.
• Enforce consistency before annotations are approved for downstream use.
• Keep schema version attached to every annotation object.

Migration and compatibility

• Support schema revisions that map older labels to new structures when appropriate.
• Make schema differences visible in review and dataset finalization rather than burying them inside export scripts.

Model-assisted proposals

• Seed boxes, segments, phase boundaries, or event markers from detectors, heuristics, or learned proposal systems.
• Preserve proposal provenance so teams know what was machine-suggested versus human-authored.
• Use proposals to reduce blank-slate annotation work rather than to bypass review.

Interpolation and tracking

• Keyframe-based propagation across dense temporal data.
• Object or trajectory tracking to avoid frame-by-frame re-labeling.
• Useful for high-frame-rate robot video and repeated motion patterns.

Programmatic annotation

• Ingest labels from vendor systems or internal automation through the Platform API.
• Attach automated reward-model outputs, heuristics, or external labeling service results to the same source entities.
• Keep machine-generated annotations inside the same review and versioning system as manual ones.

Multi-annotator workflows

• Assign the same episode to multiple labelers when agreement matters.
• Track disagreement and escalate conflict cases to reviewers.
• Measure annotation quality using approval state and inter-annotator agreement where relevant.

Review queues

• Route by task, modality, site, operator, model proposal source, or customer scope.
• Support staged review such as draft → reviewer approved → production ready.
• Make quality assurance explicit rather than dependent on informal spot checks.

Annotation completeness

• Track which required labels exist for each dataset purpose.
• Surface incompleteness to Dataset Finalizer and Workflow Engine gates.
• Prevent training-ready datasets from being built on partially labeled assumptions unless explicitly allowed.

Upstream

• **Data Explorer** provides the synchronized review environment.
• **Data Processing Pipeline** provides canonical episodes, frame indices, and alignment context.

Downstream

• **Data Curation Engine** uses label completeness, class balance, phase tags, and quality signals.
• **Dataset Finalizer** gates snapshot readiness on required annotation coverage.
• **Evaluation & Release** can promote labeled failure windows and scenario tags into replay and benchmark packs.