Inventory source differences first
List the robots, tasks, observations, action definitions, and timing conventions for every proposed source. Include missing fields and source-specific limitations. Starting with a single output schema can conceal important differences if mapping decisions are made before those meanings are understood. A common file shape is a convenience, not evidence that each collection describes equivalent behavior.
Document every mapping
Record unit conversions, frame changes, component reordering, image transforms, and task-label mappings. Identify values that cannot be represented without assumptions. Preserve original fields or references where practical so conversions remain reviewable. A normalization process should make differences explicit rather than silently replacing unknown meanings with a familiar default that looks valid to a training pipeline.
Keep identity through pooling
Retain source dataset, embodiment, configuration, episode, and version identifiers. These allow problems to be traced and coverage to be analyzed. Inspect which sources contribute which tasks and camera views. Otherwise, an aggregate model result may reflect one dominant collection while hiding a source that has incompatible labels or contributes only a narrow set of episodes. Preserve original task names alongside any common taxonomy.
Validate each source through the final pipeline
Review representative examples after conversion and loading, including missing fields and difficult boundary cases. Keep related copies grouped for evaluation and check source-specific use requirements independently. The Open X-Embodiment project provides a primary reference for cross-embodiment dataset research; a particular buyer’s compatibility and permitted-use decisions still require their own source and project review.
Review checklist
- Create a source-by-field inventory
- Version mappings and assumptions
- Preserve source identity in every episode
- Review converted samples from each source