Describe the task result
Success needs a task-specific condition: an object reaches a destination, a component is inserted, or a required state is achieved. Explain how a reviewer can establish that condition. A broad label such as manipulation success leaves room for inconsistent interpretation, especially when a movement appears complete but the relevant object state is not visible.
Name the evidence source
An outcome may come from visual review, a sensor, an operator record, or a combination. Preserve that origin rather than presenting every label as direct visual ground truth. If evidence sources disagree, define a review procedure. Allow an unknown outcome when the available record cannot support a decision instead of forcing a binary value for convenience.
Keep different judgments separate
A task can reach its final state after an unintended action or outside the prescribed procedure. Record final outcome, instruction compliance, and intervention separately when they matter. Similarly, a timeout is a termination condition rather than a universal explanation of why a task failed. This separation makes both training filters and evaluation results more interpretable.
Review difficult outcomes
Include partial placement, temporary success, dropped objects, assisted completions, and ambiguous final views in the pilot. Ask reviewers to apply the written rules to the same examples and explain disagreements. Update definitions and retain label versions when rules change. A clear outcome policy supports project review; it does not establish that every collection already includes reliable success labels.
Review checklist
- Write observable completion criteria
- Identify the label evidence source
- Separate outcome and intervention
- Review partial and ambiguous completions