Egocentric How-To & Manipulation Video
First-person how-to video, curated for hands-on manipulation density and annotated for action understanding.
See the annotation, live
Real clips with annotation JSON rendered live over the video. Toggle the overlay off to compare against the raw clip — the difference is the product.
Interactive samples for this line are being prepared. The record schema on this page shows exactly what each record contains.
Two layers, scoped separately
The curated base layer is always included. Dense annotation is an optional add-on on the same clips — take depth only where your training recipe needs it.
- Viewpoint class (head_cam / hands_only_static) with continuous egocentric score
- Domain classification and manipulation-density metrics (hand presence, sustained work runs)
- Per-record provenance manifest: source, license, fetch time, transform hash
- Dedup and leakage screening against 2.02M+ video IDs from COIN, CrossTask, YouCook2, HowTo100M, HTM-AA, VidChapters and other public corpora
- Two-level temporal structure: procedure steps + atomic action segments
- Action verbs mapped to the EPIC-100 taxonomy (97 verbs / 300 nouns) with confidence
- Per-segment captions and ASR with word-level timestamps
Machine-readable contract: annotation.schema.json
Measured on golden sets
Numbers below come from real annotation runs measured against human golden labels — with sources, not adjectives.
| Metric | Value | Context | Source |
|---|---|---|---|
| Viewpoint accuracy | 1.0 | 25/25 on the delivered set with the shipping stack; an earlier v1 debug run scored 0.941 on 17 clips | annotation run, August 2026 |
| Action-segment density | 9.64 / min | measured on the delivered set: 858 segments, 112 distinct verbs, 2.9% vague | annotation run, August 2026 |
| Step coverage | 0.99 | no clip below 0.7, no uncovered tail longer than 15 s | annotation run, August 2026 |
| ASR agreement (WER) | 0.145 | median against reference subtitles, 12 clips | internal debug run, July 2026 |
| Leakage screening | 2.4% | of candidates removed as overlap with COIN / CrossTask / YouCook2 / HowTo100M / HTM-AA / VidChapters IDs | curation funnel report |
What we tell you before you ask
- Action-segment density is 9.64 segments/min on the delivered set — comparable to reference egocentric corpora, but below EPIC-100's expert density of ~15/min.
- Auto-annotation is measured against golden labels and human-QA'd; it is model output, not frame-by-frame human labeling. Where the taxonomy has no honest match, the record abstains rather than guesses — 23.7% of verbs are marked out-of-vocabulary.
- The reference batch is 27.2 h across 500 clips; volume, domain mix and annotation depth are scoped per brief rather than taken off this shelf.
- Source licenses are mixed and documented per record; license filtering to your requirements is part of every delivery.
Sourced from the open public web through a staged curation funnel — channel-level discovery, metadata and frame-probe filters, embedding-based viewpoint checks, full download of the shortlist only, lossless cutting — followed by GPU annotation and human QA. Cheap metadata, thumbnail and embedding probes filter every candidate first: of 3,479 videos discovered, 732 were ever fetched at full resolution.
License class: Mixed, documented per record. Per-record provenance — source, license, fetch time, transform hash — ships with every delivery. Methodology →
- WebDataset (verified on delivery)
- JSON provenance manifest
- LeRobotDataset v3 — on request
- RLDS / GR00T export — on request
Loader snippets for HuggingFace datasets, WebDataset and
PyTorch are in the docs.
Three steps
- 01 Explore the public samples
Right on this page — schema, QA numbers and live viewer where cleared.
- 02 Request an eval batch
A private sample matched to your domain, under NDA, usually within 48 hours.
- 03 License the batch or scope the build
Take the current batch as it stands, or scope volume, annotation depth, license constraints and refresh cadence to your training recipe.