Multilingual Talking-Head
Verified on-camera speech across 10 languages — face tracks, 478-point landmarks, phoneme-to-viseme timelines, emotion.
See the annotation, live
Real clips with annotation JSON rendered live over the video. Toggle the overlay off to compare against the raw clip — the difference is the product.
Interactive samples for this line are being prepared. The record schema on this page shows exactly what each record contains.
Two layers, scoped separately
The curated base layer is always included. Dense annotation is an optional add-on on the same clips — take depth only where your training recipe needs it.
- Identity-stable face tracks with an active-speaker track id — overlays follow the person who is talking, not every detected face
- 478-point facial landmarks
- ASR transcript with word-level timestamps
- AV-sync score and offset, active-speaker verification
- Explicit verdicts from all seven quality gates stored on every record
- Phoneme timeline (IPA) mapped to viseme classes via multilingual forced alignment, frame-accurate — each record states its alignment backend
- Emotion label and intensity
- Head-pose track (yaw / pitch / roll)
- Speaker turns
Machine-readable contract: th.schema.json
Measured on golden sets
Numbers below come from real annotation runs measured against human golden labels — with sources, not adjectives.
| Metric | Value | Context | Source |
|---|---|---|---|
| Dedup against public corpora | 0 / 3.21M | zero overlap with AVSpeech, VoxCeleb1/2, VoxBlink+2, VoxLingua107, CelebV-HQ, CelebV-Text, TalkingHead-1KH and HDTF manifests | dedup run, August 2026 |
| Gate attribution | 2,205 / 2,515 | every rejection names the gate that fired: geometry 1,726 · sync 1,469 · speaking 1,216 · resolution 1,063 · language 507 · audio 501 | funnel report, July 2026 |
| Golden negatives rejected | 4/4 | poisoned probes caught in acceptance testing — a one-off audit, not a standing control | funnel report, July 2026 |
| Language coverage | 10 × 6 | six gate-passing clips per language (seven for ja) in the current delivery | delivery manifest |
What we tell you before you ask
- ASR quality varies by language and we do not publish a per-language WER, because we have not measured one on a golden set — it is scoped per engagement rather than quoted here. Viseme coverage, which we did measure, ranges from 66% (zh) to 86% (ar) of speech time.
- The seven gates are the code's verdicts on each clip; two of them (audio SNR, AV-sync) depend on the source's own audio quality, so a clip can pass every gate and still be unremarkable footage.
- Clips contain identifiable people. Sourcing screens out minors and synthetic media, and delivery terms for people-centric data are agreed per engagement. The samples on this page come from a dedicated CC BY 3.0 demonstration set — one reference video per language, attributed under the player; the main corpus documents license per record.
Sourced from the open public web across 10 target languages, then screened by the seven-gate cascade; sourcing removes content featuring minors and synthetic media.
License class: Documented per record, person-screened. Per-record provenance — source, license, fetch time, transform hash — ships with every delivery. Methodology →
- WebDataset
- Parquet manifest
- JSON sidecars
- landmark arrays (npz)
Loader snippets for HuggingFace datasets, WebDataset and
PyTorch are in the docs.
Three steps
- 01 Explore the public samples
Right on this page — schema, QA numbers and live viewer where cleared.
- 02 Request an eval batch
A private sample matched to your domain, under NDA, usually within 48 hours.
- 03 License the batch or scope the build
Take the current batch as it stands, or scope volume, annotation depth, license constraints and refresh cadence to your training recipe.