ContentRoofLab

ContentRoof Lab · Research notes

Notes from inside a video engine

ContentRoof turns one sentence into a narrated, animated video in eleven languages. These notes are written by the people who build it: the problem, what we tried, what failed, what we measured, and where it still falls short.

4 notesTranslationSpeechEvaluationRendering

3D simulation · the heart
A 3D simulation of a jet engine's combustor3D simulation · a jet engine
An Arabic explainer about volcanoesExplainer · Arabic
A frame of a cinematic story about the dog HachikōCinematic story
Specimens. Frames from videos our engine made from a single sentence each. Every picture in these notes comes from our own videos.
04 Translation5 Nov 20264 min A billion is not a billion in Spanish Spanish billón, German Billion and French billion mean a million million. How we check every number when a video is rebuilt in another language: 8 of 8 planted errors caught. 8 of 8planted number errors caught 03 Speech5 Nov 20263 min Sixty seconds is a different length in every language Asked for 60 s, our videos came out 44–105 s. Measuring each language's voice per letter now lands 43–62 s in eleven languages, and a cloned voice took Arabic errors from 10% to 0. 44–105 s → 43–62 sasked for 60 s, in eleven languages 02 Evaluation5 Nov 20264 min Balanced benchmarks hid a 95% error rate Our sentence classifier scored 79–95% on balanced tests. On real narration it gave 95% of plain statements a shape they don't have. What we measured and what fixed it. 5% → 88%plain statements left alone 01 Rendering5 Nov 20264 min We took the browser out of video export Our exports spent 80% of every frame waiting for Chrome's screenshot. We draw the same React components with Skia instead: 2.4× faster on a server, 5× on a laptop GPU. 2.4×faster export, the same frames
  1. Exports carry on while ContentRoof updates
  2. 3D Simulations: a new camera angle on every moment
  3. Cinematic stories: every shot moves in its first second
  4. Arabic stories keep their pictures
  5. More than 300 explainer animations, picked by what each sentence says