Captions, dubbing, voices and the product page

Four features, all built and verified.

1. Closed captions

Where: the CC control under the player. Try it on /theater/watch/… for a title with tracks — npm run seed:captions creates one with English, Spanish, Japanese and Arabic.

Thirty languages

src/lib/languages.ts is the table. Each entry carries more than a label, because a label is not enough to render a subtitle:

• nativeLabel — a Japanese viewer scans for 日本語, not "Japanese". Both are

kept: the English name for creator tooling, the endonym for the viewer picker.

• rtl — Arabic, Urdu and Persian. Not cosmetic: a caption layer that

hardcodes left-to-right renders Arabic with its punctuation on the wrong end and its clauses reordered. It looks like a font bug and is unreadable text.

• script — drives the font stack. A caption in a font with no glyph for its

script renders as a row of boxes, which reads as a broken player.

• lipSync — whether the phoneme model covers it. Sixteen of thirty do.
• avgCharsPerSecond — reading speed, for the fit check below.

Ordering is by streaming audience, not alphabetical, so a viewer finds their language near the top of the picker without scrolling.

Colour, size and background

White and yellow lead — decades of broadcast convention. Four more exist because a fixed colour is an accessibility failure: yellow over a bright scene is invisible.

Every colour carries a contrasting outline, drawn with paint-order: stroke fill so the stroke sits behind the glyph rather than eating it. Without an outline a caption vanishes the instant the frame behind it matches its fill, and films are not shot to accommodate subtitles.

A live sample sits in the menu over a dark-to-light gradient, so a colour can be judged before it is committed to.

Why the captions are drawn, not handed to <track>

A native <track> would be simpler and is not stylable in any cross-browser way: ::cue colour support is uneven, size is uncontrollable in Safari, and position is the browser's decision. Since the request was "let the viewer choose the colour", the captions have to be ours to draw.

The WebVTT endpoint still exists — ?format=vtt — for native players and for export to an editor.

Where a track comes from

Three sources, one shape, and a viewer cannot tell them apart:

1. The original language, derived from dialogue and overlaid with hand edits.
2. A translated language, from the rows a dub produced. Translating for a dub

gives you subtitles for free.

1. A hand-authored track in a language with no dub at all.

Translated cues inherit the ORIGINAL cue timing — the picture has not moved. Matching is by stored order, which is why DialogueTranslation.order exists: relying on insertion order would offset the whole track from the first mismatch onward, which reads as subtitles drifting out of sync partway through a film.

Preferences

localStorage immediately (so the choice applies on the first frame and works signed out) and on the account when signed in (so it follows to another device). A language the current title lacks shows as Off rather than a selection that silently renders nothing.

2. Making the film again in another language

Where: Studio → a project → Dub. Requires a published cut.

Two products, not one setting

Localization.fullRender records which was produced and the UI says so. A dub sold as a re-render would be discovered by the audience rather than the creator.

Why this is possible here at all

Most platforms hold a finished video file, so pasting new audio over old picture is the only thing they can do. Firefly Reels holds the script, the cast's voice profiles and the shot graph — so a dub is a re-render, in the same voice per character.

Refusals and downgrades, decided before anything runs

• No lip-sync coverage → downgraded to an audio dub with the reason stated. A

full re-render would regenerate every shot at full cost and produce the same mouths.

• No translation model → refused outright. Shipping source dialogue under a

translated label is worse than an error.

• Timing overrun → reported. A line 30% longer in German either overruns its

shot or must be rushed until it cannot be followed. translateForDub gives the model a per-line maxSeconds budget, and the plan lists the risky lines.

Batched translation, and a partial result is a failure

Translation runs in batches of 60. Two hundred lines in one request risks truncation at the token ceiling, and a truncated JSON array loses every line after the cut — a dub that silently stops being translated halfway through.

If any line comes back untranslated the whole dub is refused rather than mixing languages.

Resumable

A five-language release is thousands of generations over hours. Dubs run on the A1.8 pipeline: per-shot checkpoints, lease-based scheduling, and a resume that runs only the gap. A deploy at hour four does not discard four hours of paid work.

3. The Voice Studio

Where: Studio → Voices.

Four routes in — library, describe, upload, record — all ending at an audition.

Two gates, both enforced in the service

castVoice refuses, rather than warns, in two cases. The gates live in the service and not the UI because the UI is not the only caller: the Director and the API can both cast a character.

1. A cloned voice without verified consent. Cloning a real person's voice is

the one thing in this product that can harm someone who never used it.

1. A described voice nobody has auditioned. "Warm" means something different

to the writer and the model, and the gap is invisible until you hear it. Casting a whole film in an unheard voice means finding out after two hundred lines of dialogue.

Consent is re-checked at use, not only at assignment — voiceRefsForCharacter verifies it every time a shot is composed, so revoking consent stops the voice reaching a provider immediately rather than at the next cast edit.

Recording your own voice is self-attested: the record still exists for the audit trail, but requiring someone to formally consent to themselves is friction with no protective value.

Describing a voice

The description is parsed into traits (gender, age, accent, pitch, pace, texture, energy) and shown back for confirmation. What the parser did not classify is still passed to the engine verbatim — dropping it would discard the detail the writer cared enough to type.

Recording in the browser

MediaRecorder, with echoCancellation, noiseSuppression and autoGainControl all off. Those flatten exactly the texture a clone needs. The read-aloud passage has varied vowels, a clipped line and a warm one on purpose.

Duration is measured client-side before upload, because the server cannot decode audio without a media library — and duration is the single best predictor of clone quality.

The refresh button does something

previewSeed increments on every audition. Without it the request dedupes against the previous identical one and returns the same audio — a refresh button that demonstrably does nothing is worse than no refresh button. Previous takes are kept so they can be compared.

Voices reach the video model, not just the TTS pass

voiceRefsForCharacter returns the sample as a role: 'performance' audio reference, and composeShotPrompt attaches it to every shot the character is in.

Models with native audio — Seedance, Veo, Kling — generate the speech as part of the shot. The mouth and the voice come out of one generation, so they cannot disagree: the mismatch is not corrected, it never occurs. Models without native audio ignore an audio ref, so it is safe to attach unconditionally and the Router picks the path.

4. The product page

Where: /product, linked from the landing hero, the landing grid and the nav.

src/lib/product/catalog.ts holds 94 features across 10 areas, one sentence each.

It is data rather than markup because three surfaces need the same list and must not disagree, and because data is testable. verify-languages.ts asserts:

• Every href resolves to a real route (checked against the filesystem).
• Every /help/… link resolves to a real article.
• Blurbs stay under 160 characters — a paragraph in a box is a paragraph nobody

reads.

• Standouts are a minority. Marking everything unique marks nothing unique.

Where a feature needs a credential, needs says so and the page renders it. The /admin/activation inventory remains the live answer for a particular install; this is the catalogue of what exists.

Bugs found and fixed during this work

A test that deleted data it did not create

verify-v27.ts finished its caption block with captionCue.deleteMany({ where: { projectId } }) — every language, not just the English cues it had written. It destroyed the seeded demo tracks on every run.

It now snapshots and restores, and asserts it left no trace.

An assertion that depended on fixture state

The same block asserted "no derived cue is marked edited", which held only while nothing had hand-authored a track. It now compares edited against the stored overrides, which is the actual behaviour rather than an assumption about the fixture.

A screen reader announced every language as selected

The tick in the caption menu was rendered for every item and merely made transparent, so innerText — and assistive technology — saw a checkmark on all of them. Now role="menuitemradio" with aria-checked, and the glyph is aria-hidden.

src="" re-downloaded the whole page

The player rendered <img src={... || ''}> when a title had no poster and no render. An empty src resolves to the current page URL and downloads the entire document again; React warns about it explicitly. It now renders an empty frame.

"Speaks slowly" parsed to nothing

The voice trait regex had \bslow\b, which does not match "slowly" — the word boundary falls after "slow". The most natural phrasing for pace was silently dropped, and the creator got a default-pace voice with no indication their instruction had been ignored.

Verification

npm run verify:lang    # 124 assertions
npm run verify         # 1292 across 12 suites
npm run smoke          # routes, content, mobile, agent API against a live server
npm run seed:captions  # a title that can actually demonstrate captions

What verify-languages.ts actually asserts, beyond the obvious:

• Arabic, Urdu and Persian are marked RTL, and WebVTT emits the RTL mark.
• Reading rate genuinely differs by language (the fit check is otherwise fake).
• A cue below the minimum on-screen time is rejected however short its text.
• An unknown colour, size or language falls back rather than rendering undefined.
• A language with no track returns null, not an empty track.
• A language without lip-sync coverage is downgraded, and the reason is given.
• A full re-render costs more and takes longer than an audio dub.
• Multi-language hours are summed, not maxed.
• A described voice cannot be cast before audition; a cloned one cannot be cast

without consent; an unconsented voice never reaches a provider.

• A refresh advances the seed, so it is a new take rather than the deduped one.
• Another account cannot see this user's cloned voices.
• Every product-page link resolves.