Moderation pipeline
Implementation: src/server/services/moderation.ts. Console: /admin. Gates: /admin/readiness.
The governing principle
Never report a check that did not run as passed.
This is the most important rule in the codebase. A moderation system that returns "clean" for a scan it did not perform is worse than no moderation system, because it manufactures false confidence and someone downstream will rely on it.
Concretely:
screenPublicationreturnscoverage: 'metadata_only'when classifiers are absent, with acoverageDetailstring naming what did not run.- Missing classifiers emit a
human_reviewflag labelled "… did not run", not silence. csamHashCheckthrows rather than returning a clean result when unconfigured.
Stage 0 — automated pre-screen
Runs on every publish path, internal and external.
| Check | Runs without a provider? |
|---|---|
| Text screening (title, synopsis, all dialogue) | Yes |
| Consent ledger (cloned voices, likeness) | Yes |
| Technical QC (resolution, zero-byte, duration) | Yes |
| Visual safety (nudity, violence, hate) | No — MODERATION_VISUAL_PROVIDER |
| Audio safety | No — MODERATION_AUDIO_PROVIDER |
| Copyright / trademark fingerprinting | No — CONTENT_FINGERPRINT_PROVIDER |
| Real-person likeness / deepfake | No — LIKENESS_DETECTION_PROVIDER |
| CSAM hash-matching | No — CSAM_HASH_PROVIDER |
Flag actions: block (hard stop, approval disabled in the console), human_review, note.
Text screening is deliberately narrow and high-precision. Broad keyword nets generate noise, and reviewers who click through noise stop reading — which is strictly worse than fewer, better flags.
CSAM
csamHashCheck has exactly two honest outcomes: a provider ran and returned a verdict, or no provider is configured and the content is unscreened. There is no third "probably fine" branch, and it is not stubbed with a fake negative.
A match must trigger: immediate block, preservation of content and account data for law enforcement, and an NCMEC report. launchReadiness() makes this a blocking launch check.
Stage 1 — human safety review
Priority queue: 50 − trustLevel × 10 + strikes × 25, ascending. Trusted creators are fast-tracked; struck accounts sink.
Console features: Stage 0 flags with confidence and jump-to-timestamp, timestamped reviewer notes, reason codes, rating changes, and the two-reviewer rule for borderline content (any flag with confidence 0.4–0.85, or any creator with an active strike). The first decision on such a case records but does not finalize.
Decisions: approve, approve_with_rating_change, request_changes, reject (issues a 180-day strike), escalate.
Approval is disabled in the UI while any Stage 0 blocking flag stands. Blocking flags are not advisory.
Stage 2 — quality bar
Deliberately separate from safety. A film can be entirely safe and still not belong in the catalog.
Rubric — visual, audio, continuity, story, technical — computed from data we own: generation quality scores, the latest Film Audit, scene structure, and asset resolution. Thresholds in QUALITY_THRESHOLDS.
Quality rejection is not a ban. A film below the bar becomes unlisted with a shareable slug, and can still be published to external platforms. That distinction is what lets the catalog stay curated without the platform being punitive.
External publishing
Part 6.5: everything leaving the platform passes Stage 0 even when it skips Theater review. Under each platform's API terms you are the publisher of record via the app's credentials. queuePost calls screenPublication and refuses on a block.
Launch gate
launchReadiness() turns Part 9 into code. Blocking checks: CSAM hashing, visual classifier, copyright fingerprinting, DMCA agent registration, media liability insurance. Non-blocking: audio classifier, likeness detection, C2PA signing certificate.
With a default .env, five blocking checks fail and the function reports ready: false. That is correct, and it is surfaced on the Studio dashboard rather than buried.