135 rows in gold v1 · 135 ready · 0 not ready
Keyboard shortcuts
Y yes · N no · 1 today · 2 alternative · E own · F flag · Enter save · J/K next or previous row · U undoOnly a member labels. You can read the rows and the result.
Fantasy · 10 labelled, 3 needed · picked automatically, the topic furthest behind · en
Your earlier answers aren't shown (someone labelled this row after you). Answer again and pick the reference to update them.
System prompt · 1 turn · no tools · max 120 tokens · temperature 0
System You review one scene of a script for an Indian audio and micro-drama app before it is recorded or released. You get the show's bible (language, age rating, characters and who is alive, the story so far) and the scene's lines. Answer five questions about the scene. foul: the strongest abuse or profanity in the scene, including disguised spellings: "none", "mild" (everyday insults such as saala, kamina, damn) or "strong". language: true if any line is in a language other than the show's (Hindi shows are in Devanagari Hindi and may use single English words; Hinglish shows are Hindi in Roman script and may use English words and phrases; English shows are English). Another Indian language in the same script counts. context: true if a line contradicts the bible: a character from outside the bible, a dead character speaking, or a fact that contradicts the story so far. artifact: true if the scene contains text that is not dialogue: AI disclaimers, placeholders, notes or instructions to the writer, or a line repeated in a loop. mature: true if the scene shows content above the show's age rating, such as graphic violence in a 13+ show. confidence: your confidence in the whole answer, 0 to 1. Reply with JSON only: {"foul": "none|mild|strong", "language": bool, "context": bool, "artifact": bool, "mature": bool, "confidence": number}
Customer
Show: Midnight Alpha Language: en Rating: 16+ Setting: Silver Creek, a pack town in Oregon Characters: Ava (lead, rejected mate); Lucas (alpha of the Silver Creek pack); Logan (Lucas's beta, died in episode 31, dead) Story so far: Ava has left the pack. Lucas is searching for her before the full moon. Episode: 7 Scene: Lucas: I need the truth, whatever it costs. Ava: Do you remember what happened that night? Lucas: This house isn't what it used to be. Ava: Tomorrow morning everything changes. Lucas: Ava, listen to me carefully. Ava: Don't tell anyone about this. Lucas: I saw it with my own eyes. Ava: Damn it, Ava, not again.
Todayanthropic/claude-sonnet-5.51 word1
{"foul":"mild","language":false,"context":false,"artifact":false,"mature":false,"confidence":0.88}
Alternative—0 words2
None (empty or expired)Check 1 of 5
foul_for_rating required
No abuse stronger than the show’s age rating allows is missed or invented.
Reference
Judge's verdict
Answer every check first. The judge's verdicts show after, so they can't bias you.
Judge check · gold v1 with rubric v1
Judge: openai/gpt-6-sol · your key
| Check | Agreement (κ)How often the judge agrees with your team, beyond chance. 1 is full agreement; 0.80 or more is trusted. | Labelled | Trust |
|---|---|---|---|
foul_for_rating | — | 0 · 0 Y, 0 N | Needs 30 more labels |
mixed_language | — | 0 · 0 Y, 0 N | Needs 30 more labels |
mixed_context | — | 0 · 0 Y, 0 N | Needs 30 more labels |
generation_residue | — | 0 · 0 Y, 0 N | Needs 30 more labels |
above_rating | — | 0 · 0 Y, 0 N | Needs 30 more labels |
A check is trusted at κ 0.8How often the judge agrees with your team, beyond chance. 1 is full agreement; 0.80 or more is trusted. or more on 30 or more labelled rows, with at least 5 Yes and 5 No. A fifth of the rows is held out of every rubric fix; κ more than 0.1 lower there marks the check possibly overfit.
Locked: gold v1 with rubric v1 by ops.member · 20 Oct · 135 of 135 rows verified
Instructions for Dvar (optional)
None saved.
Rubric v15 checks by ops.member · 20 Oct
foul_for_ratingNo abuse stronger than the show’s age rating allows is missed or invented.
mixed_languageA line in another language or script is flagged, and none is invented.
mixed_contextA character from another show, a dead character or a contradiction of the story so far is flagged.
generation_residue“As an AI…”, template slots, notes to the writer or loops are flagged.
above_ratingContent above the show’s age rating, such as graphic violence in a 13+ show, is flagged.
Version historyv1 approved by risk.member · 20 Oct
Rubric v1by ops.member · approved by risk.member · 20 Oct
foul_for_ratingNo abuse stronger than the show’s age rating allows is missed or invented.
mixed_languageA line in another language or script is flagged, and none is invented.
mixed_contextA character from another show, a dead character or a contradiction of the story so far is flagged.
generation_residue“As an AI…”, template slots, notes to the writer or loops are flagged.
above_ratingContent above the show’s age rating, such as graphic violence in a 13+ show, is flagged.