Vision AI · Policy · Human Review
Every album cover uploaded to a music platform ships straight to listeners. You can't review them all by hand, and you can't auto-approve the ambiguous ones. I built a two-stage pipeline that spends money only where the answer is actually in doubt.
The Problem
Album artwork is user-generated content with no gap between upload and audience. A platform can't put a human in front of every cover, and it can't send every cover to an expensive model either — most of them are obviously fine, and paying to think hard about obviously fine images is how a moderation budget disappears.
But the interesting part isn't detection. It's that the two ways of being wrong don't cost the same. A false approval puts harmful content in front of users — an app store violation, brand damage, a regulatory problem. A false rejection just adds an image to a review queue and annoys one artist. Those are not the same mistake, and a system that treats them as symmetric is mistuned by design.
Users & What I Learned
Doesn't want a verdict handed down with no reasoning. Wants to open a queue item and see what the model saw, which policy line it applied, and what the argument on each side was — then make the call quickly and move on.
The obvious cases don't need AI judgment, they need routing. All of the cost, all of the risk and all of the design work live in the band where the model isn't sure — so that band is where I put the expensive machinery.
This reframed the whole build. Moderation reads like a detection problem — get the classifier right. It's actually a routing and cost problem: decide which cases deserve expensive thinking, decide where the line sits, and be honest that where you put the line is a risk decision, not a technical one.
Decisions I Made and Why
A single careful pass over every image would be accurate and unaffordable. A fast vision pass scores each cover across six policy categories; anything clearly clean or clearly in breach exits immediately and costs nothing further. Only the middle band reaches the second agent — which is what makes the second agent affordable at all.
The score is confidence that an image breaks policy, so a 10% approve threshold means requiring 90%-plus confidence that a cover is clean before it passes unattended. The images in between aren't rejected — they get a second look. Tighter costs more, because more images reach the expensive path. That's the trade the dial makes, and it should be set per platform rather than baked in.
The second agent reads the policy document, then works through five fixed steps — observation, policy check, the case for approval, the case for rejection, verdict. Without the forced counter-argument it just agrees with the first pass. It runs at temperature 0 so the same image always reaches the same verdict, and if its output can't be parsed it retries once and then hands the image to a person instead of guessing.
Outcomes
The result I'm proudest of is a bug I found in my own system. An illustrated cover scored 10% and auto-approved — the model was treating painted artwork more leniently than photography. I root-caused it, revised the prompt so artistic medium doesn't reduce severity, re-tested at 40% where it correctly routed to human review, and promoted the fix into the policy document as a standing principle.
Where the Lines Sit
The first pass returns a 0–100 score: its confidence that the image breaks policy. Three bands, two thresholds, and the thresholds are deliberately asymmetric.
2-Agent Moderation Pipeline
Most images never reach the second agent — that is the point.
Try the Demo
Pick a cover and watch it move through the pipeline — the fast scan, the routing decision, and, if it lands in the gray zone, the deliberation agent arguing both sides before it rules.
These four covers are invented artwork and invented artists, and the scores and reasoning are scripted to mirror what the real system produces — this is a walkthrough of the pipeline, not a live model call. The working portal runs locally against GPT-4o mini Vision.
What This Demonstrates
Two-tier routing is a product decision before it's an engineering one. The measured $0.002 / $0.007 split is the whole argument for building it this way.
Asymmetric thresholds, set per platform rather than baked in. Naming which mistake you'd rather make is the part most moderation demos skip.
Found my own false negative, root-caused it to how the model read illustrated art, fixed the prompt, re-tested, and wrote the fix into the policy.