What to log when an AI rewrite fails QA before send
A human rejects an AI rewrite. The draft dies in the review queue. Nobody writes down why.
Next week the same failure ships again. Or worse: someone force-approves a cousin of the rejected line because the audit trail is empty.
If you run supervised send at agency scale, QA failures are training data. Log them or you are paying humans to leave no memory.
Why the log exists
IBM's human-in-the-loop framing is useful here: keep people where ambiguity and edge cases still bite. That only works if the human decision is captured. A silent reject teaches the system nothing and teaches the next reviewer nothing.
Buyers who ask for an audit trail before trusting AI send do not want a vibes memo. They want timestamps, reasons, and which brief was live.
MailReach and ActiveCampaign treat deliverability as a checklist sport. Pre-send QA is the content twin of that checklist: catch bad claims and bad links before they become placement problems.
Minimum fields on every QA failure
Log these every time a reviewer blocks send:
- Timestamp (send-window timezone + UTC)
- Client workspace + campaign + sequence step
- Prospect id (not just email) and data sources used in the write
- Voice brief version / ICP snapshot pinned for that draft
- Model or agent id that produced the rewrite (no secrets in the log)
- Rejected draft hash or immutable snapshot (what they saw)
- Failure reason code (controlled vocabulary — see below)
- Free-text note (one or two sentences; optional but useful)
- Reviewer seat and whether supervised vs attempted autopilot
- Disposition — rewrite again, suppress prospect, escalate to client, kill step
Without reason codes, your "log" is a novel. Without the draft snapshot, disputes become he-said-she-said.
Reason codes that stay useful
Keep the list short and enforceable:
| Code | Meaning |
|---|---|
HALLUCINATED_FACT | Claim not in prospect evidence or brief |
VOICE_MISMATCH | Tone or phrasing breaks client voice brief |
DO_NOT_SAY | Hit legal/brand/competitor landmine |
CTA_POLICY | Booking link or ask violates step policy |
COMPLIANCE | Missing unsubscribe, bad verdict, bad identity |
STALE_DATA | Enrichment too old or job-change unresolved |
WRONG_ICP | Prospect should not have entered campaign |
FORMATTING | Broken tokens, bad merge, unreadable mess |
Amazon SES multi-tenant guidance stresses isolation so one noisy tenant does not poison the pool. Logging WRONG_ICP and COMPLIANCE per client workspace is how content QA stays isolated too.
What you do with the log each week
- Top reason codes → fix brief, do-not-say list, or write prompts
- Same step failing repeatedly → kill or rewrite the step (not the reviewer)
- Same prospect pattern → tighten campaign entry rules
- Autopilot candidates → only graduate when failure rate is quiet for the right reasons
ProspectingManual's multi-client scaling notes keep returning to repeatable SOPs. A QA failure log is that SOP for the write step.
What GTM expects you to keep honest
GTM writes from prospect evidence, keeps a compliance gate in the database (no send without verdict + unsubscribe URL), and supports supervised vs autopilot sending for agency teams. Logging QA failures is how supervised mode improves instead of becoming theater.
If you are building that motion now, join the waitlist.