Problem
Campaign ops is still manual work.
4–6 hrs
average time to set up a single programmatic campaign end to end
23%
of campaign launches delayed due to trafficking or setup errors
60%
of ops team time spent on repetitive setup tasks vs strategic work
In programmatic advertising, translating a client brief into a live campaign requires a surprising amount of manual, error-prone work. A trader or ops manager receives a brief (often unstructured, via email or a shared doc) and must manually parse it, set up line items in the DSP, configure targeting, apply brand safety rules, set pacing and budget caps, and hand off trafficking instructions. Each step is a potential point of failure.
The cost isn't just time. Trafficking errors and misconfigurations directly impact campaign performance and client trust. And with ops teams stretched thin, strategic work gets deprioritized in favor of setup and QA.
My mandate: design an AI agent that takes a campaign brief as input and produces a structured, validated campaign setup, with a human-in-the-loop approval gate before anything goes live.
Discovery & Research
Where the workflow breaks down.
I mapped the end-to-end campaign setup workflow across a typical DSP environment, identifying where time is lost and where errors originate. Two artifacts anchored the discovery.
Current State: Campaign Setup Workflow
01
Brief receivedUnstructured email or doc from client or sales team
15 min
02
Brief interpretationOps manually parses objectives, targeting, budgets, flight dates
45 min
03
DSP campaign buildManual line item creation, targeting configuration, bid strategy
90 min
04
Creative traffickingAssigning creatives, verifying specs, setting click URLs
45 min
05
QA & reviewManual checklist verification before launch
60 min
06
LaunchCampaign goes live. Errors often surface post-launch.
15 min
Not actual artifact, generated for portfolio purposes
| Error Type | Frequency | Avg Impact |
|---|
| Wrong targeting params | 34% | High |
| Budget misconfiguration | 22% | High |
| Creative spec mismatch | 19% | Medium |
| Flight date errors | 14% | High |
| Brand safety gaps | 7% | Critical |
| Other | 4% | Low |
| Total avoidable errors | 96% | — |
Not actual artifact, generated for portfolio purposes
The key finding: nearly all errors stem from the brief interpretation and manual data entry steps, which are exactly the steps an AI agent can own. The human judgment required is concentrated in the approval gate, not in the setup itself.
AI Product Design
Designing the agent architecture.
This project required thinking through AI agent design end to end: not just what the model does, but how it is structured, where humans stay in the loop, and how outputs get validated before anything consequential happens.
Why an agent, not a single prompt
A single LLM call would produce unstructured output with no validation layer. The live prototype gives Claude five tools and lets it decide which to call and in what order: check_duplicate_campaign (flags overlapping bookings for the same client), check_inventory_feasibility (checks whether budget and targeting are actually supportable), check_brand_safety (a hard stop on policy conflicts), request_clarification (halts and asks a human instead of guessing at missing fields), and finalize_setup (the only path to the human approval gate). request_clarification and finalize_setup are not separate mechanisms bolted onto the loop. They are tools the model itself chooses to call, exactly like the three validation checks. Testing confirmed Claude calls the validation tools in varying order depending on the brief, and only reaches finalize_setup once every check has passed. This mirrors how a human ops team works, and makes the system auditable: every tool call and result is logged and visible before the human approval gate.
Model selection
Claude Sonnet for its strong instruction-following on structured output tasks and its ability to handle the domain-specific language of programmatic advertising (CPM, CTR, viewability thresholds, frequency caps, deal IDs). The model is prompted with a system context that grounds it in DSP workflow conventions.
Human-in-the-loop design
The agent produces a complete campaign setup proposal but does not submit it. A human approval gate sits between generation and trafficking. The ops manager reviews the structured output, can edit any field, and explicitly approves before the system hands off to the DSP. This is a core product decision: AI owns the setup, humans own the launch decision.
Structured output & evals
Rather than parsing free text into sections, the agent calls discrete tools (check_duplicate_campaign, check_inventory_feasibility, check_brand_safety) and only reaches finalize_setup once those checks pass. Testing confirmed two specific failure-handling behaviors: the agent correctly calls request_clarification rather than guessing when a brief omits required fields like budget or flight dates, and it enforces a hard retry limit rather than looping indefinitely. In one test run, it retried an inventory check twice with adjusted targeting before escalating to a human rather than continuing to retry.
Cost & infrastructure
A single campaign generation uses approximately 3,000 input tokens + 2,000 output tokens at Claude Sonnet pricing. Estimated cost per campaign setup: ~$0.03. At 500 campaigns/month for a mid-size trading desk, that's ~$15/month in inference cost, a 99% reduction vs the ops labor cost it replaces. Infrastructure mirrors
NutriCoach: Cloudflare Worker proxy for secure API access.
Risk & guardrails
The system is explicitly designed to never auto-submit to a DSP. All outputs are read-only proposals until human approval. Brand safety rules are injected as hard constraints in the system prompt. Budget caps are validated against client contract values where available. Any missing required field triggers a QA flag rather than a best-guess fill.
What I found while building this
Real-world testing surfaced a genuine bug, not a hypothetical one. When a brief gave dates without a year (e.g., "11/15 - 12/31"), the agent's date parsing defaulted to the year 1970, which caused check_inventory_feasibility to report valid inventory as infeasible since the requested window never overlapped with any real inventory record. The fix was an explicit instruction anchoring date resolution to the current year whenever a brief omits one. This is the kind of failure mode that only shows up when you actually run a system against real inputs rather than reasoning about it on paper, which is exactly why testing the live agent mattered more than writing the architecture diagram.
Verification, not just design
All five tools now have confirmed positive-path test evidence, not just a designed-but-unverified schema. check_duplicate_campaign correctly flagged a true duplicate when tested against a brief matching the mock Acme Retail campaign's client and overlapping dates. check_inventory_feasibility and check_brand_safety were each confirmed returning real conflicts and clean passes across multiple briefs. request_clarification was confirmed halting the loop on a brief missing budget and flight dates. finalize_setup was confirmed only firing once every check passed. The distinction matters: a tool schema that looks correct on paper and a tool confirmed to behave correctly against a real test case are not the same claim, and this case study only makes the second one.
Strategy & Architecture
The agent pipeline.
The strategy centers on one core insight from the discovery: human judgment is required at the approval gate, not throughout the setup process. The agent owns everything before that gate; the human owns everything after.
- Brief intake. Free-text brief input that can be pasted directly from an email, a shared doc, or a client conversation. No structured template required on input.
- Agent reasoning loop. Claude receives the brief along with all five tools and decides which to call, in what order, and whether to escalate or finalize. This is a real multi-turn tool-calling loop, not a scripted sequence. Testing confirmed the agent calls the validation tools in varying order depending on what the brief actually requires.
- Conflict resolution or escalation. If a check returns a partial or infeasible result, the agent can retry with adjusted parameters (e.g., a different targeting segment) up to a bounded limit before escalating, or call request_clarification if required information is missing entirely. It never loops indefinitely and never guesses at missing fields.
- Hard stop on brand safety. If check_brand_safety returns any unresolved conflict, the agent is instructed to never call finalize_setup, regardless of how clean every other check is. This is enforced as a rule, not a suggestion.
- Human approval gate. Once finalize_setup is called, the ops manager reviews the structured setup across 5 tabbed sections, including the agent's own qa_notes documenting any judgment calls it made. Any field can be edited. Nothing is submitted to the DSP without explicit sign-off.
- Trafficking handoff. Approved output is formatted as a structured trafficking spec ready for DSP import or manual entry.
Agent Architecture — Multi-Turn Tool-Calling Loop
Input
Campaign Brief
Free-text, unstructured
↓
↓
↓
↓
↓ branches to one of two tools ↓
↓
Human Gate
Ops Manager Review
Reviews, edits, and explicitly approves before any DSP action. Only reached after finalize_setup is called.
↓
Output
Trafficking Spec
Structured handoff ready for DSP import
Not actual artifact, generated for portfolio purposes. All 5 tools, including both true-positive and request_clarification/escalation paths, confirmed by direct testing rather than designed and assumed to work.