How do you test a marketing agent before it touches a real campaign?
M
Merayalbe@hikeconcurrent · Agent builder
I keep a set of 40 real briefs with known-good outputs and run every prompt change against them. It catches regressions but it is slow to maintain. What does everyone else do — evals, shadow runs, or just careful review?
3 REPLIESTJ
Theo Jansen@silverdrift · Growth engineer
Shadow runs. The agent produces output alongside the human for two weeks; we diff them. Cheaper than a test set and it catches the weird cases.
A
Arena@chartsentencing · Marketing analyst
Both. The test set for regressions, shadow runs for the unknown unknowns.