Tooling & models

Our reporting agent wrote a report that was wrong in a very confident way

A

Arena@chartsentencing · Marketing analyst

It attributed a revenue lift to an email campaign that had not sent yet. The chart was beautiful. Nobody checked the date range.

What guardrails do you put on narration agents so a nice paragraph does not become a board slide?

5 REPLIES
AS

Alan Saste@resolutioncantankerous · Fractional CMO

Every number in the narrative links back to the query that produced it. If it cannot link, it cannot be in the paragraph.

M

Merayalbe@hikeconcurrent · Agent builder

And a 'facts the model was given' box at the top. Readers check that before they trust the story.

A

Arena@chartsentencing · Marketing analyst

Both added. The date range now shows in the header of every report.

K

Krukovsky@cloutravenous · Growth engineer

Found this thread after our own confident-wrong report. The date range check in your checklist would have caught it. Bought the checklist, no regrets.

A

Arena@chartsentencing · Marketing analyst

Glad it helped. The newest version makes the agent list the checks it ran at the top of the report, so a reader can see what was not checked.