Two techniques this edition, and they belong together: how to brief an AI properly, and how to know whether the thing you briefed actually works.
Brief it like a new intern — don’t prompt it like a search box
If you have rewritten the same prompt four times, the problem usually isn’t the wording — it’s that you’re prompting when you should be briefing. Gabriel Tan, who designs AI workflows for regulated communications agencies, suggests treating the model the way you’d treat a capable new starter. Give it the situation and the outcome you want, not just the task. Then — and this is the step most people skip — ask it to play the brief back in its own words before it drafts anything. The gaps it reveals are the gaps that would otherwise surface as a wrong draft twenty minutes later. Only let it write once the played-back brief matches your intent. For health communications work, where the brief carries regulatory and clinical constraints the model won’t infer, this is less a productivity trick than a quality control step moved to where it’s cheapest: before the drafting starts.
How do you know your AI tool actually works? Borrow NIH’s answer
“It seems good” is not a validation strategy — particularly not in regulated content. Udith Vaidyanathan has distilled the NIH Nature Protocols tutorial on using LLMs in medical research into a workflow any team can run. Build a test set of roughly a hundred real examples from your own work, each with an accepted output, and score the tool against it before trusting it with anything live. Require reasoning before answers, so a reviewer can check how the model got there rather than just whether it landed somewhere plausible. And before anyone mentions fine-tuning, point retrieval at the sources that already govern your content — guidelines and systematic reviews — because most failures are grounding problems, not model problems. The appeal for our field is that this is defensible: when someone asks how you validated the tool in your workflow, “we benchmarked it against a hundred accepted outputs using an NIH-published approach” is an answer that survives scrutiny.
That’s it for this edition. Try the played-back brief this week — it costs you one extra message.
— Ned
Follow the conversation: #IrreplaceablesHealth