In government communications, clarity, accuracy, and trust are paramount. Running a close second: efficiency – giving federal communicators good reason to consider whether AI is a viable tool.
At Spire, we wanted to see for ourselves what AI could do for our clients. To get a sense of AI’s strengths and weaknesses, we used a publicly available large language model (LLM) to source information for an ebook about responsible AI for government communicators.
The result left us feeling optimistic. But we also experienced firsthand the very shortcomings our AI assistant warned us about, and our content strategists and editors had to intervene.
The story behind our ebook demonstrates the vital role of human oversight in producing quality content.
Early experiments and tempered enthusiasm
My experiments with AI date back to 2012. I’ve used tools to generate content, analyze voice tracks, and categorize content.
I’ve experienced the drawbacks, and I understand the risks. I see the same headlines our clients do – Google Bard hallucinating in its own promo video, ChatGPT putting bogus citations in a legal brief, educators scrambling to AI-proof their classes.
And then there’s bias. Even AI seems to know its potential for harm on this front.
Still, my enthusiasm for generative AI is rooted in its transformative potential to empower even the smallest teams to communicate more efficiently.
Can AI write a good first draft?
Every writer knows that a good first draft helps establish mutual trust and confidence with SMEs and can save a ton of time in review cycles. If generative AI can produce solid draft content for government agencies, that’s a strong value proposition.
To hedge my bet, I used Claude, an LLM explicitly marketed as being trained with ethics, non-discrimination, and inclusivity in mind.
What does a strong prompt look like?
A good content-generation prompt follows a structure and provides key inputs– including what to do and what to avoid. Bots often need follow-up prompts to refine the first draft. Specificity is critical, but including open-ended language can guide the bot away from self-imposed restrictions.
To illustrate this, here are the prompts I used to guide Claude’s first draft, along with a text file with the outline:
Priming and context
I’m writing a blog article for the agency website https://spirecomm.com/news-and-insights/ and need help producing my first draft. See below for details. Working Title: AI for Government Comms
Complexity
Audience: Professionals working in US government organizations such as communications managers, website managers, etc.
High-level summary
Topic: While it’s tempting to look at the cool things AI can do for businesses, applying AI for government comms requires additional consideration and planning to make sure you’re staying compliant. Here are some things to consider before you get too excited about integrating AI tools into your content workflow.
Tone
Tone: Professional and authoritative, yet approachable. Written by an expert who is so well-informed they can casually speak about a topic in a way that feels fresh, insightful, and exciting. But not overly casual or full of slang that would be inappropriate for a government employee to read at work.
Structure
Here is the outline I’d like you to turn into a draft of the article: [uploaded outline]
Claude’s first draft aligned well enough to my outline, but the treatment – only 400 words – was thin. The bot also got the perspective wrong, writing as if it were an agency employee, not an external strategist who specializes in government communications.
So I provided additional feedback in a series of follow-up prompts:
Correction
The article length should be 1000 to 1500 words. Go add more information about each of the topics.
Areas for expansion
Feel free to expand on some areas as you see fit. Give examples or case studies of how AI is being used, especially in ways that might apply in government spaces.
Point of view
Write from the point of view of an outside vendor/agency advising government leaders, not as the government itself.
Within seconds, the bot generated a ~1500-word article, and this one was better! Unfortunately, it referenced information without providing a source: “AI chatbots like Tay and Eugene Goostman have recapitulated harmful stereotypes and spread misinformation, illustrating issues with biased datasets.”
Like our clients, when Spire goes on the record, we pair claims with confirmed sources to avoid inaccuracies – especially when those claims are potentially defamatory. So I asked Claude to include links for all sources mentioned. That’s where my AI-generated article went from questionable to unusable.
The bot spit out a fresh draft with 18 cited sources. But 17 of the links netted a 404 error – broken, with no redirect. Were all these pages taken down in the last six months? Probably not.
More likely, the bot hallucinated – presenting falsified information with full confidence and seeming credibility.
This was the point of diminishing returns. I knew that my remaining time on this assignment would be best spent fact-checking and editing without Claude’s help. Trying to craft more follow-up prompts to net a better version would take more time than the likely result warranted.
Generative AI needs human oversight
Despite using a reputable, paid AI tool trained on recent content, our experiment left us only moderately impressed with the state of generative AI.
Its rapid-fire output can certainly help bypass any initial creative block. It followed rules (when given examples), and its language was indeed inclusive. Claude also offered up alternate intros, which can help lift a writer out of a creative stall-out, and it successfully transformed the article into an infographic structure.
But the weaknesses were undeniable:
- 17 bogus citations is quite simply unacceptable.
- Despite a clear prompt on tone, the AI veered toward informal language.
- It took multiple prompts and when given additional input, the bot overcorrected.
Bottom line: Despite the promise of generative AI, human writers and editors are crucial to creating trustworthy content, representing the voice of an organization, and delivering what an agency needs.
Advice for government communicators
Generative AI may not be ready to lead a public affairs office, but by using writing bots responsibly, agencies can reap the value of rapid idea generation and text transformation.
As AI algorithms continue to advance swiftly, agency communications teams can lay the groundwork to effectively leverage AI when they mature, including:
- Refining workflows with a view toward slotting AI in where it can add value
- Updating style guides to include clear language for AI prompts
- Assembling content examples to train future models
In the meantime, human review of all AI-generated content is imperative to ensure quality and relevance. In line with executive guidelines for government use of AI, chatbots may augment, but should not replace, the role of humans in agency communications.
At Spire, our writers and content strategists – humans all – bring the expertise it takes to use communications technology effectively, including AI. To learn more, contact us today.
