How we read nonprofit emails with AI

Giving Radar signs up to nonprofit newsletters and keeps everything they send. A few hundred emails is too many to read closely by hand every week, and reading them loosely misses the point: the details of how an organization asks are the whole story. So every email that arrives is now read twice, once by plain code and once by an AI model. This post explains what each one does, how we check the AI's work, and what it costs.

01 - What happens when an email arrives

Every few minutes the archive picks up new mail. Then, in order:

1. We save it, with its original HTML, so it can be seen exactly as it was sent.
2. Code reads the links and buttons: which ones ask for money, which ask you to read, sign or answer a survey.
3. Code measures what can be measured exactly: personalization, design and template mistakes (next section).
4. Code finds the email's main words, the terms that make it different from everything else in the archive. This is statistics, not AI.
5. An AI model labels the email: what it wants, how it asks and who it says it comes from.

Only the last step uses AI. Everything that can be counted is counted by code, which is exact, free and never changes its mind.

02 - What plain code measures

Some questions have a right answer that a program can find without guessing:

• Does it use my name? In the subject, in the text, or on a button.
• Does it say "Dear Friend"? A generic greeting instead of a name.
• Did the template break? Unfilled codes like ${FirstName} left in the text.
• Have we seen it before? An exact duplicate, or the same email sent again with a new subject line.
• How is it built? Word count, images, tables, and which email tool sent it.

In the 222 emails labelled so far, 156 (70%) carry at least one link that asks for money, but only 33 (15%) put the reader's first name in the subject line.

03 - The questions the AI answers

The AI reads the text of the email and the list of its buttons, and answers the same set of questions for every email:

Question	Example answers
What is the email for?	fundraising, advocacy, engagement, information, event
Where is it in the relationship?	onboarding, cultivation, conversion, upgrade, retention
What format is it?	letter, story, newsletter, alert, survey, invitation
What is the ONE thing it wants you to do?	donate once, give monthly, sign, watch or read, take a survey
Which persuasion devices does it use?	matching gift, deadline, suggested amounts, named story, social proof
Which feelings does it aim for?	hope, urgency, compassion, guilt, belonging, gratitude
Who does it say it comes from?	the organization, a staff member, the CEO, a celebrity
Does it hang on a date?	Giving Tuesday, year end, an awareness month, a news event
Did the sender make a mistake a reader would notice?	an old year, garbled text, a fact about you that isn't true

It also writes the topic in a few words and one sentence explaining its choices.

Three rules keep the answers useful:

• Fixed answers only. Each question has a closed list of allowed answers, and the model can't return anything outside it. A label can never be misspelled or invented, so the emails can be counted and compared.
• A clear point of view. The model is told who is reading: someone who signed up recently and has never donated. That matters. "Thanks to your support" means something different when the reader has given nothing.
• The email is data, not instructions. If an email contains text addressed to an AI, the model is told to ignore it and label it like anything else.

An admin can correct any label by hand, and the correction is kept when an email is labelled again.

04 - What the labels show so far

These numbers come from 222 emails, mostly welcome series from organizations we joined recently. They will change as the archive grows.

• Most emails are not about money. Only 27% are mainly fundraising. Nearly half (48%) build the relationship with a story, a video or a survey. The most common single ask is "watch or read something" (31%), ahead of "donate once" (21%).
• "Thanks to your support" is everywhere. 40% of emails thank the reader for help they never gave. It's the most common persuasion device we see, ahead of social proof (22%) and deadlines (14%).
• Belonging and hope lead. Two-thirds of emails aim to make you feel part of something (67%), and 61% offer hope. Fear shows up in 7%.
• Most emails come from the organization. 59% are signed by the organization, 23% by a named staff member and 15% by the CEO or founder.
• Mistakes are rare but real. 10% of emails show an outdated year or name, 9% have garbled text, and 4% state something false about the reader.

05 - How we check the AI

An AI model that sounds confident can still be wrong, so we don't take its word for it. We labelled 200 emails by hand, then asked each model to label the same emails and counted how often it agreed with us:

Agreement with the hand labels	Claude Opus 5.5	Claude Sonnet 5.5	Claude Haiku 4.5
What the email is for	89%	88%	86%
The one thing it wants you to do	90%	90%	81%
Who it comes from	96%	96%	92%

Our bar is 85%. Most disagreements are honest borderline cases: an email that tells a story and ends with a Donate button can fairly be called fundraising or engagement. The persuasion devices and feelings are softer. The models agree with us less on those, and we treat them as a rough guide.

We run this check again every time we change the instructions or the model.

06 - Choosing a model, and what it costs

Every call to an AI is recorded with its cost, so we know exactly what this reading costs:

Model	Cost per email	Verdict
Claude Opus 5.5	about 1 cent	the most careful, and the most expensive
Claude Sonnet 5.5	about half a cent	as accurate as Opus on the main questions
Claude Haiku 4.5	a little less than Sonnet	below our bar on the main ask

We use Claude Sonnet 5.5. It matched Opus on the questions that matter most at about half the price. Haiku was only slightly cheaper for this job and missed too many main asks. At our volume, labelling costs a few dollars a month.

Two more things keep the bill small: a daily limit on how many emails are labelled, and reusing the long instructions between calls instead of sending them fresh each time.

07 - What the AI does not do

• It doesn't write this blog. People decide what the numbers mean and what gets published.
• It doesn't reply to, forward or act on any email. It reads and labels, nothing else.
• It doesn't replace reading. The labels point us to the emails worth reading closely; they don't read them for us.

Next, we will try models from other AI companies on the same 200 hand-labelled emails and publish how they compare.