The R.O.B.O.T. Test
A Framework for Evaluating AI-Generated Information
TECHNOLOGY & INNOVATION
Bernard Millet
10/9/20253 min read
Using ChatGPT, Claude, or other AI tools? Here's how to know if you can trust what they tell you.
Building on a Proven Foundation
The ROBOT Test was originally created by Sandy Hervieux and Amanda Wheatley at McGill University in 2020 to evaluate AI tools and technologies. Their framework has been adopted by academic libraries worldwide as a cornerstone of AI literacy.
What's changed?
With the explosion of generative AI, we now need to evaluate not just the tools themselves, but the content they produce. That's why I've adapted the ROBOT Test specifically for evaluating AI, generated information, the articles, code, advice, and answers we get every day.
The ROBOT Test for AI-Generated Content
Use these five questions to critically assess any AI output:
R - Reliability
Can this information be independently verified?
Does the AI provide specific claims you can fact-check?
Can you find corroborating information from authoritative sources?
Or are you looking at vague, unfalsifiable statements?
Red flag: "Studies show 73% of companies use this method" (Which studies? What companies? When?)
What to do: Google key claims and look for confirmation from multiple authoritative sources.
O - Origin
Where did this information likely come from in the training data?
Does this sound like it came from reliable sources (academic papers, official docs)?
Or from less reliable sources (forums, opinion pieces, social media)?
How recent vs. outdated might the source material be?
Red flag: Medical advice that sounds like WebMD forums rather than medical journals. Legal information reflecting popular understanding rather than actual law.
What to do: Consider the likely quality of sources in the AI's training data for this topic.
B - Bias
What perspectives or viewpoints might be overrepresented or missing?
Cultural bias (Western/English-language perspectives dominating?)
Demographic bias (whose voices are excluded?)
Commercial bias (favoring certain products or companies?)
Selection bias (what types of sources appear most online?)
Red flag: Business advice heavily reflecting Silicon Valley startup culture while ignoring practices from other regions or industries.
What to do: Ask yourself: whose perspective is this? Whose perspective is missing?
O - Objectivity
Are multiple viewpoints being considered, or is there hidden opinion?
Does the AI acknowledge controversies or debates?
Is opinion being presented as fact?
Are alternative interpretations mentioned?
Is the language neutral or loaded with value judgments?
Red flag: "Everyone knows..." or "It's obvious that..." statements. Historical events presented with only one interpretation. Lack of acknowledgment of legitimate disagreements.
What to do: Look for phrases like "some argue," "perspectives differ," or "there is debate about"—or their absence.
T - Timeliness
Is this information current and relevant now?
When was the AI's training data collected? (Check the knowledge cutoff)
How quickly does this type of information change?
Have there been recent developments the AI wouldn't know about?
Red flag: AI describing COVID policies from 2021, company leadership from 2023, or election results from months ago as if they're current.
What to do: Always check dates on statistics and verify current events independently.
How to Apply It?
Quick Version
Ask yourself these five questions about any AI-generated content:
R: Can I verify this with a quick search?
O: Does this sound like expert knowledge or internet opinion?
B: Whose perspective is represented here?
O: Is this presenting facts or opinions?
T: Could this information be outdated?
If any answer raises concerns, dig deeper before trusting the information.
Thorough Version
For important decisions or critical information, use this systematic approach:
Step 1: Identify Key Claims
Pick 2-3 specific, verifiable statements from the AI's response
Focus on claims that would impact your decision
Step 2: Cross-Reference
Search for each claim using multiple sources
Look for authoritative, primary sources (not just other AI-generated content)
Check if reputable sources agree or disagree
Step 3: Check Currency
Note any dates, statistics, or time-sensitive information
Verify these haven't changed since the AI's knowledge cutoff
Look for recent developments the AI wouldn't know about
Step 4: Seek Alternative Perspectives
For any controversial or debatable topics, actively look for opposing views
Consider what viewpoints might be missing from the response
Ask yourself: "Who might disagree with this?"
Step 5: Evaluate Impact
Consider who benefits if this information is true
Think about potential consequences of acting on incorrect information
Decide if you need additional expert consultation
When to Worry
If 1 element raises red flags: → Verify that specific aspect before using
If 2-3 elements raise red flags: → Seek alternative sources entirely
If 4-5 elements raise red flags: → Assume information is unreliable; start research from scratch
The Golden Rule
The more important the decision, the more rigorous your R.O.B.O.T. test should be.
Casual curiosity? Quick check is fine
Work project? Verification needed
Health/legal/financial decisions? Verification is essential / Consult human experts
Remember
The R.O.B.O.T. test isn't about distrusting AI completely, it's about being an informed user who understands both the power and limitations of these tools. AI is incredibly useful, but it's not infallible.
Think of AI as a brilliant but overconfident intern: great at gathering information quickly, but everything needs to be double-checked before making important decisions.
References of the original ROBOT test
Original ROBOT Test for evaluating AI tools: Hervieux, S. & Wheatley, A. (2020). The ROBOT test [Evaluation tool]. The LibrAIry.
This framework adapts their pioneering work to focus specifically on evaluating AI-generated content and information. Learn more about the original ROBOT Test at McGill University Library.
