Tag

AI Agents

2 posts tagged AI Agents.

September 9, 2026 RAG & LLM Systems

How Often Is a Research Agent's Confidence Label Right? 156 Claims Checked Against the EU AI Act

My research agent marks every claim as high, medium or low confidence, based on its sources. I had 156 of its claims checked against official sources. The low-confidence claims were right as often as the high ones, most wrong claims cited good sources but got a detail wrong, and the model's own stated confidence did better than the source-based score.

AI Agents LLM Evaluation Calibration EU AI Act Claude Python

September 8, 2026 RAG & LLM Systems

How I Check an AI Research Agent's Reports, and Test the Checks: An EU AI Act Example

An AI research agent's reports read well whether or not they are right. These are the checks I built into an agent that writes cited reports on the EU AI Act, how I tested them with planted errors, and what they still cannot tell you.

AI Agents LangGraph LLM Evaluation Multi-Agent Systems EU AI Act Claude Python