Twelve out of fifteen is not a pass
We hid fifteen errors in a fictional quality system and asked a local AI model to find them. Here is what it found, what it missed, and what it invented.
Test 1 · synthetic data25 documents of a fictional company, "Demo Pharma". No real company's documents were used. Model: Qwen 3.6 27B (4-bit), run with Ollama on a laptop with 48 GB memory, offline. All 25 documents in one prompt; 3.4 minutes.
The result
12 / 15errors found
3missed
2false findings
≥ 13our target
Below our own pass mark. Easy errors 6 of 7, medium 4 of 5, hard 2 of 3. A bonus gap (no complaints SOP at all, EU GMP Chapter 8) was not reported.
The answer key
| # | Level | Planted error | Result |
|---|---|---|---|
| 1 | Easy | SOP refers to a complaints SOP that does not exist | Found |
| 2 | Easy | SOP refers to form FRM-QA-006; the forms stop at 005 | Found |
| 3 | Easy | Reference to version 2 of an SOP; version 3 is in force | Found |
| 4 | Easy | Wrong title given for a QC SOP | Found |
| 5 | Easy | Reference to Appendix 2; the SOP has only Appendix 1 | Found |
| 6 | Easy | Quality manual lists an SOP under the wrong title | Missed |
| 7 | Easy | Two SOPs point to each other for numbering rules; neither has them | Found |
| 8 | Medium | Deviation closure: 30 calendar days vs 20 working days | Found |
| 9 | Medium | Batch certified by "QA manager" instead of the QP | Found |
| 10 | Medium | Assay limits 90–110 % vs specification 95–105 % | Found |
| 11 | Medium | SOP uses an autoclave retired from service | Found |
| 12 | Medium | CAPA effectiveness sent to self-inspection, which does not cover it | Missed |
| 13 | Hard | SOP based on the 2008 Annex 1 | Found |
| 14 | Hard | Grade A limit "1 CFU" vs "no growth" (Annex 1 2022, Table 6) | Found |
| 15 | Hard | Aseptic process simulation once a year instead of twice (Annex 1 §9.38) | Missed, and called it correct |
What it invented
- It called the record retention rule wrong, with a reason that does not match EU GMP: the periods in the SOP are those of clause 4.11 for batch documentation. The SOP applies them to all GMP documents, which the model did not point out.
- It said three consecutive aseptic process simulations are "outdated". Annex 1 (2022) still asks for at least three for initial validation (§9.38).
Both were stated with confidence. A tired reviewer could have accepted them.
What we learned
- It is good at contradictions between documents: numbers, deadlines, roles and equipment. Fast, too.
- It is dangerous when it answers regulation from memory. The two false findings and the worst miss (#15) all came from regulatory "knowledge" the model did not have right. This is the risk the draft Annex 22 warns about. Read more →
- Mechanical checks belong to rules, not AI. A simple script that compares document codes, titles and versions would have caught #6 every time.
- The raw output is not a report. It repeated findings and "thought aloud" about non-errors. A person had to sort it.
What we change for test 2
- A rule-based pre-check for codes, titles, versions and appendices.
- The relevant clauses of Annex 1 and EU GMP Chapter 4 given to the model as text, so it quotes instead of remembering.
- A stricter output format: one line per real finding, each with its source.
- The same test on a second model, one change at a time.
We will publish test 2 the same way, including what it misses. Read test 2 →