AI Testing Theory · ISTQB CT-AI

AI doesn't break like normal software.

So testing it is a different job. Sixty-three short lectures on how these systems actually go wrong - bad data going in, behaviour that quietly drifts once it is live - and what you do about it. The ISTQB CT-AI certificate comes with the territory, if you want the paper.

63
Daily lectures · 5-15 min
300+
Practice questions
13
Modules · 8 weeks
24/7
AI mentor · Telegram
How it works

Closer to a game than a textbook.

Fifteen minutes a day, and the questions get mean on purpose.

1
Step 1 · Learn

Watch the lecture

One idea, five to fifteen minutes, explained the way you'd explain it to a colleague over lunch. No ML degree assumed.

2
Step 2 · Practice

Answer a few questions

Straight after, while it is still fresh. Get one wrong and it tells you why, not just "nope, try again".

3
Step 3 · Ask Kai

Your AI mentor, 24/7

Stuck at 11pm on what "fairness" even means for your model? Ask Kai in Telegram. He answers, any hour.

Bonus · Exam

Sit the cert, if you want

There is a full mock exam to drill against. You book the real ISTQB one yourself - the paper is a nice bonus on top of knowing your stuff.

Curriculum

Eight weeks.

Thirteen modules: an intro, eleven that follow the official ISTQB CT-AI syllabus, and a final assessment. It opens with what these systems actually are and where they go wrong, then moves into how you test for that. One lecture a day; the mock exam at the end runs under real timed conditions.

Full syllabus →
  • M-00IntroductionStart here
  • M-01Introduction to AI TestingFoundations
  • M-02Quality characteristics for AI-based systemsISO 25059
  • M-03Machine learning - overviewRead a model card
  • M-04Machine learning - dataLeakage, splits
  • M-05ML functional performance metricsConfusion matrix
  • M-06Neural networks and testingAdversarial
  • M-07Testing AI-based systems - overviewStrategy
  • M-08Testing AI-specific quality characteristicsBias, drift
  • M-09Methods & techniques for testing AIMetamorphic, A/B
  • M-10Test environments for AI-based systemsData, isolation
  • M-11Using AI for testingTooling
  • M-12Final assessmentMock exam
Pricing · simple

Two doors.

- Free taster

The first twelve.

free
No card · no signup wall
  • The first 12 lectures, end to end
  • Practice questions for each
  • A session with Kai, the AI mentor
  • A taste of the mock exam
Start free
If it is not for you · zero cost
Focus group · public posts

What the testers said publicly.

Forty-six QA folks went through the free Module 1 before launch. These are the ones who wrote about it on LinkedIn - names link to the original post.

Anastasia, course author
Who is teaching

Anastasia, QA for 12 years.

I'm a QA lead, not an AI researcher. Someone handed me "the new AI thing" and I had to work out how to test it.

Twelve years in QA, ISTQB Foundation since 2018, and these days I work alongside about twenty other testers. When AI landed on my desk I started building the course I wish someone had handed me. It took two years and around €35,000.

It is theory first: what these systems actually are, and what words like fairness and drift come to mean once the thing is learning on its own. The CT-AI certificate sits on top, for when you want it. The theory is the part that makes people in an AI standup stop and actually listen to you.

- AnastasiaCourse author · QA Tech Lead · ISTQB