Artificial intelligence-based systems are entering more and more areas of our lives, including medical diagnostics, autonomous vehicles, and recommendation systems in online stores. However, as these solutions become more complex, the testers facing challenges increase. Traditional quality assurance approaches, developed over many years for deterministic systems, are inadequate in a world where models can alter their behaviour over time, operate based on statistical dependencies and learn from imperfect data. This is why AI system testing requires a new approach, tools, as well as a new way of thinking.
What does this mean in practice?
One of the first challenges a tester encounters is the unpredictability of the system’s behaviour. In conventional software, the same user action should always produce the same result. In machine learning-based systems, however, there can be many ‘correct’ answers. For instance, a chatbot may respond differently to the same question at different times; however, as long as its response makes sense and reflects the user’s intent, it is not considered a bug. Therefore, tests must allow for variations in the results and must be repeated multiple times to statistically confirm that the system is functioning correctly.
The second fundamental difficulty is a lack of a test oracle, i.e. an unambiguous source of truth. Often, the tester does not know what result should be returned, only that the behaviour is ‘within acceptable limits’. This is where techniques such as metamorphic testing are useful, as they test the relationship between input and output rather than the result itself. For instance, if the input value is doubled, the result is also expected to double. It’s like checking whether different arrangements of LEGO blocks have the same functional effect. Another method is back-to-back testing, which involves comparing the performance of two versions of the system. Sometimes, the best option is A/B testing, which includes checking which version of the system better meets business goals in practice. The next part of the article will take a closer look at these techniques.
Let’s get practical
Imagine our task is to test an AI system that recognises emotions based on text, such as customer comments at an online store. This is called sentiment analysis. We have been presented with the following sentence: ‘The product arrived late, but it was well packaged.’ We don’t know what the ‘correct’ result should be, but we can check whether the AI behaves consistently when we modify the data. We can do this by inverting the order of the sentence: ‘The product was well packaged, but it arrived late.’
As the emotions are the same, and only the order has changed, the model’s result should be the same or very similar. If AI considers the sentence to be very positive instead of neutral, then the system is not behaving correctly. Similarly, this technique can be applied to other transformations, such as:
- synonyms: ‘I am very satisfied with the service’ becomes ‘I am pleased with the service’;
- adding neutral information: ‘The phone does not work’ is changed to ‘The phone does not work. The colour of the case is black’;
- paraphrasing: ‘It was the worst decision of my life’ is changed to ‘I regret this decision more than any other’;
- removing irrelevant words: ‘I have to say that I was really not satisfied’ is changed to ‘I was not satisfied’.
The second recommended technique is back-to-back testing. It which involves comparing the AI system with another system. This can be another AI system, or a conventional system. It is important that both systems perform the same function. For example, we might take two models: one new (improved by the ML department) and one old that is already operating in the production environment. Both are fed the same data set containing 1,000 customer comments, and the results are then compared. If the new model classifies 200 comments differently, we check whether this is an improvement or a regression compared to the production system.
A/B testing is the third recommended technique for testing AI systems. Unlike back-to-back testing, which focuses on the consistency of the results of two models, A/B testing is used to compare effectiveness — that is, to check which version of the system performs better under specific conditions. Rather than looking for identical answers, the tester analyses the performance of each model separately and compares the results.
Let’s take autonomous cars as an example. The Machine Learning Department provides two versions of the control system: the old Model A and the new Model B, which is optimised for faster pedestrian recognition. Both systems receive the same input data (route, traffic and weather conditions), and our task is to compare how effectively each model responds. The tester runs both versions in a driving simulator in which a pedestrian crosses the road under identical conditions. The response time of each system is checked, as well as the braking distance and whether the manoeuvre was safe and compliant with regulations. If model B identifies the threat faster and reacts more predictably, the A/B test results will speak in its favour.
No entry for intruders!
Safety aspects must not be overlooked either. AI systems are vulnerable to adversarial attacks, which involve making small changes to the input data to confuse the model. For example, a few seemingly insignificant pixels can cause a system to fail to recognise a STOP sign. In the case of an autonomous car, this can have disastrous consequences. Therefore, it is essential to test models’ resilience to such manipulations, as well as analyse whether the training data has been deliberately contaminated in so-called poisoning attacks.
Conversely, the vast number of input data combinations typical of AI necessitates the use of techniques such as pairwise testing, which reduce the number of test cases while achieving good coverage. For instance, when testing an autonomous vehicle, it is unnecessary to check all possible combinations of weather conditions, road surfaces and lighting conditions; it is sufficient to design the tests so that each pair of these variables appears at least once. This approach helps to manage the vast number of combinations in AI that can quickly make our work seem impossible.
Data is key
However, testing AI systems is not just about correctness. The qualitative characteristics that are specific to this type of software are equally important. For example, is the system fair, or does it favour one group over another? Can the decisions made by the model be explained? Does the system know when to hand control back to humans? In the case of an AI system used by recruiters in the HR industry, for example, it is crucial that users do not favour candidates simply because the training data was dominated by men. Such issues can be identified by analysing the model’s performance and thoroughly checking the data it was trained on.
When the system is opaque or its full specification does not exist — as is often the case — the tester must adopt an exploratory approach. Techniques such as exploratory testing and exploratory data analysis help to identify patterns, anomalies or errors in labels. This approach requires technical knowledge, intuition, and experience.
Each of these techniques plays a different role, like different tools in a workshop. The best results come from combining them wisely with scepticism and curiosity. Testing AI is no longer just a question of ‘does it work?’, but rather ‘does it respond clearly, honestly and reasonably?’. Testing AI-based systems requires testers to develop new skills, both technical and analytical. In a world where the line between the code and data is getting blurred, and systems are learning from unpredictable sets of information, the role of QA is evolving from a control function to a co-creative one. Understanding techniques such as metamorphic, adversarial and A/B testing is fundamental to a tester’s toolkit, not just a fun fact. This is just the beginning — AI systems will become increasingly prevalent, not only in our products, but also in the testing process itself.
