What we typically think when we hear ML or AI?
We do typically think about the areas with the most advertising — cars and robots.
At some point in time most of us was at least a bit interested in the evolution of Tesla and the hopes for autonomous, self-driving cars. I guess some of your either own or at least drove one. 😊 Slowly, but steadily, we are getting closer and closer to what seemed like a dream or a wild image taken from ‘The Jetsons’ — a cartoon for kids first aired in 1962!
On the same note, I was astonished when I saw what the Boston Dynamics crew did years back and even more flabbergasted with what Atlas, their robot, can do now. In case you did not follow it lately, ten years ago Cheetah Robot ran a bit faster than 28 mph.
Five years ago the Atlas Robot was able to do a backflip:
A year ago the new generation of Atlas Robots could finish a parkour track or even dance
Quite a nice summary of the company and their products can be view here:
We tend to focus on areas of general significance, where the biggest amount of funding goes to — transportation, military and logistics (in random order). We sometimes imagine if, how and where we could benefit from AI and ML in every most important aspect of our day-to-day lives or activities.
But isn’t our work in Quality Assurance one of those vital aspects of our lives as well? After all, QA specialists spend a great deal of our adult life at work…
Could we benefit from AI and ML in Quality Assurance?
Short answer is: YES, definitely!
Someday, we will most likely be able to simply write a test scenario in a BDD (Business Driven Development) style, sit back and relax while the computer does everything for us until we can analyse the results and performed actions.
For the time being, that’s a pipe dream, but what if we could get some assistance? Aren’t there areas which we find mundane or tedious? Repetitive tasks, other than regression testing, we would like to be able to focus a bit less on?
I’m sure you can think of a few examples, but for now let’s concentrate on the two mentioned below:
- Analysing test run reports more efficiently
- Creating a basic frontend automated test across multiple platforms (Web, Mobile)
ML in Test Reports Analysis
Typically, the longer a project under tests lasts, the more regression tests are to be reviewed. Depending on the frequency of the regression runs, one might even end up having to review hundreds or even thousands of tests on a daily basis.
Assuming that the regression is never green, due to various reasons (e.g. integrations failing, low environment performance, defects to be resolved), you often need multiple people reviewing the results over a couple of hours before it can be clearly stated if the given version meets the threshold.
Speaking of tiresome activities, regression results analysis comes definitely within the top 3 in my own list. Thankfully, necessity is the mother of invention!
ReportPortal is unquestionably one of the test tools to support test analysis and reporting. Powerful, fast, reliable and easy to configure. Works with a lot of test frameworks.
There is quite a number of similar test reporting tools, for instance HP ALM or qTest. So, what is so different in ReportPortal that it makes it so outstanding? It’s Machine Learning!
ReportPortal allows to configure multiple different reasons for failures and then compares the regression runs using a number of different factors like test names and their logs. If the test fails with nearly the same error in the log each time, the ReportPortal’s Machine Learning will take over and begin to automatically attribute a given failure to a selected reason after a couple of manual reviews. By doing so, it will save everyone’s time and let you focus on only the failures that are new or not consistent with what you taught it before.
AI in simple frontend Test Automation
Wouldn’t it be awesome not to constantly have to maintain simple frontend test automation scripts for all the different test platforms? Web, mobile, multiple browsers, multiple devices, screen resolutions — you name it.
Last year a company called Micro Focus released a new version of their star product, UFT One. It’s the first, or definitely one of the first tools available on the market that use Artificial Intelligence to help in the development of basic frontend automated tests.
It allows you to ‘name’ the objects you see with your own eyes and then let the AI recognize them on various test platforms and environments. What then remains is to make sure that, for example, the shopping cart icons bear close resemblance in each of those.
It is not yet a very versatile solution, as each element needs to be taught to AI, so out of the box you will not be able to get all the benefits of the UFT One on each and every screen under test, but it’s a really good start.
Unfortunately, same issues apply to AI frontend testing tools as to any AI mechanism — the size of data subsets needed to teach as well as the computing power required, are very large. Due to that, it takes time to get UFT One to recognize new screen icons. It’s a great start though, especially if in the future we will invent a way to get the neural networks to use/need less resources and by doing so greatly shorten the AI/ML’s learning curves.
Here’s a glimpse of what UFT One is capable of as well a glance at how to use it:
Summary
Machine Learning or Artificial Intelligence already provides help in various areas. What definitely started with the intent of military use, then evolved into cars or civilian robots. Now it starts providing support in more and more areas, testing is and will continue be one of them.
Checkout what the Boston Dynamics robots are used for already today!
