The Science of Software Testing

The Science of Software Testing

By Thomas Haver, Test Automation Architect at AEP.

Software Testing is similar to science and here’s what we can learn from the subject;

There is a marked difference between how people view science and how they view software testing – with many placing science on a pedestal – despite the fact that the two are quite similar, especially when you consider the methodology involved and the value they deliver.

Not only do both involve adapting quickly to new systems or environments, they both require a critical perspective which relies on looking at issues from multiple angles and take a step-by-step approach when it comes to identifying and addressing problems.

So, how exactly are science and software testing similar?

James Bach, software tester and author, said software testing relates to “questioning a product in order to evaluate it”, while software engineer and professor Cem Kaner described it as the “empirical technical investigation of the product done on behalf of stakeholders, intended to reveal quality-related information of the kind that they seek”.

Elisabeth Hendrickson, agile consultant and trainer, took this notion one step further, stating that software testing involves “designing an experiment to gather empirical evidence to answer a question about risk”.

Throughout these descriptions, two common themes that arise in relation to software resting are observing information and conducting experiments. Similarly, science is defined as the intellectual and practical activity encompassing the study of the physical and natural world through observation and experiment.

Due to the fact that the basis of software testers and scientists is similar, it is not surprising that the skills and personality traits required of individuals within these industries also overlap. People working in these fields need to be analytical and technical in order to fulfil the tasks associated with such roles.

Furthermore, both testers and scientists need to be naturally inquisitive and passionate. Moreover, in order to then be able to draw conclusions from experiments and findings, and transform these into actionable steps, testers and scientists must be reflective and communicative.

The processes involved?

Both software testing and science involve the close inspection of and solution of problems. Within software testing, this means not only thinking about the different ways in which users will interact with the given product but also finding unique ways in which the product can fail. Meanwhile, the scientific mindset is underpinned by a pursuit of knowledge which aims to illuminate the situation and obtain information.

Therefore, when reviewing at the process involved in both, it can be broken down into a three-stage process: collecting empirical evidence via observation; proposing a hypothesis and making predictions using that evidence; and running tests or experiments to refute or corroborate a hypothesis.

Through constant learning and continual assessment, testers can thus observe the behaviors of the product, identify risks, predict failures and test same. Similarly, through constant learning and continual assessment, scientists can make review on how the world works, make new discoveries and identify solutions to problems.

Can software testers add a more scientific structure to their methodology?

There are a couple of options in this regard, the first of which is peer review whereby the work performed is reviewed by peers to ensure it meets specific criteria and standards. Having a peer check the test strategy or review the tests designed and executed by others add rigour and structure to the testing process.

Another methodology that could offer an added layer of analysis is deduction and/or induction. The former means making specific conclusions based on general knowledge while the latter is where general conclusions are made based on specific knowledge.

Hypothesis Driven Development (HDD) is another concept worth considering within the world of software testing when thinking about the development of new ideas, products and services. Within HDD, one can make observations about the customer or their usage of the given product, from which a hypothesis is formulated. An experiment is then designed to test this hypothesis with empirical indicators which confirm or refute if the experiment has been successful, after which modification can be performed on the product or feature.

During the testing process, it is also important for testers to balance the expectations of both producers and customers. This means keeping all aspects in mind and trying to close the gap between the expectations of the producer, including the timing and budget of a project, and the expectations of the customer, including whether a product is fit-for-purpose and how enjoyable the user experience is.

In conclusion…

Science and software testing share parallels in premise and process, both relying on the concepts of observation and experimentation. Of course, there are ways software testing can be enhanced, with practices such as peer review and HDD. By adopting the more rigorous, scientific approach, the quality of software testing can be improved, alongside how such work and its value is viewed on a broader scale.

To learn more about software testing and related challenges, get early bird tickets to attend Quest for Quality 2020 Online Conference today. Find out more here!

Don’t be left out!