Automatic exploratory performance testing using a discriminator neural network

Research output: Chapter in Book/Conference proceedingConference contributionScientificpeer-review

8 Citations (Scopus)
56 Downloads (Pure)

Abstract

We present a novel exploratory performance testing algorithm that uses supervised learning to optimize the test suite generation process. The goal of the proposed approach is to generate test suites that contain a large number of positive tests, revealing performance defects or other issues of interest in the system under test. The key idea is to use a deep neural network to predict which test could be positive and to train this network online during the test generation process, designing and executing the test suite simultaneously. The proposed algorithm assumes that the system under test is stateless and the outcome of the tests is deterministic. Also, only integer and floating point inputs are supported. Otherwise, the approach is completely automatic and it does not require any prior knowledge about the internals of the system under test. It can also be used effectively in a continuous integration setting where small variations of a system are tested successively. We evaluate our algorithm using two example problems: searching for bottlenecks in a web service and searching for efficient hardware configurations in a single-board computer. In both examples, the presented algorithm performed several times better than a random test generator and significantly better compared to our previously published algorithm, producing test suites with a large proportion of positive tests.
Original languageEnglish
Title of host publication2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
Publisherthe Institute of Electrical and Electronics Engineers, Inc.
Pages105-113
Number of pages9
ISBN (Electronic)9781728110752
ISBN (Print)9781728110769
DOIs
Publication statusPublished - 2020
MoE publication typeA4 Article in a conference publication
EventIEEE International Conference on Software Testing, Verification and Validation Workshops -
Duration: 24 Oct 2020 → …

Conference

ConferenceIEEE International Conference on Software Testing, Verification and Validation Workshops
Abbreviated titleICSTW
Period24/10/20 → …

Fingerprint

Dive into the research topics of 'Automatic exploratory performance testing using a discriminator neural network'. Together they form a unique fingerprint.

Cite this