The requirement for test automation has never been greater, with many organizations moving towards Continuous Delivery and DevOps regimes. With these rapid Build/Deploy cycles, test automation has become critical to success, removing the manual testing bottleneck is critical to keeping shorter cycle times.

“Test Automation has always faced one fundamental challenge, that of knowledge capture and representation”

Fundamentally, the challenge of test automation can be described as; how to articulate a test script (the output of the test analysis process to exercise test cases) in a way that can be interpreted by a machine for execution purposes. Essentially – creating machine-readable test scripts.

Since their inception, traditional automation tools have relied on coding languages as the means to represent test scripts for execution. But there exists the fundamental problem, most organizations rely on human-derived test scripts, the output from test analysts who define and articulate them. The analysts are experts in their subject matter and in the field of testing, interpreting system specifications and expertly deriving test cases from that knowledge, scripts to exercise those cases are typically documented in natural language and stored in a repository for later execution manually.

It is fair to say most test analysts are not skilled in writing and maintaining code, the language required of test automation tools, so how can we capture their output in a form that can be interpreted by machines or alternatively converted into code for automation tools?

Commonly, skilled automation engineers or SDETs are hired who take test scripts and rearticulate them in coded form. Rightly or wrongly automation for most organizations is a translation exercise from human-readable form to machine-readable form, essentially re-articulating the output from the test analysis process in another form. There are clearly efficiencies to be made in this process.

WHAT IS LOW-CODE TEST AUTOMATION?

In brief, Low-Code Test Automation is removing the translation step from the process described above. Providing a means of defining test scripts directly in a form readable by both humans and machines. Low-Code Test Automation is aimed at solving the fundamental knowledge capture problem, empowering test analysts, not conversant with code and software development to deliver test scripts in a machine-readable form, skipping the secondary translation step.

“Learning to code is only half of the battle, with coding, comes a plethora of tooling and ancillary skillsets“

The appeal of low-code test automation comes down to cost. The traditional script-to-code process requires duplication of effort, articulating the same information in two forms, one readable by humans, the second translated into a machine-readable form, code. This requires additional resources, if both steps could be combined it lowers the cost considerably. So what options are there?

WHY NOT TRAIN EVERYONE TO CODE?

Give the individuals responsible for deriving the test scripts in the first place the ability to directly articulate them in machine-readable form, code. The barriers to this approach are obvious. We are adding a third skillset to those of domain expertise and test expertise. Many will have no desire to take that journey or indeed the mindset to do so, it is often said that the skills of a tester and a developer are very different, the combination of the two is still a rare commodity. Needless to say, many test analysts will make that journey successfully, hats off to those that do.

There is still the problem that even if test cases can be directly derived and articulated in code, that all stakeholders in the software development process need to be able to read those test cases. This is acutely obvious in risk acceptance, audit, and compliance processes, those responsible for signing off on test cases and assessing business risk in going live. It is unlikely that those responsible will want to learn to interpret code in order to assess risk.

It must be said, that learning to code is only half of the battle, with coding, comes a plethora of tooling and ancillary skillsets, the use of Integrated Development Environments (IDEs), the build orchestration tools, frameworks, and third-party libraries. Setting up a simple Java Selenium test could require knowledge of numerous components; Eclipse, Maven, TestNG, and Selenium on top of the Java language itself. Extending that to improved reporting with Extent, DB calls with JDBC, and data abstraction with Excel with Apache POIs, quickly the learning curve becomes logarithmic!

ABSTRACTION

Providing an abstraction layer has two benefits, the first that if articulated correctly it should be readable by all stakeholders in the software development process, secondly it should be machine-readable. Abstraction has a price, the payoff is in flexibility, whilst humans can overcome limitations in interpretation, can an abstraction articulate the nuances and precision of instruction required by automated execution tools?

Numerous solutions to approach this problem have been seen over the years, and yet we still see automation rates as low as 15-20% across many organizations. So, let’s look into the history a little and how test automation approaches have evolved abstraction layers.

CAPTURE-REPLAY

Capture-Replay (Record and Playback) – A process whereby a coded test script is created by recording user actions whilst they perform steps manually on an application, directly skipping the translation step. The backbone of test automation tools for decades, the little red button is seen by many as contributing to the over-expectation placed on automated testing in organizations. Many vendors have pushed the ease of use of capture-replay tools as a major selling point. Allow your test analysts to instruct the automation tools by demonstration.

“We still see automation rates as low as 15-20% across many organizations“

In reality, there are numerous pitfalls in using capture-replay; Reliability of recorded scripts, often fragile or incorrect code is produced in this fashion. The code doesn’t incorporate accepted best practices of modularity for maintenance and parameterization for varying test data, also requiring an amendment to incorporate validation actions. Maintenance and code refactoring overheads mean this approach alone rarely scales and projects will resort to direct coding or other more structured approaches.

KEYWORD FRAMEWORKS

Stemming from the TestingwithActionWordsmethodology proposed by Hans Buwalda in 2002. The process of defining reusable parameterized test actions that can be sequenced together to form larger processes. Often architected using Excel as a ‘front-end’, enabling non-technical testers to be core to the automation process using the Excel interface to articulate test scripts.

Each action word still requires implementation as code so the team needs to be comprised of both technical and non-technical roles, with the workload split. Action words and Keyword approaches still require coding and maintenance so additional resource beyond the analyst role is required.

“BDD requires wholesale buy-in from all business stakeholders to be successful”

Whilst many traditional automation tools have their own sophisticated action/keyword solutions built-in, it is still very commonplace to see organizations creating and maintaining their own, often equally sophisticated, Keyword Frameworks in-house. Sadly, in many cases, these do not offer the return on investment promised, typically requiring teams to maintain and support them and are often vanity projects.

BEHAVIOURAL DRIVEN DEVELOPMENT

First proposed by Dan North, the use of a ‘ubiquitous language’ for articulating use cases (typically seen implemented in Gherkin form, Given, When, Then, etc.) that all Stakeholders in the software development process can interpret,  and more importantly can be executed directly by BDD frameworks like Cucumber and SpecFlow. Often interpreted incorrectly as a test automation solution, BDD requires wholesale buy-in from all business stakeholders to be successful and has not always delivered on its promise when this isn’t the case. Moreover, the ubiquitous language clauses, similar to Keywords, still require implementation in code form. BDD sits in the Hybrid camp, requiring both domain expertise and coding skill sets to achieve success.

THE RISE OF LOW-CODE TEST AUTOMATION TOOLS

Low-Code Test automation has in fact been around for many years but is recently seeing a resurgence. This can be put down in part to the widespread adoption of Open Source libraries such as Selenium and Appium. The execution ‘engines’ previously the prized IP of traditional tool vendors are open for all to build abstraction layers upon. Improvements in UI and visualization technology have allowed for creative solutions to appear outside of the traditional vendor space.

With these Open-source libraries, the same fundamental script to code translation exercise applies, even more acutely so, as they do not come with any form of UI in most cases. So Let’s explore some typical low-code abstractions:

INCREASING COMPLEXITY

Both Table and TreeView abstractions make for compelling solutions for the simple examples outlined above. In reality, test scripts are not entirely comprised of simple step-by-step interactions. It is commonplace to find easily articulated natural language steps, such as:

• “Check the Account ID in the third column of the user table is a combination of Name and DOB”
• “Enter a date in the policy field 30 days from now”
• “Repeatthe previous steps until the shopping basket contains 10 items”

Here the Table and TreeView abstractions break down or become overly complex. Essentially test scripts are simple algorithms in their own right, just articulated in natural language form. They never become as complex as application code but still have elements of algorithmic constructs and operators. Repeat, If, Until, Combine, plus, etc.

ENTER VISUAL PROGRAMMING

Visual programming languages (VPLs) are not new, they “let users create programs by manipulating program elements graphically rather than by specifying them textually”, “programming with visual expressions, spatial arrangements of text and graphic symbols”.

“This really is the problem that abstractions need to address head-on, the requirement for simple algorithmic constructs occurring in test scripts”

A quick look at Wikipedia shows they have been around for many years in different forms, primarily applied in Robotics, Process Automation, the Internet of Things (IoT), Embedded Systems, Multimedia, Games, and Education.

Let’s look at some examples.

FROM FLOW DIAGRAMS TO BLOCKS

There are a number of different visual programming styles primarily based on visual canvases where elements can be dragged and connected in some fashion on a design canvas. Some common forms are shown below, executable flow diagrams, from Flowgorithm and Node-red, and block coding from Grasp.io:

The last of these three forms, Block Coding, was the creative output of Google research and MIT projects into code abstraction which led to a way to represent simple algorithmic constructs in a form that is easy to understand, explore and apply.

Based on the indent structures of text-based code, statements and structures are represented in visual form as a series of interconnecting blocks. These can be connected in a flexible way into sequences and with inputs and outputs to articulate an algorithm.

“The application of Visual programming in robotics and process automation is telling, much like test scripts these applications do still require algorithmic constructs but not to the level of complexity that typical UI based application code requires”

Adoption of block coding has rapidly expanded in robotics, home automation embedded systems design, and IoT development, but is more commonly known in a colorful form for educating aspiring coders, with the wonderful Code.org and Scratch projects

We are already seeing vendors adopting visual programming forms for Test Automation applications and it is our belief that they hit the sweet spot in providing enough of an abstraction to be easy to adopt quickly, whilst maintaining the flexibility to cope with algorithmic constructs. Visual Programming is making ‘Low-Code’ Test Automation a reality now…

ABOUT SCRIPTWORKS

Scriptworks is the industry’s first automated testing platform based on the principles of block coding. Users can quickly define test scripts for automation without the requirement to become familiar with an IDE, set up a build an execution framework, and more importantly without the need to learn the syntax of a new and complex coding language.

Best practices of Modularity and Parameterization are built-in. Importantly Scriptworks does not do away with code altogether, a key feature in flexibility and portability, code is still present, generated automatically from the visual abstraction.

A comprehensive set of predefined blocks make up the abstraction with complete Selenium and Appium functionality, including common requirements of working with Browsers, Mobile Devices, etc. as well as ancillary functionality for dealing with Excel Data Sources, APIs, and more. Custom blocks can easily be added to the abstract layer with their own corresponding custom code generated. It is easy to extend the already comprehensive existing capabilities.

Scriptworks combines its own comprehensive execution and reporting frameworks based on Node.JS. This is entirely portable, callable from CLI, CI/CD tools like Jenkins and TeamCity and enables high levels of concurrent execution.

A DASH OF ARTIFICIAL INTELLIGENCE MAGIC

Scriptworks adopts a pragmatic approach in the adoption of AI through the integration of features from leading AI providers like Applitools for Visual Testing and Google’s powerful ML-based OCR to hit traditional pain points in the test automation field as well as introducing new solutions of its own to aid in script creation and maintenance.

Brittle element identification and poor resilience to change have led to maintenance difficulties and is a primary contributor to low automation adoption rates. The introduction of Smart selectors, based on semantic identification and combined Heuristic techniques can remove this brittleness from the outset.

Scriptworks aims to allow users to create tests without the need to learn and maintain large Selector Libraries and complex Page Object Models.

Software Testing Expert Talks (STET) on Scriptworks

STET Session Part 1

STET Session Part 2

STET Session Part 3

More STET Session on Scriptworks Coming Soon.

Scriptworks Tutorial

The following are the different contents in our Scriptworks tutorial:     

Section: GraphQL API’s

1)   GraphQL API Testing

2)   GraphQL – Single Value Response Validation

3)   GraphQL – Multiple Value Response Validation

4)   GraphQL – Full Response Body Validation

Section: REST API’s

1)   REST API Testing

2)   REST – GET Request 200

3)   REST – POST Request 201

4)   REST – PUT Request 200

5)   REST – PATCH Request 200

Section: Mobile Test Automation

1)   Getting Started with Mobile Test Automation (Android)

2)   Sample E2E Test Run on Native Mobile App (Android)

More articles coming soon