Testing a RESTful API using Supertest

Standard

This post explains how to test a RESTful API using Supertest.

There are many great blog posts that explain how to test NodeJS Express API’s using Supertest. For example:

This post explains how Supertest can be used to test any RESTful API even those that don’t run through express.

Setup

You will need a couple of node packages to get going including:

  • Supertest – “High-level abstraction module for testing HTTP”
  • Mocha – “Mocha is a simple, flexible, fun JavaScript test framework for node.js and the browser”
  • chai – “Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.”

Once installed your package.json should look something like this:

Testing API

To try out Supertest i tested http://postcodes.io/ – Postcode & Geolocation API for the UK

Searching for a single Post Code (GET Method)

APIhttp://api.postcodes.io/postcodes/SW1A1AA

In this method the user can query the Post Code data for a single post code. If valid, the API returns further detail.

In the tests below you can see a basic example that calls the API with a given Post Code and then performs assertions on the response. You can see in all three cases i’m checking the Status Code of the response and also inspecting the JSON response to ensure the correct data has been returned.

Searching for multiple Post Codes (POST Method)

To search for multiple Post Codes we need to POST an array of Post Codes to the API. To do this we pass in a JSON object to the service, in this example three post codes.

You can see we are again checking the status code of the response. In addition as multiple post codes are returned we iterate through the response to ensure the results match our search.

Summary

Supertest is really easy to use. You can very quickly write a series of tests for your API. Posting data and inspecting responses is very easy.

At this point i haven’t tried with a secure API that requires authentication but the documentation suggests that this shouldn’t be a problem.

Using Gulp for Testing

Standard

Last year I shared some posts on testing AngularJS web apps:

Getting started with Protractor to automate E2E tests for AngularJS apps

How to steep a unit test environment for AngularJs Application

I thought i’d start 2016 by sharing some other tips in that area.

What is Gulp?

gulp.jpg

Gulp is a JavaScript Task Runner. Now it took me a while to actually get my head around what Gulp actually does. Gulp enables you to encode in JavaScript repetitive tasks that you perform regularly. This could be starting a web server, JavaScript minification or Testing which will be the focus of this blog.

One of the core benefits of Gulp is that it encodes these tasks in a JavaScript file which is typically named gulp.js. This means all of these encoded tasks can be shared across the team and kept in synch using version control.

Another benefit is that gulp utilises the wide variety of packages available via NPM(Node Package Manager) which means you can automate tasks incredibly easily.

In this blog I won’t go through how to setup Gulp from scratch or how to install Node Packages as the official documents do this very well. Instead my focus will be how you can use Gulp to support testing.

If you want to get started from scratch in addition to the official docs i’d highly recommend John Papas Gulp course on Pluralsight – JavaScript Build Automation with GulpJs

Note: if you don’t currently have a Pluralsight account you can get 3 Months FREE by creating a free Visual Studio Dev Essentials account

How to use Gulp for Testing

Gulp can be used for many tasks but in this blog post I’ll focus on tasks that can be used to support testing.

  1. Run Unit Tests

This task runs your JavaScript unit tests using Karma. It utilises your karma configuration file which allows you to configure run parameters, such as single run and also what outputs we want to generate, such as Istanbul Code Coverage reports and jUnit XML. You can read more about configuring your unit tests in my previous post

Using Karma you don’t need to install any additional gulp packages as you can call Karma directly through Gulp as explained here

The gulp task looks as follows:

Snip20160103_2

2. Run Protractor E2E Tests

As detailed in a previous blog post, Protractor, is an E2E testing framework mostly used for AngularJS web apps. This gulp task allows you to execute your protractor tests. You’ll need to make some additional tweaks to the task to allow a test suite to be specified. We use protractor test suites to allow control of what tests are run. To do this we used an NPM package called yargs. This blog helps.

What is cool with this task is that in a single task it will get the latest web driver jar, start a selenium server, run the tests and generate the output.

To run your protractor tests using gulp you’ll need to install a couple of node packages:

Gulp Protractor

Yargs (Used for specifying testSuites)

The gulp task looks as follows:

Snip20160103_3.png

 

3. Linting

Linting is an activity to discover common mistakes in code and is incredibly useful for JavaScript as its not a compiled language. It’s a great example of static testing that can yield great benefits. Here is an example of the output you get:

reporter_problems.png

Using this gulp task you can Lint your code to identify any simple mistakes or errors keeping the code base clean.

To do the linting you need to install the node package JsHint-Stylish

The gulp task looks as follows:

Snip20160103_5

 

Putting it all together

 

To run these tasks you simply start up a nodeJs command prompt and enter gulp followed by your task name. For example:

gulp protractor –testSuiteName

This will run your specified e2e protractor test suite.

As NPM gives you such a wide variety of options there are many more tasks you can add to support your testing. These are probably the three most common. Another example could be starting your mock services.

Local vs CI

You obviously want to avoid duplication so you should use the same Gulp file both locally and on your CI tool such as Jenkins. To achieve this you just need to learn about file paths and ensure your gulp.js file is located sensibly, such as within the project directory.

When running any of the three testing tasks outlined above there is one main difference locally vs CI. Locally you should set Karma and JsLint to watch your files. This provides instant feedback when developing your code. On your CI tool you will want these tasks to execute once, exit and produce the relevant outputs. Executing your gulp tasks through a CI tool such as Jenkins is super easy. In the case of running your Protractor tests the CI job essentially has two commands:

npm install –> Installs your node packages at build time

gulp e2e –testSuiteName –> Executes Protractor tests

You then just need to configure your CI tool to pick up any outputs generated.

Challenges

So this all sounds pretty straightforward but for me it did take a little while to get everything working smoothly. The main challenges I had:

1. What the hell is Gulp!?!?

Sounds simple, but it actually took me a few hours to actually work out the concepts behind Gulp. Hopefully this post has helped clear that up for you.

2. Handling Node Packages Correctly

Gulp in the most part uses node packages to “Do Stuff” so you’ll need to have a good grasp of how that works. Take the time to understand how to use a package.json, devDependencies, npm install and the differences between installing globally v locally. Interestingly, you’ll also need to install gulp both locally and globally.

3. Folder Structres and File Paths

You’ll need to make sure you have your node packages and gulp.js file in a sensible location.The standard is to install dev node packages within the project and your gulp file in the project root. Your gulp file will reference underlying configure files such as karma.conf.js and also declare output directories so you’ll need to understand how to move around the directory structure using JavaScript.

Alternatives

If you keen to crack on and use Gulp, at this point i’d highlight that there are alternatives out there which I haven’t used, the most popular is Grunt.

Finally, a heads up that Gulp 4 is currently in beta and will be launched soon

How does it feel? The difference between Checking and Testing

Standard

Leaving the office late one evening, after spending most of the day writing automated Checks for our application, i started to reflect on how my emotions. Earlier in the week i’d spent most of the day Testing and Exploring our application. The emotions i’d felt on those days were distinctly different. I decided to keep a track of my emotions to see if there was a correlation between how i felt when doing Checking or Testing.

Let me firstly define the terms Checking and Testing:

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

(A check is an instance of checking.)

Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.

(A test is an instance of testing.)

http://www.satisfice.com/blog/archives/856

What is my Emotional Context?

Before i bare my emotions, let me share my context. I work as a Tester in an Agile Team where everyone is encouraged to be a T-Shaped cross functional team member. This means that both Checking and Testing are very much part of my average week(in addition to other activities). I’ve been working in Software Engineering industry for around 9 years. My early career was focused on Testing but in the past few years i’ve been spending more and more time in front of a Development IDE, such as Microsoft Visual Studio, writing code. This could be automated Checks using WebDriver, writing utility scripts using PowerShell or even developing production code. In terms of my “Programming” ability i’m still on a steep learning curve, especially as i’ve jumped around different development languages from Java, C#, JavaScript to name but a few.

Understanding your Emotions

So reflecting back after a few weeks there was some impact of my activities on my emotions. My next question was “How do i make sense of these emotions?” I’m by no means an expert on emotions but luckily there are some models to help us. Recently i attended Stephen Janaways excellent talk on Testing your emotions at Belgium Testing Days. He talked through the emotions he has experienced in Testing and presented us with the Plutchick Emotional Model:

Plutchik-wheel-2

Using this model i’ve been able to start to identify the different emotions i’ve experienced when Checking or Testing.

At some point in time i’ve experienced all of these different emotions in both activities however there are certain emotions i feel more regularly between the two:

Emotions in Checking Emotions in Testing
Fear “That i wont be able to code the solution”

Apprehension “I’m not sure how to do this”

Ecstasy “I’ve finally cracked it! This coding is easy!”

Boredom “Fixing checks, AGAIN!”

Amazement “It worked!”

Trust “I trust that the team have tried to do their best”

Anger “Why doesn’t it work?!?!”

Joy “We’ve built a really great application’

Annoyance “We had this problem two days ago!”

Anticipation “There must be some problems somewhere”

As well as Plutchicks model i encountered this interesting research paper:

“What Emotions Do Novices Experience During their First Computer Programming Learning Session?” – http://pnigel.com/papers/bosch-csl1-aied13.pdf

Some of the findings resonated with the emotions i’ve experienced writing code for Checks.

“flow/engaged (23%), confusion (22%), frustration (14%), and boredom (12%) were the major emotions students experienced, while curiosity, happiness, anxiety, surprise, anger, disgust, fear, and sadness were comparatively rare.”

When doing Checking i also usually suffer from confusion, frustration, boredom and flow. This is in contrast to Testing where i often feel the emotions of curiosity, happiness, surprise and anger.

Lets dig a little deeper into these differences……

Feeling Engaged and “In the Flow”

Flow was originally coined by Mihály Csíkszentmihályi who said creativity is a central source of meaning in our lives.

“When we are involved in creativity, we feel that we are living more fully than during the rest of life.” He is the architect of the notion of “flow” — the creative moment when a person is completely involved in an activity for its own sake.

http://psygrammer.com/2011/02/10/the-flow-programming-in-ecstasy/

As the study found, Flow/Engaged was a prominent emotion found when programming. This is something i’ve experienced too. Quite often when i’m working on complex coding problems i’ll get into Programming Flow which also has a link to the emotion of Ecstasy.

With Testing, i don’t get that sense of Flow/Engaged, as profoundly. This worries me as Flow is defined as being “absorbed into your activity completely”. Upon reflection i’ve thought of a couple of competing emotions that may prevent me from getting Flow state in Testing. Firstly is being distracted which often happens when encountering a potential problem or a behaviour that interests me. Do i investigate? Do i Stop? Do i keep on my testing path? If the system under test is problematic the “distraction” is intensified. My creative thoughts can also be a distraction. I’m often overwhelmed with the number of potential ideas for Testing.

Confusion

Another interesting difference is the feeling of confusion when Checking compared to Testing. More worryingly there is evidence to suggest that confusion correlates to learning. If i’m not feeling confused regularly in my Testing then potentially my learning has stalled?

This video explores confusion in learning in more detail(Thanks to Llewellyn Falco for the tip):

Why are understanding our emotions important?

Emotions can be used as useful heuristics in everything we do in life. Michael Bolton says that:

Emotions and feelings are signals. Look into what they’re signalling.

http://www.developsense.com/presentations/2013-05-STAREast-EmotionsInSoftwareTesting.pdf

In my case i can look at the emotions i feel between Checking and Testing.

  • I can accept that because i’m on a learning curve i’m bound to be confused and frustrated when Checking.
  • I want to be more confused by Testing. I want to feel confusion in my Testing so that i can be confident my learning is continuing.
  • I want to feel happiness in my Checking knowing the quality and coverage of my code is improving.
  • I want to experience more flow state in my Testing to ensure my creativity is maximised.

How do you feel?

With the increasing emphasis on Testers, as Team Members having T-Shaped skill set, how do you feel?

Maybe like myself you are a Tester who is doing more coding of automated Checks? How do you feel compared to when you Test?

Or maybe you are a developer who has been asked to do some Exploratory Testing. How do you feel compared to when you code?

Testers are second class citizens.

Standard

This week I attend the London Agile Discussion Group which had an interesting debate topic:

“Testers are second class citizens”

To give the evening some structure we were split into two groups, the proposition and the opposition. Each group had 15 minutes to prepare their statement which they would then present back to the audience.

The Proposition

Up first the proposition who agreed testers are second class citizens.

As they began I was actually surprised how passionately they put across their statement. I bit my tongue and resisted the urge to shout “You are wrong!” but at the same time i found it fascinating to hear this side of the debate. The debate spurred up a strange sense of emotions. It made me thankful that i don’t have these discussions in my own team otherwise i’d be a very angry man!

These are some of the arguments put forward by the Proposition.

Testers are second class citizens because……….

– Testers are paid less
– Testers are generally more junior members
– Testers are less skilled than other team members such as developers
– Testers are just a “distraction”
– Testers don’t add any value to the software
– You can make money without testers
– If you do TDD or BDD you don’t need testers
– Why use testers when your customers can do testing for you in Production
– Testing can be done by anyone
– It is commonly sent off shore
– Testers/Testing is an entry level for IT industry
– Testing is not seen as a career
– Testers are failed developers or individuals that want to eventually become developers
– We should aim for generalising specialists not specific roles. So based upon this the “tester” role itself doesn’t exist, as doesn’t “developer”, “BA”. We are all just team members.
– If you had to sack all your developers or all your testers. Who would you sack?

The Opposition

Second up was the opposition. I’d conveniently been placed in this group. There were 3 other testers in my group and as you can imagine we passionately believed in the value testers can add however being testers we did have questions around the debate topic:

– What do we mean by “Tester”?
– To whom are testers second class citizens? Developers? Business Owners? God?
– Who may think this?

To present our opposition we first looked at why people might think testers are second class citizens. We all had experiences where testing had been sent “off shore” maybe to cut costs because testing is seen as an unwanted cost, we also had experiences where developers had thought anyone could test and that as testers don’t add value to the product they are not needed.

These were the arguments we put forward as the opposition:

Testers are NOT second class citizens because……….

– We think its important to look beyond roles in Agile Teams. We are “Team Members”
– There should be no “class system” within agile teams
– Whole Team approach to quality is important
– We value that idea of “T-Shaped” team members and having team members with a deep knowledge and understanding is very, very important. We feel that its important for the following skills to be present(many of which are found in testers):
— Communication
— Problem Solving
— Analysis Skills
— Creativity
— Development/Programming skills
— Understanding the domain/business
— Put the customer first
— Perseverance/Determination

We also identified a relationship between ability and second class citizenship in teams. Often team members, sometimes testers, with less ability can be given less challenging tasks within a team, this can create a “class system” within the team. An example from the testing world would be “Lets get the testers to do the regression testing, we’ve got more important things to do.” This in turn becomes a self fulfilling prophecy, rather than bringing these “second class” citizens into “first class citizenship” we further push them away.

It was a really fascinating debate but upon reflection I was disappointed at how i struggled to clearly articulate my position. Over the past year I’ve worked in a team were everyone is treated with the same level of respect and valued equally(or at least that’s my perception) so i rarely have to justify my position in the team.

I’m planning to spend sometime thinking about how in future i can more clearly articulate why testers are not second class citizens.

I’d be interested in hearing other thoughts on the debate topic, either those for or against. Please leave a comment or tweet me!