Protractor: Tips for better End-to-End Tests



So last year I shared my initial experiences of getting started with Protractor. Here are some things i’ve learnt since then.

1. Use testSuites

As your Protractor tests grow you are undoubtably going to make use of testSuites. This allows a specific group of tests to be run by passing in the testSuite name into the run parameters.

2. Use only one protractor configuration file

Protractor uses a configuration file which contains various settings for your tests such as baseUrl, tests to run, web driver port etc. When running tests in different places it is tempting to have multiple configuration files. For example a local config and one for your CI server. Avoid this temptation and only have one config file.

3. Manage your Protractor package and dev dependencies

Keeping Protractor up-to-date as well as other packages, such Screenshot Reporter, which you depend upon is really important. If you don’t manage your packages they will quickly become out of date. Luckily using a package.json to store all your packages makes this super easy. You can declare if you want to auto update or use a specific package version when you run npm install.

4. Use a Task Runner such as Gulp

Your protractor tests will just be a small part of your JavaScript eco system. Using a Task runner can help to keep all your tasks in a single place. It also means it’s very simple for anyone in the team to run the Protractor Tests.

5. Learn how to Debug Protractor Tests

My experience is that writing E2E WebDriver tests in Java or C# is a lot easier because the IDE, such as Visual Studio, gives you lots of guidance. As Protractor Tests are JavaScript you get less help so need to understand how to debug. The two main options are browser.pause() and browser.debugger(). Aside from the Protractor debug commands I’d also recommend learning how to use browser development tools such as Chrome Dev tools to help debug JavaScript.

6. Use Protractor for non-angularJs sites only when necessary

Protractor can be used to test non-AngularJS sites but i’d only recommend this when testing a user-flow which jumps out of your AngularJS site, for example page redirects. I would suggest not using this capability to mean you can test non-Angular sites using Protractor.

7. Use other node packages to do stuff

The joy of using Node is that Node Package Manager allows you to do loads of stuff. An example is http-request which you can use to call API’s in your test flow. This is particularly useful if you want to setup some data via a direct API call.

8. Use Page Object Model

Just do it.

9. Multi Thread your Tests

In Protractor it’s so easy to run your tests in parallel and across multiple browsers. Even with a handful of tests you are going to have to look at running tests in parallel for time saving benefits. Running tests in parallel also forces you to have independent tests which is a good way to keep tests maintainable.

10. Handling Cookies and Sessions

Protractor by design does not open/close the browser between specs. This is useful in one way because it speeds up your tests. Closing and Opening browsers between tests has a significant footprint. There are cases though when you need to potentially close a browser to at least clear cookie or session history. In this case you have a couple of options. Firstly WebDriver can interact with cookies or alternatively you can add restartBrowserBetweenTests: true to your protractor config file.

11. Learn AngularJS

Having an understanding of the underlying technology is a huge help in writing good tests. As you’ll probably be using Protractor to tests AngularJS sites then you should definitely learn Angular. Build your own site in Angular and write some tests for it. There are tonnes of great sessions on Pluralsight to help. Learn also how to unit test AngularJS code as this will give you a better appreciation of where to place your test efforts.

12. Use ES6 Generators

At AngularConnect Uri Shaked did a great talk on how to use ES6 generators to control the flow of your Protractor Tests. This is particularly useful if you need certain tasks to occur in a  particular sequence.

13. Understand what Protractor does

Protractor is a wrapper around WebDriver specifically designed for AngularJS apps. This means most of the benefit will come when using AngularJS apps. The two primary benefits i’ve found are:

  • Page synchronisation – As protractor hooks into the angular digest cycle it makes one of the typical problems with E2E tests, When has the page loaded?, problems a lot easier to manage.
  • Angular Specific Locators – by.Binding, by.Model, by.Repater gives you more options for finding stuff on a page.

With this in mind i wouldn’t ever consider using Protractor for a site which isn’t built in AngularJS.

Thats more than enough tips for now.

Enjoy Protractor.

Are you a Super Recogniser?


In 2015 I made a discovery; I have a super power. Unfortunately it’s not the power of invisibility, the power of ultimate strength or the power to fly. Actually, it’s not really a Super Power but if you’ll let me indulge, it does have Super in the name. My power is “Super Recognition”

Super Recognisers are the people who quite literally never forget a face, in possession of an extraordinary ability to recognise men, women and children they barely know.

Dr Josh Davis, a forensic facial identification expert from the University of Greenwich, is conducting a study into super-recognisers and their abilities.

“We have tested them on passport images taken 10 years [ago] and they are still able to recognise where they’ve seen faces before,” he said.

“We think super-recognition is nature, rather than nurture, but I can’t say 100%. People tend to emerge in their 20s and 30s, we’re not really finding any super-recognisers in their teens so far.” [1]

My “Super Power” was confirmed after taking a test by Greenwich University. The test presents you with an array of faces and you need to match them in a subsequent sequence of photos. Incredibly, I only got 1 answer wrong which suggested I may be a “Super Recogniser”

The results of the test were not really a  suprise. Since my early 20’s I’d always had a strange ability to recognise faces with incredible accuracy. For me, it doesn’t stop at faces actually, most objects and places I can quickly match to a visual memory.

A perfect example is travelling on holiday. We’ve all been in that situation, waiting with a few hundred other people in the departure lounge to board a flight. To the frustration of my partner, every time i travel with my partner I recognise faces of people I’d seen for only a few seconds in the departure lounge. On the return flight I’d be able to identify people who’d been on the same flight and sometimes even people who had been on a previous flight maybe the year earlier.

The UK Metroploitan police have been using “Super Recognisers” to fight crime:

New Scotland Yard deploys an elite team of 140 officers across London to try to capture the most wanted criminals.

PC Gary Collins is the Met’s top super-recogniser and has identified more than 800 suspects from photographs, CCTV and his time policing the streets.

His beat is Hackney, one of the capital’s worst areas for crime.

“Whenever an incident happens they’ll call me in and show me the footage straight away.

“I’ll look [at it] and say, ‘Yeah I know that person, I know him from this area or I stopped him on this occasion,’ and it’s just putting a name to the face.” [1]

Using the Power

I started thinking;

“Have I been using my “Super Recognition” power in my day job within the IT Industry?”

Most of my experience has been involved with Software  Testing which encompasses exploring software to discover important information for my employers and stakeholders. Part of obtaining information is recognising when something isn’t right. I do believe my super recogniser power has given me a slight advantage. A quick recognition of unexpected changes means I can obtain useful information quicker and more frequently. It also means changes potentially undetected by others might be picked up when I get involved.

In many cases, automation can be applied to check differences introduced to software in the same way facial recognition software can be used to detect faces. 

Super-recognisers are not the only tool open to the authorities when chasing a face.
The ever-expanding field of facial recognition software offers the mechanical alternative to human talent, the science against the art.

But which offers the best chance of catching the criminal, now and in the future? [1]

The Metropolitan Police have found human “Super Recognition” can be highly complementary to automatic approaches which matches my experience in Testing.

Thinking Differently

It’s quite common for people in the software industry to say Developers and Testers think completely differently. Whilst I agree and disagree with that statement I do believe that the power of “Super Recognition” does help in my day-to-day job as a tester.

With every super power, there are downsides of course. Now knowing my super power is likely to lead me more towards confirmation bias. In the same way superman doesn’t think “Can I still fly?” before he jumps off a building maybe I’ll forget to think “Have I recognised this correctly?”

Are you a Super Recogniser?

Take the test to find out here and compare your score to mine!

If you score highly you can take a longer test to really put your super power to the test!

[1] –

Using Gulp for Testing


Last year I shared some posts on testing AngularJS web apps:

Getting started with Protractor to automate E2E tests for AngularJS apps

How to steep a unit test environment for AngularJs Application

I thought i’d start 2016 by sharing some other tips in that area.

What is Gulp?


Gulp is a JavaScript Task Runner. Now it took me a while to actually get my head around what Gulp actually does. Gulp enables you to encode in JavaScript repetitive tasks that you perform regularly. This could be starting a web server, JavaScript minification or Testing which will be the focus of this blog.

One of the core benefits of Gulp is that it encodes these tasks in a JavaScript file which is typically named gulp.js. This means all of these encoded tasks can be shared across the team and kept in synch using version control.

Another benefit is that gulp utilises the wide variety of packages available via NPM(Node Package Manager) which means you can automate tasks incredibly easily.

In this blog I won’t go through how to setup Gulp from scratch or how to install Node Packages as the official documents do this very well. Instead my focus will be how you can use Gulp to support testing.

If you want to get started from scratch in addition to the official docs i’d highly recommend John Papas Gulp course on Pluralsight – JavaScript Build Automation with GulpJs

Note: if you don’t currently have a Pluralsight account you can get 3 Months FREE by creating a free Visual Studio Dev Essentials account

How to use Gulp for Testing

Gulp can be used for many tasks but in this blog post I’ll focus on tasks that can be used to support testing.

  1. Run Unit Tests

This task runs your JavaScript unit tests using Karma. It utilises your karma configuration file which allows you to configure run parameters, such as single run and also what outputs we want to generate, such as Istanbul Code Coverage reports and jUnit XML. You can read more about configuring your unit tests in my previous post

Using Karma you don’t need to install any additional gulp packages as you can call Karma directly through Gulp as explained here

The gulp task looks as follows:


2. Run Protractor E2E Tests

As detailed in a previous blog post, Protractor, is an E2E testing framework mostly used for AngularJS web apps. This gulp task allows you to execute your protractor tests. You’ll need to make some additional tweaks to the task to allow a test suite to be specified. We use protractor test suites to allow control of what tests are run. To do this we used an NPM package called yargs. This blog helps.

What is cool with this task is that in a single task it will get the latest web driver jar, start a selenium server, run the tests and generate the output.

To run your protractor tests using gulp you’ll need to install a couple of node packages:

Gulp Protractor

Yargs (Used for specifying testSuites)

The gulp task looks as follows:



3. Linting

Linting is an activity to discover common mistakes in code and is incredibly useful for JavaScript as its not a compiled language. It’s a great example of static testing that can yield great benefits. Here is an example of the output you get:


Using this gulp task you can Lint your code to identify any simple mistakes or errors keeping the code base clean.

To do the linting you need to install the node package JsHint-Stylish

The gulp task looks as follows:



Putting it all together


To run these tasks you simply start up a nodeJs command prompt and enter gulp followed by your task name. For example:

gulp protractor –testSuiteName

This will run your specified e2e protractor test suite.

As NPM gives you such a wide variety of options there are many more tasks you can add to support your testing. These are probably the three most common. Another example could be starting your mock services.

Local vs CI

You obviously want to avoid duplication so you should use the same Gulp file both locally and on your CI tool such as Jenkins. To achieve this you just need to learn about file paths and ensure your gulp.js file is located sensibly, such as within the project directory.

When running any of the three testing tasks outlined above there is one main difference locally vs CI. Locally you should set Karma and JsLint to watch your files. This provides instant feedback when developing your code. On your CI tool you will want these tasks to execute once, exit and produce the relevant outputs. Executing your gulp tasks through a CI tool such as Jenkins is super easy. In the case of running your Protractor tests the CI job essentially has two commands:

npm install –> Installs your node packages at build time

gulp e2e –testSuiteName –> Executes Protractor tests

You then just need to configure your CI tool to pick up any outputs generated.


So this all sounds pretty straightforward but for me it did take a little while to get everything working smoothly. The main challenges I had:

1. What the hell is Gulp!?!?

Sounds simple, but it actually took me a few hours to actually work out the concepts behind Gulp. Hopefully this post has helped clear that up for you.

2. Handling Node Packages Correctly

Gulp in the most part uses node packages to “Do Stuff” so you’ll need to have a good grasp of how that works. Take the time to understand how to use a package.json, devDependencies, npm install and the differences between installing globally v locally. Interestingly, you’ll also need to install gulp both locally and globally.

3. Folder Structres and File Paths

You’ll need to make sure you have your node packages and gulp.js file in a sensible location.The standard is to install dev node packages within the project and your gulp file in the project root. Your gulp file will reference underlying configure files such as karma.conf.js and also declare output directories so you’ll need to understand how to move around the directory structure using JavaScript.


If you keen to crack on and use Gulp, at this point i’d highlight that there are alternatives out there which I haven’t used, the most popular is Grunt.

Finally, a heads up that Gulp 4 is currently in beta and will be launched soon

Pedestrian Crossings, Continuous Integration and Culture



You have just arrived into a new country, checked into your hotel and ready to head out exploring the area. Soon after leaving the hotel you will face your first challenge; the Pedestrian Crossing.

I’ve been lucky to visit a few different countries and not so lucky to experience some of the world’s wildest Pedestrian Crossings (Could make a great TV Show!). I live in the UK where Pedestrian Crossings are, at least to me, well signposted and generally observed both by pedestrians and drivers(with exceptions).Across the rest of Europe you’ll find experiences, in particular Italy. My strategy when i visit; Step out onto the pedestrian crossing, hope the driver notices you and stops, if he doesn’t get ready to run! Seems like i’m not alone! Other parts of the world such as India, Thailand, Vietnam introduce other examples of even riskier behaviour to cross the street.

Japan is one of the most fascinating countries i’ve visited. Incredible cities, beautiful countryside and very obedient pedestrians. Late one night, at around 3am, i was returning to my hotel. The roads were empty and not a vehicle in sight. Regardless to the apparent lack of danger, if the “Wait” sign was shown, pedestrians did not cross.


Pedestrian Crossings and Culture

As you travel the world you realise culture has a huge impact on pedestrian behaviour. An international research team did a study into the impact of Culture; Different risk thresholds in pedestrian road crossing behaviour: A comparison of French and Japanese approaches.

“The group set up observations posts at various crossings in the French city of Strasbourg and the Japanese city of Inuyama. The results were both clear and striking. At the legal crossing in Strasbourg, France, about 67 percent of pedestrians crossed against the red light. Only 7 percent of walkers in Inuyama did the same. At the unmarked crossing, the researchers measured how much time between cars a pedestrian needed before deciding to walk. In Strasbourg that time clocked in at 9 seconds, on average; in Inuyama, walkers felt free from risk at 16-second gaps.”

These findings match my anecdotal experience across Europe and in Japan.

Continuous Integration

After reflecting on a near death experience at one crossing in Bangkok recently, Continuous Integration came to mind. Continuous Integration is a practice now widely adopted across the industry. Join a development team and they will probably be using tools such as Jenkins, Bamboo and Team City to monitor the build process. All these tools use indicators very similar to a Pedestrian Crossing.

Lets take Jenkins for example:


Blue Ball = Everything appears Good.

Red Ball = Something went wrong.

These coloured indicators look simple but in the same way as pedestrian crossings, they influence behaviour in different ways.

Lets take the following two teams:

Team A

The build fails and a Red light is shown on the Jenkins dashboard. The team stops, identifies the source of the failure, perhaps by nudging the last person who checked in. The source of the problem is identified, rectified and build is run returning to normal.

Team B

The build fails and a Red light is shown on the Jenkins dashboard. The team ignores the failure, continues checking-in and the builds continue to fail. Later that day, a developer in the team identifies a compile issue locally, checks-in a fix. The build returns to normal.

How do you cross the street?

What actions we take based upon the build indicator is influenced  by a number of factors. I’ve seen teams confidence of build indicators eroded by false positives/negatives leading to ignorance of builds. However, the biggest factor in my experience is team culture.

In the example of Team A, they are clearly influenced by their CI tool and a culture to investigate problems quickly and implement solutions. However in the example of Team B, their culture is to ignore the build failure potentially with the view that it will be resolved at a later date.

So what is your teams behaviour towards your Continuous Integration tool?

Are you like the Japanese? Do you stop when the build goes Red?


Are you like the French who in most cases ignore the warning signs?

In a perfect world we’d all agree that if the pedestrian light says stop, we would all agree thats the best action to take. However, i’m sure we have all crossed the road, ignoring the sign.

Maybe it’s the same for build indicators, if the build is Red, maybe its OK to ignore……….sometimes?

Ignorance is bliss

Ignoring builds might be ok in some cases but the danger comes when we grow a team culture that blindly ignores the warning signs. The occasional ignored failed build can often develop into a culture where build failures are always ignored. This culture might yield some good results but it’s likely to result in some near misses, like this gentleman: