Protractor: Tips for better End-to-End Tests

Standard

protractor-logo

So last year I shared my initial experiences of getting started with Protractor. Here are some things i’ve learnt since then.

1. Use testSuites

As your Protractor tests grow you are undoubtably going to make use of testSuites. This allows a specific group of tests to be run by passing in the testSuite name into the run parameters.

2. Use only one protractor configuration file

Protractor uses a configuration file which contains various settings for your tests such as baseUrl, tests to run, web driver port etc. When running tests in different places it is tempting to have multiple configuration files. For example a local config and one for your CI server. Avoid this temptation and only have one config file.

3. Manage your Protractor package and dev dependencies

Keeping Protractor up-to-date as well as other packages, such Screenshot Reporter, which you depend upon is really important. If you don’t manage your packages they will quickly become out of date. Luckily using a package.json to store all your packages makes this super easy. You can declare if you want to auto update or use a specific package version when you run npm install.

4. Use a Task Runner such as Gulp

Your protractor tests will just be a small part of your JavaScript eco system. Using a Task runner can help to keep all your tasks in a single place. It also means it’s very simple for anyone in the team to run the Protractor Tests.

5. Learn how to Debug Protractor Tests

My experience is that writing E2E WebDriver tests in Java or C# is a lot easier because the IDE, such as Visual Studio, gives you lots of guidance. As Protractor Tests are JavaScript you get less help so need to understand how to debug. The two main options are browser.pause() and browser.debugger(). Aside from the Protractor debug commands I’d also recommend learning how to use browser development tools such as Chrome Dev tools to help debug JavaScript.

6. Use Protractor for non-angularJs sites only when necessary

Protractor can be used to test non-AngularJS sites but i’d only recommend this when testing a user-flow which jumps out of your AngularJS site, for example page redirects. I would suggest not using this capability to mean you can test non-Angular sites using Protractor.

7. Use other node packages to do stuff

The joy of using Node is that Node Package Manager allows you to do loads of stuff. An example is http-request which you can use to call API’s in your test flow. This is particularly useful if you want to setup some data via a direct API call.

8. Use Page Object Model

Just do it.

9. Multi Thread your Tests

In Protractor it’s so easy to run your tests in parallel and across multiple browsers. Even with a handful of tests you are going to have to look at running tests in parallel for time saving benefits. Running tests in parallel also forces you to have independent tests which is a good way to keep tests maintainable.

10. Handling Cookies and Sessions

Protractor by design does not open/close the browser between specs. This is useful in one way because it speeds up your tests. Closing and Opening browsers between tests has a significant footprint. There are cases though when you need to potentially close a browser to at least clear cookie or session history. In this case you have a couple of options. Firstly WebDriver can interact with cookies or alternatively you can add restartBrowserBetweenTests: true to your protractor config file.

11. Learn AngularJS

Having an understanding of the underlying technology is a huge help in writing good tests. As you’ll probably be using Protractor to tests AngularJS sites then you should definitely learn Angular. Build your own site in Angular and write some tests for it. There are tonnes of great sessions on Pluralsight to help. Learn also how to unit test AngularJS code as this will give you a better appreciation of where to place your test efforts.

12. Use ES6 Generators

At AngularConnect Uri Shaked did a great talk on how to use ES6 generators to control the flow of your Protractor Tests. This is particularly useful if you need certain tasks to occur in a  particular sequence.

13. Understand what Protractor does

Protractor is a wrapper around WebDriver specifically designed for AngularJS apps. This means most of the benefit will come when using AngularJS apps. The two primary benefits i’ve found are:

  • Page synchronisation – As protractor hooks into the angular digest cycle it makes one of the typical problems with E2E tests, When has the page loaded?, problems a lot easier to manage.
  • Angular Specific Locators – by.Binding, by.Model, by.Repater gives you more options for finding stuff on a page.

With this in mind i wouldn’t ever consider using Protractor for a site which isn’t built in AngularJS.

Thats more than enough tips for now.

Enjoy Protractor.

Advertisements

Getting started with Protractor to automate E2E tests for AngularJS apps

Standard

Update 17-01-2016: Added Protractor: Tips for better End-to-End tests

In my previous blog post I mentioned that our team had recently started building an AngularJS application. Well we are a few more weeks further along and we’ve spent the last few weeks overcoming some initial challenges such as using spies, injecting services and debugging the tests.
When we started developing our AngularJS app during my research into testing options I came across Protractor:

“Protractor runs tests against your application running in a real browser, interacting with it as a user would.”
Protractor

image

In this post i will talk through our decision process for automating, why we chose protractor and how we initially got started.

Why automate through front end?

At this point i think it’s important to share our decision process for choosing why and what to automate. As a team we have used automated E2E tests using WebDriver in the past for one of our other products, but we didn’t really get much return for our effort. Our main product is quite hard to automate and at the time we weren’t really in a great position to automate. Since building our AngularJS app we wanted to revisit our E2E automation. Thankfully this time round we had the following in place:

  • Good foundations through Unit Tests and Integration Tests
  • Testable application through sensible element naming and markup
  • At least one E2E journey developed
  • Critical user journeys to be automated are agreed with team

This meant we were in a much better starting position. We were also clear on our objective for the automated front end tests:

“After each deployment ensure our critical user flows continue work E2E”

We initially identified 4 user journeys. We also made the decision that our E2E automation would not be part of every user story. Instead we would automate at a “Feature” level for complete user flows. This doesn’t mean we don’t automate on each story, we still of course do lots of unit and integration tests which are backed up by lots of exploratory testing.

This article by Brian Marick also gives a good approach on deciding what to automate and when.

So now you know a bit more about Why, What and When lets go back to Protractor….

Why use Protractor rather than WebDriver out of the box?

So most people have probably heard of WebDriver. You can automate tests in a variety of different languages and we could have automated our AngularJS app without Protractor. One of the main draws of Protractor for our team was its synchronization which can be one of the challenges when automating websites.

“You no longer need to add waits and sleeps to your test. Protractor can automatically execute the next step in your test the moment the webpage finishes pending tasks, so you don’t have to worry about waiting for your test and webpage to sync.”
Protractor

Installing Protractor

Thankfully it’s a pretty similar process to other NodeJS stuff:

  1. Install packages
  2. Do your Config
  3. Run through NodeJS command prompt

The Protractor site has a good guide on how to do the basic setup.

Page Object pattern

Design of automated tests is really important and being familiar with the Page Object Pattern we wanted to follow a similar approach for our Protractor tests. Thankfully we came across this excellent article.

Data

Another important part of any automated testing is handling your data. Thankfully you can do this quite easily with protractor. You can declare your test data in your protractor config and then read it into your test. Here’s how to do it.

Parallel running of tests

Speed of tests is also important to give the team quick feedback. Although we only have 4 user journeys automated we wanted to get parallel running of the tests in early. Again Protractor supports this out of the box. Here is how you do it.

With this and the synchronization benefit of protractor mentioned earlier we find the tests run super quick. To run our 4 scenarios in Firefox it takes around 3 minutes.

Reports

Knowing what’s passed and more importantly failed is of course a must. We have used the Protractor-HTML-Screenshot-reporter for this. It gives you a basic report template showing your tests and a screenshot if the tests fail.

image

Jenkins

The final part of getting everything up and running was to create a Jenkins job to run our tests. This was again super easy. Just install everything you need on your Jenkins server and push the relevant configuration files. Then all you do is add a command to the job to run protractor. Our tests run after every deployment to our test environment and we also display our reports within Jenkins for each test run. Super.

Summary

So far the tests are giving us some good feedback. A big benefit we have found with protractor is the synchronization. We haven’t got any thread.sleeps or waits within our test code and so far we haven’t had any synchronization problems which has meant a lot less false negatives. Our next steps are to abstract our data out further in JSON files, change our tests so they run in a specific firefox profile we control and also introducing Internet Explorer. At this point we probably won’t expand the number of scenarios right away but instead invest time to improving the capabilities of the test code and making it more robust.