In this blog I hope to give an insight into the tools we are using to automate testing at GameSparks.
However I think we should first address why we are using automated testing.
The Benefits of Automated Testing
Automation Eliminates Human Error – An automated test will perform the test the same way every time. No steps will be overlooked or misinterpreted.
Automation Ensures the Test Suite is Up to Date – As the platform grows the test suite can become out of date as changes come through. Changes to the system will be reflected quickly in the Automation results and highlight the areas that require maintenance.
Automation Speeds Up Testing – Automated Test Tools can run through tests much quicker than a manual Tester. This reduces the amount of time it takes to complete a Test suite.
Automated Tests Can Be Run at Any Time – Automated tests can be run outside of normal office hours. A large suite of tests can run overnight or on weekends or at anytime when a manual tester is not available.
Automation Allows for More Effective Use of Test Resources – The time taken to run tests is reduced. The quality of the software and testing process is more reliable and can be easily repeated. This frees up the Testers to concentrate on finding the more complex bugs through ad hoc and exploratory testing.
Automated Tests Can Be Reused – Test scripts can be written to be agnostic of the device/platform they are running against. These scripts can then be reused. Also the UI may change but the scripts should hold up if the underlying behaviour is still the same.
Automated Testing Compliments Continuous Integration – An Automated Test suite can be included in the project’s Continuous Integration. For example a suite of regression tests can be kicked off after the nightly build, allowing the team to easily check the stability of the build the next day.
Automation is a Magic Bullet – the initial setup and step creation will take time and effort. It will also require effort to maintain. This needs to be factored into any planning or estimation.
Everything Should Be Automated – Not everything is suitable for automation. If you have a test that takes 30 seconds to perform but a week to automate it isn’t worth automating. Also some things such as the look and feel of the UI can not be easily automated an will require a manual check.
Automation Replaces Manual Testing – Automation will never replace manual testing. Automation is very good for regression but it can not replace the insight, experience and understanding of a tester. Automation frees up testers to spend more time on exploratory testing, finding defects that automation alone would not encounter.
Cucumber is a tool that executes plain-text functional descriptions as automated tests. The language that Cucumber understands is called Gherkin. Cucumber itself is written in Ruby, but it can be used to “test” code written in Ruby or other languages including but not limited to Java, C# and Python
An example Gherkin script:
Feature: Search courses
In order to ensure better utilisation of courses
Potential students should be able to search for courses
Scenario: Search by topic
Given there are 240 courses which do not have the topic "biology"
And there are 2 courses A001, B205 that each have "biology" as one of the topics
When I search for "biology"
Then I should see the following courses:
| Course code |
| A001 |
| B205 |
As you can see it is very easy to understand what the test is trying to acomplish.
Capybara is a library written in the Ruby programming language which makes it easy to simulate how a user interacts with your application. Capybara can talk with many different drivers which execute your tests through the same clean and simple interface. You can seamlessly choose between Selenium, Webkit or pure Ruby drivers
An Example Capybara feature
When /^I login with "(.*?)" username and "(.*?)" password$/ do |user, password|
fill_in 'username', :with => user
fill_in 'password', :with => password
Ruby is an open source programming language with a focus on simplicity
For those not familiar with Ruby the following links may be of use:
First of all you will need to install Ruby. I would recommend using Ruby Version Manager to do this. Use the following command to install RVM and Ruby:
\curl -L https://get.rvm.io | bash -s stable --ruby
To install Cucumber use
gem install cucumber
And the following command will install Capybara
gem install capybara
Next you will need to create a feature directory. This will hold all of your features (the Gherkin scripts). This folder should also contain two further Directories ‘step_definitions’ and ‘support’.
Cucumber automatically loads all files found in support, we need to create a Ruby file in this folder which has some settings for Cabybara. The contents of this file should be as follows:
Capybara.default_driver = :selenium
The first line lets Cucumber know that we wish to use the Capybara functions, the second line sets the default driver to Selenium. Selenium is used to make calls directly to the browser.
Okay we are ready to create our first feature. For this test I would like to use google to find out more about GameSparks. We will be keeping this as simple as possible.
So in the ‘features’ folder I will create the following Gherkin script:
Feature: Find the GameSparks Website
Scenario: Search for the website
Given I am on the Google homepage
Then I will search for "GameSparks"
Then I should see "Gamesparks"
Then I will click the about link
Right lets run this feature with the Cucumber command:
We get the following output:
Cucumber is not aware of any of the steps within the feature. But it gives us an outline of what it expects the steps to look like. Copy the snippet and then copy it into a new Ruby file in your step_definitions folder. We can then edit the steps to make them work.
Lets start with the first step. Capybara allows you to navigate with the ‘visit’ method. So if we insert the following line into the step it should take us to the google homepage:
Next we need to enter our search term. Again Capybara provides a method for entering text into input fields – ‘fill_in’. However we need to know what the name of the input field is. To do this I use FireBug to inspect the element.
As you can see the name we require is ‘gbqfq’ so lets use this with fill_in for the next step:
fill_in 'gbqfq', :with => searchText
I have also replaced arg1 from the snipper with searchText to save any confusion.
In the next step we are checking that the GameSparks site is returned in the search results. To do this we will use the ‘page.should have_content’ method:
Finally we want to click the link to visit the GameSparks site. Keeping this simple we can assume that the GameSparks site would be the first result. So we just need to find the name of the About link. Again we can use Firebug to inspect the element:
We will then use the ‘click_link’ method:
Putting this all together we should have the following steps in the Ruby file:
Given(/^I am on the Google homepage$/) do
Then(/^I will search for "(.*?)"$/) do |searchText|
fill_in 'gbqfq', :with => searchText
Then(/^I should see "(.*?)"$/) do |expectedText|
Then(/^I will click the about link$/) do
Lets run the feature again.
Here is a video of the test in action
Finally here is a video of a new game being created on GameSparks with Capybara