Feature files

Where feature files should go

A sub-directory of: testing/behat/features

It is recommended that you place them in a sub directory based on the main area of Rogo you are testing, for example if you were testing that the details of a paper could be updated and had the desired effect you might place it in the papers sub-directory.

Feature file coding standards

All Feature files must have a Feature definition, all tests in the file must fit into this description. We prefer more files to unrelated tests in the same file.

Indenting in the tests must be two spaces per level of indentation.

There should be no trailing white space

Template feature file

@feature_wide_tags
Feature: Short descriptive title
  Description of what the users want to do
  As a type of user
  Description of what the user needs to be able to do
 
  Background:
   Given this step runs in all scenarios in the file

  @scenario_specific_tags
  Scenario: Descriptive scenario title
    Given the following "users" exist:
      | username |
      | student1 |
    When I login as "student1"
    Then I should see "No exams found" "content"

For more details about the Gerkin language used in feature files see http://docs.behat.org/en/v3.0/guides/1.gherkin.html

Tags

Tags can be used to define the area of functionality a feature or scenario is testing, feel free to use them liberally. For example if your test is about the creation of creating Likert questions you may tag the test with @question and @likert, this would allow us to easily run any tests for questions in general or Likert questions in particular.

There are some tags that have special meanings beyond this:

Tag	Description
@javascript	The Scenario will use the Selenium2 webdriver and be able to use Javascript.Tests that use this tag will be slower than tests that do not. This tag should not be mixed with the @backend tag
@backend	The test will be run by the backend TestSuite.
@attachfile	Any test that uploads files should use this tag. It will allow us not to run them if we are doing testing in an environment that cannot do uploads (or a driver that does not support attaching files).
@iframe	Any tests that run on pages within rogo that have iframes should have this tag. It will allow us not to run them if we are doing testing using a driver that does not support iframes.
@jsevaluation	Any tests that use steps that require javascript evaluation. It will allow us not to run them if we are doing testing using a driver that does not support javascript evaluation.
@mousemanipulation	Any tests that use steps that require mouse manipulation. It will allow us not to run them if we are doing testing using a driver that does not support mouse manipulation.

Finding available steps

There are two commands you may use to find available steps:

The just lists the commands:

vendor\bin\behat --config <path_to_rogo>\testing\behat\config\behat.yml -dl

The output first tells you which test suite the step belongs to and then after the pipe tells you the step. note that Given, Then, And, But, etc. are interchangeable. If a step starts with /^ and ends with $/ it is a regular expression.

Example step listing output

frontend | Given /^the following "([^"]*)" exist:$/
frontend | Given /^I login as "([^"]*)"$/
frontend | Given /^I click "([^"]*)" "([^"]*)"$/
frontend |  Then /^I should see "([^"]*)" "([^"]*)"$/
frontend | Given /^I set the field "([^"]*)" to "([^"]*)"$/


backend | Given /^there are "([^"]*)" records in the "([^"]*)" table$/
backend | Given I store the database state
backend | Given I reset the database state
backend | Given /^the following "([^"]*)" exist:$/

The second gives a detailed list that includes information about what the step does:

vendor\bin\behat --config <path_to_rogo>\testing\behat\config\behat.yml -di

The first line of each block has the same meaning as the simple list, the next few lines describe what the step does.

Example step listing output

frontend | Given /^the following "([^"]*)" exist:$/
         | Adds records to the database using an appropriate data generator.
         | at `RogoBehatFrontend::the_following_exist()`

frontend | Given /^I login as "([^"]*)"$/
         | Log the user into Rogo.
         | at `RogoBehatFrontend::i_login_as()`

frontend | Given /^I click "([^"]*)" "([^"]*)"$/
         | Click on an element on the page.
         | at `RogoBehatFrontend::i_click()`

frontend | Then /^I should see "([^"]*)" "([^"]*)"$/
         | Checks for the presense of text.
         | at `RogoBehatFrontend::i_should_see()`

frontend | Given /^I set the field "([^"]*)" to "([^"]*)"$/
         | Fill in a form field.
         | at `RogoBehatFrontend::i_set_field()`

backend | Given /^there are "([^"]*)" records in the "([^"]*)" table$/
        | Checks that there are an expected number of rows in a database table.
        | at `RogoBehatBackend::there_are_records_in_the_table()`

backend | Given I store the database state
        | Saves the database state using the testing\behat\helpers\database\state class.
        | This step cannot be called twice without "I reset the database state" being
        | used between the calls.
        | at `RogoBehatBackend::i_store_the_database_state()`

backend | Given I reset the database state
        | Resets the database state using the testing\behat\helpers\database\state class
        | The "I store the database state" step must previously have been called.
        | at `RogoBehatBackend::i_reset_the_database_state()`

backend | Given /^the following "([^"]*)" exist:$/
        | Adds records to the database using an appropriate data generator.
        | at `RogoBehatBackend::the_following_exist()`

Test suites

The version of Behat used in Rogo allows us to define multiple test suites we have two defined.

frontend

This is the default test suite, it uses a browser to simulate student interaction with the user interface of Rogo, either via a real browser or a headless browser

backend

This suite is used if the test is tagged with @backend. It allows test to use PHP unit functions. It should also allow steps to share data between each other using variables that are automatically cleaned between tests with a custom data store

Using the custom data store

The backend test suite will automatically interpret any calls to any non-class defined properties as being a call to the custom data store. So for example if in a step you do:

$this->myvariable = 5;

In a later step of the same test you could call $this->myvariable and it would return 5

isset() and unset() will work on variables stored in the custom data store

Step definitions

The steps definitions hold the code that makes the feature files work.

In Rogo we define our steps in PHP traits

Where step definitions should be located

A sub-directory of: testing/behat/steps

Currently there are 3 sub-directories:

Directory	Notes
backend	The steps that use PHP Unit functions to test code directly should be placed here
database	Steps that generate data for Rogo by adding it directly into the database should be placed here
frontend	The steps that interact with Rogo via a web browser should be placed here

Each file in the sub directory should contain a single trait that defines steps related to a specific area, for example the frontend forms trait should contain steps for interacting with html forms.

The frontend and backend sub directory also have an include_ trait that should use all the other traits.

Creating data

There is a special step that can be used to add data to the Rogo database.

Example of generating data

Given the following "users" exist:
  | username |
  | student1 |

The data type is passed as a parameter i.e. "users" above.

The table passed must include data for all required fields for the data generator, it may include any number of optional fields. If an optional field is omitted a default or random value should be generated for it.

Currently the following data generators are available via this method:

Name	Required fields	Optional fields
users	username	surname, first_names, title, email, roles, gender, special_needs, yearofstudy, user_deleted, password_expire, grade, initials, password
questions	user, type + question type specific requirements	are question type specific from the following list: theme, leadin, notes, display_method, ownerID, q_media_id, q_media, q_media_width, q_media_height, q_media_alt, q_media_owner, q_media_num, creation_date, last_edited, bloom, scenario, scenario_plain, leadin_plain, checkout_time, checkout_authorID, deleted, locked, std, status, q_option_order, score_method, settings, quid, keywords, options
modules	moduleid, fullname	active, schoolid, vle_api, sms_api, selfEnroll, peer, external, stdset, mapping, neg_marking, ebel_grid_template, sms_import, timed_exams, exam_q_feedback, add_team_members, map_level, academic_year_start, externalID
module team members	moduleid, username	not applicable
module keywords	moduleid	keyword
papers	papertitle, papertype, paperowner, modulename	startdate, enddate, labs, duration, session, timezone, externalid, externalsys, calendaryear, remote, password

Where data is expected as an error in a data generator or step we pass this in the feature file as a json string i.e.

@question_true_false
  Scenario: Create a true_false question
    Given I login as "teacher"
    And I follow "TEST1001"
    And I select a "true_false" question type
    And The upload source path is "questions/media"
    And I create a new "true_false" question:
      | theme | textbox theme |
      | notes | textbox notes |
      | scenario | textbox scenario |
      | leadin | textbox leadin |
      | true | 0 |
      | file | {"filename":"frog.jpg","alt":"alternate text"} |
    When I click "Add to Bank" "button"
    Then I should see "Module: TEST1001" "paper_title"

Browser not supported