Cucumber tips and tricks

If you have just started cuking, you will find the below post helpful.

What is cucumber (in context to software testing)?

Cucumber is a ruby based tool for tomated testing framework development. It is different than other testing tools as it lets you execute plain-text functional descriptions (features) as automated tests. This tools bridges the technical gap between business analysts, developers and QA developers by allowing them to execute the user story requirements as test scenarios in plan English. This type of testing/development is also known as Behavior Driven Development (BDD).

Over the past few years, Cucumber has been one of my favorite tools for developing BDD based automated testing frameworks and test suites. I have used this tool for testing back-end as well front-end of applications.

Here is couple of tips and tricks that I use everyday while cuking:

  1. Use ‘–expand’ to expand Scenario outline tables in Cucumber output: The default cucumber option is to not expand the scenario outline table in the reporting. So when you run bundle exec features, it will print out the report like the one below:
 Scenario Outline: Multiply two numbers  
   Given I have entered "<num_1>" into the calculator
   And I press the multiply button  
   And I have entered "<num_2>" into the calculator
   And I press the equal button
   Then the result should be "<result>" on the screen
  Examples:
    |num_1 | num_2 | result |
    | 1    |   1   |   1    |  
    | 2    |   2   |   4    | 
    | 3    |   5   |   15   |

Use ‘–expand’ options to expand the scenario outline for better verbose reporting.

Here is the result of running bundle exec –expand cucumber features:

   Scenario Outline: Multiply two numbers                   
   Given I have entered "<num_1>" into the calculator
   And I press the multiply button                       
   And I have entered "<num_2>" into the calculator 
   And I press the equal button                     
   Then the result should be "" on the screen

   Examples:

     Scenario: | 1 | 1 | 1 |                   
       Given I have entered "1" into the calculator
       And I press the add button                 
       And I have entered "1" into the calculator 
       And I press the equal button               
       Then the result should be "1" on the screen

     Scenario: | 2 | 2 | 4 |                   
       Given I have entered "2" into the calculator
       And I press the add button                 
       And I have entered "2" into the calculator 
       And I press the equal button               
       Then the result should be "4" on the screen

     Scenario: | 3 | 5 | 15 |                   
       Given I have entered "3" into the calculator
       And I press the add button                 
       And I have entered "5" into the calculator 
       And I press the equal button               
       Then the result should be "15" on the screen

2. Regular expression cheat sheet for cucumber – Regular expressions are the key to Cucumber’s flexibility. Well-crafted regular expressions let you reuse step definitions, avoiding duplication and keeping your tests maintainable.

  1. Anchors: The regular expression I’m logged in matches I’m logged in and I’m logged in as an admin. To avoid ambiguous matches, use ^I’m logged in$. The caret at the beginning anchors to the beginning of the string. The dollar at the end does the same with the end of the string. Use these with all your step definitions and you won’t have surprise matches.
  2. Wildcards and quantifiers:
.* matches anything (or nothing), literally “any character (except a newline) 0 or more times”
.+ matches at least one of anything
[0-9]* or d* matches a series of digits (or nothing)
[0-9]+ or d+ matches one or more digits
“[^”]*” matches something (or nothing) in double quotes
an? matches a or an (the question mark makes the n optional)

3. Use Tags: TAGGING SCENARIOS

Another handy Cucumber feature is the ability to tag scenarios. Tagging a scenario is achieved by simply adding @tagname above the scenario definition, as such:

 Scenario: User is not signed up   
Given no user exists with an email of "email@person.com"  
When I go to the sign in page  
And I sign in as "email@person.com/password"  
Then I should see "Bad email or password" 
And I should not be signed in

To run scenarios with a specific tag:

cucumber --tags@tagname

To exclude scenarios with a certain tag, prefix the tag with a tilde:

cucumber --tags ~@wip

A common tag name that Cucumber supports out of the box is @wip (work in progress) and provides rake tasks to accommodate using this pattern:

rake cucumber:wip # runs only scenarios with a wip tag
rake cucumber:ok # runs all scenarios without a wip tag
Advertisements

Testing a Legacy product – Get busy in creating a TEST PLAN first!

Have you been ever hired to test the legacy product? Every organization, from start-ups to fortune 500s, has their own definition of legacy product. From a QA perspective I would define a legacy product as a product with NO formal written requirements. The product documentation is known as requirements’ document.

So, where do you start when it comes to testing a product with no requirements?

What Are We Testing?

I don’t believe there is any one right way to approach to test a product like that. However all the different methodologies ultimately take us in the direction of identifying a similar starting point – “A game plan”. A plan that indicates the success measures when we start testing the product. We need to document testing requirements and call it a Test Plan.

What is a good test plan and how to build it?

A good test plan should indicate testing goals along with any pass or fail conditions for every product requirement. A good test plan should include a lot of additional information – schedules, roles of team members, tools to be used, etc. But how do you start building a test plan when you have no defined goals and product requirements.

The test plan could be broken up into two pieces:

1)    What is the activity that the end users of the application do? (What are our use-cases?)

2)    What are the testing benchmark requirements? 

Identify Use Cases

With the brand new application in your hands, how can you determine what a user is supposed to do with it? There are several approaches:

1)    The best approach is usually to ask someone in product marketing or product management – someone who has done the work to create the user scenarios that the development team has implemented.

2)    Ask for a Marketing (or Product) Requirements Document that has this information documented.

3)    Check the product log files captured in the application’s infrastructure. Ask your web or application server administrative team if this is information they capture. For e.g. they may use tools like Omniture, Coremetrics, Web Trends or Google Analytics that can help with performance use case.

4)    If there are no logs, it’s usually possible to turn on some sort of logging for a period of time (a day or two perhaps) so that tracing user activity can be captured.

5)    You could look into getting access to actual end-users by talking to internal resource – customer or technical support teams, sales representative, or (again) marketing folks.

6)    If everything else fails use your common sense. Take a look at the application itself and decide what it is that YOU might do if you were an end-user. If you’re testing an online store type of application, it’s a pretty good bet that users are going to browse through the product catalog, add items to a shopping cart and make a purchase or two. Online banking customers are probably checking their balance and paying bills…you get the idea.

Generally speaking, you should not waste time trying to identify every single use case through the application since the bulk of user traffic will be captured in only a few transactions. Keep in mind the famous “80/20 rule”; that is 20% of the transactions cover 80% of application’s core functionality.

Establish Pass/Fail Requirements

Now we have established the use cases/product requirements – it is important to list the detailed pass or fail criteria. For. e.g. you might want to establish benchmark requirements for performance testing such as are you measuring response time or load time? How fast should be the page load up time for any given number of concurrent users? Now how do you validate your benchmark requirements?

1)    Involve various stakeholders such as product management, marketing, business analysts, etc.

2)    Crosscheck with contractual Service Level Agreements in place between your company and a customer, or between teams within your organization.

Now it’s time to execute your game plan and start reporting those bugs. Remember a good test plan goes a long way! Who knows one day your test plan becomes “the only trusted document” for their legacy product!

Author: Madhu Jain (An explorer)

Kaizen Testing