#1) It tries to define the behavior of the system or feature being developed through an example or scenario. For instance, if you are building a simple Calculator Application then the different behaviors include addition, multiplication, division, etc.
Hence through BDD, all the stakeholders will first meet to decide the behavior of the application like Addition and will have scenarios as shown below.
1 | Given, I have 2 numbers 30 and 50 as input |
2 | When I add these 2 numbers |
3 | Then I should get an output of 80 |
If you see the above representation it’s a scenario in plain English that is clearly understandable by anyone and makes the requirements for a feature clear (as per the acceptance criteria). Hence the first step is to articulate these requirements.
#2) Now with a set of these scenarios, the QA writes tests against these and this will initially fail as the feature is not yet developed.
#3) Now, the developer writes a feature code and executes these tests again.
#4) The tests may pass or fail. If they fail – refactor code and repeat the process
#5) Once code refactoring is complete all the scenarios/tests should pass.
Hence, in essence, BDD uses TDD approach and takes it to the next level by having some common easily understandable specifications in the form of scenarios. They also represent the feature documentation in itself.
There are various tools for writing tests in the BDD approach like Cucumber/JBehave for Java, Lettuce for
Python, Jasmine for Javascript,
Specflow for .NET.
In this tutorial, we will be focusing on Specflow.
The Keywords – Given, When & Then
From the unit testing world, most of us are familiar with 3 A’s i.e. Arrange, Act and Assert. Now, Given, When and Then are the replacements for these in the BDD world.
Let’s take an Example for understanding each of these. Suppose you are listing down a scenario for validating a product that gets added to the shopping cart of an e-commerce application which requires you to be logged in as a pre-requisite.
The specification can be written as follows:
1 | Scenario: Products get added to cart for a logged in customer |
2 | Given I have a logged-in customer on my application |
3 | When I add 2 quantity of a product to my shopping cart |
4 | Then the shopping cart should get updated and have the right product and quantity |
Given: This is used for describing a set of pre-conditions for the scenario being defined. For instance, in the example, the scenario’s pre-requisite is a logged-in customer. Hence comparing to the Arrangeanalogy in a unit test, the step implementation will need to ensure that there is a logged in customer.
When: This is used to describe an action or execution step. In the example, it shows that the customer is trying to add a product to his shopping cart. Hence the step implementation for this step will take care of the simulation code to add a product to the cart. This can be compared to the Act step in the Unit tests.
Then: This is used to describe the Outcome of the scenario and essentially where the validations should be placed in. It can be compared to the Assert step in the Unit testing world. In the example here, the step implementation will assert whether the product got actually added and the quantity is the same as that was chosen by the customer.
The Feature File
The feature file is essentially a grouping of multiple scenarios for the application under development or test. It can also be simply thought of as different modules of the application by which the application can be logically separated.
For Example:
An e-commerce application can decide to have different high-level feature files like:
- Login/Logout functionality
- Shopping Cart
- Payment etc.
What Is Specflow?
Specflow is a tool supporting BDD practices in .NET framework. It’s an open source framework hosted on GitHub. It aids in using ATDD (Acceptance test driver development) for .NET Applications.
Binding business requirements for an application using Specification By Example paradigm helps in a better understanding of the application behavior by all the stakeholders and thereby results in shipping the product with correct expectations.
It makes use of
Gherkin syntax for creating features & scenarios. It also has an active discussion/developer
forum.
Specflow – Getting Started
In this section, we will explore installing specflow in the Visual Studio IDE and creating feature files for a simple String Utility Application.
About Sample Application
We will be illustrating different features of the Specflow framework in this tutorial using a Calculator Application which has functions/interfaces to provide different operations like:
- Adding 2 numbers.
- Subtracting 2 numbers.
- Dividing and Multiplying 2 numbers.
- Finding the Square root of the given number.
Specflow Installation Guide
Specflow installation is a 2 step process
#1) Installing the required plugins in the Visual Studio IDE.
- To install the specflow plugin navigate to Tools -> Extension & Updates.
- Now click “Online” on the left panel.
- Now search for specflow in the right panel.
- From the search results select “Specflow for Visual Studio 2017”.
#2) Setting up the project with feature files and step definitions.
- Create a simple new project in Visual Studio. We can create any kind of project like Class Library / Console Application / Unit test project etc. For simplicity, we are taking up a Class Library project. Name the project as “SpecflowBasic”.
- In order to run the Specflow scenarios that we are going to create, we need a test runner. Specflow provides a runner out of the box called Specflow + Runner (which is a paid version and the free version introduces a delay).
(Other runners are also available for NUnit and MsTest which we will see in the further articles in this series).
To install Specflow + Runner – Navigate to Tools -> NuGet Package Manager -> Package Manager Console.
Once the Package Manager Console opens up – Run the command.
1 | Install-Package SpecRun.SpecFlow |
- Also, in order to Assert the values, we will need the help of a test framework. NUnit can be one of the options and the others include MsTest, etc. To install the NUnit framework to the application, open the Package Manager Console and type command.
#3) Create a new class named “CalculatorApplication” which will become our application under test. This is a simple class having functions to perform addition/multiplication/division/square root etc., for the given input. This is how the CalculatorApplication class looks like.
#4) Once the package gets installed, create 2 folders in the project and name them as Features and Step Definitions for storing the feature files and step bindings respectively. We will discuss in detail the reason for this folder organization for Feature & Step definitions.
#5) Now in the features folder, add a new Feature file and name it as CalculatorFeature.
You would see that by default the feature file has some description in Feature and Scenario.
Replace that with what we are going to test.
1 | Feature: CalculatorFeature |
2 | In order to test my application |
4 | I want to validate different operations of the application |
6 | Scenario: Add two numbers |
7 | Given I have provided 70 and 20 as the inputs |
9 | Then the result should be 90 |
11 | Scenario: Substract two numbers |
12 | Given I have provided 70 and 20 as the inputs |
14 | Then the result should be 50 |
16 | Scenario: Multiply two numbers |
17 | Given I have provided 70 and 20 as the inputs |
19 | Then the result should be 1400 |
21 | Scenario: Divide two numbers |
22 | Given I have provided 70 and 20 as the inputs |
24 | Then the result should be 3.5 |
26 | Scenario: SquareRoot of number |
27 | Given I have provided 70 as input |
28 | When I press squareroot |
29 | Then the result should be 8.37 |
#6) Generating Step Definitions: Specflow provides an automated way to generate bindings/implementation for the different steps in feature file scenarios. This can be achieved by right-clicking on the feature file and clicking “Generate Step Definitions”.
This step does not guarantee an implementation for all the steps, but it tries its best to group the common steps in scenarios and re-use as many bindings it can. However, it makes the job of avoiding boilerplate code every time when a scenario step needs to be implemented.
After clicking “Generate Step Definitions”, A window will show up listing the identified step implementations that the processor has detected. One can select or de-select as per the requirements.
In the later sections, we will look into more details about the Style dropdown shown in the above screenshot.
For now, let’s keep all of them selected with default settings. Clicking on the Preview will show a snapshot of how the implementation will look like.
After creating Step definitions, still, if there are some unimplemented steps, the Feature files have a visual way of identifying the un-implemented applications. It shows those steps in a different color by making it absolutely simple to know that there are some steps which don’t have an implementation yet (or are having any ambiguous step definitions).
A Sample Screen Depicts that Below:
Note: The Step definitions can be created manually as well – Any .cs file having [Binding] Attribute is a Step implementation class and the Gherkin syntax will look for matching the implementation of the given scenario step
Execution
As we have already added Specflow+ Runner in the above section, executing the Scenarios is pretty straightforward (since it’s an evaluation version of Specrun, it introduces a variable delay of 10-20s before the scenarios execute. , This delay is not present for registered variants and other flavors of Specrun runner like NUnit and MsTest).
If all the steps have not been implemented and if there are still bindings that have a pending state. Then the output will show as pending.
Let’s try to run these tests/scenarios at this point when there is no implementation for the bindings, and the scenarios are all pending.
Now let’s try to implement the CalculatorApplication class with the methods that we want to test i.e. add, subtract, multiply, divide and sqrt.
Given below is a code sample of how our CalculatorApplication class looks like:
2 | using System.Collections.Generic; |
5 | using System.Threading.Tasks; |
9 | class CalculatorApplication |
11 | public int add(int input1, int input2) |
13 | return input1 + input2; |
16 | public int subsctract(int input1, int input2) |
18 | return input1 - input2; |
21 | public int multiply(int input1, int input2) |
23 | return input1 * input2; |
26 | public double divide(double input1, double input2) |
28 | return input2 != 0 ? Math.Round(input1 / input2, 2) : 0; |
31 | public double squareRoot(int input1) |
33 | return input1 != 0 ? Math.Round(Math.Sqrt(input1), 2) : 0; |
Once the application is ready, let’s try to figure out the ways to implement the bindings for each of the scenario steps.
Let’s see the step by step approach to implement these:
- First, we need to have an instance of the application that needs to be tested. For simplicity, we can instantiate the AUT (Application Under Test class) in step bindings and use the instantiated instance to actually call different methods/functions as per the step that’s implemented.
- To capture the input and output we are declaring variables to hold these values in order to call functions on the Application instance.
Let’s see the end to end implementation for all the bindings involved in validating the Add functionality (Rest of the scenarios are simply extending this).
The Add scenario looks as shown below:
1 | Scenario: Add two numbers |
2 | Given I have provided 70 and 20 as the inputs |
Let’s see the step implementation for each of these individual steps. For use of all the step implementations, we are declaring an instance of Application under test as well as variables to hold input and output variables as shown below:
2 | CalculatorApplication app = new CalculatorApplication(); |
Let’s see the implementation of scenario steps one by one.
Step 1: Given I have provided 70 and 20 as the inputs.
1 | [Given(@"I have provided (.*) and (.*) as the inputs")] |
2 | public void GivenIHaveProvidedAndAsTheInputs(int p0, int p1) |
Here, we have just initialized the input variables with the values passed in from the scenario steps. p0 and p1 are the values that are passed in from the scenario step and will be initialized as 70 & 20 respectively.
Step 2: When I press add.
2 | public void WhenIPressAdd() |
4 | output = app.add(input1, input2); |
This is the Execution (or Act) step where the actual method is called on the Application under test. Notice that since the input variables input1 and input2 already contain the values passed in Step1 the application instance can call the method with these variables.
Step 3: – Then the result should be 90.
1 | [Then(@"the result should be (.*)")] |
2 | public void ThenTheResultShouldBe(double p0) |
4 | Assert.AreEqual(p0, output); |
This is the Validation (or Assert) step where the output is generated by the method call on and the Application instance is validated against the expected output.
Notice, that the Assert keyword used is from NUnit Framework, which returns true or false depending on the validation/expectation that is set. In case it returns false, it will cause the Step implementation to fail and that will show the scenario result as fail.
Also, please note that the output variable gets the value from the previous step where the actual method was called on the application instance.
Similar to the above, Step implementations for rest of the scenario steps are performed in the same way, the difference is in calling different methods on the application instance and asserting different output values.
Once all the Scenario steps are implemented, the tests can be executed.
The resultant output will look as shown below:
You can also view the output of the individual scenario which lists down the output of individual steps as well:
Conclusion
Hope this article would have given you a basic understanding of what BDD is and what are the tools that support BDD for .NET where we covered Specflow.
We also discussed installing and executing Specflow feature files with the help of a sample application.
Code Files
The code files used in the application are shown below:
CalculatorFeatureSteps.cs
2 | using TechTalk.SpecFlow; |
6 | namespace SpecflowBasic.StepDefinitions |
9 | public class CalculatorFeatureSteps |
12 | CalculatorApplication app = new CalculatorApplication(); |
18 | [Given(@"I have provided (.*) and (.*) as the inputs")] |
19 | public void GivenIHaveProvidedAndAsTheInputs(int p0, int p1) |
25 | [Given(@"I have provided (.*) as input")] |
26 | public void GivenIHaveProvidedAsInput(int p0) |
32 | public void WhenIPressAdd() |
34 | output = app.add(input1, input2); |
37 | [When(@"I press substract")] |
38 | public void WhenIPressSubstract() |
40 | output = app.subsctract(input1, input2); |
43 | [When(@"I press multiply")] |
44 | public void WhenIPressMultiply() |
46 | output = app.multiply(input1, input2); |
49 | [When(@"I press divide")] |
50 | public void WhenIPressDivide() |
52 | output = app.divide(input1, input2); |
55 | [When(@"I press squareroot")] |
56 | public void WhenIPressSquareroot() |
58 | output = app.squareRoot(input1); |
61 | [Then(@"the result should be (.*)")] |
62 | public void ThenTheResultShouldBe(double p0) |
64 | Assert.AreEqual(p0, output); |
CalculatorApplication.cs
2 | using System.Collections.Generic; |
5 | using System.Threading.Tasks; |
9 | class CalculatorApplication |
11 | public int add(int input1, int input2) |
13 | return input1 + input2; |
16 | public int subsctract(int input1, int input2) |
18 | return input1 - input2; |
21 | public int multiply(int input1, int input2) |
23 | return input1 * input2; |
26 | public double divide(double input1, double input2) |
28 | return input2 != 0 ? Math.Round(input1 / input2, 2) : 0; |
31 | public double squareRoot(int input1) |
33 | return input1 != 0 ? Math.Round(Math.Sqrt(input1), 2) : 0; |
packages.config
1 | <?xml version="1.0" encoding="utf-8"?> |
3 | <package id="Newtonsoft.Json" version="10.0.3" targetFramework="net461" /> |
4 | <package id="NUnit" version="3.11.0" targetFramework="net461" /> |
5 | <package id="SpecFlow" version="2.4.0" targetFramework="net461" /> |
6 | <package id="SpecRun.Runner" version="1.8.5" targetFramework="net461" /> |
7 | <package id="SpecRun.SpecFlow" version="1.8.5" targetFramework="net461" /> |
8 | <package id="SpecRun.SpecFlow.2-4-0" version="1.8.5" targetFramework="net461" /> |
9 | <package id="System.ValueTuple" version="4.3.0" targetFramework="net461" /> |
No comments:
Post a Comment