Home .NET SpecFlow and an alternative approach to testing

SpecFlow and an alternative approach to testing

by admin

SpecFlow and an alternative approach to testing
Testing with SpecFlow has become firmly embedded in my life, on my list of must-have technologies for a "good project." Furthermore, despite SpecFlow’s focus on unit tests, I’ve come to believe that even unit tests can benefit from this approach. Of course, in writing such tests the people from BA and QA will no longer be involved, but only the developers themselves. Of course, for small tests this brings considerable overhead. But how much nicer to read a human description of a test than a bare code.
As an example, here is a test converted from the usual MSTest test type to the SpecFlow test type
initial test

[TestMethod]public void CreatePluralName_SucceedsOnSamples(){// setupvar target = new NameCreator();var pluralSamples = new Dictionary<string, string>{{ "ballista", "ballistae" }, { "class", "classes"}, { "box", "boxes" }, { "byte", "bytes" }, { "bolt", "bolts" }, { "fish", "fishes" }, { "guy", "guys" }, { "ply", "plies" }};foreach (var sample in pluralSamples){// actvar result = target.CreatePluralName(sample.Key);// verifyAssert.AreEqual(sample.Value, result);}}

test in SpecFlow

Feature: PluralNameCreationIn order to assign names to Collection type of Navigation PropertiesI want to convert a singular name to a plural name@PluralNameScenario Outline: Create a plural nameGiven I have a 'Name' defined as '<name> 'When I convert 'Name' to plural 'Result'Then 'Result' should be equal to '<result> 'Examples:| name | result || ballista | ballistae || class | classes || box | boxes || byte | bytes || bolt | bolts || fish | fishes || guy | guys || ply | plies |

Classical approach

The example above does not refer to the alternative approach I want to talk about in this post, it refers to the classical approach. In this very classical approach the "input" data for the test is specially created in the test itself. This phrase can already serve as a clue as to what is "alternative".
Another, slightly more complicated example of classic data creation for a test, with which you can then compare the alternative :

Given that I have a insurance created in year 2006And insurance has an assignment with type 'Dependent' and over 70 people covered

These lines, which I will further call steps, correspond to the following lines with the code :

insurance = new Insurance { Created = new DateTime(2006, 1, 2), Assignments = new List<Assignment> () };insurance.Assignments.Add(new Assignment { Type = assignmentType, HeadCount = headCount + 1 });

There is also code in the definition of the test steps to pass these objects between steps, but for the sake of brevity this code has been omitted.


I want to mention right away that this approach was found and applied not by me, but by a colleague of mine. On hubra and on github he is registered as gerichhome I took it upon myself to describe it and publish it. Well, maybe, according to the Hubra tradition, a comment will appear which will be more useful than the article and it will turn out that I didn’t waste my time.
In some cases, as in the case of our project, to display the portal page you need a considerable amount of data. And to test some particular feature, only a small portion is needed. And to make sure the rest of the page won’t crash because of lack of data, we’ll have to write a lot of code. And what’s worse, you will most likely have to write a certain number of SpecFlow steps. So you end up with a whole page to test, not the part you need at the moment.
And to get around this, you can not create data, but search among the existing ones. The data can be either in a test database or in moc files compiled and serialized on a slice of some API. Of course, this approach is more suited to the case where we already have a lot of functionality, at least the part that will allow us to manipulate that data. So if you don’t have the right set of data for a test, you can first walk through the test script "by hand", make a mold of the data, and then automate it. This approach is convenient to use when there is a desire and/or need to cover existing code with tests and then refactor/rewrite/expand and not be afraid that functionality will break.
As before, the test requires an Insurance object created in 2006 with a Dependent type and more than seventy people covered. Any of the insurances stored in the database contains many other entities, in our project the model took up more than 20 tables. For the demonstration I didn’t use the full model, my simplified model includes only three entities.
In order to find the right insurance, you need to somehow define the source as IEnumerable<Insurance> and change the step definitions to the following :

insurances = insurances.Where(x => x.Created.Year == year);insurances = insurances.Where(x => x.Assignments.Any(y => y.Type == assignmentType y.HeadCount > headCount))

The next steps are to open the portal by passing the ID of the first insurance found in the HTTP request line and actually test the desired functionality. Specifics of these actions, of course, are far beyond the scope of this topic,

Search algorithm

So, the insurance is found, we have opened the portal, we have checked that the name of the insurance is displayed in the right section of the UI, now we need to check that the other section displays the right number of deponents. And then the question arises, how do we know in this step which Assignment allowed our insurance to pass under this condition? Which one of, say, five to compare its HeadCount with the number on the UI?
That would require repeating that condition in the "Then" step, and duplicate code is obviously a bad thing. In addition, duplicating conditions would also have to be repeated in SpecFlow steps, which is totally unacceptable.
A lyrical digression – we already had something similar on one of our projects, there was a sql-query (for simplicity let it come from configs), which searches for people, returning a list of SSNs. These people, in the scenario, were supposed to have a child at least 18 years old. And the business people were discussing for a long time, they could not understand why we could not decompose this request to find for a particular person those children who fit the condition. They could not understand why we needed a second query. And since there is a vision of a bright future in which BAs are the ones writing the test text, explaining why we need duplication in steps is much harder than eliminating that duplication, and that’s the first problem the search algorithm solves.
In addition, in the normal search given in the previous paragraph, the second step cannot be split into two SpecFlow steps. This is the second problem solved by the algorithm. We will talk about this algorithm and its implementation.
The algorithm is schematically shown in the following picture :
SpecFlow and an alternative approach to testing
The search works quite simply. The initial collection of root entities, in this case insurances, is represented as IEnumerable<IResolutionContext<Insurance> > . Only those insurances that satisfy their own conditions, and have satisfying collections of child entities, get into it. In order to designate these child entities, you need to register a so-called provider with a lambda of type Func<T1, IEnumerable<T2> > .
Subsidiary collections can also have conditions, the simplest being Exists. With this condition, a collection will be considered valid if it has at least one element that satisfies its own conditions.
Eigenconditions, aka filters, are lambdas like Func<T1, bool> in the simple case.
The picture shows a case where two insurances were found that fit all conditions, the first one has some number of Assignment objects, of which 4 fit conditions, and also some number of Tax objects, of which two fit conditions. Also for the second insurance there are 3 suitable Assignment and 4 Tax.
And, although it seems obvious enough, it is worth pointing out that in addition to the insurances that did not fit their own conditions, the list also did not include those insurances that could not find suitable Assignment objects or could not find suitable Tax objects.
The red arrows indicate the interaction tree of concrete elements. A particular Assignment element has only one link up, it "knows" only about the particular Insurance element that produced it, and "does not know" about the Assignments collection it is in, much less about the Insurances collection; there can be no collection of parent elements for the element.
From here on there are descriptions and examples of how to use the implementation I developed and published in NuGet (link at the end). My colleague, on the other hand, created his own implementation of the search algorithm, which differs mainly in that it pulls the registration network into a chain, i.e. a tree with a single branch. His implementation has some features that I don’t have and some disadvantages. Also its implementation has some unnecessary dependencies, which slightly hinder to put the solution into a separate module. In addition, either for political reasons or personal, he’s not particularly eager to publish his solutions, which would make it somewhat difficult to use it in other projects. I had free time and a desire to implement some interesting algorithm. That’s how I approached writing this article.

Entity and filter registration

For maximum granularity, the steps are broken down as follows.

Given insurance A is taken from insurancesSource #1And insurance A is created in year 2007 #2And for insurance A exists an assignment A #3And assignment A has type 'Dependent' #4And assignment A has over 70 people covered #5

This granularity may look a bit redundant, but it allows to achieve a very high level of overuse. When writing tests on a search algorithm, having 5-6 written tests, all the following tests consisted of overused steps by a good three quarters.
In order to register such sources and filters, the following syntax is used

context.Register().Items(key, () => InsurancesSource.Insurances); #1context.Register().For<Insurance> (key).IsTrue(insurance => insurance.Created.Year == year); #2context.Register().For<Insurance> (insuranceKey).Exists(assignmentKey, insurance => insurance.Assignments); #3context.Register().For<Assignment> (key).IsTrue(assignment => assignment.Type == type); #4context.Register().For<Assignment> (key).IsTrue(assignment => assignment.HeadCount > = headCount); #5

The context used here is (TestingContext context) injected into classes containing definitions. All lines in the SpecFlow test are numbered only to indicate the correspondence with definitions, the order of these lines can be anything. This can be useful when using the "Background" feature. Such freedom of registrations is achieved by the fact, that provider tree is built not during the registration itself, but when result is first received.

Retrieving search results

var insurance = context.Value<Insurance> (insuranceKey);var insurances = context.All<Insurance> (insuranceKey);var assignments = context.All<Assignment> (assignmentKey);

The first line returns the first policy that fits all conditions, i.e. it was created in 2007, and has at least one Assignment of type Dependent with 70 people in it.
The second line returns all policies satisfying these conditions.
The third line returns all matching Assignment objects from matching policies. That is, the result does not contain matching Assignments from unsuitable policies.
The "All" method returns IEnumerable<IResolutionContext<Insurance> > , not IEnumerable<Insurance> . To get the latter, you need to extract the Value field via Select. The IResolutionContext interface allows retrieving a list of matching child entities for the current parent. Example :

var insurances = context.All<Insurance> (insuranceKey);var firstPolicyCoverages = insurances.First().Get<Assignment> (assignmentKey);

It is important to mention here that, for any two pairs T1-key1 and T2-key2, the following condition is true – If from context T1-key1, let’s say it is a variable

IResolutionContext<T1> c1

, get a collection of T2-key2 contexts

IEnumerable<IResolutionContext<T2> > cs2 = c1.Get<T2> (key2)

, you can call back to T1-key1 from any of the elements in this collection, and the resulting collection will contain the original element.

cs2.All(x => x.Get<T1> (key1).Contains(c1)) == true

In addition, between c1 and elements of cs2, all the conditions designated in the combined filters will be satisfied.

Combination filters

In some cases it is necessary to compare fields of two entities. An example of such a filter :

And assignment A covers less people than maximum dependents specified in insurance A

The condition is of course unrealistic, as are some others. I hope this doesn’t embarrass anyone, an example is an example.
Definition of such a step :

context.For<Assignment> (assignmentKey).For<Insurance> (insuranceKey).IsTrue((assignment, insurance) => assignment.HeadCount < insurance.MaximumDependents);

The filter is assigned to the furthest from the root of the entity tree, in this case Assignment, and at runtime it looks for insurance by going up the red arrow(in the first picture).
If there are two entities of the same type, with different keys, an inequality filter is automatically applied between them.
That is, in the case of .For<Assignment> ("a") .For<Assignment> ("b"), the lambda (a1, a2) => will never have the same entity in both arguments.
I limited myself to using two entities in the filter, because most likely a condition that uses 3 entities can be beaten to pieces. There is no technical limitation, I can add a "triple" filter if desired.

Filters per collection

Several filters on the collection are already in place, and one of them, Exists, has been used before. There are also filters DoesNotExist and Each.
What they really mean is that the parent entity, in this case insurance, is considered a matching entity if there is at least one child Assignment entity which fits the condition. This is for Exists. For DoesNotExist – if there is no Assignment that fits the condition, and Each – if all Assignments of that insurance fit the condition.
In addition, you can set your own filters for collections. For example :

context.ForAll<Assignment> (key).IsTrue(assignments => assignments.Sum(x => x.HeadCount) > 0);

Only matching Assignment, that is, those that have first gone through their own filters, get into the collection filter, of course.
The second example involves comparing two collections.
SpecFlow text for the example :

And average payment per person in assignments B, specified in taxes B is over 10$

and the corresponding definition :

context.ForAll<Assignment> (assignmentKey).ForAll<Tax> (taxKey).IsTrue((assignments, taxes) => taxes.Sum(x => x.Amount) / assignments.Sum(x => x.HeadCount) > average);

Another testing technique

The trick is to first prepare a fully populated object, check that the page (assuming we’re testing a web application) runs the happy-path script successfully and then use the same object in each subsequent test one by one to break something and check that the page gives the appropriate warning to the user. For example, the happy-path user must have a password, email, address, access rights, etc. And for bad tests the same user is taken and his password is broken, for the next test the mail is zeroed, etc.
The same technique can be used for data retrieval :

Background:Given insurance B is taken from insurancesSourceAnd for insurance B exists an assignment BAnd for insurance B exists a tax BScenario: No assignment with needed count and typeGiven there is no suitable assignment BScenario: No tax with needed amount and typeGiven there is no suitable tax B

I haven’t given the text in full, just the relevant lines. In the example, the Background section specifies all registrations required for happy-path, and in specific "bad" scenarios, one of the filters is inverted. The happy-path test will find an insurance that has a matching Assignment and a matching Tax. For the first "bad" test the insurance with no matching Assignment will be found, for the second one, the insurance with no matching Tax will be found. Such an inversion can be included as follows :

context.InvertCollectionValidity<Assignment> (key);

Besides, you can assign some key to any filter, and then invert that filter with that key.

Logging failed search

When many filters are set, it is not immediately clear why the search returns nothing. Of course, this is easy enough to deal with by the dichotomy method, that is, comment out half of the filters and see what changes, then half of the half, etc. However, for convenience, and to save time, it was decided to let the filter that handicapped the last available entity be printed in the log. That means that if the first of the three filters disabled half of the entities, the second filter disabled the second half, and the third filter is not even touched, the second filter will be printed. To do this, you need to subscribe to the OnSearchFailure event.
That’s all for now. The project is available on github github.com/repinvv/TestingContext
You can get a ready-made build on NuGet www.nuget.org/packages/TestingContext

You may also like