As Alec’s recent roadmap post stated, the Sharp Arch team have decided to switch over to Machine.Specifications (or MSpec) for their unit testing tool of choice going forward. This means, from v2.0, all new tests will be written using MSpec and all existing tests will have been migrated over to use MSpec as well.
MSpec has been around a little while now, and has a good following and support in the .NET community, but it’s fair to say that it’s probably not in everyone’s day-to-day tool box and, even for those where it is, there’s probably variations in usage – e.g. naming conventions, formatting style etc. As Sharp Arch is open-source and community driven, it’s probably a good idea that the community understand the why’s and how’s behind this decision so that everyone is on the same page when it comes to contributing to the project.
The goal of this post is therefore two-fold:
- Provide a brief introduction to Behaviour Driven Development.
- Provide a brief introduction to MSpec and why it’s our tool of choice.
I’ll then follow it up with another post outlining some specific conventions and guidelines for how we’ll be using MSpec within the Sharp Arch project.
Behaviour Driven Development
Hopefully most people are familiar with the term Behaviour Driven Development (or BDD) by now.
It was originally named in 2003 by Dan North as a response to Test Driven Development, including Acceptance Test or Customer Test Driven Development practices as found in Extreme Programming. It has evolved over the last few years.
[…]
BDD focuses on obtaining a clear understanding of desired software behaviour through discussion with stakeholders. It extends TDD by writing test cases in a natural language that non-programmers can read. Behavior-driven developers use their native language in combination with the ubiquitous language of domain driven design to describe the purpose and benefit of their code. This allows the developers to focus on why the code should be created, rather than the technical details, and minimizes translation between the technical language in which the code is written and the domain language spoken by the business, users, stakeholders, project management, etc.
Source: Wikipdia - http://en.wikipedia.org/wiki/Behavior_Driven_Development
Now, this is all well and good when building an application, as ultimately the application provides some business value, which means the business requirements can be expressed in their natural language in executable specifications (tests) in code. But what about when you’re building a framework like Sharp Arch? We’re not directly providing any real business value, but a platform or API for other developers to build on in order to provide value to their businesses.
Well, I’ve already talked about how I believe there’s value in a BDD style approach at all layers of an application. Sure, the language and domain of business stops making sense the further you dig down into the guts of the application, but those components still have behaviour, no? And someone is going to look at them and want to know what they do, why they do it and how to use them? This may not be a business user (the holy grail of BDD), but a developer or other technical person would surely benefit from an easy to understand, executable specification of the API or functionality they’re going to work with. I think the key thing is that as the audience or end-user changes, the domain and, therefore, language changes too. So, in the Sharp Arch specs, you’ll find us talking about entities and mappers and modelbinders and all the other good things that you expect a framework to contain.
So this is why we’ve chosen to go down this route.
- It helps the team build out the framework and API in the mindset of somebody that wants to use it – i.e. you guys.
- It helps the team focus on the best language and terminology used in the framework, so everyone is on the same page.
- It describes the intention of the code, self documenting the expected usage.
- It produces an easily readable auto-generated specification output.
- It makes understanding and fixing bugs easier due to the declarative nature of the specs.
A lot of the benefit of BDD (and TDD) comes in writing the tests first – i.e. the first couple of points in that list. However, the other points justified (in our minds) migrating the existing NUnit tests over to MSpec and a BDD style too. Also, this gives us a consistency throughout the solution and helped the team get up to speed on working in this way.
Machine.Specifications and other frameworks
There seems to be two styles of BDD test frameworks currently in use within the (.NET) software development community:
The first are those frameworks that aim to produce unit tests in a BDD style syntax in C#. These could be described as an “internal DSL” for BDD testing. They tend to have unit test runner support from within Visual Studio, build tools for integrating into CI processes and produce some form of human readable output, either in HTML format, or within Visual Studio.
The second are those frameworks which are more aimed at QA or BA by providing a non-technical scenario based language for defining BDD style specifications. These could be described as an “external DSL” for BDD testing. These specs tend to be written in text files and suit high-level user story style specifications.
On a very crude level, the first set of frameworks tend to suit how developers work and the second tend to suit QA and business people. A project would benefit from both styles of testing – at a unit level within an automated CI process, and at a UI or integration level via an automated UI testing tool or runner.
Sharp Arch doesn’t really have any business users – the team is made up of developers and we’re building a tool for developers, so we’re gonna chose what works best for us.
We believe that MSpec is the most mature, featured framework, with the best integration into Visual Studio and the biggest community using it – i.e. it’s the best choice for us. We also played around with Spec Flow, which uses a more plain text style (ala Cucumber) specification format, but found it hard to write specs at a low (unit) level. We’re still keeping this on the back burner, maybe for use in with the sample applications where we have real business requirements that could be expressed in this way.
It’s worth stating that MSpec is by no means perfect, and has it’s quirks and subtleties. At first glance, it looks crazy if you’re not used to it, but most of the team have used it now in anger in their day jobs for a over a year or so, so are fairly comfortable that it’s the tool for the job and that it’s worth getting used to the new (somewhat crazy) syntax.
Subscribe