Archive for March, 2016


User Stories and Testing with Black Boxes

It’s common to hear the term black box in software development. Often this simply means that when we do testing, we only consider the external behaviour of a system. There are, however, other interesting benefits to considering your system a black box as well as some gotchas.

Often when writing user stories for, or for testing, a system it is helpful to consider that system as a black box. That means not only that do you not care about the internal workings of that system, but also that you should consider the insides unknowable.

User Stories

A user story that has no effect that can be observed from outside the black box should set off an alarm bell. There are three reasons why this may happen:

  • An incorrectly specified user story. This could mean that the writer of the story doesn’t fully understand the reasons for the planned changes. If this is the case then the story should certainly not be implemented in the current state.
  • A multi-part user story. This means that the story cannot be easily implemented in one go. Try to simplify the feature into stories that still provide (in isolation) a benefit to the stakeholder. While there are really some stories that may be irreducibly complex (requiring some intermediate user stories that provide no immediate value), however these are probably very rare. Such stories are bad as if the benefits cannot be measured from outside the black box, how will you test it? Likely you will be accumulating testing debt as you won’t be able to fully test until the full feature is complete. If such situations arise regularly, consider making the iterations longer.
  • A technical improvement.Technical improvements are a topic unto themselves. It suffices to say that if you have many technical improvements tickets, you probably have an accumulation of technical debt. This happens if a development team doesn’t have the time to develop in a sustainable way. If each user story took the time it actually needed then there would be almost zero technical debt.

You should think carefully before accepting such stories into a software iteration.

The Boundary

Whether you are writing user stories, or writing black box tests, you should define the boundary of your black box properly. While it may seem obvious, it is often more complicated that it first appears.

For example… is the database inside the system boundary? Well, it depends! There are probably database tables that are used internally by the system, and others that may constitute a kind of system output (event logging or report generation for example). These output tables may be outside your system boundary as they are consumed by stakeholders (operations staff, management etc.).

It would not make sense for a user story to change an internal database table with no other measurable effects. It would, however, be acceptable for a user story to generate a report in such an output table without any other side effects.

The same situation applies for black box testing. A test should never look inside a database table that is used internally, but it would be acceptable for a test case to look inside a table that it outside the system boundary.


We’ve all heard the term black box, but how do you use this principle? It is very helpful for writing user stories as well as for testing.
Have you really defined the boundary of your black box correctly? Do all your team agree? An ambiguously defined black box boundary will create lower quality user stories and test cases.