Posts Tagged ‘Scrum’


User Stories and Testing with Black Boxes

It’s common to hear the term black box in software development. Often this simply means that when we do testing, we only consider the external behaviour of a system. There are, however, other interesting benefits to considering your system a black box as well as some gotchas.

Often when writing user stories for, or for testing, a system it is helpful to consider that system as a black box. That means not only that do you not care about the internal workings of that system, but also that you should consider the insides unknowable.

User Stories

A user story that has no effect that can be observed from outside the black box should set off an alarm bell. There are three reasons why this may happen:

  • An incorrectly specified user story. This could mean that the writer of the story doesn’t fully understand the reasons for the planned changes. If this is the case then the story should certainly not be implemented in the current state.
  • A multi-part user story. This means that the story cannot be easily implemented in one go. Try to simplify the feature into stories that still provide (in isolation) a benefit to the stakeholder. While there are really some stories that may be irreducibly complex (requiring some intermediate user stories that provide no immediate value), however these are probably very rare. Such stories are bad as if the benefits cannot be measured from outside the black box, how will you test it? Likely you will be accumulating testing debt as you won’t be able to fully test until the full feature is complete. If such situations arise regularly, consider making the iterations longer.
  • A technical improvement.Technical improvements are a topic unto themselves. It suffices to say that if you have many technical improvements tickets, you probably have an accumulation of technical debt. This happens if a development team doesn’t have the time to develop in a sustainable way. If each user story took the time it actually needed then there would be almost zero technical debt.

You should think carefully before accepting such stories into a software iteration.

The Boundary

Whether you are writing user stories, or writing black box tests, you should define the boundary of your black box properly. While it may seem obvious, it is often more complicated that it first appears.

For example… is the database inside the system boundary? Well, it depends! There are probably database tables that are used internally by the system, and others that may constitute a kind of system output (event logging or report generation for example). These output tables may be outside your system boundary as they are consumed by stakeholders (operations staff, management etc.).

It would not make sense for a user story to change an internal database table with no other measurable effects. It would, however, be acceptable for a user story to generate a report in such an output table without any other side effects.

The same situation applies for black box testing. A test should never look inside a database table that is used internally, but it would be acceptable for a test case to look inside a table that it outside the system boundary.


We’ve all heard the term black box, but how do you use this principle? It is very helpful for writing user stories as well as for testing.
Have you really defined the boundary of your black box correctly? Do all your team agree? An ambiguously defined black box boundary will create lower quality user stories and test cases.


Fixed Sprint Lengths Considered Harmful

Scrum advocates the use of fixed length sprints. The  length of the sprint can be adapted to balance between the need to release rapidly and allowing enough time to complete a useful amount of work in each sprint. Once an appropriate length is found, it generally stays the same.

While fixed length sprints are an improvement over waterfall style development in many respects, it imposes an artificial deadline for teams and this can cause inefficiencies and technical debt.

Sprints may have fixed lengths but user stories, on the other hand, are not homogeneous at all. Stories differ wildly in complexity and the time required to complete them. Teams will estimate their size and then commit to completing a certain number in a sprint. Seems reasonable enough but….

A scrum team will ALWAYS either over-estimate or under-estimate the amount of work that can be completed in a sprint.

This leads to one of four possible scenarios occurring.

Scenario 1: Team under-commits a lot

Team under commits a lot

If a team under-commits a lot, it is usually not serious as a new story can easily by in-scoped, which will lead to one of the other 3 scenarios being triggered.

Some teams may in fact work more slowly in order to prevent a new user story being in-scoped, either consciously or unconsciously. This risk, however, probably only applies to highly unmotivated teams.