User Stories and Testing with Black Boxes

It’s common to hear the term black box in software development. Often this simply means that when we do testing, we only consider the external behaviour of a system. There are, however, other interesting benefits to considering your system a black box as well as some gotchas.

Often when writing user stories for, or for testing, a system it is helpful to consider that system as a black box. That means not only that do you not care about the internal workings of that system, but also that you should consider the insides unknowable.

User Stories

A user story that has no effect that can be observed from outside the black box should set off an alarm bell. There are three reasons why this may happen:

  • An incorrectly specified user story. This could mean that the writer of the story doesn’t fully understand the reasons for the planned changes. If this is the case then the story should certainly not be implemented in the current state.
  • A multi-part user story. This means that the story cannot be easily implemented in one go. Try to simplify the feature into stories that still provide (in isolation) a benefit to the stakeholder. While there are really some stories that may be irreducibly complex (requiring some intermediate user stories that provide no immediate value), however these are probably very rare. Such stories are bad as if the benefits cannot be measured from outside the black box, how will you test it? Likely you will be accumulating testing debt as you won’t be able to fully test until the full feature is complete. If such situations arise regularly, consider making the iterations longer.
  • A technical improvement.Technical improvements are a topic unto themselves. It suffices to say that if you have many technical improvements tickets, you probably have an accumulation of technical debt. This happens if a development team doesn’t have the time to develop in a sustainable way. If each user story took the time it actually needed then there would be almost zero technical debt.

You should think carefully before accepting such stories into a software iteration.

The Boundary

Whether you are writing user stories, or writing black box tests, you should define the boundary of your black box properly. While it may seem obvious, it is often more complicated that it first appears.

For example… is the database inside the system boundary? Well, it depends! There are probably database tables that are used internally by the system, and others that may constitute a kind of system output (event logging or report generation for example). These output tables may be outside your system boundary as they are consumed by stakeholders (operations staff, management etc.).

It would not make sense for a user story to change an internal database table with no other measurable effects. It would, however, be acceptable for a user story to generate a report in such an output table without any other side effects.

The same situation applies for black box testing. A test should never look inside a database table that is used internally, but it would be acceptable for a test case to look inside a table that it outside the system boundary.


We’ve all heard the term black box, but how do you use this principle? It is very helpful for writing user stories as well as for testing.
Have you really defined the boundary of your black box correctly? Do all your team agree? An ambiguously defined black box boundary will create lower quality user stories and test cases.


Fixed Sprint Lengths Considered Harmful

Scrum advocates the use of fixed length sprints. The  length of the sprint can be adapted to balance between the need to release rapidly and allowing enough time to complete a useful amount of work in each sprint. Once an appropriate length is found, it generally stays the same.

While fixed length sprints are an improvement over waterfall style development in many respects, it imposes an artificial deadline for teams and this can cause inefficiencies and technical debt.

Sprints may have fixed lengths but user stories, on the other hand, are not homogeneous at all. Stories differ wildly in complexity and the time required to complete them. Teams will estimate their size and then commit to completing a certain number in a sprint. Seems reasonable enough but….

A scrum team will ALWAYS either over-estimate or under-estimate the amount of work that can be completed in a sprint.

This leads to one of four possible scenarios occurring.

Scenario 1: Team under-commits a lot

Team under commits a lot

If a team under-commits a lot, it is usually not serious as a new story can easily by in-scoped, which will lead to one of the other 3 scenarios being triggered.

Some teams may in fact work more slowly in order to prevent a new user story being in-scoped, either consciously or unconsciously. This risk, however, probably only applies to highly unmotivated teams.

Read the rest of this entry »


Removing elements from dynamic arrays in D

Removing elements from associative arrays in D is easy, as we can use the handy “remove” method. When it comes to regular dynamic arrays, however, the story is a little different. There is no such method for removing elements and the documentation doesn’t make it completely obvious.

To complicate the situation more, the technique required depends on the version of D you are using.
DMD version 2.060 is out and with it comes the deprecation of std.algorithm.indexOf. Instead, std.algorithm.countUntil should be used in its place.

Here is an example:

import std.stdio;
import std.algorithm;

void main(string[] args) {

	auto items= ["item1", "item2", "item3"];
	auto index = countUntil(items, "item2");
	writefln("Index of 'item2' is: %d", index);
	auto mutated = remove(items, index);


It generates the following output:

["item1", "item2", "item3"]
Index of 'item2' is: 1
["item1", "item3"]

QTD compile error using DMD (Solved)

I’ve just been struggling to build QTD on my Ubuntu box.  QTD is the D programming language binding for the QT framework.  The error I was getting was:

CMake Error at cmake/FindD.cmake:41 (message):
  D compiler is not found
Call Stack (most recent call first):
  CMakeLists.txt:65 (FIND_PACKAGE)

This is strange because DMD is installed. After some investigation I found that the compiler isn’t found because the cmake file (cmake/FindD.cmake) searches for the version using a regex in the output to “dmd” (around line 13):

string(REGEX MATCH "(Digital Mars|DMD32) D Compiler v[0-9]\\.[0-9]+" dmd_version "${d_output}")

This is the guilty line. When I type “dmd” on my command line, the first line looks like this:

DMD64 D Compiler v2.060

This clearly doesn’t match the above regex. To fix the problem you can modify the regex above in the file “cmake/FindD.cmake” to look like the following:

string(REGEX MATCH "(Digital Mars|DMD32|DMD64) D Compiler v[0-9]\\.[0-9]+" dmd_version "${d_output}")

This fixes the problem.


Hiking in the Alps

Here is a panorama of Walchensee in the German Alps (taken from the top of Jochberg).

Walchensee from the top of Jochberg


Happy Birthday ZX Spectrum

Sir Clive Sinclair’s ZX Spectrum is now 30 years old.

I have fond memories of playing Manic Miner and Jet Set Willy on this machine, as well as writing out computer programs from magazines.  This little box of circuits is probably what launched my interest in computing and inspired me to learn how to do programming.

Notice that the chiclet keyboard has come back into fashion, albeit without the rubbery keys that would sometimes get stuck, or just wouldn’t respond.

Look how things have changed over 30 years and imagine where we’ll be in 30 years time.

Anyway, enough reminiscing… happy birthday!


RESTful API Design

It seems that everybody loves REST and many people are quick to espouse the virtues of this light weight form of web services.  Implementing a good RESTful API isn’t, however, always as easy as it seems and there are many issues such as paging or supporting multiple versions of your API that can stump the avid developer.  This is an excellent video explaining pragmatic solutions to these and other problems when designing your own RESTful interface.


Regex Performance in D Programming Language

Note: The problem described in this post was for DMD v2.054 and it no longer occurs with the more recent DMD v2.058  (see below for details or view the discussion at the D Forums).

I am currently working on a Ruby project that uses a lot of regexes on large volumes of text.  It is currently running too slowly, so I decided to try to optimise it by implementing the regex matching code in the D programming language.  D has given me a lot of joy (compared to C or C++)  by making things like string (with Unicode) handling a breeze without taking the performance hit of supposed “productivity” languages.  I painstakingly reimplemented my Ruby functions in D expecting a huge performance boost (actually I expected an order of magnitude performance jump) but instead I was shocked to see that Ruby outperformed my D code by a significant margin.  The Ruby implementation was finished after 80 seconds, whereas the D program required around 280 seconds using the exact same regexes and the exact same input.

Read the rest of this entry »


The Renaissance of Indy Gaming


In the olden days you could pick up a tape cassette containing a Spectrum or Amstrad game for a few pounds (yes, I’m British).  At this time, the gaming industry was in its infancy and the business side of the game was highly underdeveloped.  Games were written by a single (or a very small group) in squalid conditions and these programmers were like rock stars, or mad scientists working alone in their lab (complete with the associated lightning and electrical apparatus).  Some games turned out to be “smash hits” (Monty Mole, Manic Miner, Dynamite Dan to name a few), others were of very poor quality but hey, that’s the price you pay for experimenting.  How things have changed!

Read the rest of this entry »


The Rise of Social Shopping

As more shoppers move to the Internet for their bargain hunting, we’re left (certainly in England) with quite a bit of vacant high street property. Many stores, even big chains, simply aren’t able to price their products competitively enough to compete with on-line stores that don’t pay inner city rents.

Once the shoppers are on-line, a wealth of information becomes available that helps the prospective buyer make their decisions. Price search engines help to find the best deals without the user even having to move from their seat (let alone hike around town). There are countless sites hosting professional and consumer reviews and opinions to help buyers find the potential pit falls of their chosen product. Other web sites help users find new and interesting products that the shopper perhaps did not even know about.

Read the rest of this entry »