Why Should We Do Test-Driven Development? Part Four

When a team practices the discipline of test-driven development, it gives them a new power.  When all of the code’s intentions are preserved by the tests, it allows anyone to change any of the code at any time without breaking it.  As far as I know, there is no other way to do this.  What this means to a developer, is that it is possible to revise existing code whenever it needs to be revised.

The practice of revising existing code without changing its intended behavior is called “Refactoring.”  Over the last 15 years or so, I have witnessed the power of refactoring.  One of the most obvious things that refactoring does is that it helps us remove duplicated code from the system.  This not only makes future change easier, it reduces the lines of code that must be maintained.  By way of refactoring, TDD makes the code itself more agile.  Refactoring has become such a major tool in my practice, that I now wonder if I have written more lines of code than I have deleted in the last five years.  When I come to a system that is completely covered by tests, I am able to remove duplicated code.  This allows me to remove duplicated tests as well. If a system hasn’t been refactored well, I may end up removing more lines of code because of duplication, than I add when I make a new features.

Refactoring improves the internal design by allowing us to do simple things like renaming variables and methods.  I do realize that some of this can be done using modern refactoring tools without TDD, but there are many cases where dynamic types, strings and reflection cannot be determined by a refactoring tool.  You can break a system by renaming when you don’t have tests.

When I first switched to TDD, this power was a game changer.  For the first part of my career, I had to be very, very careful not to change things.  I looked for ways to not touch things in order to improve my ability to finish a new feature faster.  When a design is good, this can still be a good thing to do, but the fear of change usually leads to system deterioration.  Doesn’t it make sense that we learn more as we spend more time in a problem domain?  Don’t you think that if we know more about a problem domain that it would help us design code that matches that domain better?  Then it only makes sense that we will have better ideas for the code’s design after some of the code has already been developed.  The more experience we gain, the more we see how the design could be improved.  TDD gives you the power to make those improvements.

As I mentioned in the previous article in this series, TDD also helps us design better interfaces because it usually forces us to build something that we can imagine.  This helps us to make small, single-responsibility classes.  That’s the “S” in the SOLID principles. In fact, TDD tends to naturally help us to follow most of those principles.  We tend to make small classes, so they tend to be closely related to a single feature and that makes them more closed to change.  That’s the Open Closed Principle.  We also tend to inject the objects because they are much easier to test that way.  That causes us to reverse the dependencies naturally.  That’s the Dependency Inversion Principle.  When we near a boundary, it becomes much harder to test, so we tend to use interfaces that represent the difficult boundaries to test.  That’s the Interface Segregation Principle.  Interesting isn’t it?

Have you ever considered how you might make a spider web?  I’m going to guess that it wouldn’t be trivial, especially if I gave you constraints that actual spiders have, such as where certain objects are placed, how much wind there is and so on.  Now what if I was able to reduce your brain to the size of a spider’s and than ask you to make those same webs?

The reason I ask such a strange question, is that it seems possible that some things we do can be done with less mental energy if we use simple, repeatable disciplines.  TDD appears to be a discipline like that.  When we practice it, it helps us to naturally produce beautiful code in a variety of unusual circumstances, much like a spider is able to make a beautiful and functional web in many different places and conditions.

Why Should We Do Test-Driven Development? Part Three

In part one of this series I discussed what I believe is the most important reason to do TDD and that is for programmers to have a way to preserve their intentions.  In the second part, I explain the benefit that TDD gives a development team in the area of documentation.  In this part, I would like to discuss the effect that TDD tends to have on software design.

When we discipline ourselves to consider the end result of our programming efforts, it focuses our minds on the reason that the software is being made.  Our thinking shifts from the effect, to the cause.  By forcing ourselves to document the test first, we must do enough investigation and thinking to determine what is being requested.  One way that this kind of thinking improves code design, is that it ensures that the developer is the first user of it.  It requires that we experience what using our software is really like.  Since we don’t like complicated interfaces, and since we can only imagine a limited amount of complication in our minds, we tend to naturally test against a simple interface and this makes the system easier to use for others too.

When we force ourselves to consider our tests first, it makes us expend more mental energy than we would if we were to just make the code.  This also challenges us to consider if what we are doing is really worth doing at all.  It has long been documented that 80% of our software is rarely if ever used.  Could it be that some of this is because software is easy to write but not easy to use? I haven’t done a scientific study to determine whether or not TDD would change the 80/20 rule, but my experiences as a test-driven developer have demonstrated to me that I have avoided unnecessary code when doing TDD because of the extra effort required to test it.

Another thing that TDD does is that it amplifies bad code designs.  That probably doesn’t sound good at first, but it really is.  It has been obvious as I have practiced TDD that when code is hard to test, it’s often because the design of the code itself is overly complicated. This is especially obvious when attempting to “back fill” tests by writing them last instead of first.  One of the common mistakes that new test-driven developers make is to assume that if a test takes a lot of mocking, it must mean that TDD is not very helpful.  Actually, when TDD is hard, it’s often pointing to a design that could use some work.  There are other reasons that TDD seems hard to new practitioners and I hope to share that in another post.  The point here is that when TDD creates ugly tests, it’s actually a benefit to us because it shows us the bad design earlier rather than later.  Good designs have clean and understandable interfaces.  Tests use interfaces and when those interfaces are not very intuitive, it tends to become obvious.

Another design benefit of TDD is the effect it tends to have on names.  How we name our classes, methods, functions and variables changes the readability of the code.  If code is easier to read, it’s much easier to maintain in the future.  That’s an important measure of software quality because software usually spends most of its life in maintenance mode.  The way that TDD improves names, is that it causes developers to think in terms of the problem domain.  When we test first, we focus on exposing a problem, so we tend to use wording that is a part of that problem’s domain.  When we write the code first, we often focus on our solutions.  We may use words like “loop” or “string”.  Those types of words don’t express the problem we are solving but the way that we are solving it.  I have found that when I use TDD, words from the problem domain find their way into public methods.  They may go deeper than that as well.  I don’t believe that TDD automatically generates a system with good names, but it appears to help us start off with better names than we would otherwise.

Why Should We Do Test-Driven Development? Part Two

The first reason that I gave for doing test-driven development in part one of the series, is that it enforces communication between developers about their intent for the software they have written.  It ensures that those intentions stay in place as new software is added to the system over time.  Another important benefit of test-driven development is that it helps us document our intentions.

It’s important for professional developers to understand that they have a responsibility to document the system’s behavior.  Software isn’t just for computers to read.  It’s for other programmers and stakeholders too.  One of the benefits of a disciplined agile approach to software engineering, is that it naturally relaxes the need for some of the documentation.  One of the reasons that this is true, is that it is assumed that a full set of tests are going to be produced that form the specification.

Without TDD, where will the design be documented for future developers to read and understand?  It is true that the code is there, but the code doesn’t preserve the intent of the original developer.  It merely satisfies that intent.  I can see how it would be possible to preserve the intent in declarative code, but object-oriented and functional code is far more difficult to be sure about.  What if a future developer considers part of the existing code to be a side-effect?  That future developer will preserve what they perceive to be the original intent as they refactor or port the code to a new design.  I have actually seen this cause a production error in an area where tests were lacking, even on an a team that was trying to do TDD.

Another issue is that without a complete set of requirements documents, the team eventually forgets how the system works.  This seems to happen a few short years.  Eventually, even the most brilliant minds on the team start to forget what the system does in certain important state transitions.  The system gets more complicated as new features are added and, at the same time, we start to forget the things we did in the first few iterations.

If we are going to take hold of the privilege of agile software engineering, then we must also embrace the responsibility of it.  If we decide not to do TDD, then we had better plan on going back to written design documentation.  We must also keep that documentation current as changes are made or else it won’t help.  As you probably imagine, that is going to slow the team down.  It’s difficult to say that a team is “agile” when it chooses to go slower because it doesn’t want to perform the discipline of TDD.  If it decides to not do TDD and not document the design, I would question whether the team is even professional.  If we are to be both professional and agile, then I know of no other way to deal as efficiently with requirements documentation, than to maintain the discipline of test-driven development.

Why Should We Do Test-Driven Development? Part One

I want to explain why I find Test-Driven Development so important to modern software development teams.  There are many reasons, and it will take a few articles for me to go through them.  I’d like to start with a reason that may be the most important and is fairly easy to understand.

I have found that one of the most important things that TDD does, is that it protects an individual developer’s ability to create new code without harming the existing code.  When TDD is a team discipline, that team must run all of the unit tests and see them pass before they merge their code in with the rest of the team’s code.  When a developer knows that they may have broken something by what they add, it creates a high level of anxiety for a truly professional developer.  We often take pride in our work as developers and when our code breaks everything, it is very shameful to us.  Obviously, it is also very unproductive.  It is one of those hidden costs that makes software take longer to make and causes it to be less reliable for those who use it.  It can actually harm people when our new code breaks some of the old code that we didn’t realize would be affected.

It may be hard for those who aren’t software developers to understand the impossibility of merely reading the code in order to figure out if what we add will break something.  Even the most confident developer makes a typo.  All it takes is one letter or even a missing semi-colon in to harm the code.  I recently broke code by having the wrong case.  The letter was right but its case was not.  Trying to write code without a way to check it, is like having a job at which you must walk a tight-rope all day.  It’s clearly not sustainable.

The discipline of test-driven development, causes all team members to write their tests before they write their code.  This isn’t just a random and cruel requirement.  It’s a practice that ensures that every programmer’s intentions are both stated and preserved.  This may be the fundamental benefit of TDD.  It’s a way to communicate a programmer’s intentions to all future programmers.  That way, when a future developer makes a change, they add their new intentions and make sure all of the existing intentions are still being met.  In this light, TDD is fundamentally a discipline that enforces communication between developers.

It’s important to note that even when a programmer works alone, they often forget their own previous intentions for the software!  Our work requires that we focus.  It doesn’t take long for us to exceed our own ability to remember the details.  By practicing TDD, the intentions we preserve may be our own.

This is just one of the benefits I have witnessed over the years as I have practiced TDD.  It turns out that it also can improve our requirements process, serve as documentation, improve software design and promote the software’s internal quality, but this one reason alone would probably be worth the effort to me.

 

 

How to Communicate with Software Developers

If you are a project stakeholder and you have software that needs to be made, you might discover that it can be difficult to communicate your needs to software developers.  You may believe that you have communicated your desires, only later to discover that the developers have no idea what you mean.  So, how do you deal with this issue?  How do you communicate what you need to have done?

The most adequate software requirements I have seen, are test scenarios.  When these are provided before software is written, it gives a developer most of what they need to know to start working on the software.

Making test scenarios as requirements documentation takes serious dedication on the part of the customer representatives, though.  It requires that they go beyond providing a few sentences about the software.  They actually must pretend that they are using it.  It requires an adequate set of test data and a good imagination.  You must detail the actual steps you would go through and what you expect to see.  These are the procedures that will prove that the system is actually working in the end.

The basic framework for these scenarios is actually very simple.  They follow a three step pattern:

  1. Set up the starting conditions
  2. Perform some work
  3. Check for expected results

To write these down, you can use a pattern like this:

  1. Given a specific situation
  2. When something is done with the software
  3. Then some new situation is the result

I have come to call this pattern: “Given – When – Then.”  It has a formal name as well.  These descriptions are what we call “finite state machines” in software engineering.  A computer program can be defined as a set of these.  The great thing about them is that they don’t require a degree in computer science in order to make.  The only difficulty is that they take time, energy and attention to detail.

When you are first learning how to write these, I suggest that you start each step with the “Given,” “When” or “Then.”  Here’s an example:

  1. Given a user that has a bank account and has already put their ATM card into the ATM and used their pin to access their account.
  2. When the user requests a balance.
  3. Then the ATM shows the user’s balance to the user.

It may look easy at first, but it usually becomes evident after you start writing these down, that things have not been completely thought through.  For instance, notice that in the given part of this ATM scenario, you haven’t really described how the ATM should use the PIN yet or when the card should be taken.  Then there’s the possible fraud cases.  You have to know when to deny access too.  After you get started, you may feel like it’s just too hard, but don’t give up.  What is actually happening, is that you are flushing out issues before a software engineer is employed.  This hard work pays huge dividends.  It makes it possible to communicate with software engineers in a way they actually understand.

Although every detail that is important should be described, not every Given-When-Then has to be done before software development begins.  This is one of the biggest benefits of agile practices.  It’s important that developers use a method of development that will support changes to requirements, though.   Test-driven development and the creation of automated tests for your scenarios make that possible.

You might be thinking that you really should write down all the test scenarios before you have a developer start working.  The problem is that we usually forget important scenarios and have to add them later anyway.  It’s good to detail as many as you can and then prioritize them.  It’s usually the focus on priority that will benefit you more than completeness.

So, I challenge you to try this with a single test scenario.  Write out your scenario using the Given-When-Then style and take it to a software developer.  You will probably be amazed at how fast they understand what you’re asking for.

Relative Estimating with T-Shirt Sizes

A variation on estimating using story points is to use what I call “T-shirt sizes”.  This table shows the sizes I like to use.  The abbreviation is a good way to refer to the size in a tracking system.  The weighting factor helps you make more precise trade-offs between stories that need to fit into a development cycle.

Name Abbreviation Weighting Factor
Freebe freebe 0
Extra-Extra Small xx-small 1
Extra Small x-small 2
Small small 4
Medium medium 8
Large large 16
Extra Large x-large 32
Extra-Extra Large xx-large 64
Too Big too big

I really like to use this method to estimate stories because they:

  1. Expresses the relative estimate idea well
  2. Are easy to explain to stake-holders

This method expresses the relativity concept well because, unlike story points, it has meaning in the “real world”.  When you use shirt sizes it easy to understand the idea of relativity and you don’t automatically compare the estimates to anything like dollars, days, or weeks.  Even though the estimate isn’t precise, it has real meaning that is easy to understand.

I found that when attempting to associate with software product stakeholders, it can be difficult to explain our unusual ways.  In some environments, this means that stakeholders won’t show up to meetings or have trouble being willing participants in discussions such as software product stories and estimates.  If developers don’t get enough communication with stakeholders and customers, it can seriously harm product value.  Using concepts that are less intrusive can be a big help.

The “Freebe” is just as it sounds.  It is a feature that won’t count as a part of your budget.  This is not because it has no size; it just means that it is so small that talking about it takes more time than doing it.  Many freebes will add up to an xx-small eventually.

“Too Big”, just means that the team cannot accurately estimate the story because it doesn’t appear to fit into a development cycle.  This is especially easy to understand when the team works in sprints or iterations.  The easy way to take care of these is to simply divide them into smaller stories.

The weighting factor is really the same thing as story points.  The nice thing about these is you can make your own project-oriented weighting factors quite easily.   The thing is that you really need to keep them the same.  Make sure that you use a exponential equation to calculate them.  I suggest starting with ones like I have here, however.  I have actually used these numbers with great success.  You could use Fibonacci numbers too.  These numbers help you to have a better handle on how many stories can fit in a given time period.  You may only be able to do a single extra large story.  If they break down that story you could actually fit in 64 extra-extra small stories and possibly get more done with a little more planning effort.

Once you have these you have what you need to make your estimates for your stories.

Telling Good Stories

Estimating the time it will take to make a software feature can be quite a challenge.  In order to have a reasonable discussion about how long a feature will take, we really need to have a reasonable idea about what the feature is.  Good estimates depend on good software descriptions.

My favorite way to describe software, for the purpose of estimating, is to develop “user stories.”  User stories are short phrases that describe a behavior that the software needs to have.  I have used two different ways of writing stories that seem to work well for me, so I’m pretty sure there are more than I describe here.

I first learned about defining a software product using stories from the book “Extreme Programming Explained: Embrace Change” by Kent Beck.  Later, more definition was added by Mike Cohn who wrote a whole book on user stories.  Cohn has a handy formula for writing them.

He suggests that you follow this form:

As a [user] I can [some behavior]

For example you could write:

“As an ATM customer I can request my current account balance.”

Another one might be:

“As an ATM customer I can enter my PIN before I am given access to my account.”

This form is very popular and I find that it works great.  Before I learned about the Cohn story form, I had developed a form myself.  I still like it because I think I am able to break stories into smaller pieces more naturally and it has a cool side effect.

Simply finish this statement:

“I will know that progress has been made when …”

What you write after this statement is usually a great user story.  The Cohn form fits this method as well.  I also found a similarity with the results of my form and the development of a “User Manual”.  The sum of all of the stories could reasonably become the product user manual.  Check out these three stories:

  • I will know that progress has been made when the software provides a place for the ATM user to enter their PIN.
  • I will know that progress has been made when there is a way for a user to ask for their account balance.
  • I know that that progress has been made when the software provides a way for an ATM user to print their account balance.

Now listen to them without the “I will know…”part:

  • The software provides a place for an ATM user to enter their PIN.
  • There is a way for a user to ask for their account balance.
  • The software provides a way for an ATM user to print their account balance.

Notice that this sounds a lot like a user manual.  Steve McConnell in his book: “Software Project Survival Guide” describes a way in which user manuals can be used as requirements documentation.  I really believe that he was on to something when he wrote that.

The best stories are isolated, meaning they don’t depend on any other story.  This may seem to be too difficult at first, but it is actually quite possible.  Don’t give up on making great stories. Using practices like this can help to make it easier.

 

Having Trouble Getting Productive on a New Team?

The practice software development is very complex.  There are  interchangeable technologies and tools at all levels of development.   Books are being outdated shortly after they are published as newer versions of software outdate the old ones.

With all this change, it’s not very likely that we are going to show up at a new job with the all of the desired expertise.  Not only do we need to know the technology, we also need to know the product, the environment and the people we work with.  So what do we do as developers when we show up at a job and get left with a new cubicle, a new computer and no help?

How can we come up to speed fast without bugging the team?  Answer: You probably can’t.

You are going to have to do what you can to learn and that’s probably going to take time.  You can try to read the outdated documentation and try to reverse engineer the code but that will only take you so far.  You will probably still have to bug people and they will probably be busy or they would not have hired you.

Now consider this.  What if you had shown up at a job that was filled with a room of people who were working side-by-side on the same features?  That’s what you find on teams that practice pair programming.  Two people work on the same code together at a comfortable workstation with two screens and two keyboards.  All you would have to do is show up and you would start learning what you need to know immediately.  You would actually be productive immediately as well.

That intimidating thing called pair programming, that you may have avoided when you tried to find a job, is now something that you desperately wish you had.  If only someone would work with you to show you how it all works, you might have a fighting chance at being productive.

It doesn’t make any sense to for us to expect that we will be productive working alone in a business in which most of the things we need to do are unique to an organization.  So, what should you do?  Well, if you are already at a job that doesn’t practice pair programming, you might try to see if someone would be willing to start pair programming with you.  Try to set up a workstation with two keyboards and screens.  You could try using some sort of screen sharing, but going to a conference room with an extra screen and a keyboard would probably be better anyway.

Hopefully you can get pair programming started right where you are.  If you can’t, at least you know what to look for if you ever need to find a new team.