In his last article on his blog (sad... it's the end of an era), Joel talks about the different perspectives on version control between classic, Subversion-like systems and the "trendy new distributed version control systems".

From his post:

"With distributed version control, the distributed part is actually not the most interesting part.

The interesting part is that these systems think in terms of changes, not in terms of versions.

That’s a very zen-like thing to say, I know. Traditional version control thinks: OK, I have version 1. And now I have version 2. And now I have version 3.

And distributed version control thinks, I had nothing. And then I got these changes. And then I got these other changes."

He also put together a Mercurial tutorial, written in his easy and fun to read style, that can be found here: hginit.com.

I stumbled upon this Programmer Competency Matrix.

It is an interesting exercise to check your level and to identify weak areas that you can develop.

Oh, and try not to fool yourself while doing it!

I used the word "help" in this post ( We can’t use Test Driven Development! ) because TDD is not a silver bullet. NOTHING is. But that is not a reason not to look for improvements in the way we work.

I'm going to try to explain why I think TDD can help you in MOST of the software development projects out there.

First, mentioning something like "zero defects guarantee" when talking about TDD or testing in general is a hint in my oppinion that a lot of people see the tests as a way to FIND defects.

Myth: The Job of Testing Is to Find Defects

The job of tests, and the people that develop and runs tests, is to PREVENT defects, not to FIND them.

(Mary & Tom Poppendieck, "Implementing Lean Software Development From Concept to Cash")

Knowing that, correct me if I'm wrong with this train of thought (and see if it applies to your case, I can't think of a project where it would not apply):

1. To have or not to have tests. (By tests at this point I mean test cases, automatic or manual, covering the whole system or just a method. Tests in general).

Not having tests leaves me with the task of guessing whether my functionality really does what it is supposed to. Remember, computers do what you tell them to do, not what you want them to do. So having to tests would mean code -> compile-> mark task as done. Unfortunately it actually happens, and way too often.

But that's not the way we do it! The developer DOES check that it works!

Ah! That IS testing, so we MUST have tests then!

2. To have a test list or not.

Testing the software by randomly executing operations will not give you enough certainty that it does all it is supposed to do. The reasonable thing to do is make a list of tests that cover all the cases you can think of. Of course, the list is not cast in concrete - tests can be added or modified.

The list can be an excel or word document. Or code. Or a table on a whiteboard. Or a scribble on a piece of paper. Keep it in your memory only if you never forget anything (hello there, R2D2!).

The important thing is that it needs to specify the setup, execution and validation for each test so it can be shared among team members.

3. Define the tests before or after implementation.

Both work, but tests defined them after implementation (especially by the developer), will tend to validate what the code DOES, instead of what it is SUPPOSED to do. More, defining the tests before can help the developer contemplate cases he could have overlooked and serve him as a progress indicator (9 out of 10 tests passing is a hint that you are close to finishing).

4. Automated or manual tests.

Being able to run your tests automatically and have the feedback in seconds instead of minutes is, of course, a big plus. And, you can setup a Continuous Integration server to give you fast feedback about integration problems.

You have to take into account the time to implement them though. And testing whole applications can could imply slow operations like setting up data in databases or connecting to services. And, depending on your frontend technology, you might find that UI testing frameworks are still limited. Which brings us to the next point:

5. Big coverage or small coverage tests.

Small coverage implies cheaper tests - cheap to develop because you can mock external dependencies and check a specific flow without worrying about "downstream" effects, and cheap (fast) to run because everything is in memory - no database access or web service calls. This type of tests are usually called UNIT TESTS.

Having a good coverage of the internal workings of your components allows the bigger tests to cover the integration parts, the communication between them, validating that they play well together.

In conclusion, if the above applies to you (and if not, I welcome comments on why it doesn't): you should ALWAYS HAVE TESTS, DOCUMENT them, preferably define them BEFORE implementation, and you get a big productivity bonus if they are AUTOMATIC which is a lot cheaper if most of them are UNIT TESTS.

Developing for true multi-touch UIs is going to be fun... all the usability problems we have for mouse and keyboard will look so simple then. My guess is that until some common standards will appear it will be like playing video games in the early days - each one had its controls that you had to get used to.

10/GUI from C. Miller on Vimeo.

Link: 10/GUIvia Eric de C#.

You just have to put some pressure on them! Set a deadline!

After all, it works for programmers, doesn't it?!?

Management is used to negotiate everything, so when the developers come with an estimation for the work to be done their first reaction is:

"2 weeks... That's ok... Now what if we put pressure on them? Tell them they have a week to get the job done."

And I can understand that... It's the job of the development managers or directors to keep things in balance and ensure the developers get the necessary time to do quality work. If not, they WILL cut corners in quality. At 10 PM they WILL choose going home over spending some more extra hours making and running tests.

What I can't understand are the people with a programming background that, as soon as they start climbing the company hierarchy, they start setting out-of-the-blue deadlines for the developers they now manage.

What über programming technique were they using in their programming days when they got unrealistic deadlines to get the job done in time AND with quality?!?

PS: Just to be clear, this article not is about deadlines in general - it is about deadlines set without taking into account the developers' estimations.

Disclaimer: this may not apply in your country. In Mexico it is mandatory to be late.

Imagine that you and your friends agree to go out for dinner. You get to the restaurant on time (well, alright, you are 5 minutes late).

Of course none of your friends has arrived... You get a table and wait. Good thing they serve free totopos so you can deceive your hunger. After half an hour the first friend appears. And then another. The last one is about 2 hours late. Of course, you cannot order until everyone is there. And they take their time to decide. You finally get your food 3 hours after entering the restaurant. It is your fault, for being on time.

The solution: next time try to guess how late everybody is going to be and arrive even later, so you don't have to wait for anyone.

So? It's dinner, why should everyone be punctual?!? Are you some kind of punctuality nazi?!?

The problem is that the same thing happens in business activities. One person arrives late to a meeting, causing everyone else to be late to other meetings, in a snowball effect that makes everybody's daily agenda as accurate as the Romanian train schedule. But that's fine, everyone understands and everyone does it.

It gets ugly though when you apply the dinner scenario to employee work hours. Most of the companies in Mexico turn a blind eye to employees that arrive late. 40 minutes, one hour, even more. You can always blame the traffic.

Why would a company do that?

Because they know they can ask them for extra hours on a regular basis, without extra pay. So they are expected to stay late daily and work weekends once in a while.

So it would seem like a WIN-WIN situation: employees can be late and the company gets free extra hours - everybody's happy!

WRONG!

The employees that DO get to work on time are the LOSERS! They get the same treatment as the other ones. Extra hours, weekends, everything. And they actually seem less "involved" with the company if they leave on time at the end of the day. Yeah, everybody knows that staying late is the measure of your commitment to the company. Being on time doesn't matter... the boss is not there to see you.

Of course, the boss arrives as late as possible too.

Meet Bob.

Bob is the Development Director a company that offers a Software-as-a-Service product with customizations built for clients on a project-by-project basis.

Bob was there from the beginning. Actually he is the only one remaining from the handful of programmers that started the software. That fact alone pushed him into the position he is in today, managing about 50 programmers. He's standing among the other directors is very important to him - he wants to be seen as the guy who gets the job done, no matter the absurd deadlines. Of course "done" is relative, if you don't care about quality - everything that happens after "delivery" is the client's problem.

Bob's philosophy is simple: "If it worked until now, we MUST keep doing things the same way!". It doesn't matter that things that barely worked with 5 programmers are literally ripping the code apart now. Although his background could be defined as using technology to automate activities, he is surprisingly blind to the opportunities of improving his team using software.

So they keep using manual source control by writing the changed files and lines in notebooks and then manually merging them.

So they keep having just one development database instance where the developers make their changes and run their tests at the same time. Of course, because nobody removes their unused stuff on one side and changes being made directly in production, only about 30% of its schema is equal to the production database.

Apart from his philosophy, Bob has his excuses ready whenever somebody tries to convince him to improve the way development is done:

"We have these projects in progress and they must be delivered on time".

Makes sense. The problem is that this is forever true. When new projects arrive, Bob conveniently forgets about the time required for improvements and keeps doing the same old again.

Why does Bob do that?

"Because we have to keep the pace of deliveries".

Actually Bob "pace" is too slow. Extreme case: if Bob stops everything for a month in order to implement a practice that would give him an 10% improvement in delivery times, he would recover his investment in one year. But he lost his long-term vision. Imagine that he had done that 2 years ago. Now he would have extra income equivalent to one month's work. Let me rephrase that: Because he didn't act in time the company is losing thousands or millions at this moment.

But it's not Bob's fault:

"The salespeople make commitments to the clients and we have to deliver, even if there is not enough time".

Salespeople will always try to do that. Whatever it takes to make the sale, even if it means cutting the delivery time in half. It is the development director's job to maintain the balance. And see the point above. He could deliver faster.

If Bob HAS to act,

"We'll do it gradually. Baby steps. This will take a loooong time".

What this means is "We'll pretend we're doing it so you'll stop bothering us". He's not committed to improving the way work is done. It results in one or a few "pilot" teams experimenting other ways of doing stuff, without adequate measurements in place to check for success. Management support lacking, the teams rapidly lose their way and revert to the old practices or they are told to do it because there are no "visible" improvements.

And then, he can always use this:

"We hired cheap people and they are not capable of learning new practices".

So you looked for inexperienced people, that's OK. You didn't look for the dumbest, did you?!? Even if they are dumb, they learned the current practices - why wouldn't they be able to learn new ones? Unless... you DO TRAIN your employees, don't you?

And what if change comes from "below"? Bob has a zero-tolerance policy for programmers that do not obey his "religion".

The result: the smart people look for other opportunities while the company loses its competitive edge. Of course it's not Bob's fault... seeing farther than tomorrow's deadline was never his job, was it?

PS:

There is a happy face in the post, so nobody should get annoyed. If, despite the happy face, you still are annoyed by this post, you probably ARE Bob.

Databator is a tool I’m working on that enables you to manage the versioning of your database based on the method described here.

This first version allows you to:

• Create an project for a new or existing database
• Add versioned changes to the database
• Use command line scripts to update, backup or restore the database (handy for Continuous Integration builds)
• Conciliate your changes with those made by tour team mates
• Work in branches and then merge your changes

The final scope of this tool will include:

• Integration with SVN
• Synchronization with Visual Studio Database projects (.dbp)
• Apply or Rollback your updates directly from the UI
• History for each DB Object (just like code files)
• Helpers for creating update scripts (specific cases)
• Automatic rollback script generation (specific cases)

I recommend that you check the Getting Started Guide to get an idea about how this tool can help you optimize your database versioning.

You can download Databator v0.1 from here: Databator0.1.zip (982.01 kb)

“We can’t use TDD! It would increase our development times / costs too much! It’s just not worth it.”

IT’S OBVIOUS!

Of course, it depends on what do you understand by “development”. If you mean the activity of writing code, tests are code, so more code needs more time / resources to be written. Duh!

Testing / bug fixing and maintenance!?! That’s not development, that’s…uh… something else… it doesn’t count.

Most of the people I’ve heard using that remark don’t even count the tests performed by the developers as development time. So, we’ve written enough code, we can call development “done”. It’s a good moment to mark the task as 100% and congratulate ourselves. Don’t forget to impress the upper management with the updated Gantt chart showing the on time completion of 80% of the project. What’s left? Just testing and bug fixing, that shouldn’t take long, right?

As a bonus, we’ve written the code “cowboy programmer” style. Time is all that matters, so we cannot be bothered to think twice about inserting into the database directly from the presentation layer or about increasing that “core” stored procedure from 10,000 to 12,000 lines (no indentation, of course – what a waste of time that would be). Of course, we’ll refactor that later. We did manage to add some would-be-nice features though. We had to do a little harmless over engineering, but we’re sure the client will appreciate them.

Does any of this sound familiar? Can you guess what happens next? Of course, the remaining 20% take more time than the “actual development”. Every bug fix creates other bugs. In the end the client gets to be the tester, reporting defect after defect in the “stabilization phase”. No, do not call it Beta testing –the client would never agree to pay and/or participate in that.

Time has passed… we get less bug reports, but fixing them takes much more time. With much of the original team gone or assigned to other projects, it’s even more difficult to know if we’re not breaking something while fixing another. Refactoring is out of the question. The over engineering we did makes debugging a nightmare and on top of that the stupid users asked us to hide some of the “would-be-nice” features. Fear of being the one that caused the whole thing to blow up in the client’s face has made us religious – we change and pray.

Fade to black… the end. Happy end actually (well, sort of)...

But IT NOT OVER! Now we have to add a new module to the existing fragile application. Time is crucial, so we do what we do best: code like crazy, spend nights at the office and it’s done with only a small delay. “Actual development” that is.

There is a hail of defects that bury us in tickets. More nights at the office. Sleeping at home is a distant memory. Did I tell anyone to feed my cat? It’s too late now anyway. And who did this STUPID change in this method?!? Oh, that was me, last night.

The client is not happy. Stupid user! Why would he expect the existing functionality to work?!? This is “STABILIZATION PHASE”!

Now imagine that another project comes along after that storm finally settles down.

For this one, we could use TDD. Yes, more coding time, but we would have our functionality in a harness. We could do refactoring without fear. The code would be less smelly. It would be difficult to write tests for huge methods or business logic-hungry stored procedures, so we would HAVE TO keep it clean. No extra features or over engineering this time – the tests would keep us on track.  We could probably have a reliable shippable product in less time than it took is to deliver the previous, unreliable one.

There will be defects, of course, but we could reproduce them with tests and ensure that the fix is correct by running all the tests.  I could even go as far as to predict that the number of defect would be significantly lower because the tests we’ve written helped us identify exceptional cases and prevented later changes from breaking existing functionality. Also, adding more functionality later should be much faster and secure. All this would translate into fewer resources needed for maintenance. No more wasted nights…

Yeah, we could do all that… but we won’t. We’ll do things like we always do and expect different results.

Well… THEY’ll do it. I just can’t afford to lose another cat.

One common problem in projects that involve a database is managing its versions.
Basically, there are two needs to be fulfilled:
- to be able to see a history of changes for a database object in a similar way to seeing the changes for a code file
- to be able to update / revert the database to a specific version without losing the data.

For the first one, a simple solution is to just script all objects in the database to a folder and add it to the source code repository (one file per object - in MS SQL Server can be done with a few clicks from SQL Server Management Studio). When there are changes, all you need to do is to repeat this process and the files will be updated. You can then review changes to database objects in the same way you review them for code files.

The second one is a little trickier. Idealy you want scripts that can be automatically used to:
- update developer database copies
- update the continuous integration database (database used for the continuous integration tests)
- update the test database
- update the production database.

There are tools that give you "diff" scripts that can be used to update a database, but, apart from the price problem, they are difficult to use for several-times-a-day automatic updates like the ones needed to update the developer copies. A second problem is that, because the developers have full freedom in their database copy, some "garbage" or unwanted changes might be picked up and included in the diff script.
The solution below is free, the updates can be run automatically whenever is necessary on any database (developer, test and even production) and ensures that the changes were really meant to be included in the update. All you need is a little discipline from the developers. More...