Skip to content

Book Review - Dreaming in Code

Yesterday I finished reading Scott Rosenberg's book Dreaming in Code - Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software (Official website / Wikipedia). Inspired by a severely problematic content management system for salon.com, Rosenberg started to think about the problems of creating complex software. Why do so many software projects fail to meet deadlines? Why do they spiral off into money black holes? Not being a software developer himself, the journalist Rosenberg started to hang out with the developers of Chandler (Official website / Wikipedia), a "revolutionary" personal information manager. He followed the Chandler team for three years, attended meetings and presentations, and asked the people involved in the project about their thoughts about software development in general and Chandler in particular. The results of this journey are written down in the book.

The approximately 350 pages of Dreaming in Code are very well-written and a very pleasant read. I managed to finish the book in just two days. The book describes the problems the Chandler project faced and compared them to similar findings from historical studies and books about the difficulty of developing software. For someone who is not actually a software developer, Rosenberg knows a lot. His research is very comprehensive and encompasses aspects of software development between the 1940s and today.

The book is divided into 12 chapters that describe the development of Chandler in chronological order. Starting with Mitch Kapor's (Official website / Wikipedia) initial idea to write a revolutionary personal information manager the book walks the reader through three years of software development (2003 - 2005). Most chapters follow a simple scheme. Rosenberg describes a concrete problem that occurred during the development of Chandler, then he generalizes his observations to software development in general and likens them to what he read during his research. One could assume that the last chapter was about the successful release of a Chandler 1.0 version but that's not what happened. Chandler was delayed endlessly and in 2005 it was obvious that there would be no "finished" version of Chandler for years to come. At this point Rosenberg decided to leave the Chandler team because he had a book to write.

Let's get to the important part of the review now. For me, a fresh out of college software developer working his first real job, this book is very scary. In fact there was one page where I even started to get goose bumps. You have to understand that the Chandler project was kind of unusual. It was a project that did not really have budget constraints since Mitch Kapor paid for everything out of his own pocket. It was a project that did not really have the pressure of looming deadlines (at first) because there was no customer who paid for it. It was a project that had dozens of people with great names, reputation, and past achievements working on it. It was a project with an excellent work environment. It was basically everything a software developer could hope for. And nevertheless all initial goals of the project were disastrous failures (it is 2008 now and there's still no version 1.0). If this is the reality of software development I might as well just become a landscape gardener now.

Having only third-hand experience with the Chandler project, it is difficult for me to assess what the main reasons for Chandler's failure were. Maybe the idea Mitch Kapor had in his head was just too ambitious and no development team could have implemented it. Beyond some nebulous idea of a revolutionary PIM program it is never really clear what Chandler was supposed to be anyway. However, there were some key problems Rosenberg mentioned or at least implied.

The first main problem was the inability of the Chandler team to make decisions. Decisions were put off for weeks, months, and sometimes years. Just reading about delay after delay was extremely frustrating and I do not even want to imagine the frustration of the developers who were left in a void because management just could not make decisions. Several people left the Chandler project for this reason. On one page Scott Rosenberg even describes the relief people had after a certain difficult decision was made. Nobody actually cared whether the decision was correct or not, people were just incredibly happy that a decision was made.

The second main problem was the inability to stick to decisions. Whenever a new technology that might benefit Chandler came out or when a new person with some special skill joined the team all discussions began anew. Indeed technologies were switched several times during the project and the result after three years of work had basically nothing in common with what was planned in the beginning.

Then there were the endless meetings. I guess that was a direct result of the inability to make decisions. If you meet once and fail to make a decision you have to meet again. And maybe again, and again, and again. There were times when certain subteams held meetings three afternoons a week.

Another issue I found weird is that so many people were hired. Once in a while developers were hired for what I perceived to be incredibly minor tasks. Sometimes I felt that each dialog in Chandler had its own developer. It's no surprise that communication time increases exponentially if you can not leave the code of your dialog without having to talk to some other developer first. Less might have been more.

Anyway, these are just the directly observable proximate causes for Chandler's problems. The real question is what were the root causes that led to the proximate causes. I don't know. I don't have the experience to know this, especially not if I only read about a failed project in a secondary source. I only know that whatever the root causes are I will try everything to identify them quickly and avoid them in software projects I'm involved in.

I recommend this book to everyone who wants to read a scary story about software development.

Trackbacks

No Trackbacks

Comments

Display comments as Linear | Threaded

Zoltan H on :

I reviewed Dreaming in code a year or so back - still one of my favourite books on open source and writing software in general. In a way it reminded me of what got into writing software.

[link]www.yyztech.ca/reviews/book/dreaming-in-code[/link]

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
BBCode format allowed
Form options

Submitted comments will be subject to moderation before being displayed.