Every development team that has ever been lucky enough to have a successful product has asked themselves this question when it came time to really evaluate how to take that product to TheNextLevel(tm). And like most interesting questions, there’s no right answer. At least, not one you can be certain about beforehand. There is one thing that is definitely true though or else you wouldn’t be facing this question. The product has to change, substantially, and even continuing with the same code base will require a significant amount of work.
How do you decide between a massive amount of rework on an existing system and a massive amount of new work on a new system? I’m certainly no expert, but I’ve witnessed and been involved several times in these discussions over the years. Some things I’ve learned that are always relevant are: market pressure, technical debt, team members, and organizational bias.
How much pressure do you have to build this new version of the product fast? If the window is small and you can’t widen it, a rewrite is most likely out as an option. The inability to widen your time window is a problem though. Short term, sure, sometimes there can be situations you need to accommodate but long term you need to be able to go dark to the rest of the world and do the hard work.
Has development of new features slowed to a halt on your product? Typically, the reason for this is due to an overwhelming amount of technical debt that has accrued in your system. Bad engineering decisions of the past are coming back to haunt you. This is a great reason to rewrite. You’ve built it once, you know more now than you did the first time you built it. You also know more about and have a clearer vision for the product’s future. Armed with this and whatever technological advances have happened since the original writing of the product (hopefully you’ve been able to keep yourself up-to-date), you should be in much better shape to deliver a top of the line product.
Do you have the same quality of personnel available as you did when you built the product the first time? Not every developer is equal. Not even close. You should consider who you have on the team before you throw out code written by former, great employees.
Organizational bias is more like internal, social pressures not to rewrite. The feeling that the product is too popular to change. Or that users won’t be happy about having the product they’re already comfortable using changed out from under them. These fears allow you to avoid the hard decisions needed to make the right product. This fear also limits the options for the product; you’ve basically talked yourself out of doing the rewrite. This thinking is how IE6 happens. Don’t let your product become another IE6. (How’s that for fear-mongering?) Give your team and product the option to become something new if that’s the right decision. You know better about your product then your users. Don’t let fear direct technical decisions.
I’ll talk about AgileZen: how it’s built now and why we felt we needed a rewrite.
We have a successful product built using a fairly typical design for web applications. We have a MVC system running in .NET. This handles all of the data for the system: creating, reading, updating, and deleting. In addition, it also handles a substantial amount of the UI rendering. Using server side templates we render partial chunks of HTML code that could then be combined together to form the whole page request.
This server design accounts for most of the content that is AgileZen. Things left for the client, or browser, to do are mostly to add functionality to the content that has been loaded. Things like drag-and-drop for the stories between phases, editing the text of a story directly on the card, or adding a card to the board are all features built completely client-side using nothing more complex than jquery with a few plugins.
That’s the How, now for the Why…
It all started with knowing we wanted real-time data access for the next version of AgileZen. It was a game changer for everything from designing the UX of the new product to how the data returned from the server. The user experience of a real time system was not the sort of thing we thought we could layer on top of the existing product and make it feel like it was a first-class experience. We wanted the whole UI to react to new data being received smoothly, and fast. No matter how many different places on screen were affected by this new data, the on screen update had to be fast. Our current client side design had no mechanism built in to handle alerting all the different parts of the UI about the arrival of new data. This led us to the client-side MVC design paradigm and now a partial product rewrite. At this point, we were only going to rewrite the client code.
We may have only intended to rewrite the client but that didn’t mean that the server didn’t need to change significantly. But first, let me back up a little bit. We had known for some time that Windows and .NET was something we wanted to get away from using. I won’t go into the why for that here, that’s a whole other blog post. Suffice it to say that we knew eventually we were going to do the work to remove that and go with, most likely, a Node.JS solution. With this in mind, we still thought the safest way to release was by keeping our existing server-side code in use and adding / removing functionality as needed.
But two things happened as this continued that caused us to change our minds. First, we built proof-of-concepts of entire working subsystems of our .NET server system using Node.JS, MongoDB, RabbitMQ, Python, Redis, Socket.IO, and others on our Hack Days (20% of the time here at AgileZen is dedicated to innovation). This experience convinced us that these other technologies were the right way to go. We started ripping off subsystems like Notifications, Revisions, and Logging from the .NET server and distributing that same work to processes running on Node.JS. We now had actual experience building parts of our system using these new tools, but still we kept thinking we should hold off on the whole server side rewrite. Second, another Hack Day yielded a working Node.JS / MongoDB based Web application API server… in just one day! Seriously, you have to love Node.JS and the entire ecosystem that has emerged there, but I digress. After this next successful innovation, we knew it was worth it to put our efforts behind new work rather than rework. For us, knowing we would ultimately do the new work anyway due to a desire to have a pure Javascript environment was the deciding factor.
Would we do it again? I don’t know. This time, with the team members and circumstances involved, we opted to go for the complete product rewrite. It’s ambitious, to say the least. We have built multiple frameworks that we intend to open source once they’re polished up. But now, the AgileZen team has a different landscape than it did when we made this decision. It’s very likely given the same question, we would not go this route; we very nearly didn’t last time either. We’ll see how it pays off.
About the new frameworks: one a client-side MVC framework written in CoffeeScript and the other an Asynchronous API server also written in CoffeeScript, both started as Hack Day experiments and prove to me that investing in innovation is the right way to build a product. I’m sure these both warrant blog posts of their own when the time is right.
