Escaping Technical Debt with Agile Methodology
By Mike Peterson on August 30, 2023
Technical debt is caused by development or implementation choices that prioritize speed over best practices, usually one-off changes that end up increasing the complexity of your system. After developing a solution, you almost always find yourself making quick changes to extend capabilities or fix a bug. Eventually, these system adaptations result in something that is hard to support. At some point, like all debts, you will have to pay that back.
Carrying technical debt is a bit like taking on personal debt. It is about making feasible commitments. Much like buying a house slightly beyond your means with the help of a mortgage, carrying some debt is okay when it allows you to expand your resource footprint without compromising other vital aspects. The technological equivalent is that, when you spend all your time managing what you’ve put in, that becomes a barrier to doing anything new.
The main goal is not to eliminate technical debt but to keep it low, which isn’t easy. When you are designing new processes, you often have to account for a lot of requirements from stakeholders of all kinds. Some features are critical, and others are just nice to have, and it can be hard to identify which is which at the start.
As the project proceeds, your technical team starts out knowing your system’s scope and assets, but maybe they’re not documenting it well. Over time it becomes hard to remember all the adaptations, and even harder to communicate them to someone else. In six months, will you be able to go back and fix or extend that system? Is someone else going to be able to come in and maintain what has been created?
The requirements the technical team focuses on are usually driven by business needs. So, the players are the technical team, their managers, and a business sponsor or an executive sponsor. To control technical debt, you have to have a proper channel to communicate about it up the chain. Leadership needs to participate so that new features can be prioritized. The groups must work together to decide whether new functionality is worth adding to the organization’s technical debt. Is solving for X worth having that debt? Is X feature worth adding to the organization’s debt?
Reducing Technical Debt from the Start
Traditional Waterfall project management leads to technical debt. In the waterfall methodology, we’re more likely to say: “Here are the requirements up front, and we need all of them in the final product.” That’s paired with a timeline for delivery and an emphasis on seeing a fast return on investment. The mindset is that you need to get everything in quickly, but when speed is your top priority, you make technical compromises. You don’t design your system intentionally or document it precisely. In the end, you have something you didn’t really mean to make and don’t even totally understand.
The system may work great for a while. And then an upgrade comes due. Let’s say you need to integrate a new line of business software, and your Application Programming Interface (API) is not working. That’s a sign you’ve handicapped yourself with technical debt. Getting clear of that may take a longer development cycle than you budgeted for.
By contrast, the Agile methodology embraces the concept of the minimum viable product. Start by gathering a comprehensive list of all the features you can imagine wanting. And then go through that and identify what you really need right away. What features are critical? Which are interdependent? The goal is to implement a system with the minimum capabilities needed to make it functional, built on a foundation that’s extensible to additional requirements. That lets you start seeing return on investment while leaving room for your solution to grow in the future.
Agile teaches us to think in phases. Even when everything is going well on a project, that’s just a phase one. There’s always another phase coming. As someone who puts in systems, I like the concept of saying: “We’re working on a phase one and we already have a phase two lined up.” That means that I’m immediately going to roll into the next requirements. Whatever is deemed the most critical, we work on that next. And after we deploy it? Next set of requirements. Work on it. Put it up. This way of working gets us out of rush mode because we’re already getting the most critical benefits from the solution.
From here you can monitor the process and get feedback to see whether the prioritized features were as critical as you thought. Maybe you discover some of the original requirements aren’t relevant anymore, or that stakeholders aren’t using them. Maybe a new business problem pops up somewhere else. Take the time to evaluate before moving on. It’s a long-term strategy.
The important thing is having those next phases already queued up. Otherwise, you’ll get to the end of phase one and everyone just wants to sit back and breathe; then you’re pulled into other projects and lose the thread. You need an active roadmap to stay focused on where you’re heading next.
In Conclusion
Some technical debt is always a reality. Take your time. Don’t rush. Work on becoming more agile and tackling bite-size chunks. Focus on the identified priorities and take care of those first. Then monitor your success and move on to the next requirements in phases.
There’s always a next phase. Being proactive and iterative are the two key elements of making sure your technical debt stays both limited and manageable.
This article is condensed and adapted from Greg Council’s interview with systems engineer Mike Petersen for the ILINX User Community. Want full access to more exclusive content like this? Join the group.