Metrics For Business Justification

I frequently give talks at conferences and architecture training workshops about techniques for architecture refactoring. After you have identified the architectural root cause of your issues and established a direction as the where to take your architecture (i.e., the future- or end-state), you must be able to justify the architecture refactoring effort in order for the business to not only agree to the refactoring effort, but also to pay for it. Too many times product owners or business users see architectural refactoring as technical debt that has little or no business value. And in fact, they are right if you cannot provide a reasonable business justification for your architecture refactoring effort.

Business justification typically takes the form of three primary benefits: reduced cost, better time-to-market, and better overall user satisfaction. Fortunately, there are some metrics you can gather prior to your architecture refactoring effort that can demonstrate added business value and help you with your business justification. One key point is to ensure that you gather these metrics well before your refactoring effort and track and demonstrate the trend associated with these metrics. Then, at designated iterations in your architecture refactoring effort, you can gather additional metrics, analyze the trend, and demonstrate that it was in fact your architecture refactoring effort than made these metrics better. 

So, let's take a look at what these metrics are and how they associate directly with business value, and hence business justification for your refactoring effort. 


Business Justification: Reduced Costs
Key Metrics: Number of Bugs, Actual Development and Testing Hours

Reduced costs as a business value can be demonstrated in two ways: bug fixes and development/testing costs. Start with a trend analysis of the number of bugs reported in the system (e.g., JIRA tickets, Bugzilla reports, etc.). Each bug reported costs the business money. However, it is not only the development and testing time to fix the bug, but also the time spend away from other development tasks such as new features. This can also impact time to market as well because developers can focus more on features rather than bugs. Fixing bugs will truly slow down the entire development lifecycle, because they are done concurrently with other development tasks, requiring increased coordination, release planning, and communication, all of which impact cost. Start tracking the number of bugs and the effort (hours) associated with fixing and testing the bug, and demonstrate the reduction through trend analysis. If possible, attach an internal rate onto these, multiply by the time spend fixing the bug, and you have real cost to the business. If you are confident that the architecture refactoring effort will make the system more reliable and easier to test, you can translate it to real dollars.


Business Justification: Better Time To Market
Key Metrics: Actual Development and Testing Hours

This metric has to do with the entire software development process, from requirements publishing all the way to production deployment. Obviously you cannot control all aspects of this, but refactoring changes should be at least able to demonstrate that you are ready for production deployment faster by reducing development and testing costs associated with changes and additional features. Time to market can be measured by starting to track (again, through trend analysis) the time it takes to get features out the door. Rate them easy, medium, and hard, and track separately by actual estimates. Original estimates may not changes, but if actuals go down, it demonstrates a quicker readiness for production. This in turn will help in scheduling more frequent production deployments, also an indicator of better time to market (i.e., moving from bi-monthly deployments to weekly deployments).


Business Justification: Better Overall User Satisfaction
Key Metrics: Number of Errors, Number of Bugs, Performance Metrics

User satisfaction is unfortunately one of the hardest values to quantify due to the subjective nature of this business value. However, in general user satisfaction can be boiled down to overall performance, number of errors, how those errors are handled, and the number of bugs experienced by end users. The better the overall application performance, the more satisfied users will be. The fewer the errors, the more satisfied users will be. In terms of performance, you can create a simple interceptor for each request that records the start and stop times and write this data to the log or a database table or file. Then perform daily or weekly trend analysis based on request - you should see these numbers go down during and after your refactoring effort, indicating that your changes did in fact help create better performance, and hence a better user experience (if performance is a concern of course). 

The other measurement for this business justification is overall reliability. Reliability can come in several forms; reliability in terms of continued failed deployments, and reliability in terms of the number of errors and outages seen in production. Capturing these metrics and establishing a benchmark is fairly straight forward. Prior to the business justification step begin capturing and recording the number of failed migration/promotion issues and use this as a benchmark. Start a trend analysis right away so that you can demonstrate it was in fact the refactoring effort that brought the numbers down. Do this on a daily or weekly basis, charting time by number of failed deployments. Once you refactor, continue to record and publish the trend analysis. This is a great way to demonstrate it was your changes that made the difference. The same goes with system and application errors. On a daily basis gather the number (and type if possible) of production errors from log files or wherever your errors are recorded. Then, doing the same trend analysis, publish daily or weekly reports showing the decline in errors (see reduced costs above). Also, as indicated in the first business justification, demonstrating a reduction in the number of system errors and reported bugs can also show better overall user satisfaction.  


Conclusion

The important thing to keep in mind is that regardless of the business justification you are using, you must be able to demonstrate that business justification through some sort of metric. Maybe its a user survey, or some of the more concrete metrics identified here. To reiterate, it is critical to establish the trend analysis early on so that you can clearly demonstrate that it was your refactoring effort (and not other factors) that lead to the increased business value. 

© Mark Richards 2016