The agile manifesto states that working software is the primary measure of progress. Whilst this makes sense, a set of key metrics can help everyone to understand how things are going and to identify areas for improvement.
Developing software or orchestration vs automation is a complex business; a healthy relationship with your supplier is an essential ingredient in the recipe for success. However, success means different things to different people. Work with your supplier to produce a set of success criteria when you first engage. These criteria will ensure that everyone is focused on what is important. Creating metrics that are aligned to your objectives will help you to understand what good looks like. Your goals will most likely change over time so be sure to review your success criteria on a regular basis.
Any metrics that you do create should be actionable. In other words, if you are tracking something that does not directly influence your behaviour then you are probably tracking the wrong thing. Be aware of vanity metrics – measurements that make you feel good but do not help to improve what you are doing.
What should you measure
The broad objectives of software development are simple enough to describe as the diagram below illustrates. Doing the right thing means implementing features that will support your business and end users. Doing the thing right is about attention to detail and focusing on quality at all times. Doing it efficiently relates to being highly productive but also working at a sustainable pace. Having metrics that in some way relate to these 3 factors will help you to track and manage the development process. Locating your position on the diagram may help you to identify where to focus your attention. For example:
|A+B+C||A team in this area is developing important features in priority order. Quality levels are high and they have a comfortable and effective working environment. The team is productive and delivers a high, sustainable output.|
|A+C||There is a focus on doing the right thing i.e. implementing features that will benefit your business. The development is also efficient and the team is be productive. There may be issues relating to quality such as a high defect rate, the approach to development or an inappropriate architectural design.|
|B+C||In this area, the team is productive and quality levels are high. There may be some difficulty in setting feature priorities or ensuring that the team only works on ‘must have’ items.|
|A+B||A team in this area is implementing the right features in the right order. Their development approach is generally sound and quality levels are high. Whilst this is a great position to be in, things may be taking a little too long to complete.|
Being able to determine where a team sits on the diagram will help to identify areas for improvement.
Good properties for metrics
The things you measure will influence your behaviour in one way or another. Carefully choose metrics that have a positive influence rather than a negative one. Aligning measurements to your core objectives will help to ensure you create the right incentives. In general, metrics should:
- Be simple to understand by anyone that is looking at them
- Alwasy be actionable
- Be easy to measure, collect and share
- Be aligned to one or more of your core objectives
- Promote positive behaviour and desirable incentives
On a final note,there’s little benefit in recording any metrics if you don’t use the information to improve the way you work. Always look for ways to adjust your sails a little if you find the wind is blowing you in the wrong direction.