Continuous Improvement blogMeasuring Benefits: Retrospective Challenges

In my previous blog, I talked about the problems I encountered when I was asked to support the evidencing of benefits for projects that were already well established.  I talked about the difficulties that I had around identifying what possible benefits there were.  In this blog, I’d like to discuss the challenges that I faced obtaining the baseline data that was required to demonstrate the potential benefits that had been identified.


Baseline Data

By not thinking about what the benefits of a project might be at the start of the project, crucial baseline data can be missed, or rather, the opportunity to capture the baseline data has passed.  I had this problem when trying to gather data around a process redesign project that focused on the delivery of quotes for work. The priority of the redesign was to improve the customer experience by providing quotes in a more timely manner. 

The project team were able to provide data on the one benefit that they had thought of measuring at the start of the project.  They were able to demonstrate that the improvements the team had put in place had significantly reduced the turnaround time, in fact they had reduced the number of days taken to produce customer quotes by 52%.  This was an excellent result for the team and would certainly help to improve the customer experience.


All done?

Providing quotes faster resulted in a hugely improved turnaround time, but it also has the potential to achieve more. What is the result of doing things faster and potentially better?  By getting high quality quotes out to customers faster, does this also have a positive impact on the conversion rate of the number of quotes to the number of orders placed?  Does it result in increased income? These were all questions that I asked the team, but it was clear that they had not thought about things in this way at the beginning of the project.  Also, had they freed up any of their own time by improving the process, thus improving productivity?


We know it has improved!

I truly believe that the team are now spending less time on the process now than they were before, and producing higher quality quotes.  However, no one is able to prove that this is the case as no baseline data was captured.  The team had not captured what the current process timings were prior to any improvements and are now in the position where they feel like they spend less time than before, but they are unable to demonstrate this, and as a result, cannot claim it as a benefit.  To avoid this in future, all potential benefits need to be investigated prior to any changes being made.

Can’t prove it

Getting data to evidence these benefits has proven to be challenging, again due to the fact that the team did not think of these as at the start of the project.  It is not something that was considered and was therefore not measured.  Trying to get them on board at this point in the project with considering other benefits is difficult, their minds have moved on.  Introducing the team to benefits at the start of the project is essential in order to ensure that the team are focused on all of the potential benefits of the project.

On a more positive note, as I mentioned above, the team did feel better about the process, which can also be considered a positive outcome.

In my next blog, I will discuss the different experience I found when I introduced the concept of benefits at the very first workshop of a new project, and how the Benefits Exploration Map and the Measureable Benefits Data Plan worked well to focus the minds of the project team on measuring as many benefits as possible.