Measuring the value
Last updated
Last updated
The ability to measure and present the value of a digital system is probably the most important part for a design team. Actual performance numbers provide a real-time view on the health of the system—and if positive—data needed to continue investing in the system.
We develop a measuring plan to evaluate the success of the digital system related to the productivity and agility to develop a product. We define the plan in two parts:
Health of the system: These are metrics related to the use of the system, success of components, etc which are valuable for the Digital System team. It allows them to understand what is functioning correctly and what needs to be updated based on real data.
ROI of the system: These are metics grounded in business goals and the KPIs of the system. This will be the data that is presented to Top Management to prove the current value and future investment in the system.
For example, we've seen savings around 10,000 hours and 280,000€ a Q in just one project by using a digital system. The method used to measure and forecast ROI is provided below.
We follow a framework of Metric, KPI, Goal to measure the digital system, starting by defining the project goals, then the KPIs that will define success, and finally the metrics we will pull to evaluate the performance. Two elements to keep in mind:
Periodically. These metrics are pulled weekly to have a week-over-week comparison and understand if the system is improving. Chose a day and stick to it, the time periods always need to be the same.
Dashboard. A dashboard in the website of the digital system as well as the raw data as a Google Sheet can always be accessed by all teams.
Here are a few examples relating to the health of the system:
Number of teams using the digital system
Number of teams/projects overall
% of team that use the system
Number of libraries being used
Number of components in production
Total components in system
% of components in production
Number of broken components
% of healthy components
Number of feedback requests
Days to resolve and issue
We use a calculation that can be implemented across any digital system to measure time saved and therefore money saved by using a digital system. It is a stepped process that we detail here:
This process measures the value of the digital system during the design phase. It does not include the development phase.
Define the value of one component in terms of time (hours) and costs (euros €)
Time (hours) = time from receiving requirements to the final approved upload to the digital system Cost (euros €) = avg hourly rate of designer x hours
7 hours = average time from receiving requirements to the final approved upload to the digital system. 196€ = 28€ x 7 hours (€52,500 average salary ➗ 1840 hours in a year = €28 hour)
Define the number of unique components in a delivered design file to development
Unique components: only measure unique components in a file, repeats of the same component will create a false, inflated calculation.
Delivered design file: define calculation after final, approved design file has been delivered to development. Using in-progress work will also create a false, inflated calculation.
35 unique components = average in a simple features 245 hours = 35 x 7 hours 6,860€ = 28€/hr x 245 hours
72 unique components = average in a complex feature 504 hours = 72 x 7 hours 14,112€ = 28€/hr x 504 hours
Add all delivered features by Q across programs to highlight hours saved and euros € saved.
5 different projects delivered 26 features to development 10.015 hours saved 280.410€ saved
Calculate the average number of features delivered in a Q per project and define the average number of unique components in delivered design files for that project.
Average number of features delivered: work with Project Manager to define this average based on past Qs. Average number of unique components: average based on simple and complex features over past Qs
18 average features delivered per Q 53.5 average unique components per feature per Q (simple 35 + complex 72 / 2)
374.5 hours = 53.5 x 7 hours 10,486€ = 28€/hr x 374.5 hours
6.933 hours saved = 374.5 x 18 features 194.140€ saved = 10,486€ x 18 features
