Illustrations
The ability to measure and present the value of a digital system is probably the most important part for a design team. Actual performance numbers provide a real-time view on the health of the system—and if positive—data needed to continue investing in the system.
We develop a measuring plan to evaluate the success of the digital system related to the productivity and agility to develop a product. We define the plan in two parts:
Health of the system: These are metrics related to the use of the system, success of components, etc which are valuable for the Digital System team. It allows them to understand what is functioning correctly and what needs to be updated based on real data.
ROI of the system: These are metics grounded in business goals and the KPIs of the system. This will be the data that is presented to Top Management to prove the current value and future investment in the system.
For example, we've seen savings around 10,000 hours and 280,000€ a Q in just one project by using a digital system. The method used to measure and forecast ROI is provided below.
Measuring the health of the system
We follow a framework of Metric, KPI, Goal to measure the digital system, starting by defining the project goals, then the KPIs that will define success, and finally the metrics we will pull to evaluate the performance. Two elements to keep in mind:
Periodically. These metrics are pulled weekly to have a week-over-week comparison and understand if the system is improving. Chose a day and stick to it, the time periods always need to be the same.
Dashboard. A dashboard in the website of the digital system as well as the raw data as a Google Sheet can always be accessed by all teams.
Here are a few examples relating to the health of the system:
Goal 1 / Increase the use of the digital system
KPI's
Number of teams using the digital system
Number of teams/projects overall
% of team that use the system
Number of libraries being used
Goal 2 / Increase the implementation in production files
KPI's
Number of components in production
Total components in system
% of components in production
Number of broken components
% of healthy components
Goal 3 / Improve the effectiveness of support and feedback channels
KPI's
Number of feedback requests
Days to resolve and issue
Measuring ROI in time and money
We use a calculation that can be implemented across any digital system to measure time saved and therefore money saved by using a digital system. It is a stepped process that we detail here:
This process measures the value of the digital system during the design phase. It does not include the development phase.
Step 1. Defining the value
Define the value of one component in terms of time (hours) and costs (euros €)
Time (hours) = time from receiving requirements to the final approved upload to the digital system Cost (euros €) = avg hourly rate of designer x hours
Example: One component
7 hours = average time from receiving requirements to the final approved upload to the digital system. 196€ = 28€ x 7 hours (€52,500 average salary ➗ 1840 hours in a year = €28 hour)
Step 2. Defining unique components
Define the number of unique components in a delivered design file to development
Unique components: only measure unique components in a file, repeats of the same component will create a false, inflated calculation.
Delivered design file: define calculation after final, approved design file has been delivered to development. Using in-progress work will also create a false, inflated calculation.
Example: Simple feature
35 unique components = average in a simple features 245 hours = 35 x 7 hours 6,860€ = 28€/hr x 245 hours
Example: Complex feature
72 unique components = average in a complex feature 504 hours = 72 x 7 hours 14,112€ = 28€/hr x 504 hours
Step 3. Delivered features by Quarter (Q)
Add all delivered features by Q across programs to highlight hours saved and euros € saved.
Example: One business unit, one Q
5 different projects delivered 26 features to development 10.015 hours saved 280.410€ saved
Step 4. Forecasting
Calculate the average number of features delivered in a Q per project and define the average number of unique components in delivered design files for that project.
Average number of features delivered: work with Project Manager to define this average based on past Qs. Average number of unique components: average based on simple and complex features over past Qs
Example: One feature "Loyalty App"
18 average features delivered per Q 53.5 average unique components per feature per Q (simple 35 + complex 72 / 2)
374.5 hours = 53.5 x 7 hours 10,486€ = 28€/hr x 374.5 hours
Total Forecasting per Q Loyalty App
6.933 hours saved = 374.5 x 18 features 194.140€ saved = 10,486€ x 18 features
Last updated
