Measuring the value

The ability to measure and present the value of a digital system is probably the most important part for a design team. Actual performance numbers provide a real-time view on the health of the system—and if positive—data needed to continue investing in the system.

We develop a measuring plan to evaluate the success of the digital system related to the productivity and agility to develop a product. We define the plan in two parts:

1. Health of the system: These are metrics related to the use of the system, success of components, etc which are valuable for the Digital System team. It allows them to understand what is functioning correctly and what needs to be updated based on real data.

2. ROI of the system: These are metics grounded in business goals and the KPIs of the system. This will be the data that is presented to Top Management to prove the current value and future investment in the system.

For example, we've seen savings around 10,000 hours and 280,000€ a Q in just one project by using a digital system. The method used to measure and forecast ROI is provided below.

Measuring the health of the system

We follow a framework of Metric, KPI, Goal to measure the digital system, starting by defining the project goals, then the KPIs that will define success, and finally the metrics we will pull to evaluate the performance. Two elements to keep in mind:

• Periodically. These metrics are pulled weekly to have a week-over-week comparison and understand if the system is improving. Chose a day and stick to it, the time periods always need to be the same.

• Dashboard. A dashboard in the website of the digital system as well as the raw data as a Google Sheet can always be accessed by all teams.

Here are a few examples relating to the health of the system:

Goal 1 / Increase the use of the digital system

KPI's

• Number of teams using the digital system

• Number of teams/projects overall

• % of team that use the system

• Number of libraries being used

Goal 2 / Increase the implementation in production files

KPI's

• Number of components in production

• Total components in system

• % of components in production

• Number of broken components

• % of healthy components

Goal 3 / Improve the effectiveness of support and feedback channels

KPI's

• Number of feedback requests

• Days to resolve and issue

Measuring ROI in time and money

We use a calculation that can be implemented across any digital system to measure time saved and therefore money saved by using a digital system. It is a stepped process that we detail here:

This process measures the value of the digital system during the design phase. It does not include the development phase.

Step 1. Defining the value

Define the value of one component in terms of time (hours) and costs (euros €)

Time (hours) = time from receiving requirements to the final approved upload to the digital system Cost (euros €) = avg hourly rate of designer x hours

Example: One component

7 hours = average time from receiving requirements to the final approved upload to the digital system. 196€ = 28€ x 7 hours (€52,500 average salary ➗ 1840 hours in a year = €28 hour)

Step 2. Defining unique components

Define the number of unique components in a delivered design file to development

Unique components: only measure unique components in a file, repeats of the same component will create a false, inflated calculation.

Delivered design file: define calculation after final, approved design file has been delivered to development. Using in-progress work will also create a false, inflated calculation.

Example: Simple feature

35 unique components = average in a simple features 245 hours = 35 x 7 hours 6,860€ = 28€/hr x 245 hours

Example: Complex feature

72 unique components = average in a complex feature 504 hours = 72 x 7 hours 14,112€ = 28€/hr x 504 hours

Step 3. Delivered features by Quarter (Q)

Add all delivered features by Q across programs to highlight hours saved and euros € saved.

Example: One business unit, one Q

5 different projects delivered 26 features to development 10.015 hours saved 280.410€ saved

Step 4. Forecasting

Calculate the average number of features delivered in a Q per project and define the average number of unique components in delivered design files for that project.

Average number of features delivered: work with Project Manager to define this average based on past Qs. Average number of unique components: average based on simple and complex features over past Qs

Example: One feature "Loyalty App"

18 average features delivered per Q 53.5 average unique components per feature per Q (simple 35 + complex 72 / 2)

374.5 hours = 53.5 x 7 hours 10,486€ = 28€/hr x 374.5 hours

Total Forecasting per Q Loyalty App

6.933 hours saved = 374.5 x 18 features 194.140€ saved = 10,486€ x 18 features