MEASURING INNOVATION

A framework for getting it done

MEASURING INNOVATION
A framework for getting it done

Pedro do Carmo Costa

May 2015

Pedro dO CARMO COSTA

Exago’s director and co-founder
Pedro leads Exago’s business development efforts. His career has centred on the art and
science of innovation for almost 15 years. He’s worked to help large companies build an
internal capacity for innovation, to ensure they grow and evolve successfully. Prior to
co-founding Exago, he worked at Strategos with innovation thought leaders such as Gary
Hamel and Peter Skarzynski. Pedro lives in London, and he’s the proud father of four.

pcc@exago.com

Exago is a pioneer in software and services that use collective intelligence to solve key
business challenges, with an innovation-led approach. From cost reduction and customer
engagement to behavioural change, we help organisations worldwide unleash people’s
hidden potential to achieve real results.

Since 2007, Exago has grown with our clients and partners worldwide. Together, we’ve
reached four continents (Europe, Asia and North and South America), spanning a variety
of industries – from major retailers and telecom operators to health organisations and
energy suppliers.

We have three office locations: Lisbon, Portugal; London, the UK and São Paulo, Brazil.
CONTENTS

I
Introduction 5

II
How innovation (and its metrics) has changed 6
1. THE EVOLUTION OF INDICATORS
2. COMPANIES PUTTING INNOVATION TO WORK

III
Measuring innovation 13
1. THE HUMAN FACTOR
2. METRICS IN THE CORPORATE WORLD
3. AVOID THE ANALYSIS-PARALYSIS

IV
Conclusion 17
‘What gets measured gets done.’
Attributed to Peter S. Drucker
 I

INTRODUCTION
Innovation has left the secluded islands and frosted glass towers of research and
development (R&D) departments, reaching all corners of organisations – even going
beyond innovation’s traditional frontiers. As it grows in both potential and intensity, how
can we efficiently measure this complex process to gather all its benefits?

Where to start? What to measure? How can we make the improvements needed to
progress? How can we prove an innovation programme’s success, or even that it’s
successful at all? With many questions and too much at stake, innovation is no longer
a whim, but simply the way business is done. We need to make it work.
A clash between more traditional models – the isolated innovation islands – and new,
more comprehensive models marked the last decade of the twentieth century:
// On the one side, innovation was seen, more specifically, as R&D, science and
technology (S&T) and the creation of comparative advantages.
// On the other side, consultancy firms such as Strategos and its founder, management
guru Gary Hamel, advocated that the concept should evolve and expand. The process
became more inclusive and multifaceted, engaging larger numbers of people to
address key business challenges.
We thus cannot discuss the evolution of innovation metrics without understanding
the evolution of innovation in the corporate world.
This is why, in the first section, I take a brief but insightful look as far back as the 50s,
to track the evolution of innovation and its main indicators. I next focus on a handful
of interesting organisations’ feats – from Intel’s research labs to Uber and then Renova,
with its black toilet paper – to grasp what innovation means today, the shapes it takes
and a common trait all these cases share.
In the second section, I highlight the ‘human factor’ as the ‘fundamental driver of
innovation’ and its implications for measuring innovation practices. I also identify the
main metrics companies use to gauge the process. To conclude, I lay out a framework
and simple steps we believe can help you define your own metrics set.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 5 | 17

HOW INNOVATION
(AND ITS METRICS)
HAS CHANGED
1. THE EVOLUTION OF INDICATORS
In the 50s or the 70s, what was generally considered an innovation? How did we gauge
II
its results? The white paper ‘Innovation Metrics: Measurement to Insight’ helps us map
this evolution, highlighting the more commonly used indicators for each decade:

1st GENERATION 2nd GENERATION 3rd GENERATION

1950-60s 1970-80s 1990s

- R&D expenditures - Patents - Innovation surveys

- S&T personnel - Publications - Indexing
- Capital - Products - Benchmarking
innovation capacity
- Tech intensity - Quality change

The authors, Egils Milbergs, president of the Center for Accelerating Innovation, and
Nicholas Vonortas, director of the Center for International Science and Technology
Policy at George Washington University, explain these categories:
// ‘First generation metrics reflect a linear conception of innovation focusing on inputs
such as R&D investment.’
// ‘Second generation complements input indicators by accounting for the
intermediate outputs of S&T activities.’
// ‘Third generation metrics focus on a richer set of innovation indicators and indexes
based on surveys and the integration of publicly available data.’

Each reflects, in this schema, the corporate spirit of its time.

Particularly during the 80s and 90s, many efforts were made to develop models to
measure innovation. For statistical purposes, in 1992, the ‘Oslo Manual’ (from the
OECD and Eurostat) first codified innovation as a set of tools and concepts restricted
to technological product and process innovation in manufacturing.
In mid-90s, innovation had definitely been added to the corporate agenda as a
specialty, gradually conquering more sectors and industries. And, by 2000, while
information technology and the knowledge economy matured, a range of new metrics
had become part of business management vocabulary, integrating the innovation
sphere – from ‘knowledge’ and ‘learning’ to ‘intangibles’. Another word became
central: ‘engagement’.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 6 | 17

Then, a new generation of indicators was needed. Milbergs and Vonortas describe this
fourth generation as ‘grounded in a knowledge-based networked economy’ (see the
table below).

1st GENERATION 2nd GENERATION 3rd GENERATION 4th GENERATION

1950-60s 1970-80s 1990s 2000s plus emerging focus

- R&D expenditures - Patents - Innovation surveys - Knowledge

- S&T personnel - Publications - Indexing - Intangibles
- Capital - Products - Benchmarking - Network
innovation capacity
- Tech intensity - Quality change - Demand
- Clusters
- Management techniques
- Risk/ return
- System dynamics

In the twenty-first century, comparative advantages have become less relevant.

Countries and companies thrive by developing competitive advantages, which rest on
‘making more productive use of inputs’. This ‘requires continual innovation’, Michael
Porter, a Harvard Business School professor, says.
As management models have evolved into what can be called the ‘2.0 Age of
Management’, CEOs, top management, business managers and all employees must be
involved in this ‘continual innovation’ effort. It’s everyone’s job.

THE EVOLUTION OF MANAGEMENT

MANAGEMENT 0.0 MANAGEMENT 1.0 MANAGEMENT 2.0

setting the challenges CEO business executives everyone
idea generation top management everyone everyone
idea building n/a n/a everyone
idea selection CEO Top management everyone
idea implementation top management Top management everyone

2. COMPANIES PUTTING INNOVATION TO WORK

Today, innovation is a shared process of ideation, evaluation, selection, development
and implementation of new or improved products, programmes or services. The ‘Oslo
Manual: Guidelines for Collecting and Interpreting Innovation Data’ (3rd Edition,
2005) defines four main types of innovation:
Product | Process | Marketing | Organisational

Besides these, the concept has gained new adjectives and additional layers of meanings
throughout the entire value chain. We can say it’s about R&D and S&T, but it’s also
disruptive, incremental and social innovation, as well as innovation competencies. In
practice, innovation has turned into a set of complex, collective dynamics. Let’s have a
look at some examples to understand what this means today.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 7 | 17

R&D – the technology labs
Investment in future capabilities and technology through R&D remains a crucial element
in the innovation process. Markets and societies’ fast rhythm of change demands it.
Intel and its research labs are a well-known case study. In 2000, the company
established six first-rate labs. Following an open collaborative model, its researchers
worked closely with top host universities such as Cambridge and Berkeley, sharing
intellectual property rights. Together, they developed notable projects.
Yet, by 2011, Intel had closed all its labs. In a time of cutbacks across industries, high
costs and low investment/return ratios may have motivated this decision, although the
company declined to comment.
Intel is still the world leader in computing innovation. In October 2014, Intel’s
venture-capital arm announced that it would be investing $28 million in five Chinese
technology startups.
Companies that invest more in R&D, over longer periods, indeed do tend to develop
the stronger competitive advantages needed, performing better than market average.
They also present higher levels of productivity, particularly high-tech firms.
Nonetheless, merely investing in R&D does not necessarily make these organisations more
innovative. Innovation ‘goes far beyond the confines of research labs to users, suppliers and
consumers everywhere – in government, business and non-profit organisations, across
borders, across sectors and across institutions’, according to the OECD.

Disruptive innovation – need a ride?

Clayton Christensen coined disruptive innovation as a ‘process by which a product or
service takes root initially in simple applications at the bottom of a market and then
relentlessly moves up market, eventually displacing established competitors’.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 8 | 17

Could Uber, the exciting new app-based, ride-sharing technology company, be
disrupting the taxi market?
Let’s consider these facts:
// Uber is particularly innovative in the way it prices its services and promotes a
culture of merit by ranking drivers (and customers).
// It has been the central cause of protests by taxi drivers and companies (which means
that it’s making significant waves in the industry).
// In some countries, it’s even been banned, as in Spain and India (with specific safety
and industry regulations under the spotlight).
// The service has been growing exponentially, and several other companies have
imitated its model, a trend branded ‘Uberification’ (What are the limits of this growth?
To what point can it interfere with the market’s dynamics?).

Though some disagree, Uber may very well serve as an example of what disruptive
innovation is. Or, it can at least feed the discussion of disruptive innovation itself: its
limits, impacts and potential.
This will surely be an interesting case to follow. Still, remember that exceptional
breakthrough innovations may promote one-shot market triumphs, and companies
cannot rely on them alone. Companies need to balance their portfolios with various
and continuous degrees of innovation.

Incremental innovation – the ongoing evolution

IPhone at Apple, Gmail at Google: these are the results of incremental innovation – of
structures, features and processes improvement – typically involving larger numbers
of people in these efforts. Thomas Edison, though typically considered an inventor,
was in truth amazingly brilliant at refining products so that these reached their
optimal condition for use and commercialisation. Here are two more examples of
incremental innovation at work.

a) Fleury – collective intelligence worth millions

The Brazilian company Fleury – a leader in clinical analyses in Latin America – has
more than 10,000 employees. They have been included in Fleury’s innovation efforts.
In 2007, the company initiated a programme to encourage suggestions for how to
improve its operations, allowing all employees to submit ideas on paper. These were
then evaluated by an innovation committee.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 9 | 17

To optimise the process, in 2011, the group adopted the Central de Ideias or Idea
Market. Created by Exago, this innovation management solution is a software
platform where all stakeholders can participate – not only collecting and harnessing
everyone’s ideas but also evaluating them.
Within a year, the results overtook even the most ambitious targets:
// Ideas submitted: escalating from 1,809, in 2011, to 7,269, in 2012, when the new
model was implemented (an increase of more than 400%)
// Ideas approved: more than doubling, from 443 to 946
// Ideas implemented: increasing from 225 to 300
// Participation: growing from 712 to 3,309 people (15 months after implementing the
platform, more than 70% of Fleury’s staff had actively joined in)

Fleury’s innovations have generated millions of dollars in value. They have reduced
operating costs and introduced more efficient, sustainable and environmentally
friendly processes.
In early 2015, almost 8,000 participants were still interconnected and actively
participating. More than 11,800 ideas had been submitted and over 500 implemented.

b) Renova – black toilet paper, really?

‘Why not?’ was the slogan the paper consumption goods Renova used in 2005 when
launching this new product. Something as unpretentious as changing toilet paper’s
colour to black transformed this small Portuguese company.
Soon, newspapers and magazines across the globe rated Renova’s black toilet tissue as
a top luxury good. The New York Times described it as a ‘must-have’, and its enthusiasts
grew in number – including famous artists and personalities.
Renova next adopted different colours. Beyoncé, reportedly, is said to be a fan of the
red toilet paper.
In just two years, Renova’s tissues and toilet paper could be found in 50 international
markets and in the best hotels and restaurants in Japan, France, the UK, the United
States, Belgium and Spain. Since then, the company has continued to multiply its trade
channels, also opening pop-up stores and new concept boutiques.
Clearly, incremental innovation promotes not only trust and collaboration but also
calculated risk-taking.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 10 | 17

Social innovation – a world of challenges and opportunities
Social innovation is an area full of endless challenges and opportunities. The concept
is relatively recent, but, throughout history, examples of social innovation abound,
from hospices through to the cooperative movement.
One more recent and celebrated example is that of microcredit. Through small loans
and low interest rates for poverty-hampered individuals, this revolutionary social
innovation model helps turn talent and ideas into businesses. Muhammad Yunus,
the ‘father of microcredit’, and the Grameen Bank he founded in 1983, were jointly
awarded the 2006 Nobel Peace Prize ‘for their efforts through microcredit to create
economic and social development from below’.
For this paper’s purposes, I use the definition of the Open Book of Social Innovation
(Murray, Caulier-Grice and Mulgan, March 2010). Social innovations are ‘new
ideas (products, services and models) that simultaneously meet social needs (more
effectively than alternatives) and create new social relationships or collaborations’.

Whirlpool – building innovation competencies

In the late 90s, faced with stagnation in revenues and market share, Whirlpool
decided to make a bold long-term investment: fostering a new corporate innovation
mindset. With the help of Strategos, the company launched a global initiative to instil
innovation as a core competency.
Thousands of employees were trained (as I-mentors and I-consultants). Whirlpool
redesigned business processes and put together a new system of innovation
management, changing the company’s culture.
Since then, the firm’s employees around the world have been contributing to
innovation efforts. They participate, share their ideas for products and services and get
involved in the process. In 2013, Whirlpool reported $19 billion of sales. It’s also the
world’s leading global manufacturer of home appliances.
From Intel and Fleury to Whirlpool, each of these organisations has put innovation to
work to address very diverse challenges:
// Developing new business models
// Increasing the number of their patents
// Improving operational efficiency
// Improving customers’ experience

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 11 | 17

// Developing a culture of innovation
// Engaging their companies’ talent
// Building new capabilities
// Creating a new marketing strategy
// Having a true social impact

Without results, every one of these efforts would be useless and irrelevant. We need
results, but we also need to assess them, taking into consideration the resources
invested and the processes’ efficiency.

To measure outcomes, we will most likely have to adopt different metrics or

different combinations of metrics, according to the goals we want to achieve. In the
next chapter, I identify and discuss some of the most relevant metrics we can use to do
this.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 12 | 17

MEASURING
INNOVATION
1. THE HUMAN FACTOR
Innovation is not a stationary object. It’s a process that transforms ideas into value – a
way of doing things that every company has to adopt at its core.
Since 2000, the focus on innovation has been moving to the value within
organisations’ people: not only employees but also stakeholders and consumers. The
emphasis is on finding new means and methods to motivate and engage these people
in the search for new paths and answers to vital business challenges. In other words,
innovation seeks to find how to unleash their talent and expertise – their full potential
to innovate.
The Global Innovation Index (GII), an annual reference publication that gauges

III
countries’ innovation results and capabilities, devoted its 2014 edition to the topic ‘The
Human Factor in Innovation’, describing the human factor as ‘the fundamental driver’
and ‘the heart of innovation’.
Developed by Cornell University, INSEAD and the World Intellectual Property
Organisation, GII surveys 143 economies around the world. In 2014, it used 81
indicators to establish countries’ ranking, based on the following framework.

Global innovation index (average)

INNOVATION EFFICIENCY RATIO (ratio)

Innovation input sub-index Innovation OUTput

sub-index
institutions market sophistication
- political environment - credit knowledge and technology
- regulatory environment - investment outputs
- business environment - trade & competition - knowledge creation
- knowledge impact
- knowledge diffusion
human capital business sophistication
and research - knowledge workers
- education - innovation linkages creative outputs
- tertiary education - knowledge absorption - intangible assets
- research & development - creative goods and services
- online creativity
infrastructure
- ict
- general infrastructure
- ecological sustainability

Switzerland, the United Kingdom and Sweden topped the 2014 rank. This index considers,
among other indicators, new product creation, the quality of infrastructure (e.g. access to
information and communications technology), and employees’ competence acquisition.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 13 | 17

2. METRICS IN THE CORPORATE WORLD
What about companies? What metrics are they using? According to McKinsey
Quarterly: The Online Journal of Mckinsey & Co (2009), the main indicators ranked
by importance by the 722 organisations surveyed are:
// Revenue growth due to new products or services 16%
// Customers’ satisfaction with products or services 13%
// Quantity of ideas or concepts in the pipeline 10%
// Expenditure on R&D as a percentage of sales 8%
// Sales share of new products/services in a given period 8%
// Number of new products or services launched 8%
// Return on investment in new products or services 4%
// Number of R&D projects 6%
// Number of people dedicated to innovation 4%
// Increase in profits due to new products/services 4%
// Potential of the complete portfolio of new products/services to meet growth targets 3%
// Changes in market share resulting from new products/services 3%
// Net present value of the complete portfolio of new products/services 2%

These choices reflect the ‘people factor’ in innovation, namely, customers’ satisfaction
with products and services and the number of people dedicated to innovation.
Nevertheless, bearing in mind companies’ specific goals, many others can be suitable.
Examples are the effort expended on cultivating an innovation culture or capabilities
development and the investments made or the wide-ranging outcomes of innovation processes.

input output
- R&D, design, financial and accounting human and organisational knowledge capital and new
research capital capital opportunities
- R&D per head, - growth/income internal: - number, percentage and quality
hours and days of new ideas/contributions
- global sales due to new - productivity/engagement/ (global and per participant)
- training products or techniques motivation levels
(1 year, 3 year, 5 year - ideas selected for
comparatives)/overall and - number of people participating implementation and speed of
- improvements
in infrastructures per product - types of behaviour and implementation
- sales percentages for changes in these - number and quality of projects
- technology
that encourages new products and services - improvements in developed
innovation developed organisational processes - return on investment of new
- others - new customers implemented ideas
external:
- product enhancements and - number of delivered patents,
new products launched - new dynamics and ecosystems granted or rejected/intellectual
including stakeholders property
- ratios of income (or profit)
from new ideas divided - improved relationships with - new investment opportunities
by the average cost of customers/consumers (image/
implementing each idea brand, increased adhesion) - acquisition of skills by
employees
- others others - expansion of innovative
products portfolio

Which of these should we choose? Do they fit the needs of our specific innovation
programmes? These questions become even trickier if we know that the output of one
phase becomes another phase’s input. Cause and effect overlap.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 14 | 17

Reviewing the literature on innovation management, Henry Edison and his colleagues,
in their paper ‘Towards Innovation Measurement in the Software Industry’ (2013),
reported 232 metrics in use by business managers and leadership, across industries.

3. AVOID THE ANALYSIS-PARALYSIS – A BUILDING BLOCKS

MODEL
You may easily find yourself overwhelmed. But, you cannot get mired in endless
reports, comparative numbers and hundreds of percentages that may have nothing
relevant to show you.
Based on experiences with our clients from across several countries and industries,
we, at Exago, have come to identify the chief indicators you should contemplate at
each phase of the process: from the investment phase through to performance and
development and, finally, output and evaluation.
These indicators consider three dimensions we believe are fundamental: human
capital and organisational culture, the ‘people factor’ as the cornerstone of any
innovation programme; financial and accounting plans, a stronger focus on growth
and sustainability; resources acquisition and development, namely, tangible and
intangible aspects, such as assets and knowledge.

EXAGO’S FRAMEWORK FOR INNOVATION MEASUREMENT

(dimensions per stage - main indicators)

INVESTMENT PROCESS RESULTS

inputs performance & development output & evaluation

human capital & - people’s availability - number of participants - engagement and

organisational (internal and external) motivation levels
- leadership and
culture management - number of ideas and insights - new competencies and
involvement per participant mindset (innovation
capability)
- training - number of participants
involved in idea development - strategy alignment
and implementation
- consumer/client
- percentage of employees satisfaction
trained in the innovation
process - stakeholder relationships

financial & - budget allocated - specific amounts spent in - return on investment

each phase of the process: for innovation projects
accounting
from ideation through to (products and service
capital improvement)
implementation
- cost reduction/increased
efficiency and savings
- net present value of new
product portfolio

resources (assets - time spent - number of ideas generated - number of projects

& knowledge) implemented (products,
- talent - quality of ideas and processes and services)
contributions
- technologies and - value of the innovation
structures adopted - number of ideas selected for portfolio
implementation and actually
being implemented - number of patents filed
- average time of - brand value
implementation

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 15 | 17

Keep in mind this framework is elastic. It’s a chart to identify your own metrics set,
according to your specific challenges.
These three steps will help you decide which to adopt:
a) The most important step – define how innovation will serve the strategy and
purpose of your organisation
b) Establish the nature of the results and stabilise a set of priority metrics (and
practices) of input, process and output
c) Make the process evolve by including new metrics and excluding aspects difficult to
measure or those that do not suggest process improvements

Most of all, remember you don’t have to start from scratch and you can’t freeze up in
any phase. Innovation is a process of trial and error, by its very nature. So, build a solid
initial plan, including top metrics to track, and be ready to learn and adapt as you
progress.

MEASURING INNOVATION - A FRAMEWORK FOR GETTING IT DONE 16 | 17

CONCLUSION
More than 70% of corporate leaders consider innovation a top-three business priority,
according to ‘Innovation and Commercialisation, 2010: McKinsey Global Survey
Results’. However, only 22% set innovation performance metrics. How can we explain
this incongruence?
To start with, if no magic formula exists for innovation, how could one exist for
measuring it? There is no single universal indicator either. And, what works in one
company may get confusing or even be counterproductive in another.
Yet, innovation is the way we do business today. From R&D to disruptive and
incremental methods, companies have to find a way to measure the innovation
process at hand and analyse its relevance.
At the end of the day, it comes down to accountability to leadership and shareholders,
as well as your own teams. It comes down to your organisation’s survival.
Innovation without results becomes useless. Therefore, as you develop your innovation
efforts, remember to adopt a set of priority metrics that allows you to monitor the

IV
process from its beginning and track major results. You need to understand what is
worth following, what could be improved and what simply doesn’t work.
Finally, do not overlook vital outcomes such as higher levels of participation and
engagement – your people’s collective intelligence is your most valuable asset. Take a
look at our basic steps and innovation measurement framework when deciding which
metrics to choose. You can learn, as you progress, how to readapt your strategy and,
most importantly, how to get innovation done.

measuring innovation - a framework for getting it done 17 | 17

exago.com
 PORTUGAL
Praça do Príncipe Real, 22, 2.º
1250-154 Lisboa
+351 210 997 443

UNITED KINGDOM
26 Cadogan Square
London SW1X 0JP
+44 (0) 203 714 07 23

BRASIL
Av. Liberdade, 602 - 2º andar
01502-001 - São Paulo - SP
+55 (11) 9555 800 30