SMR703

t
os
RESEARCH
REPORT

rP
yo
FINDINGS FROM THE 2018 STRATEGIC MEASUREMENT GLOBAL
EXECUTIVE STUDY AND RESEARCH PROJECT

Improving
op
Strategic
Execution with
tC

Machine
Learning
No

By Michael Schrage and David Kiron

Do

Sponsored by:

SUMMER 2018 #MITSMRreport

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
REPRINT NUMBERor60181
Permissions@hbsp.harvard.edu 617.783.7860
R E S E A R C H R E P O R T I M P R O V I N G S T R AT E G I C E X E C U T I O N W I T H M A C H I N E L E A R N I N G

t
os
AUTHORS

rP
MICHAEL SCHRAGE is a research fellow at the MIT DAVID KIRON is the executive editor of MIT Sloan
Sloan School’s Initiative on the Digital Economy, Management Review, which brings ideas from the
where he does research and advisory work on how world of thinkers to the executives and managers
digital media transforms agency, human capital, who use them.
and innovation.

ACKNOWLEDGMENTS

yo
Hannah Grove, chief marketing officer, State Street
Andrew Low Ah Kee, chief revenue officer, GoDaddy
Amit Shah, chief marketing officer, 1-800-Flowers.com
op
Kelly Watkins, vice president, global marketing, Slack

CONTRIBUTORS
tC

Carrie Crimins, Masha Fisch, Jennifer Martin, Allison Ryder, Deborah Soule, and Barbara Spindel
No

The research and analysis for this report was conducted under the direction of the authors as part of an MIT
Sloan Management Review research initiative, sponsored by Google, in collaboration with Think with Google.

To cite this report, please use:

M. Schrage and D. Kiron, “Improving Strategic Execution With Machine Learning,” MIT Sloan Management
Review, August 2018.
Do

Copyright © MIT, 2018. All rights reserved.

Get more on strategic measurement from MIT Sloan Management Review:

Read the report online at http://sloanreview.mit.edu/kpi2018-ml

Visit our site at http://sloanreview.mit.edu/strategic-measurement

Contact us to get permission to distribute or copy this report at smr-help@mit.edu or 877-727-7170

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
Permissions@hbsp.harvard.edu or 617.783.7860
 t
os
Improving Strategic
Execution with

rP
Machine Learning

M yo
achine learning (ML) is changing how leaders use metrics to drive
business performance, customer experience, and growth. A small but
growing group of companies is investing in ML to augment strategic
decision-making with key performance indicators (KPIs). Our research,1
based on a global survey and more than a dozen interviews with
op
executives and academics, suggests that ML is literally, and figuratively,
redefining how businesses create and measure value.

KPIs traditionally have had a retrospective, reporting bias, but by surfacing hidden variables that
anticipate “key performance,” machine learning is making KPIs more predictive and prescriptive.
With more forward-looking KPIs, progressive leaders can treat strategic measures as high-octane
tC

data fuel for training machine-learning algorithms to optimize business processes. Our survey
and interviews suggest that this flip — transforming KPIs from analytic outputs to data inputs —
is at an early, albeit promising, stage.

Those companies that are already taking action on machine learning — investing in ML and ac-
tively using it to engage customers — differ radically from companies that are not yet investing in
ML. They are far more likely to:
No

• Develop a single, integrated view of their target customer.

• Have the ability to drill down to see underlying KPI data.
• Check their KPI reports frequently.

These differences all depend on treating data as a valuable corporate asset. We see a strong correlation
between companies that embrace ML and data-driven decision-making.
Do

Augmenting Execution With Machine Learning

Nearly three quarters of survey respondents believe their organization’s current functional KPIs
would be better achieved with greater investment in automation and machine-learning technolo-
gies. Our interviews with senior executives identified a variety of innovative ML practices. Without
exception, the companies with the most intriguing and ambitious ML initiatives were the ones

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
IMPROVING STRATEGIC EXECUTION
Permissions@hbsp.harvard.edu WITH MACHINE LEARNING • MIT SLOAN MANAGEMENT REVIEW 2
or 617.783.7860
R E S E A R C H R E P O R T I M P R O V I N G S T R AT E G I C E X E C U T I O N W I T H M A C H I N E L E A R N I N G

t
os
with the most serious commitment — cultural and
organizational — to managing data as a valuable
corporate asset.
ABOUT THE RESEARCH
The marketing function is often an early adopter of

rP
This report explores some of the key findings from the authors’ machine learning in the enterprise. Applications
2018 research study of KPIs and machine learning in today’s in advertising, customer segmentation, and cus-
corporate landscape. The research, which involved a survey of tomer intelligence have become common.2 Even
4,700 executives and managers and interviews with more than among marketers, however, slightly less than half
a dozen corporate leaders and academics, has far-reaching of surveyed companies have incentives or internal
implications for modern businesses. We focused our analysis on functional KPIs to use more automation and ML
3,225 executive-level respondents; more than half were technologies. (See Figure 1, page 4.) It is highly

yo
marketing executives. unlikely this finding reflects ML saturation in the
enterprise. Most of the executives we interviewed
The study strongly suggests that data-driven organizations that for our study are focused more on ML’s potential
align incentives, KPIs, and ML capabilities have distinct than its actual development or deployment.
advantages over those that move too slowly to develop their
data capabilities. For business leaders serious about succeeding Kelly Watkins, vice president of global marketing
op
in digital market environments, these shifts offer a clear and at Slack, is exploring machine-learning solutions.
urgent call to action. For Slack, an essential KPI is determining which
businesses using the company’s free workplace col-
laboration app are good candidates for converting to
paid subscriptions for premium features. “This is an
effort that the marketing organization, product orga-
tC

nization, and the sales organization are working on

together,” Watkins says. “Can we train lead-scoring
algorithms to really get a sense of, based on a variety
of criteria, what’s the best place for sales reps to start
among the options that they have for outreach?”
Nearly three quarters of
Watkins also envisions implementing machine learn-
survey respondents believe
No

ing to handle routine tasks currently performed by

their organization’s current Slack employees. She says her intention is to “enable
folks in my organization to use their minds to solve
functional KPIs would be strategic problems and to be more consistently look-
ing for insights in the data that can shift the strategy
better achieved with greater and shift execution, up-leveling their daily mode
investment in automation of operating.” In short, Watkins sees an ML future
transforming both efficiency and strategy.
and machine-learning
Do

technologies. Having incentives to use machine learning and au-

tomate processes is a strong signal that a company is
either readying itself or already prepared to compete
in digital-first environments.

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
3 MIT SLOAN MANAGEMENT REVIEW • IMPROVING STRATEGIC EXECUTION WITH MACHINE
Permissions@hbsp.harvard.edu LEARNING
or 617.783.7860
 t
os
The ML leaders in the marketing function distin-
guish themselves from the laggards in the following
three ways:
FIGURE 1: INCENTIVES TO USE
• They use their KPIs to develop a single, inte- MACHINE LEARNING

rP
grated view of their customers. Slightly less than half of surveyed companies have incentives or
• They can digitally drill down to see their KPI internal functional KPIs to use more automation and machine-
components. learning technologies to support marketing initiatives.
• They check their KPI reports with greater
frequency.
Do you have incentives or internal functional KPIs to use
more automation and machine-learning technologies to

yo
drive marketing activities?

Developing a Single, 100%

Integrated View of Customers 75%

50%
At a time when business leaders generally want a
25%
more data-driven, holistic view of their customers,
op
more than 80 percent of respondents with incentives 0%
Yes No
to use machine learning report that their functional
Percentages do not total 100 due to “Don’t know” responses.
KPIs help them develop such a view. Among organi-
zations without ML incentives, that number drops
below 50 percent. (See Figure 2.) This focus greatly
enhances an organization’s ability to segment and
tC

engage its customer/client base. In digital market-

places, enhanced awareness and anticipation of FIGURE 2: DEVELOPING A SINGLE,
customer needs can ultimately help turn customers INTEGRATED VIEW OF CUSTOMERS
into champions for the brand. Several executives More than 80 percent of respondents with incentives to use
told us they see ML as essential to that end. machine learning report that their functional KPIs help them develop
an integrated view of target customers.
Amit Shah, CMO of 1-800-Flowers.com, captures
No

the enhanced value of aligning smart algorithms

Do your functional KPIs help your function develop a single,
with greater corporate goals: “All of our AI efforts integrated view of your target customers?
are highlighting for us the central learning we have
Yes No Don’t know / not sure
had, that all of this is helping us learn about our
customers, learn about ourselves, and ultimately
learn about how we leverage technology. It has less 82%
6% 13%
to do with, ‘What are the workplace savings because 14%

we have bots?’”
Do

39%

For some leading companies, machine learning 47%

effectively expands the role of KPIs. Where KPIs Organization has incentives to use more Organization does not have incentives to use more
automation to drive marketing activities automation to drive marketing activities
were once analytic outputs to inform human
decision-making, they are fast becoming data Percentages do not total 100 due to rounding.
inputs to train machines to deliver better business
outcomes. The technical and business importance of
this inversion could be profound.

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
IMPROVING STRATEGIC EXECUTION
Permissions@hbsp.harvard.edu WITH MACHINE LEARNING • MIT SLOAN MANAGEMENT REVIEW 4
or 617.783.7860
R E S E A R C H R E P O R T I M P R O V I N G S T R AT E G I C E X E C U T I O N W I T H M A C H I N E L E A R N I N G

t
os
Drilling Down to See What’s more, companies can use high-frequency
Underlying KPI Data data and analytics to empower software and systems
to anticipate customer and business needs and move
Those with machine-learning incentives are more smarter decision-making upstream. Andrew Low
than twice as likely as counterparts that lack such Ah Kee, chief revenue officer at GoDaddy, says his

rP
incentives to agree or strongly agree that they can organization is “extremely bullish” on the possibility
easily drill down to see the underlying data or ana- of its systems’ learning from its huge stores of data.
lytic components aggregated into their KPIs. (See “We are very excited about using outcome measures
Figure 3.) The ability to see and interpret factors to help personalize and improve an individual’s on-
driving KPI outcomes creates greater transpar- line experience to help them find the right products
ency around operational effectiveness. This fosters for whatever they want to do,” he says. For early
greater shared understandings of corporate data, adopters, machine learning enables a shift in focus

yo
which, in turn, supports a data-driven, decision- to metrics that are forward-looking and predictive
making culture. Not surprisingly, companies with rather than retrospective and reactive.
ML incentives agree at an above-average rate with
the statement that they are better than their rivals at
making data-driven decisions.
Checking KPI Reports
Frequently
op
Survey respondents whose organizations have
incentives to use machine-learning technologies to
drive marketing activities are roughly twice as likely
FIGURE 3: ABILITY TO DRILL DOWN to check their KPI reports hourly or daily than those
tC

TO UNDERLYING KPI DATA whose organizations aren’t investing in ML. (See

Respondents with incentives to use machine learning for marketing Figure 4, page 6.) The frequent monitoring is an
purposes are more than twice as likely as those lacking incentives to indication that those companies’ KPIs are not merely
agree that they can easily drill down to see the underlying data or “key” in name only; they reflect the enterprise’s
analytic components aggregated into their KPIs. strategic priorities and enable daily or more frequent
assessments of organizational effectiveness.
No

I can easily drill down to see the underlying data or analytic components that are
aggregated into my KPIs. The ability to assess organizational performance
on a daily basis can be extremely useful for organi-
Agree Disagree
zations seeking to increase their responsiveness to
sudden market shifts. These may include abrupt
changes in customer behavior, disruptive arrivals of
3%
unexpected competition, or even macroeconomic
factors — from inflation to political crises to climate
15%
events. In fast-moving markets, KPIs that anticipate
Do

28% timely business risks can provide critical infor-

65%
mation for strategic decision-making in between
Organization has incentives to use more
planning review meetings. “I’m looking at market-
Organization does not have incentives to use more
automation to drive marketing activities automation to drive marketing activities ing automation almost daily to understand what
kinds of things are resonating,” notes Hannah Grove,
Percentages do not total 100 due to “Don’t know” responses and rounding.
CMO of financial services company State Street.

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
5 MIT SLOAN MANAGEMENT REVIEW • IMPROVING STRATEGIC EXECUTION WITH MACHINE
Permissions@hbsp.harvard.edu LEARNING
or 617.783.7860
 t
FIGURE 4: FREQUENCY OF

os
What’s Next CHECKING KPI REPORTS
Respondents with incentives to use machine learning for marketing
A small proportion of companies are currently activities are twice as likely to check their KPI reports hourly or daily.
investing in machine learning to improve strategic
execution across their business. While a significant

rP
How often do you check your KPI reports?
number of marketing executives say their function
Check at least weekly
is using machine learning for marketing activities, Organization has incentives to use more
8% 20% 33% 33%
automation to drive marketing activities
even within the marketing function, more executives 1% 5%
2% 1%
believe ML can help their function achieve its KPI Organization does not have incentives
to use more automation to drive
5% 12% 30% 33% 17%

outcomes. Fewer say that their function is investing marketing activities Check at least weekly

in ML in this way.

yo
Don’t know / Less than Quarterly Monthly Weekly Daily Hourly
not sure quarterly
Machine-learning capabilities are fast becoming a
baseline measure of a company’s ability to compete.
To improve these capabilities, executives should
consider three questions:

Are you making the right levels of investment in

op
1.
data and its governance and trustworthiness?
Data-driven decision-making and machine
learning all start with data. Questionable data
quality will undermine or corrupt mahine-
learning initiatives. Are you treating data as a
tC

2.
valuable corporate asset?
Does your organization have incentives to
“It’s going to be very difficult
use machine learning to address pressing to overcome one-on-one
business problems? Internal pilot programs
are a good place to start. But, scaling these ini- learning that is potentiated by
tiatives is key. If the pilots are successful, what
incentives will enable broader adoption in the
machine learning outcomes
and algorithms.”
No

enterprise? If the pilots fail, does your culture

penalize or support reasonable failure?
– Amit Shah, CMO, 1-800-Flowers.com
3. Are your KPIs more forward-looking or
backward-looking? What would your strategy
meetings look or sound like if your KPIs were
more forward-looking? How can you use ma-
chine learning to create more adaptive KPIs
that enhance your ability to predict and pre- have been slower to move have some catching up to
Do

scribe your future performance? do. Those that continue to fall behind may find the
playing field tilted evermore steeply against them.”3

The 2017 MIT Sloan Management Review/Boston We see parallels with regard to machine learning in
Consulting Group report “Reshaping Business With the current business landscape. Given the imbalance
Artificial Intelligence” concludes with a warning: between ambition and action, there is a clear need to
“Just about any company today needs a plan with improve access to ML. One option is to bake it into
respect to AI. Most do not have one, and those that interfaces and dashboards already in use.

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
IMPROVING STRATEGIC EXECUTION
Permissions@hbsp.harvard.edu WITH MACHINE LEARNING • MIT SLOAN MANAGEMENT REVIEW 6
or 617.783.7860
R E S E A R C H R E P O R T I M P R O V I N G S T R AT E G I C E X E C U T I O N W I T H M A C H I N E L E A R N I N G

t
os
We have observed that as more companies move to REFERENCES
adapt to changes demanded by digital disruption, 1. M. Schrage and D. Kiron, “Leading With Next-
others have followed in response. The early adopters Generation Key Performance Indicators,” MIT Sloan
of machine learning may well inspire their competi- Management Review, June 2018.

tors in a similar way. As Shah of 1-800-Flowers.com

rP
2. M. Yao, “14 Ways Machine Learning Can Boost Your
warns, “I think what we will find, five years down Marketing,” Forbes, April 10, 2018.
the road, is that the people who took the early bets
3. S. Ransbotham, D. Kiron, P. Gerbert, and M. Reeves,
in artificial intelligence actually achieve the learn-
“Reshaping Business With Artificial Intelligence,” MIT
ing that cannot be copied. I don’t think you can Sloan Management Review and The Boston Consulting
short-circuit your way the way you can do with Group, September 2017.
other channels. It’s going to be very difficult to over-
come one-on-one learning that is potentiated by

yo
machine-learning outcomes and algorithms.”

There are methods to using KPIs, individually and

collectively, to teach machine-learning systems to
improve and optimize their performance. Our anal-
ysis, both here and in our June 2018 report, strongly
op
suggests that early adopters explicitly use KPIs to
reinforce analytics best practices. Opportunities
to link these technologies to today’s robust data
capabilities are plentiful. To take advantage, many
organizations will need to transform their cultures
and operations to some degree. But companies that
tC

do not invest in machine learning now, and don’t

implement the array of changes that investment en-
tails, risk being left behind.

Reprint 60181.
Copyright © Massachusetts Institute of Technology, 2018.
No

All rights reserved.

Do

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
7 MIT SLOAN MANAGEMENT REVIEW • IMPROVING STRATEGIC EXECUTION WITH MACHINE
Permissions@hbsp.harvard.edu LEARNING
or 617.783.7860
 t
os
rP
PDFs ˜ Reprints ˜ Permission to Copy ˜ Back Issues
Articles published in MIT Sloan Management Review are
copyrighted by the Massachusetts Institute of Technology
unless otherwise specified at the end of an article.
MIT Sloan Management Review articles, permissions,
and back issues can be purchased on our website:
sloanreview.mit.edu or you may order through our

yo
Business Service Center (9 a.m. - 5 p.m. ET) at the phone
numbers listed below. Paper reprints are available in
quantities of 250 or more.
To reproduce or transmit one or more MIT Sloan
Management Review articles by electronic or
mechanical means (including photocopying or archiving
in any information storage or retrieval system) requires
written permission.
op
To request permission, use our website:
sloanreview.mit.edu
or
Email: smr-help@mit.edu
Call (US and International): 617-253-7170
Fax: 617-258-9739
tC

Posting of full-text MIT SMR articles on publicly

accessible Internet sites is prohibited. To obtain
permission to post articles on secure and/or password-
protected intranet sites, email your request to smr-
help@mit.edu.
No
Do

This document is authorized for educator review use only by SUNITA KUMAR, Christ University until Aug 2025. Copying or posting is an infringement of copyright.
Permissions@hbsp.harvard.edu or 617.783.7860