White paper
QoE-driven network & business optimization
QoE-Driven Network & Business Optimization
CONTENTS
03
IN TROD U C TI ON
04
L E VERAGI N G QoE I N D I C ATO R S
04
QoE I M PA C TI N G FA C T O R S
04
Global Satisfaction Index (GSI) per service or per application
05
Differentiated QoE per customer segment, per subscriber, per service
07
Correlation TCPDR and radio measurements
08
Moving from NPS-only to predictive satisfaction indicators
10
QoE, everyones business
11
MEETIN G THE N EE D F O R M EAN I N G F U L M ETRI C S
12
Voice
13
V ideo
13
Inter net browsing
13
Gaming
14
CON C L U S I ON
15
GL OS S ARY
It works ! I have network
coverage at last !
How are you ?
Introduction
Communication Service Providers (CSPs) are faced with
rising operating costs, flattening ARPU and the need to
address an increasing demand for data. On top of that,
according to research1, 50% of subscribers are at risk
of churning in the next 12 months. The survey shows
that the quality of mobile broadband experience is the
leading driver for mobile operator churn, with 37% of
consumers citing slow connection speeds as the main
reason for churning. CSPs recognize these figures and
know that churn corrodes their business.
Delivering superior customer experience has therefore
become a central concern and a constant battle for
CSPs. The implementation of a Service Operation
Center (SOC) puts QoE at the center of the organization
and breaks the silo-based organization. A SOC provides
the means to monitor services in an end-to-end context
and to leverage actionable data in order to deliver the
best possible QoE.
This paper explores the different factors that impact
QoE, proposes different methods for measuring QoE
and leveraging this information intelligently to different
teams. We shall introduce meaningful QoS metrics to
monitor, through customer case studies, we shall see
how QoE-oriented network optimization contributes
to an intensified consumption of mobile data services,
enhanced customer loyalty and increased revenues.
Quality of Experience (QoE) measures the quality
perception from an end users point of view of the
delivered services and is therefore a good indicator of
a subscribers satisfaction level. So, in order to prevent
churn and differentiate themselves from competitors,
understanding and assuring the quality of subscribers
experience is paramount.
O v um 2014 Te le c oms Cust ome r Ins i ghts
QoE-Driven Network & Business Optimization
QoE IMPACTING FACTORS
QoE is all about how satisfied a subscriber is with a service in terms of accessibility, responsiveness, quality and
retainability. Subscriber QoE is based on factors such as:
The ability to make a phone call when on the move
The time required to download a webpage
The responsiveness of a mobile application
The amount of stalling in the video being viewed
The performance of a particular mobile device
A multitude of elements have an impact on QoE: device and OS type, latency, congestion levels, application type,
network topology, location, time etc. In order to decide how to optimize their networks CSPs require a holistic view
of the subscriber, network, device and application usage.
Leveraging QoE indicators
Global Satisfaction Index (GSI) per service or per application
Based on its thorough experience with more than 200 CSPs world-wide, Astellia has developed a real-time global
satisfaction index which gives detailed information on QoE per service for a group of subscribers or an individual.
This GSI takes into account QoE of voice, messaging and selected data services, as well as access & mobility.
Alarms can be triggered when a metric is out of range and hence impacting the subscriber experience.
#1
Case study : QoE of Top10 apps
At a Middle East CSP, Astellia monitors the QoE of the top 10 most used
applications and evaluates its evolution over time.
This allows the CSP to create a partnership with the right application provider
and to optimize the QoE of these most used applications.
Through alarms the CSP is informed of any quality degradation and is then able
to locate the issue (RAN / Core / content server).
If the quality regression comes from the application provider, the CSP can send
them a trouble ticket.
#1
#1
1
2 3
Case study : Customer-centric network performance indicators
A European CSP wanted to use a customer-centric indicator to measure network performance and customer
experience while benchmarking service perception across the different regions in the country.
Astellia and the CSP created together a GSI based on the correlation of 3 main KPIs:
Call set-up efficiency
Call drop
Throughput data for data sessions to identify the worst performing areas
Optimization campaigns were prioritized on these areas, followed-up by regional marketing campaigns to
reduce churn.
The vendor independent indicator was also used to assess operations efficiency and allowed to follow-up
SLAs with Managed Services Providers.
This GSI was used afterwards to assess customer experience of different MVNOs and inbound roamers.
Differentiated QoE per customer segment, per subscriber, per service
Data explosion and the increased number of applications create usage diversity that should be better leveraged to
service the end user and is creating the need for adapted customer experiences.
It is of no use to provide high throughputs to subscribers using only Twitter, while a subscriber watching a video on
YouTube requires a seamless data service. The issue is all the more important if it concerns high value subscribers
rather than occasional service users. CSPs therefore need to monitor a differentiated QoE based on subscribers
main centers of interest. They have to define and weigh application and services KPIs differently for each customer
segment and adapt network optimization campaigns accordingly.
Case study : Differentiated QoE per customer segment
12%
39 USD
Subscribers belonging to the Media Lovers
customer segment at a Middle East CSP are
consuming on average 8,8GB per month by
watching videos and streaming music.
Video delays and frozen video are having
a higher impact on these subscribers than
on those surfing only social media sites like
Facebook and WhatsApp.
Astellia therefore puts more weight on selected
KPIs like throughput, packet retransmission,
jitter, buffering delay to evaluate the customer
experience of this segment.
Astellia provides per segment a list of subscribers
with bad QoE and hence with a potential to
churn. By taking into account the ARPU of the
impacted subscribers, optimization activities are
being prioritized on high revenue generating
customers prone to churn.
8,8 GB
M edia lovers
Other
14%
OS
Apple
27%
Android
57%
QoE-Driven Network & Business Optimization
Case study : Optimizing CX on high speed trains
A high speed train is a very complex
environment due to the high speed
at which customers are travelling.
So special challenges require
special solutions.
One of Juan Serrano Sanchez,
Quality Manager at Orange Spain,
objectives is preventing any quality
degradation in the customer
experience during traveling on a
high speed train. For this, central
and regional teams are focused
on monitoring KPIs, analyzing
root causes, detecting gaps and
proposing improvement plans.
Before, Orange Spain mainly used
drive tests to optimize network
conditions along the railway.
Now, with Astellia, they adopt a
new and very innovative approach
which manages to filter out those
people that are actually on the train.
The optimization of high speed
train routes is particularly important
for business travelers who need to
stay connected at any time. This
allowed Orange Spain to become
the best service provider on high
speed trains in Spain and created
a real competitive differentiator.
Correlation TCPDR and radio measurements
When a subscriber experiences bad throughput, the cause can be manifold: device, content server, radio, core, etc.
Correlating application usage and radio measurements allows root cause problem elimination: having good radio
measurements doesnt necessarily mean providing good QoE. However, if radio measurements are bad (coverage,
congestion, etc) it will result in poor QoE.
Methodology: Correlating probe-based data and call traces
Different data services and applications require adapted levels of throughput and latency. So with exactly the
same radio conditions, customer experience might be completely different. Therefore, it is very important to
be able to correlate user plane information (including application usage info) provided by probes with radio
measurements coming from BSC, RNC and eNodeB call traces.
PROBE
+
CALL
TRACES
Because of bad radio conditions,
these customers stopped their
web browsing session, leading to
immediate loss of revenue for the
operator.
The poor radio conditions didnt have
an immediate impact on youtube
usage since part of the videos had
been buffered. After that, customers
stopped using the service.
QoE-Driven Network & Business Optimization
Moving from NPS-only to predictive satisfaction indicators
Marketing departments often use
customer surveys including the Net
Promoter Score (NPS), a customer
loyalty metric, to measure customer
satisfaction based on a simple
question Would you recommend
our company to a friend?
Subscribers are then categorized as
promoters, passives or detractors.
bad QoE, better competitive offer,
etc.). Therefore it is impossible for
operators to troubleshoot and solve
issues and therefore retain these
customers.
Astellia has developed Satix, a
powerful customer-centric indicator,
that correlates radio network
metrics and end-user perception
information
collected
through
customer surveys.
The Net Promoter Score translates
customer satisfaction at a given
point in time for a sample of
Customer experience feedback
customers but it doesnt allow
such as blocked calls, dropped
operators to identify precisely
calls, coverage issues, voice and
why the customer is not likely to
data quality are correlated with
recommend their brand nor does
order
technical radio measurements such
it allow them to Inidentify
the
reason
to ma
a gene
would(billke shock,
of their dissatisfaction
r
a
app
l stuas RSCP, Ec/N0, throughput, CQI,
reciate
BLER, time spent on 4G, 3G vs
time spent on 2G.
The result is an advanced customer
satisfaction index that not only
provides
objective
customer
perception indicators but also
precise
location
information
where radio optimization has
to be reinforced. In case of a
bad customer experience radio
optimization teams will know exactly
which parameters to fine-tune to
improve QoE. This customer-centric
optimization allows prioritizing of
network investments based on high
value customers perception.
Surve
y
dy of th
could e mobile ph
answe
o
r the fo ne market in
llowing
S
H
o
questi pain, we
w likely
COVE
ons :
would
R.IN
you re
comm
How li
URBA
end O
k
e
ly
N COV
PERA
would
TOR to
ER
you ra
te OP
a frien
And th
RURA
E
d, a re
R
e
ATOR
covera
L COV
lative
s cove
ge in u
ER
or a coCustomer
r
a
r
g
b
e
a
lleagu
n area
And th
indoor
BLOC
e?
s
e
?
?
c
o
KING
verage
in rura
l areas
In are
as
?
the ca with a goo
d
p
a
c
c
o
it
v
y to es
e
DROP
tablish rage, how w
S
o
a call
on the uld you rate
A
nd the
first att
SPEE
y
capac
CH
empt ? our level of
ity to k
satisfa
QUAL
e
e
p a ca
ITY
ction r
And th
ll witho
egard
e quali
ing
ut losin
ty of s
g
o
conne
OVER
und du
ction ?
ALL
ring th
e phon
NETW
ORK
e calls
Regar
ding a
?
ll the p
netwo
reviou
rk ?
MOBIL
s
it
e
ms, w
E BRO
hat is
SING
Wyour o
SPEE
Only if
verall
D
youre
satisfa
satisfa
using
ction w
your m
ction r
ith OP
egard
o
b
il
ERATO
e
ing the
phone
Rs
to bro
surfing
wse th
speed
e net,
?
what is
your le
vel of
NPS
if you
E X A M P L E
survey
Case study : Correlating network performance and NPS
A European Tier 1 operator that relentlessly pursues
network performance excellence to provide the
best customer experience called upon Astellia for
customer-centric optimization.
Satix radio network metrics at this Tier 1 operator
are based on traffic
of 12 million subscribers
covered by 40 000 2G/3G/4G cells
CX Survey
Correlation
These metrics are then compared with user
perception information based on weekly automatic
customer surveys amongst a panel of 3000
subscribers. The correlation between Satix and
Survey scores enabled the operator to discern
whether a customer was dissatisfied due to poor
network quality (low Satix score) or due to nonnetwork related reasons i.e. commercial reasons
(high Satix score).
SatiX CX index
Network performance
indicators
QoE scored by subscribers
Technical metrics
Coverage
Blocked calls
Dropped calls
Voice quality
Data quality
RSCP
Throughput
CQI / BLER
Ec/No, ...
Calibration
CX index & triggering
optimization actions
To improve QoE
& network performance
The survey score identifies detractors, i.e. customers on the verge of churning. The Satix score however
shows that these customers have not experienced any network quality degradation.
The survey scores detractors. The Satix score is very low and indicates that the customer has experienced
several network quality issues.
IMSI
IMSI 1
IMSI 237
IMSI 3004
IMSI 412
IMSI 5492
IMSI 6057
IMSI 7054
IMSI 486
IMSI 9
IMSI 8956
SATIX ATTRIBUTES
Attribute Voice
Blocks
4,98
4,97
4,95
4,84
5,00
4,99
4,99
5,00
4,98
4,96
Attribute Voice Attribute Speech
Quality
Drops
5,00
2,58
5,00
2,21
4,8
2,38
4,83
2,29
5,00
1,98
4,96
3,42
4,96
3,51
5,00
3,65
4,86
3,50
5,00
2,82
Attribute Data
Quality
1,33
1,75
1,23
1,26
1,04
3,64
4,07
3,39
3,97
3,61
Attribute
Coverage
1,63
1,74
1,79
1,94
1,35
3,03
3,16
2,27
2,95
2,66
SATIX score
3
3
2
3
1
9
10
7
9
7
Customer
survey
score
0
0
0
0
0
0
0
2
0
2
Promoter
Neutral
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Detractor
Reason for
disatisfaction
Network QoS related
Network QoS related
Network QoS related
Network QoS related
Network QoS related
Non-technical reason
Non-technical reason
Non-technical reason
Non-technical reason
Non-technical reason
Correlation between Satix and NPS Survey
This information was then used by marketing teams to launch win-back campaigns and by network
operations teams to better target network optimization campaigns based on customer location and value.
QoE-Driven Network & Business Optimization
QoE, everyones business
Understanding, managing and ensuring a good QoE is everyones concern within a CSP.
Network performance and radio engineers can perform proactive QoE monitoring by
diagnosing QoE & service degradation and drilling down to the root cause of the problem.
For optimal ROI, they can prioritize their investigations on high value customers, corporate
subscribers and key areas.
Marketers can improve campaign effectiveness by including QoE as a dimension for
understanding their target audience. It will also allow them to improve their ability to predict
which subscribers are likely to churn. Moreover, they will need to adapt the timing of their upsell and cross-sell campaigns to send the right offer at the right time.
Device managers need to understand device behavior to select the best performing ones
for their new data plans and inform device manufacturers or network equipment vendors in
case of a device issue.
Customer Care needs information to ascertain in real-time the level of satisfaction for every
subscriber and for each type of service (voice, messaging, data). They need to know if the
customer is the only one impacted by the issue or if it is the issue that impacts all subscribers
in that area.
Service Operation Centers (SOC) manages E2E service quality and is at the heart of a CSPs
customer focused strategy. A SOC breaks down organizational barriers and has the role of
an intermediary between the operations, engineering, performance, marketing and customer
care teams.
10
Meeting the need for meaningful
metrics
There are numerous elements which can degrade QoS and might affect QoE. As mentioned before, it is often a
combination of network availability, capacity, throughput, location, time-of-day, usage profile, and device. To deliver
consistent quality, there must be consistency in the metrics used to determine that quality. CSPs therefore need
QoS (KPIs) & QoE (KQIs) metrics that take into consideration all of the impairments that affect the quality of each
service.
SRVCC Success SRVCC
Speech
CSFB Success
CSFB Setup Time
RTP Packet Loss
RTP Jitter
Voice
Call Setup Time
Call Setup Delay
Call Setup Success
Call Drop
Call Block
Connection Gap
Video
Jitter
Latency
Throughput
Stalling Rate
Stalling Duration
Packet loss
Transaction Delay
Frame Rate
Buffering Delay
Session Drop
Session Start Time
Cut-off Rate
Internet
browsing
Http Efficiency Uplink
Efficiency Downlink
Efficiency Response
Time Packet Loss
Throughput Transfer
Transfer Duration
Bitrate duration
Page Refresh Delay
Page Download Delay
Packet Loss
E-mail
Session Setup Success
Send/Receive Delay
Abort Rate
Gaming
Jitter
Latency
Ping
Packet Loss
Delay
11
RTP Delay
Bearer Establishment
Success
MOS
Establishment Success
Establishment time
Used codecs
QoE-Driven Network & Business Optimization
Voice
Voice still remains a vital service CSPs run over their networks. The destabilizing impact of OTT
providers is putting pressure on CSPs to deliver quality voice services. Room for error round
voice services is therefore slight to non-existent.
Delivering a quality voice service in an LTE environment is a major goal. For VoLTE to succeed,
end-to-end QoS is essential. CSPs will need to provide seamless fallback (SRVCC) to 2G/3G
networks where LTE is unavailable.
Case study : Launching VoLTE service
Astellia helped a European operator launch
its VoLTE service focusing optimization
activities on 3 areas:
Fast Voice call
establishment
Voice call
continuity with 3G
& 2G
Astellia analyzed the SRVCC efficiency and delay per
handset and indicated the best performing devices for
VoLTE
Voice & data
quality
Astellia could show that voice call setup
with VoLTE was clearly faster than with
CSFB: 200-300ms vs 8 sec
Astellia provides the MOS score of each VoLTE call.
Our solution analyses a call every 5s, from start to end,
to depict the real quality of experience of the voice
service. This provides a much more reliable vision of
customer experience than competitors who calculate
the average MOS.
12
Video
Ciscos report estimates that video will account for 70% of traffic by 2018, driven by VOD and
P2P streaming and even mobile games containing video. So, there is no application that is more
important to the consumer experience than mobile video, especially for video addicts. Ensuring a
consistently great video experience by preventing video blackouts, frozen video, silence, buffering
delays etc. will have an immediate impact on customer satisfaction and loyalty.
Methodology: Analyzing video quality
When measuring, for instance, the user experience of on-demand Internet streaming service Netflix, it is
important to make the distinction between
> Browsing Netflix navigation menu
> Watching a movie
The KPI server response time reflects the user experience for menu browsing but throughput is a better
KPI for evaluating the movie watching experience.
Browsing
Watching
KPI
Server response time
Throughput
Internet browsing
Subscriber waiting time is the key determinant of web browsing QoE: the longer users have to
wait for the web page to arrive, the more dissatisfied they tend to become with the service.
Gaming
QoE of so called First Person Shooter games poses strict requirements with respect to network
quality. Having a low ping is desirable because lower latency provides smoother gameplay by
allowing faster updates of game data.
13
QoE-Driven Network & Business Optimization
satisfy
analyze
improve
Conclusion
Operating mobile networks today is expensive for
CSPs, competition from OTTs is fierce and revenues are
flattening. Subscribers increasing reliance on mobile
networks and the exponentially rising demand for mobile
data services are putting pressure on the network.
for everyone. Measuring QoE is difficult and CSPs rely on
different QoE indicators to do so.
Focus on QoE is worth the effort and holds a great
opportunity to improve loyalty, reduce churn, raise
ARPU and margins. In other words, QoE is the key to
drive customer lifetime value and emerges as the new
battleground for CSPs.
However, CSPs cannot afford to continue simply adding
more bandwidth, especially in cases where bandwidth
alone isnt the cause of the customers poor experience.
Optimizing QoE has therefore become a prime concern
14
Glossary
ARPU
Average Revenue Per User
BLER
Block Error Rate
BSC
Base Station Controller
CQI
Channel Quality Indicator
CSFB
Circuit Switched Fall Back
CSP
Communication Service Provider
GSI
Global Satisfaction Index
KPI
Key Performance Indicator
KQI
Key Quality Indicator
MOS
Mean Opinion Score
NPS
Net Promoter Score
P2P
Peer to Peer
QoE
Quality of Experience
RAN
Radio Access Network
RNC
Radio Network Controller
ROI
Return on Investment
RSCP
Received Signal Code Power
RTP
Real-time Transport Protocol
SOC
Service Operation Center
SRVCC
Single Radio Voice Call Continuity
TCPDR
Transmission Control Protocol Data Record
VOD
Video on Demand
VoLTE
Voice over LTE
15
ZA Airlande | 2 rue Jacqueline Auriol
CS 69 123 | 35091 Rennes Cedex 9 | France
Tel : +33 299 048 060 | Fax : +33 299 048 061
infos@astellia.com | www.astellia.com
BRAZIL | CANADA | INDIA | LEBANON | RUSSIA | SOUTH AFRICA
SPAIN | USA
