Computers in Human Behavior 61 (2016) 478e487

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh

Full length article

The Social Media Disorder Scale: Validity and psychometric properties

Regina J.J.M. van den Eijnden a, *, Jeroen S. Lemmens b, Patti M. Valkenburg c
a
Interdisciplinary Social Science, University of Utrecht, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
b
Amsterdam School of Communication Research, University of Amsterdam, Nieuwe Achtergracht 166, 1001 NG, Amsterdam, The Netherlands
c
Amsterdam School of Communication Research, University of Amsterdam, Spui 21, 1012 WX Amsterdam, The Netherlands

a r t i c l e i n f o a b s t r a c t

Article history: There is growing evidence that social media addiction is an evolving problem, particularly among ad-
Received 9 December 2015 olescents. However, the absence of an instrument measuring social media addiction hinders further
Received in revised form development of the research eld. The present study, therefore, aimed to test the reliability and validity
10 March 2016
of a short and easy to administer Social Media Disorder (SMD) Scale that contains a clear diagnostic cut-
Accepted 11 March 2016
off point to distinguish between disordered (i.e. addicted) and high-engaging non-disordered social
media users.
Three online surveys were conducted among a total of 2198 Dutch adolescents aged 10 to 17. The 9-
Keywords:
Social Media Disorder
item scale showed solid structural validity, appropriate internal consistency, good convergent and cri-
Social media addiction terion validity, sufcient test-retest reliability, and satisfactory sensitivity and specicity. In sum, this
Problematic social media use study generated evidence that the short 9-item scale is a psychometrically sound and valid instruments
Pathological social media use to measure SMD.
Social media use 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND
Internet addiction license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction (APA, 2013), social media addiction still has no status in the DSM-5.
While the exclusion of social media addiction from the DSM-5
The research eld of Internet addiction continues to suffer from may give the impression that social media addiction is not a
denition and measurement problems. The concept Internet legitimate mental disorder, there is a growing body of evidence
addiction, also referred to as compulsive (Meerkerk, Van den suggesting otherwise (Pantic, 2014; Ryan, Chester, Reece, & Xenos,
Eijnden, Vermulst, & Garretsen, 2009) or problematic Internet use 2014). Moreover, there is empirical evidence indicating that
(Caplan, 2010), is multi-dimensional by nature and can refer to compulsive social media use is a growing mental health problem,
different forms of compulsive online behaviors. Individuals do not particularly among adolescent smartphone users (Van Rooij &
seem to be addicted to the Internet itself, but rather to certain Schoenmakers, 2013). However, the absence of a clear denition
online activities (Grifths & Szabo, 2013). Some of these activities, and a measure for social media addiction hampers research on the
however, appear to elicit more compulsive tendencies than others. prevalence of this type of disordered behavior, thereby obstructing
Among adolescents, the age group that rapidly adopts new tech- vital next steps in the research eld of social media addiction.
nologies and is expected to be most vulnerable to possible negative Therefore, the present study aims to develop and validate a new
inuences of these new technologies (Valkenburg & Peter, 2011), instrument for measuring social media addiction e that is, the
Internet addiction has most convincingly been linked to gaming Social Media Disorder (SMD) Scale.
and to social media use (Rumpf, Meyer, Kreutzer, John & Meerkerk, Currently, the research eld of social media addiction largely
2011; Van Rooij, Schoenmakers, Van den Eijnden, & Van de Mheen, lags behind research on game addiction. Whereas research on
2010). Although the latest version of the Diagnostic and Statistical game addiction has a long history dating back before online games
Manual of Mental Disorders (DSM-5) recognizes Internet gaming were available (e.g., Shotton, 1989; Soper & Miller, 1983), the social
disorder as a tentative disorder in the appendix of this manual media addiction eld is relatively young, with the rst studies
appearing after 2010 (for a review, see Ryan et al., 2014). Further-
more, while there are several validated instruments for measuring
* Corresponding author. Department of Interdisciplinary Social Science, Utrecht game addiction (e.g. Grifths, 2005, Lemmens, Valkenburg, & Peter,
University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands. 2009; Lemmens, Valkenburg, & Gentile, 2015; Van Rooij,
E-mail addresses: R.J.J.M.vandenEijnden@uu.nl (R.J.J.M. van den Eijnden), j.s.
Schoenmakers, Van den Eijnden, Vermulst, & Van de Mheen,
lemmens@uva.nl (J.S. Lemmens), p.m.valkenburg@uva.nl (P.M. Valkenburg).

http://dx.doi.org/10.1016/j.chb.2016.03.038
0747-5632/ 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 479

2010), no validated instruments exist measuring social media addiction measures (King, Haagsma, Delfabbro, Gradisar, &
addiction. Instead, the eld of social media addiction is character- Grifths, 2013).
ized by an abundance of measurement instruments tapping into Based on thorough consideration by a multidisciplinary expert
particular forms of compulsive social media use, such as Facebook group (see Petry et al., 2014), the APA decided to include three
addiction (Ryan et al., 2014), addiction to social network sites additional criteria when dening the criteria for the DSM-5 diag-
(Grifths, Kuss, & Demetrovics, 2014), Twitter addiction (Saaid, Al- nosis of IGD, namely deception (e.g., Demetrovics et al., 2012;
Rashid, & Abdullah, 2014), and microblogging dependence (Wang, Gentile et al., 2011), displacement (e.g., Huang, Wang, Qian,
Lee, & Hua, 2015). Zhong, & Tao, 2007; Rehbein, Kleimann, & Mo ssle, 2010), and
The fragmentation in the social media research eld, along with conict (e.g., Lemmens et al., 2009; Young, 1996). Moreover, several
the proliferation of measures targeting specic forms of social authors in the eld of IGD refer to relapse as persistence, to mood
media addiction, is problematic for two reasons. First, the social modication as escape, and to external consequences as problems
media landscape is characterized by rapid changes, whereby (Lemmens et al., 2015; Petry et al., 2014).
existing social media platforms are expanded with new interactive According to the DSM-5 denition, someone is diagnosed with
functions or simply replaced by new platforms. Instruments tar- having IGD if he or she meets ve (or more) of the nine criteria for
geting specic forms of social media addiction may thus be IGD during a period of 12 months. Since SMD and IGD are regarded
outdated easily. Second, existing measures tend to use slightly as two specic forms of the overarching construct Internet addic-
different criteria for eor operationalization ofe social media tion, it is reasoned that the nine criteria for IGD, which is the rst
addiction, thereby hampering the comparability of research data internet-related disorder included in the DSM, can also be used to
and stimulating further fragmentation of the eld. Hence, to dene SMD. The development of a SMD scale will thus be based on
accomplish the necessary progress in the eld of social media the DSM-5 diagnostic criteria for IGD and will include the same
addiction, it is vital to develop and validate a general measure of nine diagnostic criteria.
social media addiction based on a solid set of existing diagnostic As suggested before, the development and validation of a
criteria. theoretically grounded and well-dened instrument to measure
There is ample ground for the development of a general social SMD is essential in order to prevent the use of a large variety of
media addiction instrument, since social media platforms share slightly different measurement instruments that do not allow for
many characteristics such as facilitating social interaction, the clear-cut off points, and may not be applicable to multiple types of
sharing of ideas, formation and maintenance of relationships and/ social media. Moreover, there is a vital need for utilizing actual
or interest groups, and development of one's presence, reputation, clinical criteria in order to differentiate between pathological (i.e.
and identity (Kietzmann, Hermkens, McCarthy, & Silvestre, 2011). addictive) and highly engaged social media users (Kuss et al., 2013).
Moreover, the nding that excessive use of different person-based Thus, the development and validation of an instrument that is using
and group-based social media applications is related to Internet a clear diagnostic cut-off point, as provided by DSM-5, is crucial for
addiction (Kuss & Grifths, 2012; Kuss, Van Rooij, Shorter, Grifths, the development of this research eld because such an instrument
& Van de Mheen, 2013; Van den Eijnden, Meerkerk, Vermulst, offers the opportunity to assess and monitor the prevalence of
Spijkerman & Engels, 2007; Van Rooij et al., 2010) justies the social media addiction in the population. Since SMD can be ex-
development of a general social media addiction instrument. pected to be particularly disturbing for the psychosocial develop-
ment of adolescents (Valkenburg & Peter, 2011), the SMD scale will
be tuned to adolescents.
1.1. Development and validation of the Social Media Disorder
(SMD) scale
1.2. The current study
The basic theoretical assumption underlying the development
of the Social Media Disorder (SMD) scale in the current study is that The main aim of the present study was to develop and validate a
social media addiction and Internet Gaming Disorder (IGD; APA, scale to measure Social Media Disorder (SMD). Since SMD and IGD
2013) are two forms of the same overarching construct Internet are conceptualized as meeting at least ve of the nine DSM-5
addiction and should therefore be dened by the same set of criteria for IGD, this study builds on a previous study testing the
diagnostic criteria. As stated before, the Internet incorporates a reliability and validity of a short (9-item) and a long (27-item) scale
variety of potential activities, and some of these activities, such as to measure IGD (Lemmens et al., 2015). Lemmens et al. (2015)
gaming and social media use, tend to elicit compulsive tendencies showed that the short 9-item scale, with a dichotomous (yes/no)
in a subgroup of users. Therefore, the measurement of SMD should response scale, provided a valid and reliable measure of IGD with
correspond with the measurement of both Internet addiction and good diagnostic accuracy, even in comparison to the long 27-item
IGD. Thus, the same set of diagnostic criteria should be used when scale. Because of the important advantages of a short and easy to
operationalizing these related concepts. administer measurement instrument, such as the possibility to
In recent years, the addiction literature has extensively reected incorporate the scale into space-limited surveys, and in agreement
on the existence of non-substance related or behavioral addictions, with the ndings by Lemmens et al. (2015), the ultimate aim of the
such as Internet addiction. In the absence of DSM-criteria for present study was to develop and validate a short 9-item scale to
Internet addiction, most instruments were based on the DSM-IV measure SMD.
criteria for substance dependence and/or pathological gambling. Our starting point was the development of a 27-item SMD scale,
More specically, most instruments developed to assess Internet consisting of three items for each of the nine DSM-5 criteria (see
addiction included at least six of the DSM-IV criteria for patho- Appendix A). After testing the factor structure and factor loadings
logical gambling, namely preoccupation, tolerance, withdrawal, of this 27-item scale, the nine items with the highest factor loading
relapse, mood modication, and external consequences (see Van per criterion were selected to constitute the short 9-item scale.
Rooij & Prause, 2014). These six criteria were also recognized as Next, the psychometric properties of the short SMD scale were
the core elements of behavioral addictions (Brown, 1993; Grifths, tested, and compared with some psychometric properties of the
1999; Marks, 1990) and used for the development of most game 27-item SMD scale. More specically, we examined (1) the factor
480 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487

structure and internal consistency, (2) construct validity, as indi- Social Media Addiction, Self-esteem, Depression, Attention Decit,
cated by convergent validity, (3) criterion validity, (4) test-retest Impulsivity, and the use of several social media applications. The
reliability, and (5) sensitivity and specicity of the short SMD second online survey contained the short 9-item SMD scale and
scale. These psychometric properties were tested across three also measured Depression, Attention Decit, Impulsivity, and the
different samples of Dutch adolescents to establish population additional variable Loneliness, in order to further test construct
cross-validation. Finally, we established the prevalence of SMD in validity. The third survey contained the 27-item SMD scale and a
the current three samples, and tested for group differences in wider range of items regarding smartphone usage than the rst
gender, age, and frequency of daily social media use between survey (e.g., WhatsApp).
disordered and non-disordered adolescents.
Construct validity is dened as the extent to which the scale 2.2.1. Social Media Disorder
measured the intended construct. Construct validity of the SMD The SMD scale consisted of 27 items (see Appendix A). Three
scale was established by testing the strength of the relationships items were created for each of the previously identied nine
between scores on the SMD scale and constructs to which it should criteria: Preoccupation, Tolerance, Withdrawal, Displacement, Escape,
theoretically be related. An important aspect of construct validity is Problems, Deception, Displacement, and Conict.
convergent validity, referring to the relation between the SMD scale
and comparable constructs. In this study convergent validity was 2.2.2. Compulsive Internet Use
tested by relating scores on the SMD scale to Compulsive Internet Compulsive Internet use was assessed in the rst sample using
Use, as measured by the Compulsive Internet Use Scale (CIUS; the 14-item Compulsive Internet Use Scale (Meerkerk et al., 2009).
Meerkerk et al., 2009), and to self-declared social media addiction. Example items are: How often do you feel restless, frustrated, or
In case of good convergent validity, we expect to nd strong cor- irritated when you cannot use the Internet? and How often do you
relations between scores on the SMD scale and scores on the CIUS nd it difcult to stop using the Internet when you are online?
and on self-declared social media addiction. Items were assessed on a 5-point scale, ranging from (1) never to (5)
Criterion validity is dened as the extent to which a measure is very often. Cronbach's alpha was 0.93 (M 2.28, SD 0.78).
related to an outcome. Criterion validity of the SMD scale was
examined by testing the relationship between scores on the SMD 2.2.3. Self-declared Social Media Addiction
scale and several psychosocial constructs that have previously Respondents were asked: To what extent do you feel addicted
been related to compulsive Internet use and (specic forms of) to social media? Answers to this question were given on a 5-point
compulsive social media use: self-esteem (Mehdizadeh, 2010; Van scale ranging from (1) not at all addicted to (5) strongly addicted.
Rooij et al., 2015), depression (Caplan, 2007; Hong, Huang, Lin, &
Chiu, 2014; Koc & Gulyagci, 2013; Yen, Ko, Yen, Wu, & Yang, 2.2.4. Self-esteem
2007), loneliness (Caplan, 2007; Odaci & Kalkan, 2010; Van Rooij The degree of self-esteem was measured using the six-item self-
et al., 2015), attention decit (Dalbudak & Evren, 2014; Van Rooij esteem scale (Rosenberg, Schooler, & Schoenbach, 1989). This
et al., 2015; Weinstein & Lejoyeux, 2010; Yen et al., 2007), and measure implies feelings of self-acceptance, self-respect and
impulsivity (Dalbudak & Evren, 2014; Wu, Cheung, Ku, & Hung, generally positive self-evaluation. Sample items are: I am able to
2013). We expect to nd weak to moderate correlations between do things at least as well as other people and I feel that I don't
scores on the SMD scale and these psychosocial constructs. Finally, have much to be proud of (reverse coded). Response categories
in line with previous studies on the relationship between ranged from 1 (totally disagree) to 5 (totally agree). The items were
compulsive Internet use and daily time online (Meerkerk et al., averaged to create the scale scores. Cronbach's alpha for this scale
2009), and between IGD and time spent gaming (Lemmens et al., was 0.84 (M 3.78, SD 0.73).
2015; Van Rooij et al., 2012), we expect to nd moderate associa-
tions between the SMD scale and frequency of daily social media 2.2.5. Depression
use. Depression was assessed using the 6-item Kutcher Adolescent
Depression Scale (LeBlanc, Almudevar, Brooks, & Kutcher, 2002).
2. Method Respondents were asked whether items were applicable to them
on a 5-point scale, ranging from never (1) to very often (5). Example
2.1. Sample and procedure items are I feel there is little hope for the future and I feel un-
happy and depressed. Items were averaged to create scale scores.
From November 2014 through April 2015, three online surveys Cronbach's alpha was 0.86 (M 2.58, SD 0.84) in the rst sample,
were conducted among a total of 2198 Dutch adolescents, who and 0.87 (M 2.51, SD 0.85) in the second sample.
were all recruited through Marketing Science Institute (MSI), an
international market research company located in the Netherlands. 2.2.6. Attention decit
In November 2014, the rst online questionnaire was distributed The extent to which respondents displayed symptoms of
among 724 teenagers (54% girls) aged 10e17 (M 14.36, SD 2.11). Attention Decit was assessed by adapting nine items from the
Respondents received credit points for participating that could later DSM-IV checklist for ADHD that focused on attention decit, or
be redeemed for prizes. A second online survey was distributed two inattention (APA, 2000). As proposed by Kessler et al. (2005), re-
months later among a sample of 873 adolescents, aged 10e17, spondents were asked to indicate how often nine situations were
(M 14.28, SD 2.15, 48% girls), of whom 238 had also completed applicable to them on a 5-point scale ranging from never (1) to very
the rst questionnaire. Finally, a third online survey was conducted often (5). Example items are I am easily distracted and I have
among a new sample of 601 adolescents aged 10e17 (M 14.05, difculties organizing tasks. Items were averaged to create scale
SD 2.18, 50% girls). scores. Cronbach's alpha was 0.89 in the rst sample (M 2.59,
SD 0.74). In the second sample, Cronbach's alpha was 0.88
2.2. Measures (M 2.58, SD 0.71).

The rst online survey included the 27-item SMD scale, as well 2.2.7. Impulsivity
as validity measures; that is, Compulsive Internet Use, Self-declared The extent to which respondents displayed symptoms of
 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 481

impulsivity was assessed using six items adapted from the DSM-IV Next, a short 9-item version of the SMD scale was developed
checklist for ADHD that focused on impulsivity (APA, 2000). that encompasses all DSM-5 criteria by selecting the highest
Example items are I have difculty awaiting my turn and I loading items from each criterion. The standardized item-loadings
interrupt or intrude on others. Respondents were asked to indicate from sample 1 were used to select a set of nine items with the
how often these six situations were applicable to them on a 5-point highest overall loadings from each of the nine rst-order factors.
scale ranging from never (1) to very often (5). Items were averaged This short version of the scale was then tested as a rst-order
to create scale scores. Cronbach's alpha was 0.84 (M 2.16, structural model using Mean- and Variance-adjusted Weighted
SD 0.67) in the rst sample. In the second sample, Cronbach's Least Square (WLSMV) estimators in Mplus. Internal consistency of
alpha was 0.80 (M 2.24, SD 0.72). the 9-item scale was calculated in all three samples by means of
Cronbach's alpha.
2.2.8. Loneliness We also investigated the population cross-validity of the one-
Feelings of loneliness were assessed with the 10-item Loneliness dimensional structure of the short 9-item scale. More specically,
Scale developed by Russell, Peplau, and Cutrona (1980); this scale we tested whether the hypothesized one-dimensional structure of
contained 5 positive and 5 negative items. Examples of items are I the short SMD scale, which was found in the rst sample, was also
feel completely alone, I have nobody to talk to, and there are found in the second and third sample. Population cross-validity is
people who really understand me. Negative items were recoded satisfactory when the results found in one sample of a population
before summing the 10 items into a scale. The internal consistency of can also be found in another independent sample drawn from the
the scale was high; Cronbach's alpha was 0.90 (M 2.18, SD 0.99). same population (e.g., Raju, Bilgic, Edwards, & Fleer, 1997).
After testing the factor structure and internal consistency of the
2.2.9. Frequency of daily social media use long 27-item and short 9-item SMD scale (rst aim), we examined
The frequency of daily social media use was measured by pre- the construct validity of these scales (second aim). More specif-
senting a list of the fteen most popular social media. Respondent ically, we assessed convergent validity, which can be established if
were asked to indicate how often they used these social media on a two similar constructs correspond with one another. To assess
daily basis. Answer options were: (0) never (1) less than once a day convergent validity, respondents sum scores on the SMD scale
(2) 1e2 times (3) 3e5 times (4) 6e10 times (5) 11e20 times (6) 21e40 were correlated with compulsive Internet use, and self-declared
times (7) more than 40 times a day. Finally, we also asked for each social media addiction. Next, we determined criterion validity
type of social media platform or app how often respondents posted (third aim), that is, the extent to which a measure is related to an
something, using the same 7-point scale. outcome that it theoretically should be related to. We assessed
criterion validity by correlating the scores on the SMD scale with
self-esteem, depression, loneliness, attention decit, and impul-
2.3. Strategy of analyses
sivity. The following criteria were used to classify magnitude of
correlations: small, r .1e.29; medium, r .3e.49; large, r .5e1.0
First of all, we tested whether the 27 items of the SMD scale,
(Cohen, 1960).
consisting of three items for each of the nine DSM-5 criteria, can
The fourth aim was to calculate the test-retest reliability of the
be accounted for by one higher-order factor: social media disorder).
short 9-item SMD scale. We investigated the test-retest reliability
This factor structure was tested in the two independent samples,
by computing Pearson correlations between scores on the short
the rst and third one. We used structural equation modeling
SMD scales among the 238 adolescents who participated in the rst
(SEM) with weighted least squares estimators to test these second-
n, and second online survey. In addition, the intra-class correlation
order factor models using CFA in MPlus (Asparouhov & Muthe
coefcient (ICC) was established, using a two factor mixed effects
2009). Although maximum likelihood is the most common esti-
model and type consistency (McGraw & Wong, 1996). The fth aim
mation method in CFA, this method assumes that observed vari-
was to determine the sensitivity and specicity of the nine items of
ables are continuous and normally distributed in the population
n, 2004). Because this assumption was not met the short SMD scale. However, before doing so, we examined the
(Lubke & Muthe
prevalence of SMD in the three samples as indicated by the ve-or-
with our skewed distribution of SMD and ordinal levels of mea-
more cut-off point of the short SMD scale, and we tested for group
surement, a weighted least squares approach was applied to our
differences in gender, age, and frequency of daily social media use
data, allowing any combination of dichotomous, ordered categori-
between disordered and non-disordered adolescents. Next, sensi-
cal, or continuous observed variables (Flora & Curran, 2004).
tivity was demonstrated by the proportion of disordered social
Although researchers sometimes correlate error terms on the basis
media users who answered yes on an indicator of SMD, whereas
of theoretically overlapping indicators in an effort to improve
specicity was indicated by the proportion of non-disordered users
model t, this should be avoided if possible, since it means that
who reported no on an indicator of SMD.
there is some other issue that is not specied within the model that
is causing the covariation (Hooper, Coughlan, & Mullen, 2008).
Therefore, the error terms associated with each observed item are 3. Results
uncorrelated (Byrne, 2001).
The goodness of t was evaluated using the chi-square value, 3.1. Social media use
the Comparative Fit Index (CFI), the Root Mean Square Error of
Approximation (RMSEA), and its 90% condence interval (CI). The reported results on social media use are derived from the
Particularly when dealing with large samples, the chi square test is combined samples 1 and 3 (N 1325) unless otherwise specied. A
not a good indicator of t, and the CFI and RMSEA indices are small group (Sample 1: 6.6%, n 88; Sample 2: 10,3%, n 90) re-
considered informative t criteria in SEM (Byrne, 2001). A good t ported not using any form of social media and was excluded from
is expressed by a CFI greater than 0.95 and a RMSEA value less than analyses. Out of all 1237 social media users, 92.2% (n 1140) re-
0.08 (Byrne, 2001; Hu & Bentler, 1999; Yu & Muthe n, 2002). In ported owning a smartphone and using it for social media. The
addition, the internal consistency of the 27-item scale was calcu- most popular social media platforms and apps are displayed in
lated by means of Cronbach's alpha. Table 1.
482 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487

Table 1
The most popular social media (N 1325).

Total users Users on smartphonea Daily posts (1 post) Daily posts (>10 posts)

Facebook 83% 68% 46% 1%

WhatsAppb 82% 82% 82% 32%
Instagram 54% 51% 41% 1%
YouTube 53% 43% 33% 1%
Twitter 34% 26% 19% 2%

Note: aProportion of the total sample (N 1325); b

WhatsApp was measured only in survey 3 (N 601).

3.2. The dimensional structure of the SMD scale 9-item model would provide an equal or even better description of
the data. In sample 1, the unconstrained rst-order structural 9-
The 27-item scale was included in the rst sample (N 724), item model using Mean- and Variance-adjusted Weighted Least
M 5.65, SD 5.5 and in the third sample (N 601), M 5.65, Square (WLSMV) estimators yielded a good t, c2 (27,
SD 6.17. For analyses, all yes-answers were summed (range 0e27). n 724) 24.846, p 0.58, CFI 1.000, RMSEA 0.000 (90% CI:
The dimensional structure of the 27-item SMD scale (3 items per 0.000e0.026). This short SMD scale was strongly correlated with
criterion) was tested using a second-order factor model. This the 27-item SMD scale (r 0.89, p < 0.001) and showed good
resulted in an acceptable model t, c2 (288, n 724) 672.424, reliability with a Cronbach's alpha of 0.81 (M 1.22, SD 1.87).
p < 0.001, CFI 0.963, RMSEA 0.043 (90% CI: 0.039e0.047) in the Items for the short 9-item SMD scale are displayed in Table 3. The
rst sample. Similarly, in the third sample (n 601), the same total time to complete the short 9-item SMD scale was about 45 s,
model also showed an acceptable model t, c2 (288, compared to about 2 min and 15 s for completing the 27-item scale.
n 601) 570.681, p < 0.001, CFI 0.973, RMSEA 0.040 (90% CI: In a next step, we examined the population cross-validity by
0.036e0.045). Moreover, the 27-item SMD scale showed good in- testing whether the one-dimensional structure of the short SMD
ternal consistency with a Cronbach's alpha of 0.90 in the rst scale that was found in the rst sample, could also be found in
sample and 0.92 in the third sample. Table 2 shows the factor the second and third sample. Again, the unconstrained rst-order
loadings and percentages of afrmative answers for all 27 items in structural 9-item model yielded a good t, c2 (27,
samples 1 and 3. n 873) 62.852, p 0.001, CFI 0.997, RMSEA 0.041 (90% CI:
0.028e0.055). Furthermore, the 9-item scale showed adequate
reliability with a Cronbach's alpha of 0.76 (M 1.94, SD 2.11).
3.3. Constructing a short SMD scale and testing population cross-
Finally, in sample 3, the unconstrained rst-order structural 9-item
validity
model also yielded a good t, c2 (27, n 601) 54.129, p 0.002,
CFI 0.989, RMSEA 0.041 (90% CI: 0.025e0.057). In this sample,
In order to facilitate incorporation of the SMD scale into space-
the 9-item scale also showed a strong correlation with the 27-item
limited surveys, and assess the prevalence of SMD among adoles-
scale (r 0.94, p < 0.001) and showed good reliability with a
cents, an important aim of this study was to investigate whether a
Cronbach's alpha of 0.82 (M 1.52, SD 2.11).

Table 2
Afrmative answers and conrmatory factor loadings of SMD items.

# Criterion Sample 1 (n 724) Sample 3 (n 601)

% yes Loadings (b) % yes Loadings (b) 3.4. Convergent and criterion validity of the SMD scales
1 Preoccupation1 44 0.686 44 0.676
2 Preoccupation2 12 0.784 13 0.860 In order to establish the convergent validity, respondents mean
3 Preoccupation3 30 0.554 32 0.631 scores on the long and short SMD scales were correlated with
4 Tolerance1 39 0.713 35 0.829 compulsive Internet use and self-declared social media addiction.
5 Tolerance2 32 0.742 30 0.833 Next, to assess the criterion validity, the SMD scales were correlated
6 Tolerance3 09 0.902 10 0.938
7 Withdrawal1 22 0.781 24 0.833
with dissimilar but related constructs, i.e. depression, self-esteem,
8 Withdrawal2 16 0.886 21 0.894 loneliness, attention decit, impulsivity, and frequency of daily
9 Withdrawal3 13 0.946 14 0.963 social media use. As Table 4 shows, all correlations were signicant
10 Persistence1 18 0.854 18 0.876 at least at p < 0.001 in the expected directions. The long (27-item)
11 Persistence2 16 0.975 17 0.896
and short (9-item) versions of the SMD scale both showed large
12 Persistence3 16 0.908 15 0.926
13 Displacement1 29 0.773 33 0.769 positive correlations with compulsive Internet use (r > 0.50) and
14 Displacement 2 18 0.919 22 0.844 medium to large correlations with self-declared social media
15 Displacement 3 13 0.903 16 0.873 addiction, (r > 0.48), indicating satisfactory convergent validity.
16 Problems1 27 0.647 30 0.684 With regard to criterion validity, the long and short SMD scales
17 Problems2 35 0.601 34 0.666
18 Problems3 09 0.889 09 0.858
showed medium positive correlations with depression, attention
19 Deception1 14 0.814 16 0.928 decit, and frequency of daily social media use and posts, and weak
20 Deception2 13 0.760 13 0.947 to moderate positive associations with loneliness and impulsivity
21 Deception3 20 0.760 20 0.936 (see Table 4). Finally, a small negative correlation with self-esteem
22 Escape1 28 0.919 27 0.905
was found. The correlations between the SMD scales and these
23 Escape2 23 0.954 24 0.819
24 Escape3 20 0.965 19 0.834 related constructs indicated good criterion validity. Overall, the
25 Conict1 08 0.814 11 0.874 strength of the correlations between the SMD scales and these
26 Conict2 08 0.877 11 0.825 similar and related constructs was somewhat lower for the 9-item
27 Conict3 06 0.842 05 0.809 scale than for the 27-item scale, but the 9-item scale still demon-
Note: Item descriptions are found in Appendix A. strated satisfactory convergent and criterion validity.
 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 483

Table 3
The 9-item SMD scale.

Criterion During the past year, have you

Preoccupation regularly found that you can't think of anything else but the moment that you will be able to use social media again?
Tolerance regularly felt dissatised because you wanted to spend more time on social media?
Withdrawal often felt bad when you could not use social media?
Persistence tried to spend less time on social media, but failed?
Displacement regularly neglected other activities (e.g. hobbies, sport) because you wanted to use social media?
Problem regularly had arguments with others because of your social media use?
Deception regularly lied to your parents or friends about the amount of time you spend on social media?
Escape often used social media to escape from negative feelings?
Conict had serious conict with your parents, brother(s) or sister(s) because of your social media use?

Table 4
Correlations Between the 9- and 27-item SMD scales and Validation Constructs.

Sample 1 (N 724) Sample 2 (N 873) Sample 3 (N 601)

SMD 27 SMD 9 SMD 9 SMD 27 SMD 9

CIUS 0.57 0.51 e e e

Self-declared SMD 0.60 0.48
Self-Esteem 0.19 0.19 e e e
Depression 0.37 0.29 0.29 e e
Loneliness e e 0.24 e e
Attention Decit 0.36 0.33 0.26 e e
Impulsivity 0.30 0.30 0.27 e e
Frequency Daily Use 0.35 0.25 e 0.25 0.20
Frequency Daily Posts 0.34 0.27 e 0.28 0.24

Note: All correlations are signicant at p < 0.001.

3.5. Test-retest reliability of the 9-item SMD scale With regard to age, no differences were found between disor-
dered and non-disordered users (sample 1, t (1,722) 1.40,
Test-retest reliability of the 9-item short SMD scale was assessed p 0.16; sample 2, t (1,781) 0.60, p 0.55; sample 3, t
among the 238 adolescents who participated in both the rst and (1,599) 0.30, p 0.77). Table 5 illustrates the differences in use
the second online survey (with an interval of 2 months between of specic social media applications between disordered and non-
these two surveys). A moderate degree of reliability was found disordered social media users. In the rst sample, all social media
between the rst and second SMD scales. The Pearson correlation are used more often among disordered users. In the third sample,
between both scales was 0.50, p < 0.001. The averaged measure ICC, signicant differences were only demonstrated for active use of
using an absolute agreement denition, was.663 (95% Facebook, Instagram, and Whatsapp.
CI:.565e.739), and the mean variation between the measures of
SMD was 0.47. An averaged measure ICC of 0.60 or higher indicates
satisfactory stability (Landis & Koch, 1977).
3.7. Sensitivity and specicity of the 9-item SMD scale

3.6. Prevalence of Social Media Disorder Finally, each of the nine indicators of the SMD scale was
examined for their sensitivity and specicity. Sensitivity of the
The 9-item SMD scale was used to assess the prevalence of short scale items was demonstrated by the proportion of disor-
disordered social media use among teenagers. In accordance with dered social media users from each sample (n's 53, 101, 63) who
the cut-off point for IGD in the DSM-5, at least ve or more (out of answered positively on an item, whereas specicity was indicated
nine) criteria must be met for a formal diagnosis of disordered by the proportion of negative responses on a scale item by non-
social media user. Among the rst sample (N 724), we found that disordered gamers from each sample. Ideally, both sensitivity and
53 teenagers met ve or more of the criteria (7.3%). In the second specicity of an item should be high in order to discriminate false
sample (N 873), 101 adolescents (11.6%) met the cut-off point for positives and false negatives (Glaros & Kline, 1988). As Table 6
disordered use of social media. In the third sample (N 601), 62 shows, the nine items show adequate sensitivity and high speci-
teenagers (10.3%) could be viewed as disordered social media users. city. The diagnostic accuracy, as indicated by the proportion of all
Also, we examined whether disordered social media users true positives (indicating sensitivity) and true negatives (indicating
differed from non-disordered users with regard to gender, age, and specicity), was highly comparable across samples. The sensitivity
frequency of daily social media use. Chi-square tests for the rst of Problems and Conict were the lowest of all items across samples,
sample indicated that there were more disordered boys (n 34, indicating that between 47% and 62% of all disordered social media
10.2%) than disordered girls (n 19, 4.9%), c2 (1, 724) 7.471, users had experienced serious problems as a result of their
p 0.006. The second and third sample, however, did not replicate compulsive social media use, and that between 50% and 61% of the
this gender difference: in the second sample, the number of boys disordered users had experienced conicts with friends, family or
among disordered social media users (n 45, 9.9%) did not differ partners because of their social media use. Conversely, the speci-
from the number of disordered girls (n 56, 13.3%), c2 (1, city of Problems and Conict was high across samples (ranging
873) 2.462, p 0.117. Similarly, in the third sample no differences between 0.92 and 0.97) indicating that between 3% and 8% of the
between boys (n 26, 8.7%) and girls (n 36, 12.0%), c2 (1, social media users who had experienced problems and conicts
601) 1.762, p 0.148, were found. were not among the disordered gamers.
484 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487

Table 5
Mean frequency of daily social media usea of disordered and non-disordered users.

Sample 1 (N 724) Sample 3 (N 601)

Non-disordered Disordered Non-disordered Disordered

*
Facebook use 3.20 (1.44) 3.70 (1.53) 3.26 (1.33) 3.57 (1.64)
Facebook posts 1.43 (0.67) 2.13 (1.15)* 1.64 (0.90) 2.08 (0.99)*
FB messenger use e e 2.44 (1.18) 3.03 (1.55)*
FB messenger posts e e 1.88 (1.08) 2.60 (1.42)*
Instagram use 3.44 (1.52) 4.06 (1.63)* 3.54 (1.37) 3.78 (1.71)
Instagram posts 1.60 (0.84) 2.31 (1.53)* 1.75 (0.91) 2.22 (0.87)*
YouTube use 1.21 (0.70) 1.57 (1.07)* 3.30 (1.33) 3.37 (1.33)
YouTube posts 1.21 (0.69) 1.57 (1.07)* 1.58 (0.99) 1.86 (1.04)
Twitter use 1.60 (1.17) 2.70 (2.05)* 2.41 (1.38) 2.88 (1,80)
Twitter posts 1.59 (1.17) 2.70 (2.05)* 1.73 (0.99) 1.88 (1.00)
WhatsApp posts e e 4.13 (1.58) 5.08 (1.71)*

Note: (SD), * signicant t-test differences at least p < 0.01; aAnswer options were: (0) never, (1) less than once a day, (2) 1e2 times, (3) 3e5 times, (4) 6e10 times, (5) 11e20
times, (6) 21e40 times, and (7) more than 40 times a day.

Table 6
Sensitivity and specicity of the nine criteria for Social Media Disorder.

Disordered users sample 1 (n 53) Disordered users sample 2 (n 101) Disordered users sample 3 (n 62)

Sens e Spec Sens e Spec Sens e Spec

Preoccupation 0.74e0.92 0.79e0.87 0.77e0.94

Tolerance 0.74e0.97 0.66e0.93 0.66e0.96
Withdrawal 0.87e0.94 0.78e0.85 0.81e0.93
Persistence 0.81e0.88 0.69e0.77 0.69e0.83
Displacement 0.62e0.97 0.76e0.83 0.73e0.84
Problems 0.62e0.92 0.55e0.93 0.47e0.95
Deception 0.77e0.91 0.66e0.89 0.71e0.90
Escape 0.79e0.82 0.66e0.83 0.71e0.82
Conict 0.59e0.97 0.50e0.95 0.61e0.94

4. Discussion Moreover, the nine items generally showed adequate sensitivity

and good specicity. The prevalence of SMD, determined on basis of
There is a growing body of evidence suggesting that social the diagnostic cut-off point of the 9-item scale, ranged from 7.3% to
media disorder (i.e. addiction) is an emerging mental problem, 11.6% in the 3 online samples.
particularly among adolescents (Pantic, 2014; Ryan et al., 2014). Convergent validity was determined by the strength of corre-
However, the absence of a measurement tool for SMD hampers lations between SMD and similar constructs. As expected, corre-
further development of the research eld. Particularly, there is a lations between scores on the SMD scales and scores on the
strong need for an assessment instrument that can distinguish Compulsive Internet Use Scale (CIUS) were strong. Also, the SMD
between disordered (i.e. addictive) and highly engaged non- scales showed strong correlations with self-reported social media
disordered social media users. Therefore, the present study, con- addiction, although the strength of this correlation was somewhat
sisting of three online surveys among adolescents aged 10 to 17, lower for the 9-item scale than for the 27-item scale. The strength
aimed to test the reliability and validity of a short and easy to of the association between the 9-item scale and self-declared social
administer SMD Scale that contains a clear diagnostic cut-off point. media addiction, however, is substantial and similar to the strength
In the absence of specic diagnostic criteria for SMD, the of the previously found association between scores on the CIUS and
development of our measurement tool was based on the assump- self-reported Internet addiction (Meerkerk et al., 2009).
tion that SMD and IGD are two forms of the same overarching Criterion validity was determined on basis of the strength of
construct Internet Addiction, and should thus be dened by the correlations with psychosocial constructs that were previously
same set of diagnostic criteria. Therefore, the development of the related to compulsive Internet use and/or social media use. In line
SMD scale was based on the nine DSM-5 criteria for IGD (APA, with our expectations, the SMD scales showed moderate relation-
2013). First, a 27-item dichotomous scale was developed assess- ships in the expected direction with depression, attention decit
ing the nine DSM-5 criteria, i.e. preoccupation, tolerance, with- and frequency of daily social media use, weak to moderate asso-
drawal, persistence, escape, problems, deception, displacement, ciations with loneliness and impulsivity. However, only a weak
and conict, with three items per criterion. Next, we examined negative association was found with self-esteem.
whether a short 9-item scale, consisting of the nine items with the The relatively stronger associations found for depression and
highest factor loading per criterion, would provide an equally valid attention decit coincide with ndings of a recent study indicating
and reliable measurement tool. that depressive mood and hyperactivity-inattention are more
This study generated evidence that the 9-item scale is a psy- strongly related to SMD than to IGD (Van Rooij et al., 2015). The
chometrically sound and valid instrument to measure SMD, and relatively strong link between SMD and attention decit corrobo-
just as valid as the 27-item version. Conrmatory factor analysis rates the public concern that the use of social media, particularly
showed good model ts, indicating solid structural validity. The 9- when used via smartphones (e.g., Whatsapp), is distracting ado-
item scale also showed appropriate internal consistency, sufcient lescents attention from their everyday activities and obligations.
test-retest reliability, and good convergent and criterion validity. A recent review indeed suggests that higher levels of media
 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 485

multitasking, i.e. the use of media while engaging in non-media media use and SMD in more detail.
activities, such as completing homework and engaging in face-to- Some limitations of the present research warrant discussion.
face interactions, is related to decits in cognitive control, in First, the nine DSM-5 criteria dened for IGD were translated to
particular to the ability to sustain attention (Van der Schuur, SMD. It should be noted, however, that these nine criteria for IGD
Baumgartner, Sumter, & Valkenburg, 2015). The relatively high are still subject to discussion (e.g. Grifths et al., 2015; Kardefelt-
correlation between SMD and attention decit thus provides some Winther et al., 2014). Consequently, using the same nine DSM-5
evidence for the scattered attention hypothesis (e.g., Ophira, Nass, criteria of IGD to measure SMD may yield similar conceptual de-
& Wagner, 2009) which states that when people frequently bates. For instance, the notion that the criterion of Deception, i.e.
engage in media multitasking, they become accustomed to con- lying about the time spend on social media, is socially or culturally
stant switching between activities and eventually lose their ability subjective and also depend on the people close to the gamer
to focus on a single activity (Van der Schuur et al., 2015; Wallis, (Kardefelt-Winther et al., 2014), also applies to the current
2006, 2010). However, because of the cross-sectional nature of conceptualization of social media disorder. In addition, some of the
the present ndings, no causal inferences can be made. Future proposed IGD criteria may be less relevant in the context of social
longitudinal and experimental research is warranted to establish media use. As suggested earlier, Conict and Problems may be less
causality between SMD and attention decit in adolescents. appropriate criteria to measure SMD, as compared to IGD, and thus
Most of the nine items of the short SMD scale showed good strongly require further investigation.
sensitivity and specicity. The items measuring Problems and Despite these conceptual shortcomings, we followed the prag-
Conict, however, showed a lower sensitivity in comparison to the matic approach of developing and validating this short and easy to
other items of the SMD scale, as well as in comparison to Problems administer tool to measure SMD, which enables the investigation of
and Conict items of the IGD-scale (Lemmens et al., 2015). Experi- trends and developments in the prevalence of SMD during this
encing conict with others about the time spent on social media period of rapid changes in the social media landscape. This study,
use may have less external validity than conicts about time spent however, is regarded a rst research step, and an important next
gaming. Social media use is more easily stopped or combined with step would be to investigate the correctness of the nine DSM-5
other activities, thereby causing fewer problems as a result of criteria as the core features of SMD, as well as to test whether the
compulsive social media use, in comparison to compulsive gaming. items covered in the short SMD scale indeed are the most suitable
Also, having to quit gaming may be experienced as more frustrating ones for diagnosing SMD in both clinical and non-clinical samples.
by disordered adolescents than having to quit social media use. It would be interesting, for instance, to test the extent to which self-
Consequently, IGD may induce more conict and arguing with declared social media addicts identify with the items of both the
family members than SMD. Previous research indeed showed that short 9-item and the long 27-item SMD scale to gain more insight
disordered gamers display more physical aggression (Lemmens, into the actual signicance of the nine DSM-5 criteria for deter-
Valkenburg, & Peter, 2011) than non-disordered gamers. Future mining SMD. After these research steps have been taken, this in-
research should address the aptness of the nine DSM-5 criteria for strument will facilitate the investigation of psychological processes
measuring SMD, and examine whether Problems and Conict are (motivational, affective, cognitive, interpersonal, and social) sus-
indeed core features of SMD, as was assumed in the present study. taining the dysfunctional involvement in social media use (Billieux,
The ndings of the third survey suggest that some types of social Schimmenti, Khazaal, Maurage, & Heeren, 2015; Dudley, Kuyken, &
media use may elicit a higher risk than others, and that disordered Padesky, 2011), and will thereby contribute substantially to un-
users differ from non-disordered users particularly in the number derstanding Social Media Disorder.
of posts that they place on Facebook, Instagram and Whatsapp.
However, these results are somewhat inconsistent with the nd-
ings of the rst survey suggesting that both passive and active use Appendix A
of social media is related to SMD. Future research should address
the strength of the relationships between different types of social 27 items for the Social Media Disorder Scale.

Preoccupation
During the past year, have you
often found it difcult not to look at messages on social media when you were doing something else (e.g. school work)?
regularly found that you can't think of anything else but the moment that you will be able to use social media again?*
often sat waiting until something happens on social media again?
Tolerance
During the past year, have you
felt the need to use social media more and more often?
felt the need to check messages on social media more and more often?
regularly felt dissatised because you wanted to spend more time on social media?*
Withdrawal
During the past year, have you
often felt tense or restless if you weren't able to look at your messages on social media?
regularly felt angry or frustrated if you weren't able to use social media?
often felt bad when you could not use social media?*
Persistence
During the past year, have you
tried to reduce your use of social media, but failed?
tried to spend less time on social media, but failed?*
been unable to stop using social media, even though others told you that you really should?
Escape
During the past year, have you
regularly used social media to take your mind off your problems?
(continued on next page)
486 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487

(continued )

often used social media so you didn't have to think about unpleasant things?
often used social media to escape from negative feelings?*
Problems
During the past year, have you
often not paid attention at school, while doing homework or at work because you were using social media?
regularly not had enough sleep because you were using social media too late at night?
regularly had arguments with others because of your social media use?*
Deception
During the past year, have you
regularly lied to your parents or friends about the amount of time you spend on social media?*
regularly hidden your social media use from others?
often used social media secretly?
Displacement
During the past year, have you
regularly devoted no attention to people around you (e.g. family or friends) because you were using social media?
regularly had no interest in hobbies or other activities because you would rather use social media?
regularly neglected other activities (e.g. hobbies, sport) because you wanted to use social media?*
Conict
During the past year, have you
had serious problems at school or at work because you were spending too much time on social media?
had serious conict with your parent(s) and sibling(s) because of your social media use?*
jeopardised or lost an important friendship or relationship because you were spending too much time on social media?

References students. Telematics and Informatics, 31, 597e606.

Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling:
guidelines for determining model t. Electronic Journal of Business Research
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental
Methods, 6, 53e60.
disorders (4th ed.). Washington, DC: American Psychiatric Pub. Association. text
Huang, Z., Wang, M., Qian, M., Zhong, J., & Tao, R. (2007). Chinese internet addiction
revision.
inventory: developing a measure of problematic internet use for Chinese col-
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental
lege students. Cyberpsychology & Behavior, 10, 805e811. http://dx.doi.org/
disorders (5th ed.). Washington, DC: American Psychiatric Association. text
10.1089/cpb.2007.9950.
revision.
n, B. (2009). Exploratory structural equation modeling. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for t indexes in covariance structure
Asparouhov, T., & Muthe
analysis: conventional criteria versus new alternatives. Structural Equation
Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397e438.
Modeling, 6, 1e55. http://dx.doi.org/10.1080/10705519909540118.
Billieux, J., Schimmenti, A., Khazaal, Y., Maurage, P., & Heeren, A. (2015). Are we over
Kardefelt-Winther, D. (2015). A critical account of DSM-5 criteria for internet
pathologizing everyday life? A tenable blueprint for behavioral addiction
gaming disorder. Addiction Research & Theory, 23(2), 93e98.
research. Journal of behavioral addictions, 4(3), 119e123.
Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E. V. A., et al. (2005).
Brown, R. I. F. (1993). Some contributions of the study of gambling to the study of
The World Health Organization Adult ADHD Self-Report Scale (ASRS): a short
other addictions. In W. R. Eadington, & J. A. Cornelius (Eds.), Gambling behavior
screening scale for use in the general population. Psychological Medicine, 35(02),
and problem gambling (pp. 241e272). Reno, NV: University of Nevada.
245e256.
Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, appli-
Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media?
cations and programming. Mahwah, NJ: Erlbaum.
Get serious! Understanding the functional building blocks of social media.
Caplan, S. E. (2007). Relations among loneliness, social anxiety, and problematic
Business Horizons, 54(3), 241e251.
Internet use. CyberPsychology & Behavior, 10, 234e242.
King, D. L., Haagsma, M. C., Delfabbro, P. H., Gradisar, M., & Grifths, M. D. (2013).
Caplan, S. E. (2010). Theory and measurement of generalized problematic Internet
Toward a consensus denition of pathological video-gaming: a systematic re-
use: a two-step approach. Computers in Human Behavior, 26(5), 1089e1097.
view of psychometric assessment tools. Clinical Psychology Review, 33(3),
Cohen, J. A. (1960). A coefcient of agreement for nominal scales. Educational and
331e342.
Psychological Measurement, 20(1), 37e46.
Koc, M., & Gulyagci, S. (2013). Facebook addiction among Turkish college students:
Dalbudak, E., & Evren, C. (2014). The relationship of Internet addiction severity with
the role of psychological health, demographic, and usage characteristics.
Attention Decit Hyperactivity Disorder symptoms in Turkish University stu-
CyberPsychology, Behavior, and Social Networking, 16, 279e284.
dents; impact of personality traits, depression and anxiety. Comprehensive
Kuss, D. J., & Grifths, M. D. (2012). Internet gaming addiction: a systematic review
Psychiatry, 55(3), 497e503.
of empirical research. International Journal of Mental Health and Addiction, 10(2),
Demetrovics, Z., Urb an, R., Nagygyo rgy, K., Farkas, J., Grifths, M. D., Pa
pay, O., et al.
278e296.
(2012). The development of the problematic online gaming questionnaire
Kuss, D. J., Van Rooij, A. J., Shorter, G. W., Grifths, M. D., & Van de Mheen, D. (2013).
(POGQ). PLoS ONE, 7. http://dx.doi.org/10.1371/journal.pone.0036417.
Internet addiction in adolescents: prevalence and risk factors. Computers in
Dudley, R., Kuyken, W., & Padesky, C. A. (2011). Disorder specic and trans-
Human Behavior, 29(5), 1987e1996.
diagnostic case conceptualisation. Clinical Psychology Review, 31(2), 213e224.
Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of
data. Biometrics, 33, 159e174.
estimation for conrmatory factor analysis with ordinal data. Psychological
LeBlanc, J. C., Almudevar, A., Brooks, S. J., & Kutcher, S. (2002). Screening for
Methods, 9, 466e491. http://dx.doi.org/10.1037/1082-989X.9.4.466.
adolescent depression: comparison of the Kutcher adolescent depression scale
Gentile, D. A., Choo, H., Liau, A., Sim, T., Li, D., Fung, D., et al. (2011). Pathological
with the Beck depression inventory. Journal of Child and Adolescent Psycho-
video game use among youth: a two-year longitudinal study. Pediatrics, 127,
pharmacology, 12(2), 113e126. http://dx.doi.org/10.1089/104454602760219153.
319e329. http://dx.doi.org/10.1542/peds.2010-1353.
Lemmens, J. S., Valkenburg, P. M., & Gentile, D. A. (2015). The internet gaming
Glaros, A. G., & Kline, R. B. (1988). Understanding the accuracy of tests with cutting
disorder scale. Psychological Assessment, 72, 567e582. http://dx.doi.org/10.1037/
scores: the sensitivity, specicity, and predictive value model. Journal of Clinical
pas0000062.
Psychology, 44, 1013e1023.
Lemmens, J. S., Valkenburg, P., & Peter, J. (2009). Development and validation of a
Grifths, M. (1999). Internet addiction: fact or ction? Psychologist, 12(5), 246e250.
game addiction scale for adolescents. Media Psychology, 12, 77e95. http://
Grifths, M. (2005). A components model of addiction within a biopsychosocial
dx.doi.org/10.1080/15213260802669458.
framework. Journal of Substance Use, 10, 191e197.
ly, O., Lemmens, J. S., Valkenburg, P. M., & Peter, J. (2011). The effects of pathological
Grifths, M., Van Rooij, A. J., Kardefelt-Winther, D., Starcevic, V., Kira
gaming on aggressive behavior. Journal of Youth and Adolescence, 40, 38e47.
Pallesen, S., et al. (2016). Working towards an international consensus on
Lubke, G., & Muthe n, B. (2004). Factor-analyzing Likert scale data under the
criteria for assessing Internet Gaming Disorder: a critical commentary on Petry,
assumption of multivariate normality complicates a meaningful comparison of
et al (2014). Addiction, 111(1), 167e175.
observed groups or latent classes. Structural Equation Modeling, 11, 514e534.
Grifths, M. D., Kuss, D. J., & Demetrovics, Z. (2014). Social networking addiction: an
Marks, I. (1990). Behavioural (non-chemical) addictions. British Journal of Addiction,
overview of preliminary ndings. In Behavioral addictions. Criteria, evidence, and
85(11), 1389e1394.
treatment (pp. 119e141). New York: Elsevier.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass
Grifths, M., & Szabo, A. (2013). Is excessive online usage a function of medium or
correlation coefcients. Psychological methods, 1(1), 30.
activity? an empirical pilot study. Journal of behavioral addictions, 3(1), 74e77.
Meerkerk, G. J., Van den Eijnden, R. J. J. M., Vermulst, A. A., & Garretsen, H. F. (2009).
Hong, F., Huang, D., Lin, H., & Chiu, S. (2014). Analysis of the psychological traits,
The compulsive Internet use scale (CIUS): some psychometric properties.
Facebook usage, and Facebook addiction model of Taiwanese university
 R.J.J.M. van den Eijnden et al. / Computers in Human Behavior 61 (2016) 478e487 487

CyberPsychology & Behavior, 12(1), 1e6. Problematic Internet use: comparing video gaming and social media use
Mehdizadeh, S. (2010). Self presentation 2.0: narcissism and self-esteem on Face- [Conference Abstract]. Journal of Behavioral Addictions, 4(1), 1e62. http://
book. Cyberpsychology, Behavior and Social Network Sites, 13, 357e364. dx.doi.org/10.1556/JBA.4.2015.Suppl.1.
Odaci, H., & Kalkan, M. (2010). Problematic Internet use, loneliness and dating Van Rooij, A., & Prause, N. (2014). A critical review of Internet addiction criteria
anxiety among young adult university students. Computers and Education, 55, with suggestions for the future. Journal of behavioral addictions, 3(4), 203e213.
1091e1097. Van Rooij, A. V., & Schoenmakers, T. M. (2013). Monitor Internet en Jongeren 2010-
Ophira, E., Nass, C., & Wagner, A. D. (2009). Cognitive control in media multitaskers. 2012. Het (mobiele) gebruik van sociale media en games door jongeren [Monitor
In Proceedings of the national academy of sciences of the United States of America, Internet and Youth 2010-2012: The (mobile) use of social media and games by
106. September 15, 2009. Available at: http://www.pnas.org/content/106/37/ adolescents]. http://www.ivo.nl/UserFiles/File/Publicaties/2013-01%20Factsheet
15583. %20Social%20media%20en%20gamen.pdf.
Pantic, I. (2014). Online social networking and mental health. Cyberpsychology, Van Rooij, A. J., Schoenmakers, T. M., Van den Eijnden, R. J. J. M., & Van de Mheen, D.
Behavior, and Social Networking, 17(10), 652e657. (2010). Compulsive Internet use: the role of online gaming and other Internet
Petry, N. M., Rehbein, F., Gentile, D. A., Lemmens, J. S., Rumpf, H., Mo le, T., et al. applications. The Journal of Adolescent Health, 47(1), 51e57.
(2014). An international consensus for assessing internet gaming disorder using Van Rooij, A. J., Schoenmakers, T. M., Van den Eijnden, R. J. J. M., Vermulst, A. A., &
the new DSM-5 approach. Addiction, 109(9), 1399e1406. Van de Mheen, D. (2012). Video game addiction test: validity and psychometric
Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1997). Methodology review: esti- characteristics. Cyberpsychology, Behavior, and Social Networking, 15(9),
mation of population validity and cross-validity, and the use of equal weights in 507e511.
prediction. Applied Psychological Measurement, 21(4), 291e305. Van den Eijnden, R. J. J. M., Meerkerk, G., Vermulst, A. A., Spijkerman, R., &
Rehbein, F., Kleimann, M., & Mo ssle, T. (2010). Prevalence and risk factors of video Engels, R. C. M. E. (2007). Online communication, compulsive Internet use, and
game dependency in adolescence: results of a German nationwide survey. psychosocial well-being among adolescents: a longitudinal study. Develop-
Cyberpsychology, Behavior and Social Network Sites, 13, 269e277. http:// mental Psychology, 44, 655e665.
dx.doi.org/10.1089/cyber.2009.0227. Van der Schuur, W. A., Baumgartner, S. E., Sumter, S. R., & Valkenburg, P. M. (2015).
Rosenberg, M., Schooler, C., & Schoenbach, C. (1989). Self-esteem and adolescent The consequences of media multitasking for youth: a review. Computers in
problems: modeling reciprocal effects. American sociological review, 1004e1018. Human Behavior, 53, 204e215.
Rumpf, H. J., Meyer, C., Kreuzer, A., John, U., & Meerkerk, G. J. (2011). Pra valenz der Wallis, C. (2006). The multitasking generation. Time Magazine, 167(13), 48e55.
Internetabha ngigkeit (PINTA). Bericht an das Bundesministerium fr Gesundheit. Wallis, C. (2010). The impacts of media multitasking on children's learning & devel-
Verfgbar ber. http://www.drogenbeauftragte.de/leadmin/dateien-dba/ opment. New York: Joan Ganz Cooney Center.
DrogenundSucht/Computerspiele_Internet/Downloads/PINTA-Bericht- Wang, C., Lee, M. K. O., & Hua, Z. (2015). A theory of social media dependence:
Endfassung_280611.pdf. evidence from microblog users. Decision Support Systems, 69, 40e49.
Russell, D., Peplau, L. A., & Cutrona, C. E. (1980). The revised UCLA Loneliness Scale: Weinstein, A., & Lejoyeux, M. (2010). Internet addiction or excessive internet use.
concurrent and discriminant validity evidence. Journal of Personality and Social The American journal of drug and alcohol abuse, 36(5), 277e283.
Psychology, 39, 472e480. Wu, A. M. S., Cheung, V. I., Ku, L., & Hung, E. P. W. (2013). Psychological risk factors of
Ryan, T., Chester, A., Reece, J., & Xenos, S. (2014). The uses and abuses of Facebook: a addiction to social networking sites among Chinese smartphone users. Journal
review of Facebook addiction. Journal of behavioral addictions, 3(3), 133e148. of Behavioral Addictions, 2(3), 160e166.
Saaid, S. A., Al-Rashid, N. A. A., & Abdullah, Z. (2014). The impact of addiction to Yen, J. Y., Ko, C. H., Yen, C. F., Wu, H. Y., & Yang, M. J. (2007). The comorbid psychiatric
Twitter among university students. In Future information technology (pp. symptoms of Internet addiction: attention decit and hyperactivity disorder
231e236). Springer Berlin Heidelberg. (ADHD), depression, social phobia, and hostility. Journal of Adolescent Health,
Shotton, M. (1989). Computer addiction?: A study of computer dependency. London: 41(1), 93e98.
Taylor & Francis. Young, K. (1996). Internet addiction: the emergence of a new clinical disorder. In
Soper, W. B., & Miller, M. J. (1983). Junk-time junkies: an emerging addiction among Paper presented at the 104th annual meeting of the American Psychological As-
students. The School Counselor, 31(1), 40e43. sociation. Toronto, Ontario, Canada.
Valkenburg, P. M., & Peter, J. (2011). Adolescents' online communication: an inte- Yu, C. Y., & Muthe n, B. (2002). Evaluation of model t indices for latent variable models
grated model of its attraction, opportunities, and risks. Journal of Adolescent with categorical and continuous outcomes (Technical Report). Los Angeles: Uni-
Health, 48, 121e127. versity of California at Los Angeles, Graduate School of Education & Information
Van Rooij, A. J., Ferguson, C., Van de Mheen, D., & Schoenmakers, T. M. (2015). Studies.