Computers & Security 97 (2020) 101924

Contents lists available at ScienceDirect

Computers & Security

journal homepage: www.elsevier.com/locate/cose

Unpacking the intention-behavior gap in privacy decision making for

the internet of things (IoT) using aspect listing
Qizhang Sun a,∗, Martijn C. Willemsen a, Bart P. Knijnenburg b
a
Human-Technology Interaction, Eindhoven University of Technology, PO Box 513, 5600 MB Eindhoven, the Netherlands
b
Human-Centered Computing, Clemson University, Clemson SC 29634, USA

a r t i c l e i n f o a b s t r a c t

Article history: Previous studies have observed an intention-behavior gap that has been labeled the “privacy paradox”:
Received 2 January 2020 people disclose personal information (behavior) despite expressing negative sharing intentions (in sur-
Revised 19 May 2020
veys). However, this phenomenon has not been studied in the Internet of Things (IoT) in which users’ per-
Accepted 3 June 2020
sonal information sharing is crucial for the functionality of the technology. We explore this phenomenon
Available online 22 July 2020
by comparing participants’ intentions (via a survey) with their actual behavior (via a privacy-setting inter-
Keywords: face) and controlling the data sharing device and storage. Furthermore, we explore the decision processes
Privacy paradox underlying these privacy decisions by measuring and manipulating these processes using an aspect list-
Risk-benefit calculation ing task. We find a reversed intention-behavior gap in IoT: participants disclosed less (rather than more)
Process-tracing information in the behavior condition than in the intention condition, an effect that was associated with
Internet of things fewer benefits than risk aspects listed in the behavior condition. The number and type of aspects listed
Decision-making
fully mediated the effect of decision type (intention versus behavior) on the decision, which suggests that
a risk-benefit calculation guided the privacy decision-making. Moreover, this reversed intention-behavior
gap vanishes if we specifically ask participants to think about positive and negative aspects of the deci-
sion, as this allows them to consider both risks and benefits, irrespective of decision type.
© 2020 The Author(s). Published by Elsevier Ltd.
This is an open access article under the CC BY license. (http://creativecommons.org/licenses/by/4.0/)

1. Introduction about privacy or showing low/negative sharing intentions. The pri-

vacy paradox involves two lines of research (Kokolakis, 2017). One
Traditionally, privacy research has used intention to study pri- line focuses on a concern-behavior gap, which describes that peo-
vacy decisions, because measuring intention is an inexpensive ple share their personal information despite their high concerns
means to study such decisions outside the context of actual sys- about privacy. For example, numerous studies have shown indi-
tems. A number of influential theories (e.g., theory of reasoned viduals reporting high levels of privacy concerns in a question-
action, theory of planned behavior, theory of attitude-behavior, naire, while on the other hand disclosing a large amount of per-
and theory of protection motivation) have argued that inten- sonal information on social networks (e.g., Min and Kim, 2015;
tion is among the most immediate and most important predic- Tufekci, 2008). The other line of research focuses on an intention-
tors of behavior (Ajzen, 1988, 1991; Fishbein, 1980; Fishbein and behavior gap, which describes that people share their personal in-
Ajzen, 1975; Rogers, 1983; Triandis, 1980). However, decision- formation despite their intention not to share. For example, people
making research typically shows a large gap between intention shared more personal information to a researcher pretending to be
and behavior: A meta-analysis showed that in previous stud- a bank representative than they claimed they intended to share
ies, intention could only explain 28% of the variance in behavior in a preceding intention questionnaire (Norberg et al., 2007). In
(Sheeran, 2002). In the field of privacy, this intention-behavior gap contrast to the general privacy concerns measured in the concern-
is labeled as the “privacy paradox” (Norberg et al., 2007). behavior gap, the intention to share is a measurement of willing-
The privacy paradox shows that people actually disclose a lot ness to share specific personal information in a specific circum-
of personal information (behavior) despite reporting high concerns stance or context with a purpose (e.g., Norberg et al., 2007), which
is a product of an explicit privacy-benefit trade-off. With a few ex-
ceptions (i.e. Kokolakis, 2017; Phelan et al., 2016), earlier research
∗
Corresponding author. on the privacy paradox has not explicitly distinguished between
E-mail addresses: Q.Sun.2@tue.nl (Q. Sun), M.C.Willemsen@tue.nl (M.C. Willem- the intention-behavior gap and the concern-behavior gap. Further-
sen), BartK@clemson.edu (B.P. Knijnenburg).

https://doi.org/10.1016/j.cose.2020.101924
0167-4048/© 2020 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license. (http://creativecommons.org/licenses/by/4.0/)
2 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

more, most privacy paradox research has focused on the concern- to lack of knowledge or control, decision-makers engage in little
behavior gap, while the intention-behavior gap has received only or no risk assessment, which results in no risk-benefit calculation.
limited attention. In the first branch, Privacy calculus (Culnan and Arm-
Studying the intention-behavior gap is especially critical in the strong, 1999) is the dominant theory of set of unbiased risk-benefit
context of privacy in the internet of things (IoT) in households. calculation theories. It states that individuals’ privacy decisions
IoT in households describes the class of connected devices such depend on the outcomes of a calculation of the privacy risks
as smart assistants, IP cameras and smart washing machines that and benefits that they would incur for disclosing the information.
provide services that strongly rely on the collection and sharing of Specifically, people only disclose personal information when the
data between devices in the home and to the servers of the man- benefit of sharing outweighs the risk of sharing. Most research has
ufacturers of these devices. The privacy decision about what per- indeed shown that users’ perceptions of benefits are positively, and
sonal information these devices are allowed to collect and share privacy risks are negatively associated with their information shar-
plays a key role in the benefits and risks the technology can bring. ing intention and behavior, respectively (e.g., Rindfleisch, 1997).
On the one hand, the technology will not function appropriately The set of biased risk-benefit calculation theories argues that al-
without this personal information, but on the other hand, the in- though people use privacy calculus to make a privacy decision, the
formation that is shared may be highly sensitive, and privacy in- privacy calculus can be moderated by various biases and heuris-
fringements can have serious consequences. Yet, according to the tics. For example, the attribution theory suggests that people un-
review by Gerber et al. (2018), no study has yet investigated the consciously fail to see themselves as being average, specifically,
intention-behavior gap in IoT in households. they tend to underestimate their own risks while overestimat-
This paper investigates the existence of the intention-behavior ing those of others (Kelley, 1967). As such, people underestimate
gap by comparing the user’s intention with their actual sharing be- the chances of being a victim of cybercriminals and hackers de-
havior (RQ1). Furthermore, we study the underlying mechanisms spite widespread reports of victimization, resulting in risky on-
leading to the potential intention-behavior gap by exploring and line behavior (Barth and de Jong, 2017). Likewise, the hyperbolic
manipulating the underlying decision processes (RQ2). Previous discounting theory (Laibson, 1997) proposes that people are near-
intention-behavior gap research has concentrated mostly on the sighted in their assessments of risk and benefits, which means
existence of the phenomenon, influencing factors, and broad de- that they can be blinded by the immediate benefit of disclosing
scriptions of the underlying decision approach (e.g., privacy cal- their personal information despite the long-term “privacy benefit”
culus). Studying the decision process is arguably the best way to of withholding it.
understand the mechanism behind users’ privacy decision-making The second branch of “little to no risk assessment” theories ar-
practices and may offer an explanation for the gap between users’ gue that little or no risk assessment takes place in the privacy
intentions and their behavior. decision-making process. For example, people are said to ignore
risks because of a lack of privacy-related information or knowledge
(Harsanyi, 2004). Another study suggested that a reason for the
1.1. Existence and mechanism of the intention-behavior gap lack of risk assessment is that a state of learned helplessness can
emerge when a self-imposed privacy boundary that defines their
Norberg et al. (2007) were the first to demonstrate the privacy behavior is invaded so often that people feel like they no
intention-behavior gap in two experiments in commerce. They longer have any control over the situation (Shklovski et al., 2014).
asked the participants to share the same information twice (i.e., Although these existing theories, which mostly concern the
two phases) with a 12 (Study 1) or 7 (Study 2) weeks’ time-lag. concern-behavior gap, propose a diversity of potential mechanisms
Participants were asked to express their information sharing will- that might potentially also explain the intention-behavior gap,
ingness (intention) in the first phase but asked to engage in actual there are two remaining questions: The first question is about
personal information sharing (behavior) in the second phase. They which of the two opposing theories (risk-benefit assessment or
found that people are more likely to share personal information “little to no risk assessment”) best describes the approach indi-
than they intend to. In contrast, other studies were not able to viduals take in making privacy decisions about IoT. Using process
show an intention-behavior gap in the domain of social network tracing, we will test whether a risk-benefit calculation best pre-
sites SNSs (Dienlin and Trepte, 2015; Van Gool, Van Ouytsel, Pon- dicts the privacy decision, or whether the “little to no risk assess-
net, and Walrave, 2015) and mobile apps (Keith et al., 2013) These ment” theories apply instead. In case the risk-benefit calculation
studies found that information sharing intention is positively re- is a predictor of privacy decisions, the second question is whether
lated to information sharing behavior with considerably large ef- this risk-benefit calculation can explain the differences between in-
fect sizes Gerber et al. (2018). These mixed results warrant more tention and behavior. Specifically, we can test if the risk-benefit
research on the intention-behavior gap, specifically in the under- calculation mediates the effect of intention and behavior on the
researched domain of household IoT. privacy decision.
Furthermore, none of these earlier studies on the intention-
behavior gap tried to study the underlying mechanisms, although 1.2. Query theory and aspect listing
the study of Norberg et al. (2007) did investigate the moderating
effect of trust and risk. By measuring the underlying decision pro- To access the decision processes participants in our study go
cesses, we aim to better understand when and how the intention- through, we will use a process-tracing technique called aspect list-
behavior gap might or might not occur. Theories in general pri- ing based on the query theory approach (Johnson et al., 2007).
vacy decision-making offer a good starting point to theorize on In earlier research, query theory has explained and subsequently
how the underlying decision processes might differ. Barth and de been used to eliminate endowment effects (Johnson et al., 2007),
Jong (2017) reviewed and structured all theories that attempted to the asymmetry between recent and future reward in intertempo-
understand the mechanism of the privacy paradox, although they ral choice (Weber et al., 2007) and the effect of attribute framing
did not distinguish between the concern- and intention-behavior (Hardisty et al., 2010). The method was also employed to study pri-
gaps. Their review categorizes the theories into two branches: The vacy decision-making (Adjerid et al., 2016).
first branch of theories proposes an explicit risk-benefit calculation The query theory approach argues that participants decompose
and can be further divided based on whether this calculation is bi- (privacy) decisions (e.g., “Should I disclose my personal informa-
ased or unbiased. The second branch of theories argues that due tion?”) into a series of queries (e.g., “What is the benefit of dis-
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 3

closing my personal information?” or “What are the risks of dis- they engage in actual behavior than when they are asked to ex-
closing my personal information?”). These queries are asked, either press their intention (H1).
implicitly or explicitly, to oneself, in a particular order. The nat- Since people might lack knowledge about IoT, we need to in-
ural order in which people ask themselves these queries reflects troduce the technology to the participants before any decision
the relative importance of the components that the queries rep- tasks. However, such an introduction influences the level of risk
resent. For example, if the question “what is the benefit of dis- and benefits people might perceive. To study this influence, we
closing my personal information” comes to mind earlier than the use 3 different introductions, which emphasize different levels of
question “what are the concerns of disclosing my personal infor- risks and benefits. As the privacy calculus suggests that people are
mation”, this means that the benefits might outweigh the con- more likely to share when they perceive more benefits than risks
cerns for this user, in this situation. Specifically, earlier queries (Culnan and Armstrong, 1999), we hypothesize that the levels of
indicate better retrieval from memory than later queries (which risks and benefits in this introduction influence the sharing de-
might suffer from interference), and therefore these earlier queries cision (H2). Specifically, when the introduction contains more in-
arguably affect people’s preference and behavior more than their formation about benefits and less information about risks, partici-
later queries. For example, if the question “what is the benefit of pants are more likely to share, compared to the “risks-benefits bal-
disclosing my personal information” comes to mind earlier than anced” condition (H2.1). In contrast, when the introduction con-
the question “what are the concerns of disclosing my personal in- tains less information about benefits and more information about
formation”, this means that the benefits might outweigh the con- risks, participants are less likely to share, compared to the risks-
cerns for this user, in this situation. benefits balanced condition (H2.2).
Aspect listing can be employed as a method to measure deci- Aspect listing involves asking people for reasons to share or not
sion queries. With aspect listing, individuals are required to list the to share, which is a measurement of the underlying decision pro-
aspects they consider for (privacy) decision-making before mak- cess. Hence, if the risk-benefit calculation branch of theories ap-
ing a decision. The content, order, and time-latency of the as- plies (biased or not), those who list more benefits are more likely
pects that are listed can show us what queries are being asked to share, while those who list more risks are less likely to share.
in the mind during the privacy decision-making process. Earlier On the other hand, if there is no relation between the aspects
research in other decision-making domains (Hardisty et al., 2010; listed and participants’ sharing decisions, the branch of little to
Johnson et al., 2007; Weber et al., 2007) has supported this in- no risk assessment theories is more likely to apply. In addition,
ference. For example, in a study by Johnson et al. (2007), people the temporal position of the aspect listing (i.e. asking people for
were asked to value a mug either as a buyer or a seller and listed reasons to share or not to share before or after the sharing de-
the “reasons why you would want to either have the mug or have cision) has an influence on the willingness-to-share. Specifically,
the money”. In this unguided natural condition, the sellers listed Adjerid et al. (2016) found that people are less willing to share
more value-increasing aspects and valued the mug higher than the when they are asked for reasons to share or not to share before
buyers. However, when the sellers were specifically asked to list (rather than after) they make the decision. We hypothesize that
value-decreasing aspects before value-increasing aspects, and buy- people are less likely to share when they are asked to express their
ers were asked to do the opposite, both sellers and buyers listed intention than when they are asked to engage in behavior, and this
about the same number of value-decreasing and value-increasing effect is stronger when people are asked for reasons to share or not
aspects, and buyers and sellers no longer showed any difference in to share before than after they make the decision (H3).
the estimated value of the mug. Therefore, by manipulating the or-
der of aspect listing, the strength of the underlying effects can be 2.1. Study design
influenced. In our case, we also expect the privacy decision to be
influenced by asking users to list “reasons to share” before or after A 3 (introduction: risk-focused, neutral, or benefit-focused) x 2
listing “reasons not to share”. (decision type: intention or behavior) x 2 (aspect listing position:
Specific to privacy, Adjerid et al. (2016) used aspect listing to before or after the main task) between-subjects design was em-
understand the impact of trust and the temporal position of the ployed in Study 1. Regarding the decision type manipulation, fol-
aspect listing, i.e. asking users for reasons either before or after the lowing previous research (e.g., Norberg et al., 2007), we used a sur-
sharing decision, on privacy decision-making. They illustrated that vey (Fig. 1 left) to measure the intention of sharing (measured on a
people are less willing to share when they are asked for reasons to 4-point scale) in the intention condition. In the behavior condition,
share or not to share before (rather than after) they make the de- we used a mock-up interface to measure behavior as realistically as
cision. More importantly, they showed that people were more will- possible (measured as a dichotomous variable, Fig. 1 right).
ing to share with research organizations (low risk) than marketing In the behavior condition, we asked the participants to make a
organizations (high risk) and that more aspects in favor of shar- yes-or-no decision in the mock-up interface using forced-choice ra-
ing with the former rather than the latter. However, they did not dio buttons (see Appendix A for all disclosure decisions). In the in-
investigate the intention-behavior gap; they measured information tention condition, we used a more granular scale (as is common in
sharing behavior but did not measure intention. They also did not a regular survey): we asked the participants to rate the statement
change the order of aspect listing by separately asking for reasons “I would share the data from my [device] with [party] for [rea-
to share or not, nor did they test whether the number of aspects son]” on a 4-point scale (“Strongly disagree”, “Disagree”, “Agree”
mediates the effect of risk level on sharing—an analysis that is cru- and “Strongly agree”) without a midpoint.
cial to gain a complete understanding of the mechanisms underly- While prior studies (e.g., Norberg et al., 2007) used a within-
ing the sharing decision. subject design, a between-subjects design is more suitable for the
nature of our study. Most importantly, Norberg et al. assume that
the intention and behavior phases of their study are independent—
2. Study 1 an assumption that is likely valid, given that they devised a gap of
7 to 12 weeks between the first (intention) and second (behav-
In this study, we first investigate the existence of the intention- ior) measurement—but their study design cannot rule out poten-
behavior gap in IoT. As the intention-behavior gap has been ob- tial demand characteristic-like spill-over effects since they could
served in other domains, we expect that it exists in IoT as well. not (realistically) counterbalance the order of presentation. Our
Hence, we hypothesize that people are more likely to share when between-subjects design rules out spill-over effects and also makes
4 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

Fig. 1. Screenshots of the measurements of the dependent variables in intention via a survey (left) and behavior via a mock-up interface (right).

the study more manageable; a within design would have been a through some general settings”. While in the intention group, the
large burden on the participants as we also needed to measure the participant was asked to “answer some statements about three In-
aspect listing for their decision. ternet of Things devices, namely a Smart Camera, a Smart Assis-
Another complicating factor is the fact that we aim to take the tant, and a Smart Washing machine”.
contextual nature of privacy decision-making into account. Sim- At the end of the study, we asked participants to self-code their
ilarly, previous studies showed that type of device, data storage aspects (cf. Johnson et al., 2007) as being risk or benefit of sharing,
principle, and purpose of sharing have an impact on the sharing followed by some demographic questions to complete their partic-
decision (Bahirat et al., 2018; He et al., 2019). Hence, to control for ipation. Fig. 3 shows the complete study design.
these potentially influencing variables and to have a more precise
measurement of sharing behavior/intention, we asked each partic- 2.3. Participants
ipant to make multiple decisions, for combinations of 3 devices
(a smart assistant, a smart security camera, and a smart washing Power analysis showed that 128 participants would be required
machine), 2 data storage principles (store data on the company’s to be able to show medium main effects of size f = 0.25 or
server versus also sharing it with a third-party), and 3 purposes larger (our effect size of interest) with a power of 0.8. In total,
(use data to automate operations, optimize the service, or provide 171 participants were recruited from our university’s online sub-
insights in the user’s own behavior). ject pool. After removing participants with invalid or incomplete
data, data from 138 participants were analyzed (75 males, 63 fe-
2.2. Study procedure males; Meanage : 24.45, SDage : 5.42). Twenty-three participants were
removed due to at least one of the following reasons: (1) a du-
Following the informed consent procedure, participants were plicated IP address; (2) incomplete data; (3) not watching the full
asked to fill out a questionnaire regarding their prior knowledge introduction video; (4) staying with the page for the script of the
of IoT. After that, a video introduced the concept of household IoT video for less than 10 s after getting no or only one correct an-
to the participants. Depending on the introduction condition, this swer out of 3 knowledge test questions. In addition, ten partici-
video either presented a balanced view or put additional emphasis pants were removed because they did not provide any aspects in
on either the benefits or the risks of household IoT. If participants the aspect listing task. Aspects of 31 participants were split up be-
failed a subsequent general IoT quiz, they were instructed to read cause they entered all aspects instead of one aspect at a time.
the script of the video. Next, participants were sent to a task intro- Overall, participants were not very knowledgeable about IoT
duction, where they learned about the smart camera, smart assis- (seven-point scale from −3 to 3: Median = −0.40, Mean = −0.43,
tant, and smart washing machine that constituted the household SD = 0.46) and most were quite unfamiliar with IoT (five-point
IoT devices in our study. scale from 1 to 5: Median = 1.00, Mean = 1.28, SD = 0.37). Table 1
All participants were asked to list aspects of why they would shows the sample size in each condition. While sample sizes of in-
or would not share their data (Fig. 2). This allows us to measure dividual conditions are small, comparisons across main effects have
participants’ relative focus on benefits vs. risks in their decision- a sample size of around 40 to 70 per group, with two-way interac-
making process, expressed in the variables “benefit superiority” tions at a sample size of around 20 to 30 per group.
and “first aspect”, which are further investigated in Section 2.4.3.
Depending on the aspect listing position condition, people made 2.4. Results
the sharing decisions before or after the aspect listing.
We followed previous studies (e.g., Bahirat et al., 2018; He et al., 2.4.1. The effects of manipulations on sharing decisions
2019) and provided the participant a realistic decision scenario To test the intention-behavior gap and the other hypotheses
in the introduction of the sharing decisions, which depended on regarding the introduction risk level and aspect listing position,
the condition. For the behavior group, the participant was asked we first run a multilevel logistic regression analysis clustered by
to “imagine that you have just bought some Internet of Things participants predicting sharing likelihood as a function of the de-
devices, namely a Smart Camera, a Smart Assistant, and a Smart cision type (centered by grand-mean), involving risk level of in-
Washing Machine. Before using these devices, you have to go troduction (dummy coded with neutral as baseline), aspect list-
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 5

Fig. 2. Aspect listing instruction and interface. The figure shows the aspect listing interface after a participant input two aspects.

Fig. 3. Study 1 design and procedure flowchart.

Table 1
The number of participants in all conditions in Study 1.

Introduction Aspect listing before task Aspect listing after task Total

Behavior Intention Behavior Intention

Benefit 16 8 10 14 48
Neutral 12 5 6 15 38
Risk 6 18 14 14 52
Aspect listing position sub-total 65 73 138
Decision type sub-total Behavior: 64 Intention: 74

ing position (centered by grand-mean), all two-way interactions, to match the setting “on” in behavior condition, while the “dis-
and a three-way interaction (full model in Table 2). We also tested agree” and “strongly disagree” are considered as “off”. As there is
for moderations and main effects of the demographic variables no midpoint in the ordinal scale, splitting in the middle is the nat-
and knowledge and level of familiarity with IoT, but we did not ural transformation.
find any consistently significant moderations, so we left these Fig. 4 shows the reverse intention-behavior gap: Across all other
out of the models in Table 2. As all the interactions are non- conditions, participants are less (rather than more) likely to share
significant, we discuss our results below based on the cleaned- their personal information in behavior than in intention (odds ra-
up model that excludes all interactions (see the cleaned model in tio = 0.32, p < 0.001), which is opposite to H1. Moreover, the as-
Table 2). pect listing position (before or after the decision; H2) and intro-
In our analysis, we transform the ordinal data in the inten- duction (risk, neutral, or benefit; H3) do not significantly affect
tion condition into binary data by splitting the 4-point scale in people’s decisions. The full model in Table 2 also shows none of
half. Specifically, the “agree” and “strongly agree” are considered the interaction effects are significant. This means that participants’
6 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

Table 2
Full results of multilevel logistic regression of aspect listing position, decision type and introduction on sharing
decision in Study 1.

Full model Cleaned model

Predictors Odds Ratios p Odds Ratios p

(Intercept) 0.42 0.003 0.40 0.001

Aspect listing position (after vs. before) 1.36 0.605 1.25 0.443
Decision type (behavior vs. intention) 0.31 0.053 0.32 <0.001
Benefit introduction 1.05 0.909 1.10 0.803
Risk introduction 0.93 0.856 0.90 0.766
Aspect listing position: Decision type 0.71 0.776
Aspect listing position: Benefit introduction 1.42 0.655
Aspect listing position: Risk introduction 0.83 0.807
Decision type: Benefit introduction 0.56 0.466
Decision type: Risk introduction 1.82 0.444
Aspect listing position: Decision type: Benefit introduction 1.05 0.977
Aspect listing position: Decision type: Risk introduction 3.71 0.401
Random Effects
σ2 3.29 3.29
τ 00 2.32 id 2.42 id
ICC 0.41 0.42
N 138 id 138 id
Observations 2484 2484
Marginal R2 / Conditional R2 0.068 / 0.454 0.055 / 0.455

Fig. 4. Grouped means of sharing probability by introduction, decision type, and aspect listing position. Error bars represent one standard error of the mean.

decisions were coherent, i.e., they were not significantly influenced Fig. 5 shows that across all conditions people are less likely to
by the aspect listing position nor the introduction. store and share their data to third-party than to the company’s
server (odds ratio = 0.07, p < 0.001). Device also has effects on
2.4.2. The effects of device and storage on sharing decisions the sharing decision, where people are more likely to share their
In this experiment, we measured sharing/intention per device smart washing machine’s data (odds ratio = 4.28, p < 0.001) and
and storage type separately. To explore the influence of device and less likely to share their smart camera’s data (odds ratio = 0.45,
storage on intention-behavior gap, we run a multilevel logistic re- p < 0.001), compared to share smart assistant’s data. The de-
gression analysis clustered by participants predicting sharing likeli- vice and storage also moderate the reverse intention-behavior gap,
hood as a function of the decision type (centered by grand-mean), where the reverse intention-behavior gap is stronger in smart cam-
device (dummy coded with smart assistant as baseline), storage era condition than smart assistant condition (odds ratio = 2.74,
(centered by grand-mean), all two-way interactions, and a three- p = 0.03), and company’s server than third-party (odds ratio = 0.07,
way interaction (Table 3). p < 0.001).
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 7

Table 3
Full results of multilevel logistic regression of decision type, storage, and device on shar-
ing decision in Study 1.

Sharing decision

Predictors Odds Ratios CI p

(Intercept) 0.21 0.13 – 0.33 <0.001

Decision type (behavior vs. intention) 0.16 0.06 – 0.41 <0.001
Storage (third-party vs. company’s server) 0.07 0.04 – 0.10 <0.001
Camera 0.45 0.33 – 0.62 <0.001
Washing machine 4.28 3.15 – 5.82 <0.001
Decision type: Storage 2.74 1.10 – 6.82 0.030
Decision type: Camera 2.46 1.29 – 4.69 0.006
Decision type: Washing machine 0.88 0.48 – 1.63 0.692
Storage: Camera 1.59 0.84 – 3.01 0.156
Storage: Washing machine 0.92 0.50 – 1.67 0.775
Decision type: storage: Camera 0.41 0.11 – 1.50 0.178
Decision type: storage: Washing machine 0.12 0.04 – 0.41 0.001
Random Effects
σ2 3.29
τ 00 id 5.50
ICC 0.63
N id 138
Observations 2484
Marginal R2 / Conditional R2 0.271 / 0.727

Fig. 5. Grouped means of sharing probability per decision type, storage, and device. Error bars represent one standard error of the mean.

Furthermore, we observe three-way interactions (camera: odds assistant and smart camera conditions, which limits the ability to
ratio = 0.41, p > 0.05; washing machine: odds ratio = 0.12, p interpret the three-way interaction and the differences found for
< 0.001). Specifically, simple effect analyses splitting devices (see these devices.
Appendix B for full result) show that when people make a shar-
ing decision for the smart washing machine’s data, the reverse 2.4.3. Process tracing and distinguishing theories
intention-behavior gap is stronger in third-party condition than Another goal of the study is to test which theory applies
in company’s server condition (odds ratio = 0.13, p = 0.018). In to the privacy decision in IoT, based on the decision processes
contrast, for the smart camera condition, no significant interac- as revealed by our aspect listing task. On average, participants
tion is found and for the smart assistant, the reverse intention- listed 2.94 aspects in total (Min = 1, Median = 3, SD = 1.95,
behavior gap is weaker in the third-party condition than in the Max = 10), showing that participants did take the aspect list-
company’s server condition (odds ratio = 4.81, p = 0.007). How- ing task seriously and that this data can be used for understand-
ever, Fig. 5 shows a floor effect for third-party storage in the smart ing the decision processes. The aspects listed by the participants
8 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

Table 4
Full results of multilevel logistic regression on sharing decision co-varied by the first
aspect in Study 1.

Sharing decision

Predictors Odds Ratios CI p

(Intercept) 0.41 0.31 – 0.54 <0.001

First aspect (benefit vs risk) 2.88 1.66 – 4.98 <0.001
Decision Type (behavior vs intention) 0.38 0.22 – 0.65 <0.001
First aspect ∗ Decision Type 4.25 1.41 – 12.79 0.010
Random Effects
σ2 3.29
τ 00 id 2.13
ICC 0.39
N id 138
Observations 2484
Marginal R2 / Conditional R2 0.114 / 0.462

Fig. 6. Moderation of the first aspect on the effect of decision type, based on mean sharing probability between decision type and first aspect listed. Error bars represent
one standard error of the mean.

were self-coded (cf. Johnson et al., 2007) as being a sharing risk tered by subjects (Table 5). Following Johnson et al. (2007), we use
or a sharing benefit. We follow query theory (Johnson et al., the difference in the number of benefits aspects versus the num-
2007) by using two indexes to trace the privacy decision process: ber of risks aspects as a measure of “benefit superiority”, where
the first aspect provided by the participants (“first aspect”), and a positive number refers to more benefit aspects listed than risks
the difference between the number of benefit and risk aspects aspects while a negative number indicates fewer benefit aspects
(“benefit superiority”). We would expect the likelihood of shar- listed than risks aspects.
ing to increase if benefits are listed first and/or more often than Table 5 shows that the benefit superiority predicts sharing deci-
risks. sions (odds ratio = 1.33, p < 0.001). Fig. 7 illustrates that the effect
We first predict sharing decision by decision type and the first of benefit superiority on sharing decisions in behavior is stronger
aspect provided with a multilevel logistic regression clustered by than intention, in line with the previous analysis for the first as-
the participant (Table 4). Fig. 6 shows that compared to those who pect listed. However, the interaction between the benefit superior-
come up with a risk aspect first, those who come up with a benefit ity and decision type is not significant.
aspect first are more likely to share (odds ratio = 2.88, p < 0.001).
In addition, the significant interaction between the decision type 2.5. Conclusion
and first aspect (odds ratio = 4.25, p = 0.010) suggests that those
who come up with a risk aspect first are more likely to experi- In Study 1, we examined the intention-behavior gap in IoT. Dif-
ence the reverse intention-behavior gap, disclosing much less for ferent from the previous studies that either found a null result
behavior than for intention. or intention-behavior gap, we find a reversed intention-behavior
Subsequently, we predict sharing decision by decision type and gap for IoT. Specifically, people are less likely to disclose their per-
the number of aspects with a multilevel logistic regression clus- sonal information when they are asked to engage in actual behav-
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 9

Table 5
Full results of multilevel logistic regression on sharing decision co-varied by the benefit
superiority (number of benefits vs. number of risks aspects) in Study 1.

Sharing decision

Predictors Odds Ratios CI p

(Intercept) 0.49 0.37 – 0.66 <0.001

Benefit Superiority 1.33 1.13 – 1.56 <0.001
Decision Type (behavior vs intention) 0.43 0.24 – 0.76 0.004
Benefit Superiority ∗ Decision Type 1.24 0.90 – 1.69 0.184
Random Effects
σ2 3.29
τ 00 id 2.16
ICC 0.40
N id 138
Observations 2484
Marginal R2 / Conditional R2 0.110 / 0.462

Fig. 7. Moderation of the benefit superiority on the effect of decision type, based on estimated model effects. gray areas represent the 95% confidence interval.

ior than when they are asked to express their intention, which 3. Study 2
is opposite to H1. Moreover, the levels of risks and benefits in
the introduction and the aspect listing position do not influence Although Study 1 showed that the decision process (measured
the sharing decision (H2 and H3), nor do they moderate the ef- by the first aspect or the benefit superiority) predicts the sharing
fect of decision type on sharing decision. The reverse intention- decision, the evidence of the study is correlational, and the effect
behavior gap is contextual, where the gap is larger in smart cam- could also have another source. To better understand how the risk-
era condition than in smart assistant condition and smart wash- benefit calculation influences the decision, we implement the same
ing machine condition. Regarding storage, people are more likely to order-manipulation task as Johnson et al. (2007). Their study illus-
store and share their information with the company’s server than trated that specifically asking participants to list one side of the
a third-party and we find some evidence that the reversed gap aspects (in our case: risk or benefit) before the other was able to
is larger for third-party storage, at least for the washing machine influence the number of aspects listed of each type, and ultimately,
scenario. the decision itself. Hence, in Study 2, we manipulate the order of
Furthermore, we used aspect listing to distinguish privacy deci- the aspects, where we hypothesize to influence the aspects con-
sion theories. We find that benefit superiority and the first listed sidered by the users by means of the benefits superiority measure
aspect predict the privacy decision, which suggests that the branch (which is positive if more benefits than risks are listed) and subse-
of risk-benefit calculation theories (rather than little or no risk as- quently influence the sharing decision.
sessment) seem to apply to privacy decisions in the context of IoT. Specifically, we propose a moderated mediation model (Fig. 8)
Specifically, when people make an IoT privacy decision, they are and hypothesize that:
more likely to share in case benefits dominate in the amount or
the order in which they mentally process these risks and benefits, H4: Those who are in the free aspect listing group (i.e., aspect
which suggests some calculation or tradeoffs of the risks and ben- listing order not manipulated) are more likely to disclose
efits occur mentally before making the decision. their personal information when asked to express their in-
10 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

1. Study 1 showed that neither the decision process nor the final
decision was influenced by the risk level of the introduction.
Hence, in Study 2 all participants were shown the same neutral
IoT introduction video.
2. Based on the comments from participants and careful inspec-
tion in Study 1, we rephrased the 7 decisions into 5 more real-
istic sharing options/survey questions.
3. Participants in the intention group completed the questionnaire
on a 2-point scale instead of a 4-point scale, to keep the scales
on the same granularity level for maximal comparability be-
tween intention and behavior.

3.2. The procedure of the study

Fig. 8. A proposed moderated mediation model for study 2. The effect of decision Fig. 9 shows the procedure of Study 2 which is similar to Study
type on the (mediating) benefit superiority and the decision to disclose is moder- 1 except for two changes.
ated by aspect listing order.
1. In study 1, participants who listed aspects before the main task
did not yet know what the task would be, so their aspect listing
tention than when asked to engage in actual behavior (i.e., could not be influenced by different considerations caused by
reverse intention-behavior gap as in study 1 that also em- the decision type (intention versus behavior). Hence, in Study 2,
ployed a free aspect listing task), which is mediated by ben- participants were told which sort of task (behavior or intention)
efit superiority; they would do after the IoT introduction but before aspect list-
H5: Those who are asked to list benefit aspects first (i.e., be- ing. Specifically, in the intention condition they were instructed
fore risk aspects) generate more benefits aspects than risks that they would fill in a survey for research purposes, while in
aspects (higher benefit superiority), compared to the free the free aspect listing condition they were instructed to com-
aspect listing condition, and the higher benefit superiority plete some IoT devices settings in a panel for their new house.
subsequently increases the likelihood of sharing; 2. Three different query orders were applied. Participants who
H6: Those who are asked to list risk aspects first (i.e., before were in the risk first condition were asked to list reasons why
benefit aspects) generate more risk aspects than benefit as- they would not like to share their information (risk) followed
pects (lower benefit superiority), compared to the free as- by a request to list reasons why they would like to share their
pect listing condition, and the lower benefit superiority sub- information (benefit), while the order was reversed in the ben-
sequently decreases the likelihood of sharing; efit first condition. In the free listing condition (neutral condi-
H7: The aspect listing order manipulation moderates the effect tion), participants were free to list any aspects regarding why
of decision type on sharing decisions via benefit superiority. they would or not like to share their information in any order,
as in Study 1.
Furthermore, we explore how specifically the aspect listing or-
der moderates the effect of decision type on benefit superiority,
3.3. Participants
which subsequently influences the sharing decision.
Power analysis demonstrated that 158 participants were re-
3.1. Study design quired to be able to show the hypothesized interaction effects of
medium size (f = 0.25, or larger) with a power of 0.8. Two hundred
A 3 (aspect listing order: risk first, free aspect listing/control, or and forty-four participants were recruited from our university’s
benefit first) x 2 (decision type: intention or behavior) between- online subject pool or known affiliates. After removing partici-
subjects design was employed in Study 2. The study design was pants with invalid or incomplete data, data from 211 participants
similar to Study 1, except for the following changes. were analyzed (117 males, 92 females, 2 others; Meanage : 26.77,

Fig. 9. Study 2 design and procedure flowchart.

 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 11

Fig. 10. Mean sharing probability by aspect listing order and decision type, for different storage types (above) and devices (below). Error bars represent one standard error
of the mean.

Table 6 cant, we clean up the model by excluding all four-way and three-
The number of participants in all conditions in Study 2.
way interactions (see model 2 and model 3 in Appendix D for
Aspect orders Behavior Intention Total full regressions results). As we are only interested in the factors
Benefit first 44 30 74 that moderate the decision type, we clean up the model by re-
Free aspect listing 35 38 73 moving all the non-significant two-way interactions as well as
Risk first 27 37 64 all the two-way interactions that do not involve decision type.
Total 106 105 211 Appendix F shows that the moderation of the camera on the ef-
fect of decision type is due to the floor effects in the sharing rate
for the smart assistant and smart camera with third-party storage.
SDage = 12.41). Thirty-three participants were removed based on Hence, we also exclude these two interactions in our final model
the same criteria as in Study 1. Due to a coding error in build- (see model 4 in Appendix D for full regression results). The model
ing the online survey, the text of question 4 was a non-revised cleanup did not qualitatively change our effects of interest, which
earlier version that was inconsistent in wording with the other are the effects of decision type and their interactions with bene-
questions, making its interpretation ambiguous (see Appendix C). fits and risks and device and storage type. We discuss our results
Hence, we removed Question 4 in the analysis. Most of the partici- below based on model 4.
pants are neither knowledgeable about IoT (seven-point scale from Fig. 10 shows that participants who are requested to list ben-
−3 to 3: Median = −0.40, Mean = −0.43, SD = 0.46) nor familiar efits first in general are more likely to share (odds ratio = 1.92,
with IoT (five-point scale from 1 to 5: Median = 1.00, Mean = 1.31, p = 0.040) compared to the free aspect listing condition. However,
SD = 0.45). Table 6 shows the sample size in each condition. surprisingly, those who are in the risk first condition are also more
(rather than less) likely to share (odds ratio = 2.47, p = 0.006) than
3.4. Results the free aspect listing condition. Across all aspect order conditions,
participants who are asked to engage in actual behavior are less
3.4.1. The effects on sharing decisions likely to share than those who are asked to express their intention
Before we test the hypotheses, in this section, we first inspect to share, although this effect is only marginally significant (odds
the effect of decision type and aspect order on sharing decisions. ratio = 0.43, p = 0.059). The result of a simple effect analysis for
We run a multilevel logistic regression clustered by participants, the free aspect listing condition shows a small overall intention-
predicting sharing decision by decision type (centered by grand behavior gap (odds ratio = 0.40, p = 0.055; Appendix E shows the
mean), aspect orders (dummy coded with free aspect listing as full results of the simple effect analysis) but the strength differs by
baseline), storage (centered by grand mean), device (dummy coded storage type (odds ratio = 0.14, p<0.001).
with smart assistant as baseline) and full interactions (see model 1 Compared to the free aspect listing condition, the interaction
in Appendix D for full regression results). We also tested for mod- effect of decision type and the benefit first condition (odds ra-
eration by knowledge/familiarity with IoT but no moderation ef- tio = 5.70, p = 0.006) and of decision type and the risk first con-
fects were significant, so we did not include these effects in our dition (odds ratio = 7.28, p = 0.003) on sharing decision are signif-
models. As all four-way and three-way interactions are not signifi- icant. As Fig. 10 shows, in the free aspect listing condition, partici-
12 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

Fig. 11. The effects of aspect listing order and decision type on the benefit superiority. Error bars represent one standard error of the mean.

pants disclosed less information in the behavior than in the inten- Table 7
Full results of the linear regression of decision type and aspect
tion condition, which is a replication of reverse intention-behavior
orders on benefit superiority in Study 2.
gap found in Study 1. However, this reverse intention-behavior gap
vanishes when people are asked to list both risks and benefits, re- Benefit superiority
gardless of the order. Predictors Estimates p
The regression result shows that across all conditions people (Intercept) –0.69 <0.001
are more likely to share personal information in smart washing Decision type (behavior vs intention) –0.94 0.004
machine condition than in smart assistant (odds ratio = 2.25, p Benefit first 0.98 <0.001
< 0.001), and in smart assistant condition than in smart camera Risk first 0.21 0.386
Decision type: Benefit first 0.46 0.312
condition (odds ratio = 0.38, p < 0.001). Regarding storage, in line
Decision type: Risk first 1.04 0.028
with Study 1, people are more likely to store and share information Observations 211
with the company’s server than third-party (odds ratio = 0.08, p <
0.001). Beyond the influence on sharing decision, storage also mod-
erates the intention-behavior gap (odds ratio = 0.17, p < 0.001),
where the regular intention-behavior gap is larger in the com- pect listing as baseline) and their interactions using linear regres-
pany’s server than in third-party across all conditions, although as sion. As the normality assumption of linear regression is not met,
in study 1, the floor effects for assistant and camera makes the in- we run a robust regression with the bisquare estimation method
terpretation somewhat harder.3 (Table 7).
Fig. 11 shows that people have a lower benefit superiority
3.4.2. Process tracing and distinguishing theories in behavior than intention across all conditions (β = −0.94,
To further understand how aspect listing order moderates the p = 0.004), which means the domination of benefit over risk in
intention-behavior gap, we test the decision process by means of behavior is weaker than in intention. Fig. 11 also shows that peo-
the listed aspects. As in study 1, we would expect the benefit su- ple in benefit first condition and risk first condition have a higher
periority to be higher in intention than behavior in the free as- (β = 0.98, p < 0.001) benefit superiority, though the main ef-
pect listing condition, while the benefit superiority for the other fect of risk first on benefit superiority is not significant (β = 0.21,
two orders will depend on benefits first (higher than free) or risk p > 0.05). Furthermore, the aspect orders moderate the effect of
first (lower than free) as H5 and H6 predict. On average, partici- decision type on benefit superiority (moderation of benefit first:
pants listed 4.55 aspects in total (Min = 1, Median = 4, SD = 1.82, β = 0.46, p > 0.05; moderation of risk first: β = 1.04, p = 0.028).
Max = 10). We predict benefit superiority by decision type (cen- Simple effect analyses splitting aspect orders (see Appendix G for
tered by grand mean), aspect orders (dummy coded with free as- full result) show that in free aspect listing condition benefit supe-
riority is lower in behavior than intention (β = −1.26, p = 0.009),
3
while benefit superiority in behavior is not significantly different
The models did not show significant three- or four-way interactions. This means
that small deviations of the data such as the (surprising) larger sharing for behavior
from in intention for the other manipulated orders. The results
than intention in the free aspect listing of the washing machine (Appendix F) are above show that the effects of the decision type and aspect or-
not corroborated with significant effects in Model 2 or 3. ders on benefit superiority are generally in line with on sharing
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 13

Fig. 12. The multilevel moderated mediation model and the estimates of path coefficients. The numbers in brackets stand for standard error. Dashed and solid lines stand
for non-significant and significant direct paths, respectively. ∗ p<05, ∗ ∗ p<0.01, ∗ ∗ ∗ p<0.001.

decisions, which suggests a mediation of decision type and aspect 3.5. Conclusion
orders on sharing decisions via benefit superiority, as was hypoth-
esized. In Study 2, we manipulated the order of the aspect listing to
We subsequently create a moderated mediation model to fur- test the causal relationship between the risk-benefit calculation
ther test our hypotheses. Fig. 12 shows the paths and the estimates and the sharing decision. In the free aspect listing condition (con-
of path coefficients of the multilevel moderated mediation model, trol group) we observe an indirect effect of decision type on shar-
clustered by participants (see Appendix H for indirect and total ef- ing decision via benefit superiority: participants who are asked to
fects). In the model, decision type is centered by the grand mean express their intention have a higher benefit superiority (i.e., think
(behavior versus intention). more benefit than risk) than the participants who are asked to en-
Fig. 12 shows that in free aspect listing, decision type directly gage in actual behavior, which sequentially leads to higher sharing
affects benefit superiority negatively (β = −1.385, p = 0.002). probability in intention than in behavior.
Moreover, benefit superiority is positively related to sharing Compared to the free aspect listing group, those who are asked
(β = 0.429, p < 0.001). In effect, users who are asked to engage to list benefits aspects first have a higher benefit superiority (al-
in actual behavior are less likely to list benefits aspects and ulti- though only marginally significant) and those who are asked to list
mately less likely to share than users who are asked to express the risk aspects first also surprisingly have a higher benefit superi-
their intention (i.e., the reverse intention-behavior gap, the indi- ority. This last result might be due to the fact that in this condition
rect effect of decision type on sharing is significant: β = −0.595, they are asked to also list benefits (after listing risks), whereas in
p = 0.010), and this result is in line with our hypothesis (H4). the free aspect listing they are not asked specifically to think about
Regarding the aspect listing order manipulation, Fig. 12 shows either and therefore mostly lists risks, especially for behavior (see
that requesting benefit aspects first, compared to the free as- Fig. 11). The higher benefit superiority in the manipulated order
pect listing condition, directly affects benefit superiority positively conditions sequentially increases the likelihood of sharing.
(β = 1.132, p < 0.001). Given the positive influence of benefit Furthermore, the effect of decision type on benefit superiority
superiority on sharing decisions, the benefit-first condition has a is smaller in both the benefit-first and risk-first conditions than in
positive indirect effect on sharing decisions via benefit superiority the free aspect listing condition. These results suggest that list-
(β = 0.486, p = 0.001), as hypothesized (H5). Opposite to our hy- ing both the risks and benefits of sharing, regardless of the or-
pothesis (H6), the risk-first condition has a positive (rather than der, makes people think more about both risks and benefits and
negative) overall effect on sharing decision (β = 0.698, p = 0.009), increases the benefit superiority in behavior, which subsequently
although the indirect effect via benefit superiority is not significant turns the reverse intention-behavior gap into a regular intention-
while the direct effect is significant (β = 0.543, p = 0.030). behavior gap.
Regarding the moderation on the effect of decision type (H7),
the benefit-first (β = 1.398, p = 0.015) and risk-first manipula- 4. Discussion
tions (β = 1.570, p = 0.003) reduce the influence of decision type
on sharing decision. These overall reductions on the effect of de- Two questions remain for future research. First of all, our stud-
cision type are contributed by the marginal indirect moderation of ies show that decision type influences benefit superiority, which
benefit-first (β = 0.433, p = 0.077) and significant indirect mod- subsequently influences the privacy decision. However, it is not
eration of risk-first (β = 0.636, p = 0.012) via benefit superiority. clear why people who express an intention and people who engage
As Fig. 10 shows, the sharing rates in the intention condition are in actual behavior think differently in the first place. Similarly, it is
similar across all aspect listing order conditions, while the shar- not clear why people think differently about privacy in IoT, where
ing rate in behavior condition is higher in both the benefit-first we find a reverse intention-behavior gap, compared to commerce,
and risk-first groups than in the free aspect listing group, which where Norberg et al. (2007) found an intention-behavior gap.
turns the reverse intention-behavior gap into a regular intention- One possible explanation of the reversed intention-behavior gap
behavior gap in these conditions. is that participants lack knowledge in IoT, especially compared
14 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

to commerce which has been exposed to the participants in real the sharing decision. The mediation analysis result demonstrates
life for a long time. Bandara et al. (2017) argued that knowl- that the risk-benefit calculation fully mediates the effect of deci-
edge deficiency can influence people’s decisions directly on the sion type (i.e. intention and behavior) on sharing, which supports
side of the risk and benefit tradeoff. For example, lack of knowl- the branch of risk-benefit calculation theories underlying IoT pri-
edge prevents people from calculating risks and benefits properly vacy decision-making.
(Bandara et al., 2017) by underestimating the likelihood of privacy One of the major contributions of this paper is that we used
violations (Acquisti and Grossklags, 2005) and ignoring risks and process tracing to distinguish theories, where privacy calculus ap-
prefer more benefits (Deuker, 2009). However, this is opposite to plies to IoT privacy sharing decision, and to unpack why we ob-
what we observed in two studies in IoT that people overestimate served the reversed intention-behavior gap, as people lack benefit
risks (rather than underestimate risks). As IoT is new to the pub- thinking in behavior if not being reminded about thinking about
lic, it is very likely that people lack knowledge of IoT, and the both risks and benefits.
demographic data in two studies also illustrate that they indeed This finding has societal relevance, as it shows that the gap be-
have very limited subjective knowledge and experience (familiar- tween intention and behavior can be bridged by reminding peo-
ity) with IoT. However, if we look closer to the aspect participants ple to think about both the risks and benefits of sharing. More
list, they specifically do not seem to come up with benefit aspects broadly speaking, this finding aligns with the recent body of
for the behavior task in the free aspect listing condition. The lim- work suggesting that system design should promote mindfulness
its of coming up with benefit aspects are not salient in intention (Thatcher et al., 2018; Zou et al., 2015). Bridging the intention-
condition or when they are queried explicitly for benefits and risks behavior gap allows people to make more consistent privacy de-
(risk or benefits first). We did test for moderation of this effect by cisions and to engage in explicit risk-benefit tradeoffs as has
familiarity and knowledge, as we would expect this effect to be been proposed in the privacy calculus literature (Culnan and Arm-
stronger for those having little knowledge about IoT, but no signif- strong, 1999). We recommend that household IoT providers give
icant moderation was observed. This might be because in general users ample guidance to critically reflect on the privacy decisions
most participants showed limited familiarity and knowledge with they make, rather than forcing them to make these decisions ad
IoT, so we might not have sufficient variance among our partici- hoc during the installation process. Future studies should focus
pants to show such moderation. The other potential explanation is more on how privacy decisions are made rather than what privacy
that people construe risk and benefit differently in different do- decisions are made in IoT, especially on a more contextual level to
mains. This concept of IoT is more abstract and less salient than understand the contextual effects we observed in these two stud-
commerce. This might also explain why the intention-behavior gap ies.
does occur when we ask people to systematically list both bene-
fits and risks (regardless of the order), making these more concrete Declaration of Competing Interest
and salient than when we ask them to list aspects freely.
Our studies also come with a limitation in that we simulated The authors declare the following financial interests/personal
the actual behavior using a mock-up interface. Although this inter- relationships which may be considered as potential competing in-
face was sufficiently distinct from the survey to observe a signif- terests: Alfred Kobsa, Yangyang He and Paritosh Bahirat involved in
icant effect of decision type on sharing decisions, users’ behavior this project.
in our mockup interface might still be different from their actual
behavior in daily life.
CRediT authorship contribution statement
5. Overall conclusion
Qizhang Sun: Conceptualization, Data curation, Formal analy-
In two studies, we test the intention-behavior gap in IoT and sis, Investigation, Methodology, Software, Validation, Visualization,
trace the decision process to further understand the mechanism Writing - original draft. Martijn C. Willemsen: Conceptualization,
of the phenomenon. Although the intention-behavior gap was ob- Funding acquisition, Methodology, Supervision, Writing - review &
served in commerce, our first study finds that in an IoT scenario, editing. Bart P. Knijnenburg: Conceptualization, Funding acquisi-
people share less information than they say they intend to in a tion, Methodology, Supervision, Writing - review & editing.
survey. More importantly, in both of our studies, we explore the
mechanism of the intention-behavior gap by using aspect listing Acknowledgments
to trace the decision process, where we also distinguish between
two branches of theories that involve risk-benefit calculation and a This work was supported by the Netherlands Organization for
theory that argues that users cannot make such a calculation be- Scientific Research ( NWO ) project 628.001.027, the US National
cause they have little to no knowledge about risk. We find that the Science Foundation (NSF) award no. 1640664 “Using Process Trac-
reason for the reverse intention-behavior gap is that the difference ing to Improve Household IoT Users’ Privacy Decisions”, and Sam-
between the number of benefits aspects versus the number of risks sung Research America. We thank Paritosh Bahirat and Yangyang
aspects people consider, is larger when they are asked to express He for the discussion on the paper. We thank Lars Middel, Pepijn
their intention than when they are asked to engage in actual be- Schnitzeler, Ken Geelhoed, Juliët van Gent, Anouk van Kasteren and
havior. This result is supported by a mediation analysis that tests Raghav Mohan for their help in setting up to the studies and data
the combined causal effect of decision type and aspect order on collection.
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 15

Appendix A. Privacy decisions in Study 1

Behavior condition Settings (binary)

Store my data remotely on the company’s server

Let it automate its operations Yes - No
Let it optimize the service Yes - No
Let it give you insight into your behavior Yes - No
Share and store my data on the server of a third party
Let it automate its operations Yes - No
Let it optimize the service Yes - No
Let it give you insight into your behavior Yes - No
Let it recommend you other services Yes - No

Intention condition Options (4-point scale)

∗ ∗
I would share my data from my device with the company’s server to automate its operations Strongly agree - strongly disagree
∗
I would share my data from my device∗ with the company’s server to optimize the service Strongly agree - strongly disagree
∗
I would share my data from my device∗ with the company’s server to give me insight into my behavior Strongly agree - strongly disagree
∗
I would share my data from my device∗ with third parties to automate its operations Strongly agree - strongly disagree
∗
I would share my data from my device∗ with third parties to optimize the service Strongly agree - strongly disagree
∗
I would share my data from my device∗ with third parties to give me insight into my behavior Strongly agree - strongly disagree

Appendix B. Results of multilevel logistic regression on sharing decision predicted by decision type and storage, clustered by
participants in Study 1

Sharing decision with assistant Sharing decision with camera Sharing decision with washing machine

Predictors Odds Ratios p Odds Ratios p Odds Ratios p

(Intercept) 0.15 <0.001 0.03 <0.001 0.94 0.890

Decision type (behavior vs. intention) 0.13 0.002 0.40 0.232 0.02 <0.001
Storage (third-party vs. company’s server) 0.03 <0.001 0.06 <0.001 0.01 <0.001
Decision type: Storage 4.81 0.007 1.12 0.858 0.13 0.018
Random Effects
σ2 3.29 3.29 3.29
τ 00 9.09 id 12.04 id 22.36 id
ICC 0.73 0.79 0.87
N 138 id 138 id 138 id
Observations 828 828 828
Marginal R2 / Conditional R2 0.248 / 0.800 0.131 / 0.814 0.269 / 0.906

Appendix C. Privacy decisions in Study 2

Behavior condition Settings (binary)

Store my data remotely on the company’s server

Let it use my data to adapt to my needs and preferences and to improve the service. Yes - No
Let it use my data to give me insight into my behavior. Yes - No
Share and store my data on the server of a third party
Let third parties use my data for further automation, added functionality and improved service. Yes - No
Allow third parties to provide further insight into my behavior. Yes - No
Let third parties use my data to recommend me other services or products. Yes - No

Intention condition Options (binary)

I would share my data from my ∗ device∗ with the company’s server to let it use my data to adapt to my needs and Disagree - Agree
preferences and to improve the service.
I would share my data from my ∗ device∗ with the company’s server to give me insight into my behavior. Disagree - Agree
I would share my data from my ∗ device∗ with third parties for further automation, added functionality and improved Disagree - Agree
service.
I would allow interconnected devices to use my data to give me insight into my behavior. Disagree - Agree
I would share my data from my ∗ device∗ with third parties to let it recommend me other services or products. Disagree - Agree
16 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

Appendix D. Results of all multilevel logistic regressions on sharing decision in Study 2

Model 1 Model 2 Model 3 Model 4

Predictors Odds Ratios p Odds Ratios p Odds Ratios p Odds Ratios p

(Intercept) 0.39 0.001 0.41 0.001 0.41 0.001 0.32 <0.001

Decision type (behavior vs intention) 0.42 0.106 0.43 0.108 0.41 0.063 0.43 0.059
Benefit first 1.42 0.350 1.41 0.350 1.44 0.320 1.92 0.040
Risk first 1.55 0.265 1.51 0.286 1.53 0.263 2.47 0.006
Storage (third-party vs company’s server) 0.05 <0.001 0.07 <0.001 0.08 <0.001 0.08 <0.001
Camera 0.41 <0.001 0.37 <0.001 0.37 <0.001 0.38 <0.001
Washing machine 1.13 0.629 1.12 0.620 1.16 0.508 2.25 <0.001
Decision type: Benefit first 6.85 0.011 6.35 0.012 5.54 0.007 5.70 0.006
Decision type: Risk first 6.43 0.018 6.10 0.020 7.84 0.002 7.28 0.003
Decision type: Storage 0.23 0.039 0.20 0.003 0.19 <0.001 0.17 <0.001
Benefit first: Storage 0.97 0.948 1.55 0.142 1.52 0.147
Risk first: Storage 0.95 0.928 0.72 0.339 0.82 0.522
Decision type: Camera 2.69 0.051 2.37 0.081 1.84 0.029
Decision type: Washing machine 0.43 0.085 0.45 0.081 0.68 0.152
Benefit first: Camera 1.15 0.684 1.22 0.552 1.12 0.733
Risk first: Camera 0.91 0.793 1.03 0.925 0.99 0.987
Benefit first: Washing machine 2.10 0.026 2.08 0.022 2.04 0.024
Risk first: Washing machine 3.93 <0.001 4.02 <0.001 3.65 <0.001
Storage: Camera 2.17 0.124
Storage: Washing machine 1.27 0.625
Decision type: Benefit first: storage 1.15 0.891 2.12 0.208
Decision type: Risk first: storage 0.54 0.550 0.39 0.161
Decision type: Benefit first: Camera 0.30 0.082 0.35 0.114
Decision type: Risk first: Camera 0.74 0.692 0.97 0.964
Decision type: Benefit first: Washing machine 1.76 0.397 1.82 0.347
Decision type: Risk first: Washing machine 1.98 0.343 2.09 0.293
Decision type: storage: Camera 0.44 0.415 0.36 0.074
Decision type: storage: Washing machine 0.88 0.899 1.51 0.440
Benefit first: storage: Camera 1.51 0.547
Risk first: storage: Camera 0.42 0.255
Benefit first: storage: Washing machine 2.41 0.186
Risk first: storage: Washing machine 0.95 0.940
Decision type: Benefit first: storage: Camera 1.90 0.640
Decision type: Risk first: storage: Camera 0.20 0.287
Decision type: Benefit first: storage: Washing machine 3.20 0.382
Decision type: Risk first: storage: Washing machine 1.49 0.781
Random Effects
σ2 3.29 3.29 3.29 3.29
τ 00 3.01 id 2.94 id 2.86 id 2.88 id
ICC 0.48 0.47 0.46 0.47
N 211 id 211 id 211 id 211 id
Observations 2532 2532 2532 2532
Marginal R2 / Conditional R2 0.310 / 0.640 0.310 / 0.636 0.303 / 0.627 0.296 / 0.625

Appendix E. Simple Effect Analysis in Free Aspect Listing condition in Study 2

Sharing decision

Predictors Odds Ratios CI p

(Intercept) 0.41 0.24 – 0.69 0.001

Decision type (behavior vs intention) 0.40 0.16 – 1.02 0.055
Storage (third-party vs company’s server) 0.08 0.05 – 0.12 <0.001
Camera 0.34 0.22 – 0.55 <0.001
Washing machine 1.22 0.79 – 1.88 0.377
Decision type: Storage 0.14 0.06 – 0.35 <0.001
Random Effects
σ2 3.29
τ 00 id 3.06
ICC 0.48
N id 73
Observations 876
Marginal R2 / Conditional R2 0.276 / 0.625
 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924 17

Appendix F. Mean sharing probability by aspect listing order and decision type, for different storage types and devices in Study 2.
Error bars represent one standard error of the mean

Appendix G. Simple effect analysis splitting aspect orders in Study 2

Benefit first Free aspect listing Risk first

Predictors Estimates p Estimates p Estimates p

(Intercept) 0.26 0.063 −0.77 0.001 −0.49 <0.001

Decision type (behavior vs intention) −0.50 0.073 −1.26 0.009 0.11 0.633
Observations 74 73 64

Appendix H. Indirect and total effect of multilevel moderated mediation model in Study 2

Indirect effect Total effect

Coefficient SE p Coefficient SE p

Decision type -0.595 0.231 0.01 −0.539 0.411 >0.05

Benefit first 0.486 0.142 0.001 0.501 0.285 >0.05
Risk First 0.155 0.110 >0.05 0.698 0.265 .009
Decision type: Benefit first 0.433 0.245 >0.05 1.398 0.576 0.015
Decision type: Risk First 0.636 0.252 .012 1.570 0.536 0.003
18 Q. Sun, M.C. Willemsen and B.P. Knijnenburg / Computers & Security 97 (2020) 101924

References
Keith, M.J., Thompson, S.C., Hale, J., Lowry, P.B., Greer, C., 2013. Information disclo-
sure on mobile devices: re-examining privacy calculus with actual user behav-
Acquisti, A., Grossklags, J., 2005. Privacy and rationality in individual decision mak-
ior. Int. J. Hum. Comput. Stud. 71 (12), 1163–1173. doi:10.1016/j.ijhcs.2013.08.
ing. IEEE Secur. Priv. 3 (1), 26–33. doi:10.1109/MSP.2005.22.
016.
Adjerid, I., Samat, S., Acquisti, A., 2016. A query-theory perspective of privacy deci-
Kelley, H.H., 1967. Attribution theory in social psychology. In: Nebraska Symposium
sion making. J. Legal Stud. 45 (S2), S97–S121. doi:10.1086/689599.
on Motivation, 15, pp. 192–238.
Ajzen, I., 1988. Attitudes, Personality, and Behavior. Keynes: Open University Press,
Kokolakis, S., 2017. Privacy attitudes and privacy behaviour: a review of current
Milton.
research on the privacy paradox phenomenon. Comput. Secur. 64, 122–134.
Ajzen, I., 1991. The theory of planned behavior. Organ. Behav. Hum. Decis. Process.
doi:10.1016/j.cose.2015.07.002.
50 (2), 179–211. doi:10.1016/0749- 5978(91)90020- T.
Laibson, D., 1997. Golden eggs and hyperbolic discounting. Q. J. Econ. 112 (2), 443–
Bahirat, P., Sun, Q., Knijnenburg, B.P., 2018. Scenario context v/s framing and defaults
478. doi:10.1162/003355397555253.
in managing privacy in household IoT. In: Proceedings of the 23rd International
Min, J., Kim, B., 2015. How are people enticed to disclose personal information de-
Conference on Intelligent User Interfaces Companion - IUI 18, pp. 1–2. doi:10.
spite privacy concerns in social network sites? The calculus between benefit
1145/3180308.3180372 New York, New York, USA: ACM Press.
and cost. J. Assoc. Inf. Sci. Technol. doi:10.1002/asi.23206.
Bandara, R., Fernando, M., Akter, S., 2017. The privacy paradox in the data-driven
Norberg, P.A., Horne, D.R., Horne, D.A., 2007. The privacy paradox : personal infor-
marketplace: the role of knowledge deficiency and psychological distance. Pro-
mation disclosure intentions versus behaviors. J. Consum. Aff. 41 (1), 100–126.
cedia Comput. Sci. 121, 562–567. doi:10.1016/J.PROCS.2017.11.074.
doi:10.1111/j.1083-6101.2009.01494.x.
Barth, S., de Jong, M.D.T., 2017. The privacy paradox – investigating discrepancies
Phelan, C., Lampe, C., Resnick, P., 2016. It’s creepy, but it doesn’t bother me. In: Pro-
between expressed privacy concerns and actual online behavior – a systematic
ceedings of the 2016 CHI Conference on Human Factors in Computing Systems,
literature review. Telemat. Inform. 34 (7), 1038–1058. doi:10.1016/j.tele.2017.04.
pp. 5372–5384. doi:10.1145/2858036.2858381.
013.
Rindfleisch, T.C., 1997. Privacy, information technology, and health care. Commun.
Culnan, M.J., Armstrong, P.K., 1999. Information privacy concerns, procedural fair-
ACM 40 (8), 92–100. doi:10.1145/257874.257896.
ness, and impersonal trust: an empirical investigation. Organ. Sci. 10 (1), 104–
Rogers, R.W., 1983. Cognitive and physiological processes in fear appeals and at-
115. doi:10.1287/orsc.10.1.104.
titude change: a revised theory of protection motivation. In: Cacioppo, B.L.,
Deuker, A., 2009. Addressing the privacy paradox by expanded privacy awareness
Petty, L.L. (Eds.), Social Psychophysiology: A Sourcebook. Guilford, London.
– The example of context-aware services. In: Privacy and Identity Management
Sheeran, P., 2002. Intention—behavior relations: a conceptual and empirical review.
for Life in IFIP Advances in Information and Communication Technology, 320,
Eur. Rev. Soc. Psychol. 12 (1), 1–36. doi:10.1080/147927721430 0 0 0 03.
pp. 275–283 Retrieved from.
Shklovski, I., Mainwaring, S.D., Skúladóttir, H.H., Borgthorsson, H., 2014. Leakiness
Dienlin, T., Trepte, S., 2015. Is the privacy paradox a relic of the past? An in-depth
and creepiness in app space. In: Proceedings of the 32nd Annual ACM Conference
analysis of privacy attitudes and privacy behaviors. Eur. J. Soc. Psychol. 45 (3),
on Human Factors in Computing Systems - CHI ’14, pp. 2347–2356. doi:10.1145/
285–297. doi:10.1002/ejsp.2049.
2556288.2557421.
Fishbein, M., 1980. A theory of reasoned action: some applications and implications.
Thatcher, J., Wright, R.T., Sun, H., Zagenczyk, T.J., Klein, R., 2018. Mindfulness in
In: Howe, H., Page, M. (Eds.), Nebraska Symposium on Motivation. University of
information technology use: definitions, distinctions, and a new measure. MIS
Nebraska Press, Lincoln, NB, pp. 65–116.
Quart. 42 (3), 831–847.
Fishbein, M., Ajzen, I., 1975. Belief, Attitude, Intention and Behavior: An Introduction
Triandis, H.C., 1980. Values, attitudes, and interpersonal behavior. In: Howe, H.,
to Theory and Research. Addison-Wesley, Reading, MA.
Page, M. (Eds.), Nebraska Symposium on Motivation. University of Nebraska
Gerber, N., Gerber, P., Volkamer, M., 2018. Explaining the privacy paradox: a sys-
Press, Lincoln, NB, pp. 195–259.
tematic review of literature investigating privacy attitude and behavior. Comput.
Tufekci, Z., 2008. Can you see me now? Audience and disclosure regulation in
Secur. 77, 226–261. doi:10.1016/J.COSE.2018.04.002.
online social network sites. Bull. Sci. Technol. Soc 28 (1), 20–36. doi:10.1177/
Hardisty, D.J., Johnson, E.J., Weber, E.U., 2010. A dirty word or a dirty world? Psychol.
0270467607311484.
Sci 21 (1), 86–92. doi:10.1177/0956797609355572.
Van Gool, E., Van Ouytsel, J., Ponnet, K., Walrave, M., 2015. To share or not to share?
Harsanyi, J.C., 2004. Games with incomplete information played by “Bayesian” Play-
Adolescents’ self-disclosure about peer relationships on Facebook: an applica-
ers, I–III. Manage. Sci. 50 (12 supplement), 1804–1817. doi:10.1287/mnsc.1040.
tion of the Prototype Willingness Model. Comput. Hum. Behav. 44, 230–239.
0270.
doi:10.1016/j.chb.2014.11.036.
He, Y., Bahirat, P., Knijnenburg, B.P., Menon, A., 2019. A data-driven approach to de-
Weber, E.U., Johnson, E.J., Milch, K.F., Chang, H., Brodscholl, J.C., Goldstein, D.G., 2007.
signing for privacy in household IoT. ACM Trans. Interact. Intell. Syst. doi:10.
Asymmetric discounting in intertemporal choice: a query-theory account. Psy-
1145/3241378.
chol. Sci. 18 (6), 516–524. doi:10.1111/j.1467-9280.2007.01932.x.
Johnson, E.J., Häubl, G., Keinan, A., 2007. Aspects of endowment: a query theory of
Zou, H.M., Sun, H., Fang, Y., 2015. Understanding post-adoption regret from the per-
value construction. J. Exp. Psychol. 33 (3), 461–474. doi:10.1037/0278-7393.33.3.
spectives of herding and mindfulness. Paper presented at the International Con-
461.
ference on Information Systems, Fort Worth, TX.