Journal of Korean Library and Information Science Society

Vol. 51, No. 2

Internet-Mediated Research in the Age of Social Distancing:

Methodological Reflections and Recommendations from
Two Online Research Projects*
사회적 거리두기 시대의 인터넷 기반 연구:
두 온라인 연구 프로젝트로부터의 방법론적 고찰과 제안점
Jisue Lee**
Jonathan M. Hollister***
<Contents>
Ⅰ. Introduction Ⅳ. How the Research Projects Addressed
Ⅱ. Literature Review Key Issues
Ⅲ. Methodology of Two Online Research Ⅴ. Practical Considerations & Useful
Projects Resources
Ⅵ. Conclusion & Next Steps
ABSTRACT: Due to the COVID-19 pandemic, scholars at institutions of higher education around the world
are transitioning their teaching, service, and research into online settings; for many this may be a new and
challenging experience. While many of the best practices associated with research in traditional or face-to-face
settings also apply to research in or via online settings, there are some additional challenges and nuances
that researchers must adequately address and plan for due to the affordances and limitations of online settings.
This paper discusses the key issues of privacy, informed consent, trust and trustworthiness, and retention
through the literature and provides practical recommendations based on evidence and experience from two
different online research projects. The reflections on and examples from these two research projects
contextualize the above issues and act as evidence to inform research as a practice. The authors hope this
evidence and practical guidance may help researchers better prepare for research in a socially distanced world.
KEYWORDS: Internet Research, Research as Practice, Ethics, Online Interviews, Ethnography
요 약: 코비드-19(코로나19)의 세계적 유행으로 인해, 전세계의 고등교육기관 학자들은 교육, 서비스, 연구를
위해 온라인 환경으로 대대적으로 이전하고 있으며, 많은 사람들에게 있어 이 시도는 새롭고 도전적인 경험이
되고 있다. 전통적인 대면 환경에서 이루어지는 연구의 모범실무 중 많은 것들이 온라인 환경 연구에도 똑같이
적용될 수 있지만, 온라인 연구 수행 시 온라인 환경의 특수한 지원성과 한계로 인해 연구자들이 철저하게 계획을
세우고 실행해야하는 추가적인 도전과 뉘앙스들이 존재한다. 이 연구는 문헌연구를 통해 사생활보호, 사전동의,
신뢰의 형성과 데이터의 신빙성, 그리고 참여자 보유와 같은 핵심적인 이슈들에 대해 토론하고, 두가지 온라인
연구의 경험과 증거에 기반한 실제적인 제안점을 제공하고자 한다. 두가지 온라인 연구에서 얻어진 고찰과 예시들
은 해당 이슈들을 실제 연구에 맥락화한 증거이자 실천으로 기능한다. 사회적 거리두기 시대에 온라인 연구를
고려하는 많은 연구자들에게 이 연구의 증거와 실제적인 가이드가 도움이 될 것으로 기대한다.
주제어: 인터넷 리서치, 실천으로서의 연구, 연구윤리, 온라인 인터뷰, 문화기술지

* The data reported here are parts of the authors’ respective dissertation projects at the School of
Information at Florida State University.
** Assistant Professor, Department of Library & Information Science, Chonnam National University
(jislee@jnu.ac.kr / ISNI-0000000474720115) (제1저자)
*** Assistant Professor, Department of Library, Archives, & Information Studies, Pusan National University
(hollisterjm@pusan.ac.kr / ISNI-000000048513885X) (교신저자)
∙ 논문접수: 2020년 5월 29일 ∙ 최초심사: 2020년 6월 3일 ∙ 게재확정: 2020년 6월 19일
∙ 한국도서관․정보학회지, 51(2), 319-353, 2020. [http://dx.doi.org/10.16981/kliss.51.2.202006.319]

- 319 -
한국도서관․정보학회지(제51권 제2호)

Ⅰ. Introduction
Due to the COVID-19 pandemic, scholars, and, arguably, most people, are adapting their
work in response to public health and wellbeing directives. In the higher education context,
faculty members and other educators are moving their meetings and, more importantly, their
courses online, a first for many. Public health guidelines, social distancing in particular, have
implications for conducting and publishing research. Scholars who have relied on more
traditional or face-to-face means of data collection may face new challenges as they venture
into online research. While many of the ethical issues and best practices associated with
research in face-to-face, offline settings also apply to online settings, the affordances and
limitations of internet-mediated research add nuance and create additional considerations that
researchers must be aware of and adequately address.
Online, internet, or internet-mediated research is not a new phenomenon and has been
increasingly popular since the early 1990s. Scholars across a wide variety of academic fields
and disciplines, such as Psychology, Sociology, Health and Medicine, History, Communication,
and many more, including Library and Information Science, have explored and described
human behaviors and activities in online spaces or by using internet technologies (Buchanan
2012, boyd and Ellison 2008, Brügger 2012, Chase and Alvarez 2000, Consalvo and Ess 2011,
Dalton 1991, Farrell 2010, Herring 2002, Jones 1999, Tuttas 2015, Zhang 2000). Researchers
have applied nearly all types of qualitative and quantitative research methods, including
mixed and multiple methods designs, such as surveys, social network analysis, focus groups,
interviews, observation, ethnographic methods, and others (Boellstorff et al. 2012, Consalvo
and Ess 2011, Fielding et al. 2017, Hunsinger et al. 2020, Murray and Sixsmith 1998, Farrell
and Petersen 2010). Online or internet-mediated research has explored and used a variety of
online spaces and technologies, including email, websites, messenger programs, virtual
conferencing (real-time online video-enabled software such as Go2Meeting, Zoom Video
Communication, Google Hangouts, and Skype) or Voice over Internet Protocol (VoIP)1),
virtual worlds, online games, social media, wikis, and more (Boellstorff et al. 2012, boyd and
Ellison 2008, Joinson, 2001, Fox et al. 2007, Murray and Sixsmith 1998, Tuttas 2015, Weller
2017, Matthews et al. 2018).
While some researchers in South Korean Public Administration conducted studies
examining methodological issues of using internet and SNS to interview surveys (Kwon and

1) For more information about technology selection discussed in terms of strengths and limitations of multiple
video conferencing technologies, refer to Tuttas (2015).

- 320 -
 Internet-Mediated Research in the Age of Social Distancing

Lee 2005, Kwon and Myong 2011,) and industrial design research analyzed the rigorousness
of conducting messenger interviews compared to face-to-face interviews (Choi 2016), a
keyword search for internet or online research in South Korean Library and Information
Science journals and databases suggests that online or internet-mediated research (such as
online interview, email interview, online focus groups, etc.) are not as common in Library
and Information Science (LIS) scholarship. Since the first qualitative research in the South
Korean LIS domain was published in 19942), more and more qualitative research investigating
users’ information behaviors, and users and librarians’ perceptions of librarians’ duties has
been published in major LIS journals3) in the 2000s and 2010s (Kim 2017). Kim (2017) found
out that among those 73 qualitative research from four major LIS journals from the 1970s to
2010s (1.4% of total 5,071 research publications), the most common data collection methods
included intensive interviews, focus groups, phenomenology, ethnography and case studies
and major analysis methods included grounded theory approach, content analysis, and
discourse analysis. However, Kim (2017) mentioned that most qualitative research had critical
issues regarding trustworthiness, such as specification of sampling techniques, saturation
point, informed consent, reliability testing, and member checking. Relatedly, Khoo et al.
(2012) found that while the use of ethnographic methods in LIS research in three major
library science databases is increasing, many researchers using ethnographic methods fail to
adequately describe or justify their approaches. Considering that qualitative research using
intensive individual and group interviews and observation based research such as
phenomenology and ethnography is increasing in the LIS field, this research expanding those
qualitative research into the various virtual settings of internet, social media, and game venues
provides the LIS researchers who plan to conduct online qualitative research with meaningful
findings and useful tips from the two empirical online research projects.
Todd’s (2015) framework for evidence-based practice recommends that practitioners inform
and better their work through three types of evidence: (1) evidence for practice, or
knowledge from existing scholarship; (2) evidence in practice, or that which is generated
through/by working; and (3) evidence of practice, or evidence gained from the results and

2) Kim and Chang’s research on the present stage of the development of public libraries in Korea - a
qualitative analysis of the consciousness of librarians in their everyday lives in Pusan-Kyoungnam area
public libraries was published as the first qualitative research in the journal of the Korean Society for
Library and Information Science in December, 1994.
3) The four major LIS journals refer to the Journal of the Korean Society for Library and Information Science,
the Journal of Korean Library and Information Science Society, the Journal of the Korean Biblia Society for
Library and Information Science, and the Journal of the Korean Society for Information Management.
Readers can find more detailed information including the analysis of qualitative research purpose, data
collection and analysis methods in Kim (2017).

- 321 -
한국도서관․정보학회지(제51권 제2호)

outcomes of said work. While Todd’s conceptualization of evidence-based practice is

presented in the context of school librarianship, the framework can also be applied to the
practice of research. When considering research as a practice, evidence for practice can be
seen as literature reviews, which provide the foundations of most research projects. Evidence
in and of practice may be partially addressed in the research design, limitations, results or
findings, or discussion sections of an academic manuscript. However, methodological
reflections can also provide evidence of all types: reflections on which methods, techniques,
or approaches work well or not, and why or why not, in a given research setting can provide
nuanced and contextualized evidence for how to overcome challenges and conduct research
more effectively in similar settings. As such, methodological reflections and evidence from
the field may supply useful guidance and practical advice for conducting research.
In this vein, this article, as many others have done (Fox et al. 2007, Janghorban et al. 2014,
Padayachee 2016, Sturges and Hanrahan 2004, Tuttas 2015, Weller 2017, Williams et al. 2012,
Zhang 2000), reflects on the processes and results of conducting research using various
methods across different contexts to provide useful information to other researchers. As
discussed below, while the usual ethical and methodological considerations for research in
more traditional, face-to-face contexts are also present in online or internet-based research,
there are additional considerations and challenges related to privacy, informed consent,
building rapport, trust, and trustworthiness, and retention, among others, that researchers must
be aware of and prepared for due to the affordances and limitations of online settings. This
article identifies and discusses key issues and provides recommendations for conducting online
research through a selected review of the literature and from evidence and methodological
reflections on two different online research projects from the authors. The goal of this article
is not to provide objective guidelines for online research, but to discuss ethical considerations
and challenges in context and to provide evidence, insights, and resources that may be useful
for other researchers interested in conducting online or internet-based research.

Ⅱ. Literature Review

1. Key Issues and Challenges in Online, Internet-Based Research

Research in online contexts shares many of the same issues and concerns with research
in offline settings. However, the technological affordances and limitations of online or

- 322 -
 Internet-Mediated Research in the Age of Social Distancing

internet-mediated research create additional considerations that must be adequately addressed

by the researcher. Of concern are the interrelated issues associated with privacy; informed
consent; building rapport, trust, and trustworthiness; and retention.

1.1 Privacy in Online Spaces

Privacy concerns the rights of individuals to control access to their information or
information about them (Sveningsson Elm 2009). In online spaces, beliefs and realities of
privacy and confidentiality may vary across different websites and online services. This variance
also positions private and public online spaces on a continuum rather than a dichotomy.
As Sveningsson Elm (2009) describes, public online spaces are open and freely accessible
to anyone with an internet connection, such as websites without sign-in or registration
requirements, open chats, or discussion boards. Semi-public online spaces are also open to
everyone but require account creation and registration to take part, such as most social
networking sites. Semi-private online environments are only accessible to those with
registration and membership within an association or organization that has its own
requirements, such as company intranets. Private online spaces are only accessible to the
creator of the space and those they have invited, such as private chats or direct messages.
Complicating this continuum is that many online spaces permit individual users to control
who can and cannot see their content or information (Sveningsson Elm 2009). For example,
users can lock their Twitter accounts so that only their followers can see their tweets and
Facebook users can limit the audience of their posts, photos, and activity to different groups
of users, such as friends only, friends of friends, excluding acquaintances, etc. Public posts
and pages on both social networks can also be viewed without a signed-in account.
Additionally, there is no guarantee that a private post or message will not be screenshot, copy
and pasted, saved, or shared without permission.
Relatedly, users may not understand the extent to which their posts or other online
information is public or private (Sveningsson Elm 2009). As such, social researchers that are
interested in collecting online data should consider both how accessible the data is and how
public or private users perceive it to be (Sveningsson Elm 2009). For example, posts on
Twitter can be seen publicly without an account or membership on the platform, and as such,
many studies on Twitter users report or include usernames or account handles.
Online spaces, such as social networking sites, are powerful platforms that hold personally
identifiable information about their users, some of which may be sensitive or wished to
remain private. However, these platforms may not store or use the data in ethical ways,
either. For example, Facebook was discovered to have granted Cambridge Analytica access

- 323 -
한국도서관․정보학회지(제51권 제2호)

to the data of over 87 million users, including personally identifiable information, which was
then used by Cambridge Analytica alongside other data to unethically influence consumers
and voters (Bruns 2019, Isaak and Hanna 2018). Following public and political outcry, many
social media platforms placed new restrictions on accessing their Application Programming
Interfaces (APIs), which researchers and developers used to collect data (Bruns 2019).
Though, limited access to APIs poses added challenges and issues for conducting ethical
research on issues of public interest (Bruns 2019).
Even though the data may be freely or publicly accessible online, researchers must still
consider how the data was collected and whether using or reporting on it would cause harm
(franzke et al. 2020; Markham and Baym 2009). In addition to following standard ethical
guidelines, franzke et al. (2020) reminds researchers to abide by laws and regulations
protecting individuals’ privacy and dignity as well as the terms of use and related policies
of the online space or platforms themselves. Cultural aspects, social norms, beliefs, and
ethical values regarding privacy, confidentiality, anonymity, and informed consent also vary
across individuals, communities, countries, and platforms and researchers to plan, prepare,
and conduct their research accordingly (franzke et al. 2020).
To prevent harm and ensure the privacy and confidentiality of informants or participants,
any personally identifiable information can be kept confidential or eliminated altogether
(Boellstorff et al. 2012, Schutt 2009). Collecting data and presenting findings anonymously
may be the best way to protect participants’ privacy and to avoid potential harm (Boellstorff
et al. 2012). In extreme or highly sensitive cases and circumstances, where potentially
identifiable information is accessible and in great quantity and the potential for harm is real,
Markham (2012) argues that using narratives, dialogues, and dramas or creating composites
of people, processes, and interactions may be successful in creatively disguising and reporting
sensitive data without identifying its sources.

1.2 Informed Consent

Obtaining informed consent is a critical step in online or internet-based research, just as
it is with research in traditional or face-to-face contexts. However, the technological
affordances and limitations of the internet and online spaces can make obtaining informed
consent more challenging in some cases.
As discussed above, the public or private nature of the online space also has implications
for obtaining informed consent. In public and semi-public online spaces, such as virtual
worlds, open forums, online games, and some social media sites, a waiver of consent may
be possible because they are considered or perceived to be public spaces and it may not

- 324 -
 Internet-Mediated Research in the Age of Social Distancing

be feasible to collect informed consent from everyone within the online space (Boellstorff
et al. 2012). Relatedly, due to the anonymity afforded to users in online places, it may be
difficult to confirm the true identity of informants. In such circumstances, informed consent
may be waived altogether or substituted with a public notice of some kind (Boellstorff et
al. 2012). Of course, this assumes that the study’s design includes proper measures to ensure
the privacy and avoid harm to participants and has been approved by a human subjects
committee and/or institutional review board (IRB). However, research that involves direct
interaction and engagement with users should still require a more traditional informed
consent process even if it is in a public or semi-public online space.
As mentioned above, researchers using social or big data sets sourced from online
platforms should consider how the data was collected and whether users were adequately
informed and indeed consented to share their data (franzke et al. 2020). However, Hutton
and Henderson (2017) notes that many users may not read or fully understand a platform’s
End User License Agreement, terms of use, privacy policy, or related policies when they sign
up and use it. Furthermore, these policies often use legal jargon to protect the company or
organization while obscuring and extending their ability to collect and use data as they wish
(Hutton and Henderson 2017). As such, researchers should consider the clarity and
transparency of these policies before collecting or using data from these platforms.

1.3 Building Rapport, Trust, & Trustworthiness

Establishing trust and rapport with participants or informants is critical to facilitate the
collection of accurate and honest data. In interviews, mutual trust between participants and
researchers helps to foster a harmonious connection that enables rich data collection from
the participants’ stories and experiences and improves data quality (Jorgenson 1992, Weller
2017). Establishing rapport suggests that “a basic sense of trust has developed and allows for
the free flow of information” (Spradley 1979, 78). Once trust and rapport has been
established, the interactions between participants and the researcher(s) can become more
comfortable and enjoyable (Jorgensen 1992).
Building rapport and earning trust are important for ethnographic work in online spaces
as well. Gillen interprets ‘ethnographic’ to mean “a commitment to use mixed methods to
endeavor to explore the research participants’ own perspectives on events, to recognise the
complexity of influences on practices and events and to seek to reflexively consider the
researcher’s development of interpretive understandings” (2009, 66). As Gillen (2009) and
others (Boellstorff et al. 2012) note, ethnographic approaches often utilize multiple or mixed
methods to explore and describe communities and cultures. Geertz (1973) equates

- 325 -
한국도서관․정보학회지(제51권 제2호)

ethnography to thick description, a discussion and analysis of human behaviors that is

mindful to account for and describe the complex contexts in which behaviors occur. Without
rapport, trust, reflections on the methods, and multiple perspectives or sources of data, it may
be difficult to accurately understand or portray the perspectives of informants or to
demonstrate the trustworthiness of the findings.
As Boellstorff et al. argue, trust and rapport form the foundations for “frank, open
conversations” (2012, 95). They go on to note that positive interactions with participants or
informants during field site visits helps to establish rapport and build trust, and that interviews
can build on those foundations to develop more powerful connections with informants.
Boellstorff et al. (2012) encourage researchers to be prepared for the interview by familiarizing
themselves with the work or activities of the information, if possible, to be non-judgmental,
be supportive, and provide positive feedback during interviews to make interviewees more
comfortable, thus building rapport and trust. Boellstorff et al. (2012, 95) note that rapport can
be developed indirectly as well, “A referral from a well-respected community member or friend
of the informant can help provide a foundation for trust.” They argue that researchers must
both say and demonstrate their care for informants and make it clear that the researcher is
seeking their perspectives to help tell their story to others. As discussed below, referrals from
established role-players were helpful during snowball sampling.

1.4 Retention
Regarding the process of collecting data through interviews and focus groups online,
another important emphasis is placed on how to maintain the number of participants.
Williams et al. (2012, 379) argue that online research may attract potential participants “who
normally would not or could not take in part in traditional studies.” However, attrition with
online data collection is thought to be higher than what was expected from traditional
face-to-face settings (Fox et al. 2007, Tuttas 2015, Whitehead 2007). Recent research reports that
real-time audiovisual web conference technology (i.e., Skype or Facetime) offers researchers a
promising alternative means to carry out focus groups with geographically dispersed populations
while decreasing time and financial constraints and to closely resemble a traditional face-face
setting (Tuttas 2015). Tuttas (2015) argues that by using web conference technology, the
researcher and participants can see each other, freely talk and type, and gain quality interaction,
engagement, and non-verbal activity during data collection and later analysis sessions as well.
Therefore, if the researcher designs and manages the process of data collection using
technologies effectively, internet-based mechanisms can foster rapid and direct interaction and
communication for building a favorable rapport and decrease attrition rate.

- 326 -
 Internet-Mediated Research in the Age of Social Distancing

However, there is also a possibility that high dependence on technologies may cause
difficulties for participants to decide not to participate in the data collection process. For
example, even skillful participants using web conferencing technologies may decide to quit
if they experience technical issues such as local internet instability or unfamiliarity with
functionality of certain software. Matthews et al. (2018) and Tuttas (2015) recommend that
researchers carefully plan testing and training sessions with other researchers (such as
research assistants and collaborators) and participants so that both parties guarantee that they
have a stable internet connection and test with software functionalities prior to their allocated
sessions.
Alternatively, asynchronous online data collection techniques can provide more flexible
options to encourage participation and retention. Williams et al. (2012, 379) found that
text-based, anonymous, asynchronous online focus groups can enable “greater self-disclosure,
increased reflexivity, and an opportunity to collect details of participant experiences over
time.” Joinson (2001) and Choi (2016) also found that messenger interviews (e.g., chat
programs) having the benefits of using computer-mediated communication and visual
anonymity provided a high level of self-disclosure that could develop close relationships.
Here, flexibility and anonymity, as discussed above, can provide a comfortable way for
participants to share their experiences when they can and without fear of judgment or
repercussions.
Providing participants with flexibility in selecting interview medium increased participation
(Sturges and Hanrahan 2004). Also using various communication tools that participants
normally use in their everyday activities also help the researcher to maintain “the contextual
naturalness” in the research design, which can increase participants' familiarity and ease to
take part in the research (Kazmer and Xie 2008). For those who often use social networking
sites and play games, it is natural for them to use the same tools when they are approached.
Also, considerately scheduling interviews or focus group sessions is helpful to increase the
response rate. Matthews et al. (2018) recommend that focus group allocation with those who
are geographically dispersed should be determined by the availability of participants
according to preferences and time zone differences expressed in pre demographic surveys.
As Tuttas (2015) advised, using web-based scheduling programs or meeting coordination
software products can mitigate challenges associated with arranging focus group sessions and
increase the efficient communication for recruitment. Lastly, good incentives influence
positively on participants’ voluntary participation. Researchers can use creativity and flexibility
when preparing meaningful incentives for targeted participants and sending them through the
manners reflecting the contextual naturalness as well.

- 327 -
한국도서관․정보학회지(제51권 제2호)

Ⅲ. Methodology of the Two Online Research Projects

The considerations discussed in Sections IV and V below are based on evidence from two
different internet-based dissertation research projects. The first is a mixed methods study on
the political information behaviors of opinion leaders on Twitter (Lee 2019). The second is
an ethnographic study on the social information behaviors and digital literacy practices of
role-players in a Massively Multiplayer Online Role-Playing Game (MMORPG) (Hollister 2019).
The data collection and analytical techniques of each project are briefly described below to
provide context for the rest of major sections that follow; please see the cited articles for
more detailed information about the full methodologies and their motivations.

1. Mixed Methods Study of Opinion Leaders on Twitter

This research employed a mixed methods design (exploratory sequential design) of social
network analysis, content analysis and semi-structured interviews with the opinion leaders
from the South Korean Twitter network. Considering that the Twitter data have both
quantitative (structure of the communication networks) and qualitative (content of texts,
images, URLs, etc. in tweets) characteristics, using multiple or mixed methods design is
desired for Twitter research in order to seek convergence (triangulation), to examine different
facets of a phenomenon (complementarity), and to add breadth and scope to a project
(expansion) (Parmelee and Bichard 2012, Greene et al. 1989). By using an exploratory
sequential design, the researcher attempted to collect quantitative data first for the general
picture of the research problem and collect qualitative data in the second phase to add more
refinement, extension, or explanations on the general picture (Creswell 2008, Creswell and
Plano Clark 2011).
Political communication research traditionally has emphasized the importance of closely
held networks where communication among individuals takes place (Ward et al. 2011). Social
network analysis (SNA) allows for studying relationships among these social entities, and the
patterns and implications of these relationships and it has been a dominant approach to
understanding and analyzing the structures, magnitude, typology of and the participants in
major subgroups from the networks (Wasserman and Faust 1994, Ward et al. 2011). Social
network analysis helped the researcher to identify the certain influential users’ position (as
opinion leaders), their social types and roles based on their relationships and behaviors
within the network. However, research studied solely by social network analysis cannot

- 328 -
 Internet-Mediated Research in the Age of Social Distancing

contribute to illuminating those users’ intentions, perceptions, interpretations, and evaluations

of a variety of political information behaviors they engage in with others. In addition to the
social network analysis, the researcher conducted content analysis for analyzing political
sentiments and links embedded in the tweet messages as well as semi-structured interviews
with key opinion leaders. By doing so, this research examined the topical and sentimental
contents, information sources shared in the tweet contents, and the intentions and
interpretations of major political information behaviors that opinion leaders engaged with,
along with investigating the structural characteristics of political communication network (in
particular, retweeting networks) on Twitter during the election campaign in South Korea.
Figure 1 below shows the explanatory sequential research design procedures. The
researcher collected the most frequently retweeted political messages filtered by keywords
related to the Seoul Mayoral election on June 4, 2014: Wonsoon Park, and Mongjoon Jung
(i.e., two names of candidates for the Mayor of Seoul, the capital city of South Korea), Seoul
Mayor Election, and General Election and created the retweeting networks. In phase I, by
using the Python Twitter API4), Tweets were collected for 22 days from May 15 (the official
registration day) to June 5 after the Election Day, June 4, 2014—including the official election
campaign period (May 22 to June 3). Among 297,388 entire retweeting activities, the top
35.3% (104,989 of 297,388) retweeting activities were chosen as a sample for analysis. The
researcher analyzed the top 300 tweets retweeted 104,989 times. For political sentiment
analysis, the researcher conducted qualitative analysis at the message level by reviewing the
full texts, emoticons, hashtags and hyperlinks considering that political tweets included a high
level of subjectivity by using irony, sarcasm, jokes, and puns.
In Phase II, for semi-structured interviews, 30 opinion leaders were selected based on
criterion sampling technique (using the number of retweets and followers, self-disclosure of
information in profile, a list of the opinion leader from previous research and opinion leaders
who participated in the pilot study) for the operationalized definition of opinion leaders. A
sample of 30 opinion leaders was initially reached for recruiting interviewees by emailing,
tweeting, and or using Facebook messaging system in July 2014. 13 out of 30 decided to
take part in the interviews by choosing between email, Voice over Internet Protocol (VoIP)
such as Skype, or face-to-face interview modes. Given the geographical and temporal
distance between the United States and South Korea, eight participants in total chose to use

4) As mentioned above, many social networking sites removed or changed their APIs after the Cambridge
Analytica scandal. The Python Twitter API has been replaced with a different standard, enterprise, and
premium search; more information here: https://developer.twitter.com/en/docs/tweets/search/api-reference/
get-search-tweets.

- 329 -
한국도서관․정보학회지(제51권 제2호)

email (7 out of 13) and Skype (1 out of 13) for the interviews. The remaining five offered
to meet for face-to-face interviews that the researcher conducted in South Korea during the
summer and winter breaks in 2014. The interview data were collected for six months from
July 2014 to December 2014.

<Fig. 1> The Explanatory Sequential Research Design and Procedures

2. Ethnographic Study of Role-Players in a MMORPG

This research project employed a hybrid ethnographic method to explore and describe the
social information behaviors and digital literacy practices of role-players in the science fantasy
themed Massively Multiplayer Online Role-Playing Game (MMORPG), WildStar (Hollister 2019,
Carbine Studios 2014). Role-players in MMORPGs create characters and engage in interactive
and collaborative story-telling while in-character both within and outside of the game world.
The hybrid aspect of the methodological design references its combination of themes from
Hine’s (2010) virtual ethnography, Gillen’s (2009) virtual literacy ethnography, Pearce’s
game/performance/ethnography (Boellstorff et al. 2012, Pearce and Artemesia 2009),
Steinkuehler’s (2007) cognitive ethnography, and Knoblauch’s (2005) focused ethnography.
The resulting hybrid ethnographic approach is multi-sited, allowing for collection of data
where the action is happening; adaptive, allowing for the exploration of relevant or emergent
behaviors and phenomena; focused, using multiple types of data and data collection
techniques within a shorter duration of study; engaging, using participant observation and

- 330 -
 Internet-Mediated Research in the Age of Social Distancing

engagement in the activities alongside participants; and context-aware, balancing the

researcher’s role as observer, participant, and researcher. Multiple data collection techniques
were used to support the hybrid ethnography, including fieldwork via overt participant
observation and engagement, collection of community artifacts, and semi-structured interviews.
Fieldwork conducted in-game happened over a period of six months shortly after the
release date of WildStar. To help in-game data collection, the researcher created two
role-playing characters, one on each faction within the game. The researcher visited in-game
field sites 42 times, with each visit ranging between from a half hour to 4.5 hours, averaging
around 2 hours per visit. In-game data was collected in a variety of formats. The chat logging
utility built into WildStar was used to collect raw chat data from public channels and any
private messages sent to or from the researcher during all game-play sessions, all of the
public role-playing events and activities that were attended, and interviews conducted
in-game. Screenshots were collected using basic print screen/screenshot features common on
most computers and were used to capture moments of relevance and to create evidence of
confirmations of informed consent for the interviews. Full high definition audiovisual
recordings of gameplay during all site visits and interviews were recorded using Open
Broadcaster Software (OBS), an open source and free software suite used for screen-capturing,
recording, and online broadcasting or streaming (OBS, 2020).
Outside of the game, community artifacts, including forum and social media posts,
community wikis and websites, were collected using NVivo’s browser addon, NCapture, or
HTTRack Website Copier (QSR International 2012, Roche 2020). In practice, HTTRack was
used primarily for capturing and archiving an entire website or wiki. Alternative, the
NCapture browser addon was used in case HTTRack was not successful. In addition to the
recorded electronic data, the researcher took brief field notes via pen and paper during
gaming, role-playing, and interview sessions.
Sets of in-character and out-of-character semi-structured interviews were conducted mostly
within the game, with one conducted over Skype chat and one other via email. A
semi-structured approach to the interviews was used to allow informants to describe their
behaviors in their own terms and allow the researcher to delve deeper and focus on emergent
themes that arose from the discussions with the informant (Kazmer and Xie 2008, Mishler
1986, Murray and Sixsmith 1998). Out-of-character interviews focused on the role-players
themselves, while in-character interviews focused on the behaviors, activities, and
perspectives of their created characters. Potential interviewees were purposively sampled and
recruited through a call for participation posted to the forums on the primary role-playing
community website. Additional interviewees were snowball sampled through references and

- 331 -
한국도서관․정보학회지(제51권 제2호)

recommendations from both interviewees and other community members.

All the electronic data and files were saved on a secured local hard drive on a computer
kept in a locked office. Duplicates of all data were created and copied onto two encrypted
and password-protected external hard drives as a safety and data recovery measure. The data
and information captured within the fieldnotes, chatlogs, screenshots, audiovisual files,
community artifacts, and interview transcripts served as the data for this study. The data,
predominantly qualitative, was cleaned, managed, and analyzed using NVivo 10 (QSR
International 2012). As per the original approved IRB application, the study’s data and all its
copies will be destroyed 5 years after the conclusion of the original study.

Ⅳ. How the Research Projects Addressed Key Issues

In the following subsections, methodological reflections and evidence from the two
aforementioned online research projects will describe the actions, strategies, and other
considerations taken to address issues and challenges associated with privacy; informed
consent; building rapport, trust, and trustworthiness; and retention.

1. Privacy

1.1 Privacy in the Twitter Study

The privacy issue can be examined for both the phase I (collection and analysis of the
network and tweet content data and the phase II (collection and analysis of semi-structured
interviews). Twitter, one of the most popular microblogging services, is largely considered
as a public online space that allows individuals to construct a public or semi-public profile,
and share their lists of connections with others and ‘tweets,’ ‘replies to,’ and ‘retweets’ in the
network (boyd and Ellison 2008, Marwick and boyd 2010). The information such as
participants’ Twitter accounts, nicknames, photos, email addresses and URLs to other
information sources they posted in their profile pages and their tweets are regarded as public
data that can be freely accessible unless users lock their accounts. Therefore, the researcher
in this study displayed users’ Twitter accounts and personal information as originally
expressed, when reporting the results of tweet content analysis. For example, one of the most
frequently retweeted messages was written by a member of a famous hip hop band called
TopDogg (of 10 boys), SangDo. He attached his photo to the tweet to verify his voting and
shared it with his fans. The original tweets written in Korean were translated into English

- 332 -
 Internet-Mediated Research in the Age of Social Distancing

and in square brackets the researcher provided additional explanations for clarity and/or
offered information about the hyperlinks included in the tweet.
[SangDo] Hi, Fans! Today is General Election Day! Hope all of you can enjoy your right to vote! All
of us (9 members in TopDogg) except for the youngest member Yano (who had no right to vote yet)
just finished voting! HaHa! [A hyperlink to an image of SangDo taken in front of the polling place is
attached to the tweet.]

For reporting the social network analysis results, the researcher did not display individual
users’ accounts as nodes in the visualization, as seen in Figure 2 below. As the focus of the
social network analysis was to identify and visualize homogenous groups’ retweeting patterns
according to their political orientation, the researcher decided to minimize display of
individuals’ personal information.

<Fig. 2> Visualization of Retweeting Network

Note. Colors stand for the political orientations (Blue for Liberalism, Red for Conservatism,
and Green for Neutralism)

In the Phase II, the researcher conducted semi-structured interviews using three different
modes: email, Skype, and face-to-face interviews. All interviews were recorded upon the
participants' permissions, carefully transcribed, and reviewed once again by themselves to
verify the trustworthiness and accuracy. When reporting the results, the researcher
anonymized participants’ names and tried to protect their privacy and confidentiality of
personally identifiable information. However, interview participants known as opinion
leaders on Twitter included some highly well-known political figures in real life as well,
which makes it easier to be recognized along with detailed information such as professional
occupation, age, gender, and other contact information captured from their interviews. In

- 333 -
한국도서관․정보학회지(제51권 제2호)

those cases, the researcher double checked and let the participants choose the parts to be
eliminated from the reports. All participants agreed that the researcher should report as much
information as they provided.

1.2 Privacy in the MMORPG Study

In the public, semi-public, semi-private, private continuum, MMORPGs teeter between
semi-public and semi-private (Markham and Baym 2009, Rosenberg 2010, Sveningsson Elm
2009). Users must register and log into the game space to communicate with other players.
Once there, other members of the community can see what each other is typing in public,
guild, party, or raid chats. However, MMORPGs could also be considered semi-private
because there are modes of communication that some players may consider to be private
since the audience is limited (e.g., party, whisper, or guild chats).
To ensure the privacy and confidentiality of role-players observed, interviewed, and
interacted within the study, personally identifiable information was omitted from the
presented findings and discussions. This was also extended to the characters of role-players,
too, as their characters and stories may also be unique and potentially identifiable. Both
role-player and character names referenced in examples or excerpts were anonymized. For
example, findings referencing role-players while out-of-character are simply labelled as “Player
1, Player 2, etc.” to protect the player behind the avatar or character. Additionally, Player
1 in one excerpt or example was not necessarily the same Player 1 in a different example
unless explicitly stated otherwise. Additionally, important plot details or the names of guilds
and venues were also omitted or removed from the reported findings. Screenshots were used
sparingly in presentation of the findings, and those that were used were edited to conceal
the identities of those who took part in the captured events or activities.
As discussed in the sections below, the study was overt and used member checking to
engage the community and ensure the accuracy and trustworthiness of the findings. Member
checking was facilitated through the main community website forums. The topic of public
vs. private did come up during a conversation about the study, as seen in Figure 3 below.

<Fig. 3> Public Data Concerns

- 334 -
 Internet-Mediated Research in the Age of Social Distancing

While the researcher took precautions to include notes in character profiles, forums
signatures, and other descriptions of the study to clearly state the researcher was recording
data and included statements asking community members to contact the researcher to opt out
or ask questions, it still could have been possible that some members were not aware of
my presence. Given the high level of activity and information exchange as well as the
persistent nature of the game world, it is easy to see how the information could go unseen.
Given the concern raised by this community member and discussions with moderators on the
community website, a public notice was posted and stickied to the news section of the
community forums to make it clearly visible to visitors to the site. The initial post was made
by a moderator and stated their acknowledgement, but not sponsorship5), of the study and
a reminder to the community that the site is indeed public. Following the moderator’s post,
the researcher responded to open the thread up for discussion in case there were other
concerns, questions, or comments about the study. After explaining the approach to data
collection and describing the precautions the researcher had been taking, there was not any
additional feedback from the community member in the figure above nor did anyone contact
the researcher to opt out of the study at any point. Furthermore, the statements from the
concerned community member in the figure above that the content “doesn’t feel like public
data” and that role-playing can be “very personal” further justify the importance of protecting
the identities and removing identifiable information from the findings and discussions of
them. Furthermore, Jackson (2012) reminds ethnographers to be mindful and sincere about
the data collected from their participants and the information they share about them because
participants have the ability to find the research on their own.

2. Informed Consent

2.1 Informed Consent in the Twitter Study

This research consisted of using Twitter network and content data as well as semi-
structured interview data with opinion leader users. The researcher submitted an application
along with the recruitment messages, cover letter, and semi-structured interview protocols
written in both English and Korean to the Human Subjects Committee of the Institutional
Review Board (IRB) at Florida State University and received an approval. However, as not
all social scientists in South Korea submitted IRB applications and obtained consent forms

5) Here, sponsorship means that the moderators did not invite or fund or were otherwise associated with the
study. However, their positive response and support were invaluable to this project.

- 335 -
한국도서관․정보학회지(제51권 제2호)

from participants, the researcher revised the initial application to streamline the process of
obtaining consent from participants. In the cover letter written in Korean, the researcher
included the section where participants could still express their willingness to take part in
the interviews and select the most favored interview mode among email, Skype, and
face-to-face interview options. By collecting cover letters with participants’ answers in, this
research dealt with obtaining consent from the participants. Figure 4 below shows the section
of the cover letter asking the participants to provide their consent.

<Fig. 4> Informed Consent Process in Cover Letter

2.2 Informed Consent in the MMORPG Study

Due to the semi-public/semi-private nature of MMORPGs, the infeasibility of obtaining
informed consent from and confirming the true identity of every player encountered with the
game, informed consent was waived for the participant observation and engagement
fieldwork. Waiver of consent in observational studies of virtual worlds and MMORPGs is not
uncommon due to these reasons, though other types of public notice may be preferable
(Boellstorff et al. 2012, Monahan 2009, Storie 2008). In lieu of informed consent during
fieldwork, the researcher posted multiple notices, including one that was approved and
stickied by forum moderators, about the study on the community’s main website forums to
increase visibility and awareness of the study within the role-playing community in WildStar.
Additionally, information about the study, a link to the researcher’s website, and a notice that
the researcher was recording and to let the researcher know if they wanted opt-out of the
recordings was also included within the researcher’s character’s in-game role-playing profile.
The role-playing profile was viewable through an add-on called PDA, which was commonly
used by the role-playing community. As discussed later, this overt approach helped to
improve rapport with the community and trustworthiness of the data.

- 336 -
 Internet-Mediated Research in the Age of Social Distancing

However, a traditional informed consent process was used for the semi-structured
interviews. Prior to beginning the interview, the researcher provided a URL link to the
IRB-approved informed consent information about the study, informed them that the
interviews would be recorded, and provided time for the interviewee to review the material
and ask any questions they had. After they completed looking over the information and the
researcher answered any questions, the researcher shared a statement for the potential
interviewee to agree or disagree with. Most of the interviews were conducted in-game via
text chat. The screenshot in Figure 5 shows the informed consent process during an interview
conducted via in-game text chat.
In Figure 5 below, Jonti Thort is one of the researcher’s characters and the interviewee’s
name is obscured to protect their identity. After the potential interviewees agreed to the
terms, the researcher acknowledged their informed consent and affirmed that they would
follow up with the interviewee with a confirmation of their participation and to arrange
delivery of their incentive, which will be discussed below. The interviews only began after
informed consent was obtained and confirmed. While out-of-character interviews were
typically conducted in private chat channels in-game, in-character interviews were conducted
in public, local /say or custom emote channels, as was the norm for most role-playing
activities in WildStar. In-game interviews were conducted in a public setting, typically in or
near a major city or outpost.

<Fig. 5> In-Game Informed Consent Process

- 337 -
한국도서관․정보학회지(제51권 제2호)

3. Building Rapport, Trust & Trustworthiness

3.1 Building Rapport, Trust, and Trustworthiness in the Twitter Study

Since Twitter's convention for making social connections with others by following does not
require building reciprocal relationships (Larsson and Moe 2011, Marwick and boyd 2010)
unlike similar social networking services of Facebook (friend-centric networks based on
reciprocity), it appears to be more sparse to maintain consistent and enduring communication
among Twitter users. The researcher developed strategies for establishing and maintaining
rapport and trust and applied them into the procedures of recruiting participants, scheduling
the interviews, and checking the trustworthiness of the findings.
Access to potential participants for recruitment for qualitative interviews can be achieved
through face-to-face, email, and social networking sites (Janghorban et al. 2014). Twitter user
accounts of participants mostly supplied the default contact information. On users’ profile
pages, some users disclosed additional contact information such as Facebook page, personal
website, and/or email address. By using this additional contact information, the researcher
reached participants via three different communication tools: email, Twitter, and Facebook
messaging systems for participant recruitment. The researcher tried to keep the Twitter and
Facebook accounts for research purposes. Candid and sufficient self-disclosure information
was provided in the profile sections: the recent photo of the researcher, information about
academic affiliation and brief description about major research areas. Also, well-written
recruitment messages were prepared and sent to the sample of 30 opinion leaders:
information about the researcher, purpose of the research, reasons for recruiting interviewees,
and the links to the related scholarly works that the researcher conducted regarding political
communication on Twitter. By professionally and consistently managing researcher’s identity
and communication in social networking sites, the researcher tried to inform participants of
the research itself and assist them with building rapport and trust.
The researcher sent out recruitment messages three times by using one communication tool
at a time so that participants would feel less pressure when receiving the recruitment message.
To manage the recruitment process, the researcher maintained the communication log with
individual participants in excel spreadsheets with accurate details of correspondence (e.g.,
dates of initial and subsequent waves of recruitment, chosen interview schedules and interview
modes, etc.). Maintaining communication log was helpful for the researcher to systematically
manage all the conversations in order and keep multiple interviews on the right track.
Another tip for increasing recruitment rate is to study and use participants’ Twitter

- 338 -
 Internet-Mediated Research in the Age of Social Distancing

activity/usage patterns. For example, the researcher found that one participant would send
out the first tweet at 6 o’clock in the morning every day, which would be the best time for
the researcher to send out a recruitment message via tweeting. When a participant clearly
indicated on his Twitter profile page that an email would be the best contact for him to
communicate, the researcher chose email as the first communication tool to send out the
recruitment message. In this manner, identifying the users’ Twitter activity/usage patterns in
advance assisted the researcher to successfully recruit the interview participants during a short
time period. Many participants showed their Twitter activities (e.g., tweeting, retweeting, etc.)
in the morning after they came to work: The researcher mainly sent our recruitment messages
in the morning (South Korea) and at night (U.S.) and received replies from the participants
via tweeting between within an hour and up to a day.
Building rapport and trust between the researcher and the participant in advance was
critical to facilitating the recruiting process and increased the response rate and quality of the
responses in the interviews. As Baym (1995) suggests, building rapport online can be done
over time. Even though the researcher did not directly interact with the participant (e.g., by
mentioning, replying or direct messaging), the researcher’s exposure on the Twitter timeline
by following them and/or retweeting other users’ tweets assisted both parties in getting
familiar with each other and building indirect interactions. For example, the researcher has
been following many opinion leaders included in a sample for this research long before
starting this research. One of the most famous opinion leaders briefly talked to the researcher
via Twitter around the 2012 Presidential Election; building upon this short interaction allowed
the participant to trust the researcher and willingly take part in the interview.
Based on the frequent and consistent communication through direct messaging on Twitter
and emails between the researcher and participants for longer than six months, rich stories
and narratives regarding participants’ political information behaviors were collected.
Participants using email interviews were instructed to freely attach additional information
sources other than just textual information, such as news articles or screenshots of their own
or others’ tweets, when talking about particular events or episodes. All seven email interviews
were terminated after a couple of additional email exchanges for probing details. Even
though the length of email interviews were a lot shorter than Skype and face-to-face
interviews, sufficient responses covering major themes were collected. One participant chose
the Skype interview mode to avoid security concerns. The researcher and participant logged
in before the allocated session to test out the audio and microphone settings in Skype before
continuing onto a ninety-minute interview. Five face-to-face interviews were conducted at
sites of participants’ choosing (e.g., their office, a coffee place, restaurant, etc.) in Seoul while

- 339 -
한국도서관․정보학회지(제51권 제2호)

the researcher was visiting South Korea in July and December of 2014. These individual
face-to-face interviews, which lasted approximately 50 to 60 minutes each, allowed the
participants to freely express and elaborate on their thoughts and feelings.
Additionally, the researcher was analyzing the email and Skype interview data and
discovered some emergent themes that informed the questions of the face-to-face interviews,
allowing the researcher to get additional feedback on the accuracy and trustworthiness of the
findings. Through this member checking process for the last three face-to-face interviews, the
major findings were verified and illuminated with more details. However, given the small
sample size of the qualitative interviews, the findings are not generalizable or transferable,
but may be insightful for similar contexts.

3.2 Building Rapport, Trust, and Trustworthiness in the MMORPG Study

The overt approach and member checking used in this project were critical for establishing
rapport, building trust, and ensuring trustworthiness of the findings. As mentioned above, the
researcher shared details about the study through a variety of channels, both inside and
outside of the game. In Figure 6 below, one community member expressed appreciation for
the transparency of the study.

<Fig. 6> Comment on Study Transparency

In addition to triangulating multiple data types and points and inter-coder reliability testing,
member checking was used to ensure the verifiability and accuracy of the findings. Member
checking was facilitated through the role-playing community’s main forums and all posts
related to the study and member-checking were included in data collection. Member checking
provided another opportunity to engage with the community and get feedback on the
accuracy and trustworthiness of the findings (Adams 2009, Schutt 2009). For example, after
sharing some of the preliminary results on the forums, one of the community members
responded to say that the findings seemed to match their experiences.

<Fig. 7> Member Checking 1

- 340 -
 Internet-Mediated Research in the Age of Social Distancing

Member checking also allowed for more in-depth conversations and discussions with
community members to help flesh out different interpretations or potential applications of the
findings. For example, Figure 8 below depicts part of a discussion between the researcher
(Thorstein) and a community member about social context and media literacy skills.

<Fig. 8> Member Checking 2

Due to time spent analyzing data and writing results, as well as outside commitments and
life events, there were sometimes long periods of times between the researcher’s posts to the
community forums. However, community members still seemed to appreciate the information,
as seen in the forum post captured in Figure 9 below.

<Fig. 9> Don’t be a Stranger

As such, member checking was critical for building rapport, trust, and ensuring
trustworthiness of the findings. While member checking was a successful strategy in this
study, it does not always go well. Carlson (2010) warns that miscommunications with
participants can harm not only the relationships between the participants and the researcher,
but also the trustworthiness of the overall study. Recalling the discussion between the
researcher and a role-playing community member described in Section 1.2 and as seen in

- 341 -
한국도서관․정보학회지(제51권 제2호)

Figure 3, the MMORPG study may have not gone as well if the discussion had turned sour,
which could have led to losing community support and interest in the study.

4. Retention

4.1 Retention in the Twitter Study

To efficiently communicate with and attract the interview participants via fast-paced
communication tools of Twitter and Facebook messaging systems, the researcher tried to keep
the communication informative, professional, and rapid so that opinion leaders build trust and
decide to spend a reasonable amount of time participating in the interview. The
well-managed prompt correspondence makes recruitment and communication processes
resemble one in a face-to-face setting. As Tuttas (2015) argued, establishment of effective
lines of communication is the key to successful participants’ recruitment and retention.
Also, the researcher provided the participants with the freedom to choose the best time
and most favored interview mode depending on the degree they want to commit, which
facilitated interview scheduling and increased the response rate. By professionally managing
the communication, the researcher could build a favorable relationship online that resulted
in retention of all thirteen participants for longer than six months until the end of data
collection and analysis.
Another way to encourage their participation was to provide an incentive. Even though
most of the participants politely refused to take an honorarium (Korean currency equivalent
worth twenty dollars), the researcher offered it to appreciate their voluntary time and effort.
One participant allowed the researcher to alternately donate the honorarium for an
independent news media that he supported, instead of taking it.

4.2 Retention in the MMORPG Study

A variety of measures were taken to aid in recruiting and retaining interviewees for
ethnographic study. As mentioned above, a purposive sampling of role-players was enabled
by posting recruitment calls on the community’s main website forum. This post attracted a
lot of attention and a variety of role-players, including some of the most active and
experienced members. Additionally, snowball sampling was helpful in finding additional
community members that may have not seen the posts or were initially shy or reluctant. In
Figure 10 below, one community member mentions in a reply to the call for participation
that they were referred to the study by one of their friends.

- 342 -
 Internet-Mediated Research in the Age of Social Distancing

<Fig. 10> Snowball Sampling

To attract potential interview participants and to compensate them for sharing the knowledge
and experiences, an attractive and meaningful incentive was used. Interviewees were
compensated with 1 C.R.E.D.D. per interview. A C.R.E.D.D. was an in-game token worth
20 USD, but it could be used in exchange for one month’s worth of game-time subscription
or sold on the in-game auction house for in-game currency. If an interviewee completed both
in-character and out-of-character interviews, they could earn 2 C.R.E.D.D.; as 16 out of the
17 interviewees completed both types of interviews, it seemed to be effective. However, an
unforeseen market and trading limitation within WildStar, that only one C.R.E.D.D. could be
mailed per day, resulted in some participants having to wait a few days to receive their
incentives. Using game-time tokens or in-game items as incentive can be a great alternative
to gift-cards or cash. Another benefit of using game-time tokens or other in-game items was
that it did not require collecting additional personal information such as personal email or
physical addresses in order to distribute the incentive, further ensuring the privacy and
confidentiality of the interviewees.
Another measure used to retain interviewees was to allow them to choose where and how
to do the interviews. Interviews were scheduled and confirmed via messages on the community
website or through the in-game mail system. Prospective interviewees were offered the option
to conduct the interviews in-game via text chat, using a voice over internet protocol or
chatting app outside of the game, or by email. Kazmer and Xie (2008) note that being able
to choose the interview medium increases the likelihood of participation. Additionally,
prospective interviewees also had the choice to do either in-character or out-of-character
interviews, or both. Most (16 out of 17) participants chose to do both types of interviews.
They were also able to choose which one they wanted to do first and to take breaks as
needed during or between the interviews, which lasted between 0.5 to 3 hours in length.

Ⅴ. Practical Considerations and Useful Resources

In addition to the methodological and ethical considerations discussed in the previous
section, the researchers also gained insights regarding research as practice that may be helpful
for other researchers starting their journey into online or internet-based research. Useful tools
and resources are also identified.

- 343 -
한국도서관․정보학회지(제51권 제2호)

1. Taking an Adaptive or Flexible in Approach

Taking an adaptive or other flexible approach to research in online spaces may provide
a better opportunity to explore and describe expected behaviors and activities as well as
emergent themes and phenomena in the dynamic and complex online spaces (Boellstorff et
al. 2012). In the MMORPG study, for example, the original goal was to explore and describe
the role-playing community in WildStar by joining and participating in a role-playing guild.
As such, the researcher applied to multiple guilds, fully stating their intentions for research,
and was accepted into two of them. The researcher joined the first guild that accepted their
application and became a member after a two-month probationary period. However, shortly
after this, disagreements in role-playing rules between subgroups within the guild, ultimately
causing the guild to disband. The researcher then turned to the second guild that accepted
their application only to find that a similar implosion had happened to that guild as well.
As such, the researcher had to re-plan and reposition the study to focus on public
role-playing activities and events, rather than guild-based ones. Of course, this change also
required the researcher to change and get their IRB application re-approved, resulting in
delays in data collection and frustration. However, the flexibility resulted in a more
representative portrayal of the role-playing community by focusing on public role-playing
rather than on guild-based role-playing.
Relatedly, the semi-structured approach to the interviews allowed for in-depth and
meaningful conversations with members of the role-playing community. As mentioned above,
some of the interview sets lasted up to 3 hours long. While intense, these interviews supplied
valuable insights into the complex rules and values of the community that may not have been
captured through observation alone.

2. Considerations for Participants and Researchers, Too

As surely all researchers can attest, good research takes a lot of time and effort, physically,
mentally, and emotionally. This, of course, extends to the participants who offer their time
and experiences to help researchers better understand the social world. Due to the COVID-19
pandemic, most people are facing substantial added strain on their physical and mental health
and wellbeing, not only of themselves, but also their family and friends, as well as additional
limitations on time, finances, and other resources. Researchers should be mindful that there
are more important things for participants to attend to than completing a survey or interview.
Now more than ever, researchers should be patient, supportive, and flexible with scheduling

- 344 -
 Internet-Mediated Research in the Age of Social Distancing

and supply generous and useful incentives.

In turn, researchers should also be mindful of their own wellbeing. Sampson et al. (2008)
found that researchers tend to pay more attention to their participants’ wellbeing rather than
care for their own. Unfortunately, universities, research project supervisors, and researchers
themselves may lack sufficient training, coping strategies, policies, or other resources, and are
otherwise unprepared or ill-equipped to handle health and occupational risks faced by
researchers (Bloor et al. 2010; Dickson-Swift et al. 2008, Moncur 2013). Physical and emotional
risks may be more common or severe for researchers focusing on sensitive topics or using
reflexive methods (Dickson-Swift et al. 2008, Sampson et al. 2008). During the MMORPG
study, the researcher lost two grandparents and got married. These events caused delays and
complications with scheduling, analysis, and writing to cope or otherwise manage these
important and emotionally exhausting events. Thankfully the researcher’s advisor was
supportive during the project, as was the studied community. As mentioned above, another
benefit of member checking is that researchers can keep the community informed on things
that may affect the research study one way or another, and the researcher did just that.
Ethical standards of and for online or internet research primarily focus on protecting the
privacy, integrity, and wellbeing of participants (Boellstorff et al. 2012, Buchanan 2011,
Consalvo and Ess 2011, franzke et al. 2020). However, these issues are also important for
the researchers, too. Studying toxic communities or risky behaviors online can result in
harassment, abuse, and much worse (franzke et al. 2020, Marwick et al. 2016). As such,
franzke et al. (2020) advocates for institutional and legal policies to protect researchers.
Marwick et al. (2016) describes strategies researchers can take to help protect themselves
from online harassment and cyber-attacks and Dickson-Swift et al. (2008) describe practical
strategies and policy changes aimed at minimizing risk to researchers at multiple levels.

3. Selected Readings, Resources, and Tools

Below is a brief list of useful readings, resources, and tools that the researchers encourage
readers to explore if they are considering online or internet research. More specific sources
can be found in the references list below as well.

3.1 Ethical Guidelines

∙ Buchanan, E. A. 2011. “Internet research ethics: Past, present, future.” The Blackwell
Handbook of Internet Studies (pp. 83-108). Oxford: Wiley-Blackwell.

- 345 -
한국도서관․정보학회지(제51권 제2호)

∘ This book chapter provides an overview of the past and recent status of internet
research ethics, and considers future directions as well.
∙ franzke, a. s., A. Bechmann, M. Zimmer, and C. Ess. 2020. Internet research: Ethical
guidelines 3.0. https://aoir.org/reports/ethics3.pdf
∘ The most recent version of the ethical guidelines for conducting online research
developed by the Association of Internet Researchers.
∙ British Psychological Society. 2017. Ethics guidelines for internet-mediated research.
https://www.bps.org.uk/news-and-policy/ethics-guidelines-internet-mediated-research-2017
∘ Oft-cited ethical guidelines for online research from the British Psychology Society.

3.2 Online, Internet Research Methodology Handbooks and Textbooks

∙ Boellstorff, T., B. Nardi, C. Pearce, and T. L. Taylor. 2012. Ethnography and virtual
worlds: A handbook of method. Princeton; Oxford: Princeton University Press.
∘ Detailed methodology handbook for research in virtual worlds and MMORPGs.
∙ Consalvo, M. and C. Ess. 2011. The Blackwell handbook of internet studies. Oxford:
Wiley-Blackwell.
∘ Edited collection of essays from established scholars from various disciplines that
explores the history, trends, methodologies, ethics, and future of internet research.
∙ Fielding, N. G., R. M. Lee, and G. Blank. 2017. The Sage handbook of online research
methods. 2nd ed. London: Sage Publications.
∘ Textbook that provides detailed guidance on the design and conduct of qualitative
and quantitative research in online settings.
∙ Hunsinger, J., M. M. Allen, and L. Klastrup. 2020. Second international handbook of
internet research. Dordrecht: Springer.
∘ Edited collection of essays from established scholars of various disciplines from
around the world to highlight the wide variety of internet research topics and approaches.
∙ Markham, A.N. and N. K. Baym. 2009. Internet inquiry: Conversations about method.
London: SAGE.
∘ Edited collection of essays that discuss a wide variety of ethical issues, methods, and
other challenges associated with internet research.
∙ Marwick, A.E., L. Blackwell, and K. Lo. 2016. “Best practices for conducting risky
research and protecting yourself from online harassment.” Data & Society.
https://datasociety.net/library/best-practices-for-conducting-risky-research/.
∘ Recommendations for researchers focusing on potentially risky topics or toxic
communities and how to protect oneself from online harassment.

- 346 -
 Internet-Mediated Research in the Age of Social Distancing

3.3 Free or Open-Source Online Research Tools

Table 1 below includes a variety of recommended software and tools to facilitate online
research activities, organized in alphabetical order of the author(s).

<Table 1> Recommended Software and Tools for Online Research

Name Author(s) Date Source Description/Purpose Platform
Gephi Bastian M. et 2017 https://gephi.org/ Free and open-source Windows, Linux,
al. tool for creating graphs MacOS X
and visualizations of
network and other
quantitative data
QualCoder Curtain, C. 2020 https://qualcoder.word Free and open-source Windows, Linux
1.9 press.com qualitative data (text,
image, video) analysis
tool available
AQUAD: Huber, G. L. 2020 http://www.aquad.de/en/ Free qualitative data Windows
Analysis of analysis tool
Qualitative
Data
Wayback Internet n.d. https://archive.org/web/ Online digital archiving Any web browser
Machine Archive tool that can search for
older versions or
missing websites
ELAN Max Planck 2019 https://archive.mpi.nl/tla/ Free and open-source Windows, Linux,
Institute for elan tool for annotating MacOS X
Psycholinguisti audio or video data
cs, The
Language
Archive.
Open OBS Project. 2020 https://obsproject.com/ Free and open-source Windows, Linux,
Broadcaster tool for recording, MacOS X
Software streaming, and
(OBS) broadcasting audio or
video
R R Foundation. 2020 https://www.r-project.org Free software tool for Windows, Linux,
statistical and graphical MacOS X
analysis
HTTrack Roche, X. 2020 http://www.httrack.com Free tool to archive or Windows, Linux,
Website create offline versions MacOS X,
Copier of websites Android
Social Kalamaras, D. 2020 https://socnetv.org Free and open-source Windows, Linux,
Network software for the MacOS X
Visualizer analysis and
visualization of social
network data

- 347 -
한국도서관․정보학회지(제51권 제2호)

Ⅵ. Conclusion & Next Steps

The authors hope that this introduction and discussion of key issues and challenges related
to online or internet-mediated research will be helpful for scholars interested in taking their
research online for the first time as it seems as if the COVID-19 pandemic may have a major
impact on research practices now and into the future. As seen in both the Twitter and
MMORPG studies above, there are a variety of measures to adequately address the complex
issues of privacy, informed consent, building rapport, trust, and trustworthiness, and
retention. However, of course, there are other issues related to internet-mediated research,
such as method validity, security, sampling, credibility, technology (Padayachee 2012), not
directly addressed here that can be explored in future work. The authors plan to continue
developing and using online research methods and hope to establish an online research
workshop or special interest group to provide opportunities for faculty in South Korea to
learn new online research skills, share their work, and expand their networks.

References

권익현, 이상원. 2005. 패널을 이용한 온라인 서베이와 오프라인 서베이 데이터 사이의 신뢰도에 대한
탐색적 연구. 경영학연구, 34(4): 1261-1278.
권용민, 명승환. 2011. SNS를 활용한 조사방법론의 가능성과 쟁점. 한국행정학회 학술발표논문집,
12: 1-20.
김갑선. 2017. 문헌정보학 학술지 논문의 질적 연구 동향 분석. 한국도서관정보학회지, 48(1):
373-396.
김정근, 장덕현. 1994. 우리나라 공공도서관 발전의 현단계에 관한 연구: 부산 경남지역 공공도서관
사서의 일상적 의식세계에 대한 질적분석. 한국문헌정보학회지, 27: 3-73.
이지수. 2019. 한국의 트위터 오피니언 리더들의 정치적 정보행동에 관한 연구: 정보세계이론을 중심
으로. 한국문헌정보학회지, 53(1): 83-108.
최석준. 2016. 메신저 인터뷰의 사용자 조사 효과에 관한 연구. 국디자인문화학회지, 22(3): 563-573.
Adams, S. S. 2009. “What Games Have to Offer: Information Behavior and Meaning-Making
in Virtual Play Spaces.” Library Trends, 57(4): 676-693.
Boellstorff, T. et al. 2012. Ethnography and Virtual Worlds: A Handbook of Method. Princeton;
Oxford: Princeton University Press.

- 348 -
 Internet-Mediated Research in the Age of Social Distancing

boyd, d. and N. B. Ellison. 2008. “Social Network Sites: Definition, History, and Scholarship.”
Journal of Computer-Mediated Communication, 13(1): 210-230.
Bloor, M., B. Fichman, and H. Sampson. 2010. “Unprepared for the Worst: Risks of Harm for
Qualitative Researchers.” Methodological Innovations Online, 5(1): 45-55.
British Psychological Society. 2017. Ethics Guidelines for Internet-Mediated Research.
<https://www.bps.org.uk/news-and-policy/ethics-guidelines-internet-mediated-research
-2017> [Cited 2020. 5. 20].
Brügger, N. 2012. “When the Present Web is Later the Past: Web Historiography, Digital
History, and Internet Studies.” Historical Social Research, 37(4): 102-117.
Bruns, A. 2019. "After the ‘APIcalypse’: Social Media Platforms and Their Fight Against Critical
Scholarly Research." Information, Communication & Society, 22(11): 1544-1566.
Bruns, A. 2018. “Facebook Shuts the Gate After the Horse Has Bolted, and Hurts Real Research in
the Process.” Internet Policy Review. <https://policyreview.info/articles/news/facebook
-shuts-gate-after-horse-has-bolted-and-hurts-real-research-process/786> [Cited 2020. 5. 6].
Buchanan, E. A. 2011. “Internet Research Ethics: Past, Present, Future.” The Blackwell Handbook
of Internet Studies (pp. 83-108). Oxford: Wiley-Blackwell.
Carlson, J. A. 2010. “Avoiding Traps in Member Checking.” Qualitative Report, 15(5): 1102-1113.
Chase, L. and J. Alvarez. 2000. “Internet Researcher: The Role of the Focus Group.” Library
& Information Science Research, 22(4): 357-369.
Consalvo, M. and C. Ess. 2011. The Blackwell Handbook of Internet Studies. Oxford: Wiley-
Blackwell.
Creswell, J. W. 2008. Educational Research: Planning, Conducting, and Evaluating Quantitative
and Qualitative Research. Upper Saddle River, N.J: Pearson/Merrill Prentice Hall.
Creswell, J. W. and V. L. Plano Clark. 2011. Designing and Conducting Mixed Methods
Research. Thousand Oaks, CA: Sage Publications.
Dalton, M. L. 1991. “Does Anybody Have a Map? Accessing Information in the Internet's
Virtual Library.” Internet Research, 1(1): 31-39.
Dickson-Swift, V., E. et al. 2008. “Risk to Researchers in Qualitative Research on Sensitive
Topics: Issues and Strategies.” Qualitative Health Research, 18(1): 133-144.
Farrell, D. and J. C. Petersen. 2010. “The Growth of Internet Research Methods and the
Reluctant Sociologist.” Sociological Inquiry, 80(1): 114-125.
Fielding, N. G. et al. 2017. The Sage Handbook of Online Research Methods. 2nd ed.
London: Sage Publications.

- 349 -
한국도서관․정보학회지(제51권 제2호)

Fox, F. E., M. Morris, and N. Rumsey. 2007. “Doing Synchronous Online Focus Groups with Young
People: Methodological Reflections.” Qualitative Health Research, 17(4): 539–547.
franzke, a. s. et al. 2020. Internet research: Ethical guidelines 3.0. <https://aoir.org/reports/ethics3.
pdf> [Cited 2020. 5. 6].
Geertz, C. 1973. The Interpretation of Cultures: Selected Essays. New York: Basic Books.
Gillen, J. 2009. “Literacy Practices in Schome Park: A Virtual Literacy Ethnography.” Journal
of Research in Reading, 32(1): 57-74.
Greene, J. C., V. J. Caracelli, and W. F. Graham. 1989. “Toward a Conceptual Framework
for Mixed-Method Evaluation Designs.” Educational Evaluation and Policy Analysis,
11(3), 255-274.
Herring, S. C. 2002. “Computer‐Mediated Communication on the Internet.” Annual Review of
Information Science and Technology, 36(1): 109-168.
Hine, C. 2000. Virtual Ethnography. London: SAGE.
Hollister, J. M. 2019. “An Ethnographic Study on the Digital Literacy Practices of Role-Players
in a Massively Multiplayer Online Role-Playing Game.” Journal of the Korean Library
and Information Science Society, 50(4): 429-467.
Hsieh, H.-F. and S. E. Shannon. 2005. “Three Approaches to Qualitative Content Analysis.”
Qualitative Health Research, 15(9): 1277-1288.
Hutton, L. and T. Henderson. 2017. “Beyond the EULA: Improving Consent for Data Mining.”
Transparent Data Mining for Big and Small Data (pp. 147-167). Cham: Springer.
Isaak, J. and M. J. Hanna. 2018. “User Data Privacy: Facebook, Cambridge Analytica, and
Privacy Protection.” Computer, 51(8): 56-59.
Jackson Jr., J. L. 2012. “Ethnography is, Ethnography ain’t.” Cultural Anthropology, 27(3):
480-497.
Janghorban, R., R. L. Roudsari, and A. Taghipour. 2014. “Skype Interviewing: The New
Generation of Online Synchronous Interview in Qualitative Research.” International
Journal of Qualitative Studies on Health and Well-Being, 9(1). <https://doi.org/10.3402/
qhw.v9.24152> [Cited 2020. 5. 20].
Jones, S. 1999. Doing Internet Research: Critical Issues and Methods for Examining the Net.
London: Sage Publications.
Joinson, A. N. 2001. “Self-Disclosure in Computer-Mediated Communication: The Role of Self-
Awareness and Visual Anonymity.” Education Journal of Social Psychology, 31,
179-192.

- 350 -
 Internet-Mediated Research in the Age of Social Distancing

Jorgenson, J. 1992. “Communication, Rapport, and the Interview: A Social Perspective.”

Communication Theory, 2(2): 148-156.
Kazmer, M. M. and B. Xie. 2008. “Qualitative Interviewing in Internet Studies: Playing with
the Media, Playing with the Method.” Information, Communication & Society, 11(2):
257-278.
Khoo, M., L. Rozaklis, and C. Hall. 2012. “A Survey of the Use of Ethnographic Methods in
the Study of Libraries and Library users.” Library & Information Science Research, 34:
82-91.
Knoblauch, H. 2005. “Focused Ethnography.” Forum: Qualitative Social Research, 6(3): 1-14.
Larsson, A. O., and H. Moe. 2011. “Studying Political Microblogging: Twitter Users in the 2010
Swedish Election Campaign.” New Media & Society, 15(5): 729-747.
Markham, A. 2012. “Fabrication as Ethical Practice.” Information, Communication & Society,
15(3): 334-353.
Markham, A. N. and N. K. Baym. 2009. Internet Inquiry: Conversations About Method.
London: SAGE.
Marwick, A. E. and d. boyd. 2010. “I Tweet Honestly, I Tweet Passionately: Twitter Users,
Context Collapse, and the Imagined Audience.” New Media & Society, 13(1): 114-133.
Marwick, A. E., L. Blackwell, and K. Lo. 2016. “Best Practices for Conducting Risky Research
and Protecting Yourself from Online Harassment.” Data & Society. <https://datasociety.
net/library/best-practices-for-conducting-risky-research/> [Cited 2020. 5. 21].
Matthews, K. L., M. Baird, and G. Duchesne. 2018. “Using Online Meeting Software to
Facilitate Geographically Dispersed Focus Groups for Health Workforce Research.”
Qualitative Health Research, 28(10): 1621-1628.
Mishler, E. G. 1991. Research Interviewing: Context and Narrative. Cambridge: Harvard
University Press.
Monahan, K. 2009. “Information Seeking in World of Warcraft.” Master's Thesis, University of
North Carolina at Chapel Hill, Chapel Hill, NC.
Moncur, W. 2013. “The Emotional Wellbeing of Researchers: Considerations for Practice.”
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp.
1883-1890). ACM.
Murray, C. D. and J. Sixsmith. 1998. “E-mail: A Qualitative Research Medium for Interviewing?”
International Journal of Social Research Methodology, 1(2): 103-121.
OBS Project. n.d. Open Broadcaster Software [Computer software]. <https://obsproject.com/>

- 351 -
한국도서관․정보학회지(제51권 제2호)

Padayachee, K. 2016. “Internet-Mediated Research: Challenges and Issues.” South African

Computer Journal, 28(2): 25-45.
Parmelee, J. H. and S. L. Bichard. 2012. Politics and the Twitter Revolution: How Tweets
Influence the Relationship Between Political Leaders and the Public. Maryland:
Lexington Books.
Pearce, C. and Artemesia. 2009. Communities of Play: Emergent Cultures in Multiplayer
Games and Virtual Worlds. Cambridge, Mass: MIT Press.
QSR International. 2012. NVivo 10 [Computer software]. <https://www.qsrinternational.com>
Roche, X. 2020. HTTrack website copier [Computer software]. <http://www.httrack.com>
Rosenberg, Å. 2010. “Virtual World Research Ethics and the Private/Public Distinction.”
International Journal of Internet Research Ethics, 3(1): 23-37.
Sampson, H., M. Bloor, and B. Fichman. 2008. “A Price Worth Paying? Considering the ‘Cost’
of Reflective Research Methods and the Influence of Feminist Ways of ‘Doing.’”
Sociology, 42(5): 919-933.
Schutt, R. K. 2009. Investigating the Social World: The Process and Practice of Research. 6th
ed. Thousand Oaks: Sage Publications.
Spradley, J. P. 1979. The Ethnographic Interview. New York, NY: Rinehart & Winston.
Steinkuehler, C. 2007. “Massively Multiplayer Online Gaming as a Constellation of Literacy
Practices.” E-Learning, 4(3): 297-318.
Storie, D. 2008. “Serious Research? Exploring the Information Behaviour of Avid Players of a
Massively Multiplayer Online Game.” Master’s Thesis. University of Alberta, Canada.
Sturges, J. E. and K. J. Hanrahan. 2004. “Comparing Telephone and Face-to-Face Qualitative
Interviewing: A Research Note.” Qualitative Research, 4(1), 107-118.
Sveningsson Elm, M. 2009. “How Do Various Notions of Privacy Influence Decisions in
Qualitative Internet Research?” Internet Inquiry: Conversations About Method (pp.
69-87). Los Angeles: Sage Publications.
Todd, R. J. 2003. “Evidence-Based Practice: Overview, Rationale, and Challenges.” We Boost
Achievement! Evidence-Based Practice for School Library Media Specialists (pp. 1-25).
Salt Lake City, UT: Hi Willow Research and Publishing.
Tuttas, C. A. 2015. “Lessons Learned Using Web Conference Technology for Online Focus
Group Interviews.” Qualitative Health Research, 25(1): 122-133.
Ward, M. D., K. Stovel, and A. Sacks. 2011. “Network Analysis and Political Science.” Annual
Review of Political Science, 14: 245-264.

- 352 -
 Internet-Mediated Research in the Age of Social Distancing

Wasserman, S. and K. Faust. 1994. Social Network Analysis: Methods and Applications.
Cambridge: Cambridge University Press.
Weller, S. 2017. “Using Internet Video Calls in Qualitative (Longitudinal) Interviews: Some
Implications for Rapport.” International Journal of Social Research Methodology, 20(6):
613-625.
Whitehead, L. C. 2007. “Methodological and Ethical Issues in Internet-Mediated Research in the
Field of Health: An Integrated Review of the Literature.” Social Science & Medicine,
65(4): 782-791.
Williams, S. et al. 2012. “Methodological Reflections on the Use of Asynchronous Online Focus
Groups in Health Research.” International Journal of Qualitative Methods, 11(4):
368-383.
Zhang, Y. 2000. “Using the Internet for Survey Research: A Case Study.” Journal of the
American Society for Information Science, 51(1): 57-68.

국한문 참고문헌의 영문 표기
(English translation / Romanization of reference originally written in Korean)

Choi, S. J. 2016. “A Study on Effect of User Researcher Through Interview on Messenger -

Through Comparison Between Online and Offline.” Journal of the Korean Society Design
Culture, 22(3): 563-573.
Kim, K. S. 2017. “Trends of Qualitative Research in Korean Library and Information Science
Journals.” Journal of Korean Library and Information Science Society, 48(1): 373-396.
Kim, J. G. and Chang, D. H. 1994. “A Study on Present Stage of the Development of Public
Libraries in Korea - A Qualitative Analysis of the Consciousness of Librarians in Their
Everyday Lives in Pusan-Kyoungnam Area Public Libraries.” Journal of the Korean Society
for Library and Information Science, 12(3): 3-73.
Kwon, Y. M. and Myong, S. H. 2011. “Possibilities and Issues of Interviewing Surveys via
SNS.” The Korean Association for Public Administration, 12: 1-20.
Kwon, I. H. and Lee, S. W. 2005. “An Exploratory Study on Reliability Between Online and
Offline Survey Data Using a Single Panel.” Korean Management Review, 34(4): 1261-1278.
Lee, J. 2019. “An Exploratory Study on the Political Information Behaviors of Korean Opinion
Leaders on Twitter.” Journal of the Korean Society for Library and Information Science,
53(1): 83-108.

- 353 -