Gender & Sex in Methods
& Measurement
Research Equity Toolkit
Tool #3: Sampling Plans & Data Analyses
Gender & Sex in Methods & Measurement: Tool #3 1
When we recruit participants to our
research studies, they become a part of our
sample – the group of people from whom
we will collect data (or with whom we will
generate data, depending on our theoretical
framework) to answer our research
questions or test our hypotheses. There are
several points to consider about sampling
in relationship to gender, sex, and sexuality.
Here, we offer questions for researchers to
ask themselves, issues to carefully consider
and balance, and some illustrative example
situations to walk through as they consider
their research samples.
Gender & Sex in Methods & Measurement: Tool #3 2
Is the sample representative?
Does it need to be?
A sample is a smaller group of people, drawing from a full population;
the sample is representative when it reflects the larger population
of interest. A key feature of representative sampling is evaluating
representativeness, to ensure that the sample is an accurate reflection
of that larger population.
Consider
We might recruit 4000 participants to our study, to build a
representative sample of Canadians. We could use Statistics Canada
census data to determine how best to achieve representativeness,
with particular focus on gender representativeness.
However
Following a period of community and expert feedback, Statistics
Canada modernized its questionnaire in time for the 2021 Census,
allowing respondents to differentiate between sex assignment at birth
and current gender identity for the very first time. Based on the Survey
of Safety in Public and Private Spaces study conducted by Statistics
Canada in 2018, and the 2019 Census Test that tested the new
measurements, trans men, trans women and nonbinary people make
up approximately 0.35% of the Canadian population – roughly 133,000
people out of 38.01 million. With this statistic in mind, a 4000-person
sample of Canadians could be evaluated as representative if 14 of
the participants identified as trans and/or nonbinary (0.35% of 4000).
Whether or not this sample is truly representative is contentious
for a number of reasons. For example, the method for evaluating
representativeness depends on a newly updated national census, and
limited reliable data exist on how many trans and nonbinary people
there are in any given country, province, territory or region. Moreover,
there are many gender and sex identities that may not be represented
in the census.
3 CGSHE Research Equity Toolkit 2022
But, beyond these contentious issues, even if 14 participants is
representative according to national frequencies, this sample
size may present statistical and interpretive challenges when
it comes time for data analysis. Representativeness is thus one
issue that needs to be balanced against others when considering
gender, sex and sexuality in research.
Remember
We are frequently without accurate benchmarks against which
to measure and assess representativeness, especially when it
comes to estimating the prevalence of intersex, trans, nonbinary,
Two-Spirit, queer and other identities in the population.
Representativeness may not be achievable – or desirable!
Moreover, whether representativeness is a requirement for
“validity” (lack of bias) in quantitative research is a matter of
debate, with some experts asserting that representativeness is
an “overrated principle,” particularly if the aim is to compare
relationships between two variables rather than estimate the
prevalence of some thing (Nohr & Olsen, 2013).
On the collection of sex assignments
That sex is classified and assigned in binary ways is not without contention.
In addition, the legal requirement of a binary birth certificate sex ignores
the reality of intersex bodies, and this legal erasure is facilitated by doctors’
acceptance of a role as arbiter of a binary sex assignment decision for
intersex children, and presumption that they should impose interventions
to change the apparent sex characteristics of intersex children’s bodies
so that they will visually conform to their assigned binary sexes. Sex
assignment is then measured in binary ways, including on the census,
which does not attend to this complexity.
Gender & Sex in Methods & Measurement: Tool #3 4
Is the sample large enough to ensure
statistical power?
If our goal is to compare participant groups to find statistically
significant differences between groups or explore effects within
a single group that has some feature of their identity in common,
we need to ensure that the sample size for each participant group
is sufficiently large to achieve statistical power. A power analysis
can be performed before data collection for planned comparisons,
or afterwards with groups that emerge, to estimate minimal
sample size requirements for each of the groups. An underpowered
group can undermine statistical analyses, their outcomes and
their interpretations. However, when power analyses are done a
priori, researchers can purposively sample and even oversample
participants with specific identities or experiences according to their
research questions or hypotheses, to ensure sufficient sample size.
Oversampling involves intentionally including more participants
who identify in a particular way than would be representative of the
general population or which we could recruit by random chance,
even with the most thoughtful and inclusive recruitment strategies.
Consider
Sometimes, we know in advance that we are going to compare
participants grouped in particular ways (e.g., comparing trans
and nonbinary participants to cisgender ones; comparing sexual
majority participants to sexual minority ones). Where a priori planned
analyses are known, researchers can determine sample size needs to
achieve power and can recruit participants accordingly. They can also
consider whether to oversample.
Sometimes, researchers engage in post hoc, emergent analyses
and do not know in advance how they will group participants for
the purposes of analyses. This can occur when we are collecting
data ourselves, and when we are conducting secondary analyses of
already existing data. We may find that we are unable to run analyses
because there are “too few” participants who identify in a particular
way, or who have a particular experience. We may find that groups
are emergent and unknown to researchers in advance.
5 CGSHE Research Equity Toolkit 2022
We might therefore consider aggregating the data differently
– for example, instead of comparing trans and nonbinary
participants to cisgender ones, it may ultimately be appropriate
to include all trans and cis women into a single group, and
all trans and cis men into a single group, if we were going to
be using current gender identity as a key or covariable. This
may mean that that the nonbinary participants would be
underrepresented and perhaps also excluded from analyses,
having been separated them from participants who identify
as trans women and men. We might, when possible, continue
or renew sampling, to address an underpowered power by
additional, targeted recruitment and oversampling.
However
We need to balance oversampling with the potential of
exhausting the sample. We can exhaust the sample by
exacerbating research fatigue among over-studied communities,
and when there are few people who identify in particular ways.
For example, it may be challenging to oversample and recruit
sufficient numbers of trans and nonbinary participants, if we are
limiting eligibility to only one small geographic region, or only to
participants of a particular age. Oversampling may require that
we expand other elements of our eligibility criteria in order to
not exhaust the sample.
Remember
Oversampling to achieve statistical power is a strategy that
requires careful consideration. Where the research findings
have important implications for the daily lives and wellbeing
of marginalized people, oversampling can be a critical and
justifiable strategy, despite associated challenges. The benefits
and drawbacks of oversampling will need to be weighed against
the alternatives – either leaving the data of marginalized people
who are insufficiently represented unanalyzed or collapsing
categories and aggregating the data after the fact to achieve
power (see a discussion of these strategies on page 14).
Gender & Sex in Methods & Measurement: Tool #3 6
Can I ensure the anonymity &
confidentiality of participants?
Where people of marginalized and minoritized genders, sexes and
sexualities are included in our samples, we may need to pay special
attention to concerns regarding anonymity and confidentiality –
especially when these individuals are few.
Consider
There may be undue, negative consequences of being identifiable as
a participant in research, and therefore confidentiality is paramount
for many – being able to guarantee anonymity will foster trust and
facilitate disclosures. Even if we have achieved representativeness, or
groups of sufficient size to run adequately-powered analyses, there
may still be ethical concerns about individual participants being
identifiable, especially where data are disaggregated and viewable
by participant (e.g., individual responses, even without names, as
opposed to averages among variables). When our sample only includes
a handful of participants who identify in a particular way, we may want
to report on their data separately from our primary or key analyses
– in another paper, a footnote, or a supplemental file. This allows
these participants’ data to be analyzed and considered, rather than
excluded; it also allows for future meta- and systematic analyses to
be run using this data, when combined with data from other studies.
However, it may be ethically inappropriate to report on that data, if
there is a risk of participant identities being deduced.
However
Threats to anonymity and confidentiality are only ethical concerns
if they are concerns for the participants. There are many reasons
why participants may prefer to be identified and identifiable within
research outputs. For example, participants may be concerned over
the loss of ownership over their stories and experiences when data are
anonymized. There is an ethical dimension to researchers deciding
that concealing identities is of primary importance, especially when
that decision may result in data being left unanalyzed or aggregated in
ways that are not reflective of participants’ unique identities, or which
erase important inter-group differences.
7 CGSHE Research Equity Toolkit 2022
Within the context of research as a colonial tool, claims of
confidentiality concern have been used as a strategy to ensure
that smaller communities, including Two-Spirit people, are
silenced and erased. Where confidentiality is privileged, those
who are underrepresented within a sample and/or minoritized
within society will have their visibility or viability in the data
undermined or erased – their concerns and experiences will be
left unexplored; their needs will be left unmet in the research and
its impacts. For example, Two-Spirit people may be absorbed
into the “LGBTQ” acronym and collapsed with other, Western
gender and sexual minority populations for the purposes of
analysis. This may be done to protect against the threat of
identification when samples of Two-Spirit people are deemed
too small. However, this decision represents and reinforces
the problematic practice of equating Two-Spirit with Western
identities. Further, the resulting analysis will be insufficient to
attend to the nuances and specifics of Two-Spirit experiences. In
this way, confidentiality can serve as a mechanism that silences
Two-Spirit participants.
Remember
We can empower participants to decide for themselves to
what extent they value confidentiality or anonymity. This is
especially important due to research fatigue and considering
the historical and ongoing harms done by the pervasive erasure
of certain people from the research landscape. Consent forms
can speak to this tension. Participants can be invited to reflect
on and sign off on how their data can be used if researchers are
concerned that data will need to be excluded from analyses to
guarantee confidentiality. Researchers are also responsible for
ensuring that they are effectively recruiting enough participants
who identify in diverse ways to alleviate this risk. Researchers
must also deidentify participants so that the sociodemographic
descriptors used cannot, alone or in combination, render
any single participant identifiable. Deidentification may be
important no matter how many participants in the sample
identify similarly.
Gender & Sex in Methods & Measurement: Tool #3 8
Will a convenience sample introduce
problematic bias?
Using a non-probability sampling strategy called convenience
sampling selects participants based on accessibility and availability,
trading these benefits with some detractions. For example,
convenience sampling may introduce bias into samples and, in most
cases, the sample will not be representative of the population of
interest. In designing research studies with the intent of including
people who are marginalized and minoritized based on their genders,
sexes and sexualities, we need to be aware of some sampling biases
that occur due to common convenience sampling techniques.
Consider
As part of our recruitment and associated oversampling strategies,
we might, for pragmatic reasons, circulate our recruitment materials
among local universities’ and colleges’ Pride clubs. We will need to
consider, however, that participants recruited in this way may be
younger, more formally educated and from higher socioeconomic
statuses than the rest of our sample. Additionally, we may utilize
facility-based sampling and attempt to recruit additional trans,
nonbinary, gay, lesbian, bisexual and queer participants who have
accessed a local sexual health clinic. In this case, we may discover that
these participants are different from those who were not recruited as
patients at a particular facility in fundamental ways – perhaps they
have more sexual partners on average, more instances of HIV/STBBI
testing, more skill at STBBI protection practices, or are simply more
likely to have a family practitioner than the rest of our sample.
However
Whether these targeted recruitment strategies introduce sampling
bias into our study will depend on several factors that can be better
elucidated with clear reporting of sampling strategies and approaches.
9 CGSHE Research Equity Toolkit 2022
Remember
Transparency in reporting is key. If we are aware that our
recruitment strategies have introduced bias into our samples,
it is prudent to acknowledge those biases, make visible to
readers their impact on validity, and highlight their limits to the
generalizability and transferability of our findings. A common
convenience sampling technique is to recruit trans, queer
and Two-Spirit participants from physical health and mental
health clinics. It would thereafter be inappropriate, invalid
and harmful to argue that these populations have increased
rates of mental health diagnoses compared to cisgender,
heterosexual participants recruited primarily through other
strategies. Acknowledging sampling bias is especially important
for how we report findings associated with people who are
already marginalized and minoritized, where the reification of
stereotypes can have profound, negative impacts on health,
wellness and quality of life. There are a range of quantitative
bias analysis methods that can be used to try to quantify these
biases in transparent ways.
Gender & Sex in Methods & Measurement: Tool #3 10
What happens when certain types of
participants are underrepresented in
our samples?
Intersex, trans, nonbinary, Two-Spirit, queer and other people who
are marginalized and minoritized based on gender, sex and sexuality
are more likely to be underrepresented in our samples as compared to
cisgender, heterosexual participants. This is because people with these
experiences or who identify in these ways are at once minorities and
minoritized (see Sotto-Santiago, 2019, for a discussion of how these
framings differ and how this difference matters). This means that the
world presents them with more barriers, and that they are fewer in
numbers. We may find that these participants are “too few” within our
samples - too few to ensure confidentiality and/or too few to achieve
statistical power for the purposes of running analyses. Apart from
purposively recruiting in advance, there are three primary strategies for
addressing the “too few” problem once the sample has been collected:
(a) excluding underrepresented participants, (b) aggregating/clustering/
collapsing data into larger groups or (c) thinking of other ways to group
participants, along other shared axes so that resulting groups are
sufficiently sized. In general, the decision of which strategy to utilize
needs to be determined thoughtfully, critically and transparently.
Be thoughtful
Oftentimes, researchers exclude underrepresented participant data
because that is what they have always done, or because doing so is easy
or makes for ostensibly cleaner data sets. Or researchers collapse certain
types of underrepresented participants into larger groups because
that is what other researchers have done and it seems that it can be
justified by the extant literature, even if those other researchers were not
transparent in their explanations and justifications of their choice.
Be critical
Excluding or collapsing underrepresented participant data will have
consequences – to the participants, to the research findings and to
secondary, meta and systematic analyses that can be conducted using
the data in the future. For example, although a single study may have
reduced statistical power, data from this study may be systematically
reviewed and meta-analyzed alongside small samples from other
studies. The impacts of the decision to exclude or collapse participant
data must be weighed against the potential benefits. This includes
impacts and benefits in terms of data quality for the present and future
research, but also to the participants and the communities from which
data are drawn.
11 CGSHE Research Equity Toolkit 2022
Be transparent
No matter what is decided, researchers need to be transparent
in their research outputs about what they did, why they did
it, and what they see as the impacts of their decision. This will
allow readers to assess for themselves whether the decision was
justified. Transparency will also allow other researchers to learn
by example, rather than replicating approaches that may be
problematic simply because that is how things have generally
been done.
On excluding underrepresented participants
When participants are deemed “too few” within a sample, these
participants’ data may be removed from the data set, excluded
from analyses or otherwise ignored. It is important to remember
that, just as there are psychological costs associated with being
excluded from research at the stage of recruitment, there are also
psychological costs of having participated in research only to have
your data ignored or to seeing people like you excluded from or
not represented in analysis.
We need to carefully consider the impacts of exclusion at various
stages of the research process, considering the wellbeing of
prospective and actual participants as individuals, as well as
the potential impacts of exclusion on the generalizability and
transferability of our findings. We also need to consider whether
that exclusion is justified at which or any stage and explain that
justification in our research outputs. It could be that we are
deeming the participants as “too few” because we are aggregating
the data in a way that does not follow from our research questions
or hypotheses – that there are, in fact, sufficient numbers of these
participants to run analyses if we were to approach the data, and
groups within the data, in a slightly different way (see the section
below on aggregating data!).
Gender & Sex in Methods & Measurement: Tool #3 12
Example 1: Where membership in a marginalized
group is not being used a key, control or covariable
Drs. Smith, Wang and Singh are researchers at Imaginary University.
They are conducting a retrospective chart review to better
understand the relationship between fibroid size, location and
quantity, and intrauterine device (IUD) expulsion among patients
of different ages. While reviewing the 279 charts that noted IUD
expulsion, they find five patients with an “M” on their health care
card, and two with an “X”. A further two charts with an “F” on their
health care card included provider notes that indicated that the
patients were trans and nonbinary people.
Decision
Since none of their planned analyses pertain to gender identity and
they are not using gender as a covariate, the researchers include
all participants in their data set, regardless of gender/sex marker.
In their report, the researchers mention the gender markers of all
participants, and state that, given the diversity of gender markers,
their findings may be generalizable to people of various genders
and legal sexes. Their literature review found that while IUD use
among trans and nonbinary people is increasing, little is known
about their IUD experiences, side effects and expulsion rates. The
researchers were surprised to find that 3.25% of their sample were
trans and nonbinary people; they therefore suggest future research
to determine whether trans and nonbinary people with fibroids are
more likely to expel IUDs as compared to cisgender women.
Remember
While there may be participants whose presence was unexpected
in your research related to gender, sex or sexuality, excluding them
from the research just because of that or because there are few of
them is rarely justifiable, especially if that variable is not part of
your planned analysis.
13 CGSHE Research Equity Toolkit 2022
Example 2: Where participant identity is key to
your analysis
PhD candidate Ravi Das is conducting a study that explores
the relationships between gender, sexual orientation and the
sexualized use of substances. She develops a survey instrument
that allows her to classify each respondent based on gender
identity and sexual orientation.
When analyzing her data, she discovers that she has relatively
equal numbers of participants by some genders – cisgender
men, cisgender women, trans men and trans women – as well as
some sexualities – heterosexual/straight or queer/bi/gay/lesbian.
However, she finds that there only a small handful of nonbinary
people, an insufficient number for statistical analysis.
Decision
In her dissertation, she explains why the nonbinary participants
were excluded from the analysis, explores why her recruitment
strategy may have been ineffective at soliciting nonbinary
participants, and explains that her findings are not generalizable
to nonbinary people of any sexuality. She explains that it did
not feel appropriate to collapse trans men, trans women, and
nonbinary people into a single category – which would have
been necessary to achieve power. She writes about how this
might not have honoured the identities of these participants
and would not have been in keeping with her research question,
which was about whether and why people of different gender
and sexual identities engage in sexualized substance use.
Though the collapsing would have facilitated the inclusion
of the nonbinary participants, it would have also meant
overlooking the inter-group differences among trans people
of different genders. Ravi identifies this as a limitation of her
project and in her subsequent work she endeavours to ensure
that this limitation is not replicated; to do so, she purposively
oversamples nonbinary participants of diverse sexualities in her
next study.
Remember
There is no one-size-fits-all approach or list of circumstances,
where excluding underrepresented data is justified or not.
Thoughtfully, critically, and transparently disclosing the factors
that contributed to the decision and reflecting on the impacts of
that decision are key.
Gender & Sex in Methods & Measurement: Tool #3 14
On aggregating/clustering/collapsing data
When participants are deemed “too few” within a sample, these
participants’ data may be aggregated, clustered or collapsed, in order
to create groups that are large enough to analyze on their own and/
or to be compared with other groups. How best to aggregate variables
will depend on your research questions, hypotheses and theoretical
framework as well as the potential impacts if data is not aggregated
(e.g., whether data will be excluded). When collapsing data categories
in this way, we need to do so based on likeness, where participants
that are understood as the same, or as sharing something important in
common are grouped together. There are some common approaches
to aggregating the data of participants who are marginalized and
minoritized based on their genders, sexes and sexualities.
Example 1: Gender and sexual minorities
Dr. Brown has been involved in a longitudinal cohort study of women
in Urban City for 10 years. In previous years, the survey was focused on
the experiences of cisgender, heterosexual women.
Two years ago, the survey was updated to be more attentive to the
experiences of women who are minoritized based on axes of sexuality
(specifically queer, bisexual and lesbian women) and based on axes
of gender (specifically trans and nonbinary women). The project’s
recruitment strategy has been updated to target these women. Dr.
Brown intends to analyze gender, sexuality and access to sexual health
services by comparing trans women and cisgender women, and then
straight women and queer/bi/lesbian women. She runs two parallel
analyses and creates two tables. She finds statistically significant
differences between the queer/bi/lesbian women and the straight
women, where marginalized and minoritized sexual identity is found
to be associated with sexual health service access. She finds a similar
trend when comparing the trans women and cisgender women in the
sample – trans women are less likely to access sexual health services
and more likely to experience discrimination when accessing these
services. However, the trend is not statistically significant. She is
cautious about how to interpret this association and worries that
publishing these results would imply that gender minority women do
not experience barriers to sexual health care in significant ways. She
runs the analysis again, this time comparing all gender and sexual
minority women to the gender and sexual majority women. The
direction of the association remains the same, and association is now
stronger – cisgender, heterosexual women experience fewer barriers
and instances of discrimination as compared to the trans, queer,
bisexual, and lesbian women.
15 CGSHE Research Equity Toolkit 2022
Decision
Dr. Brown produces a manuscript of her findings – she
provides a detailed rationale of her first analysis, why she
is collapsing gender and sexual minority women together,
and details the limitations of this decision, including that the
important differences between these two groups of women
are left unclear and in need of further exploration.
Remember
Sexual and gender minorities (SGM), a commonly use phrase
in research and community, also represents a common
approach to collapsing and categorizing participant data.
These groups are often stigmatized on some shared bases
including presumptions around proscriptive gender norms, so
there can also be some “likeness” as articulated above. But,
collapsing them is not always appropriate or justified. When
data are aggregated in this way, it suggests to readers that
these distinct groups of people are fundamentally the same for
the purposes of the analysis and findings. This may represent
a conflation between gender and sexuality and a failure
to consider the unique differences between these groups.
Research that looks at sexual and gender minorities must be
done thoughtfully, critically and transparently, acknowledging
the assumptions, limits and impacts of this classification.
Example 2: The ‘Everybody-but-Cis-Men’
Approach
Masters student Romesh de Silva is working as an RA at
the Fictious Centre for Studying People and Stuff. They are
working with an existing data set, where the participants are
described as men, women and trans people – with the fewest
participants representing this latter group. They have been
asked by their PI to run a correlational analysis to explore the
relationship between gender and daily incidents of sexualized
street harassment.
Gender & Sex in Methods & Measurement: Tool #3 16
Romesh is not sure whether they should exclude data from the trans
group from the analysis or recode these participants’ data as belonging
either the men’s group or the women’s group. They cannot now
determine precisely how these participants identified as they did not
collect this data, nor do they know what the survey instrument looked
like. Accordingly, they ask the centre’s statistician for advice on what
do with the trans group. The statistician recommends that the trans
participants get collapsed with the women, and thereafter described
as two groups – cis men and everybody else.
Decision
The statistician’s suggestion was based on data utility rather than a
nuanced understanding of the relationship between gender, gender
expression and sexualized street harassment. Romesh nevertheless
moves forward with this plan and recodes the data accordingly.
In their report to the PI, Romesh explains their choice – that it was
more appropriate to compare cisgender men to all other participants
than to exclude the trans participants from the analysis, even if this
means that the few trans men in the sample are not included in the
“men” category. Romesh relabels the “men” group as “cisgender
men,” accordingly. They suggest to the PI that future data collection
more carefully attend to the complexity of gender identity as well as
to gender expression, which the research shows is also an important
contributing factor to sexualized street violence, even among
cisgender men.
Remember
Collapsing all trans participants, regardless of gender identity, into the
same category as cisgender women may erase important differences
between these diverse people, and as such is typically inappropriate
and unadvisable. We always need to ask ourselves whether doing so is
appropriate and justifiable based on our research question, hypothesis
or theoretical framework. In Romesh’s case, the participants do not
share gender in common, but the PI’s interest is in the relationship
between gender and sexualized street harassment. However, the
participant group may share some experiences of gender oppression
in common, even if not the same kind, which may be a reasonable
justification for their being collapsed in this way given the limitations
of the data set and the costs associated with other approaches.
17 CGSHE Research Equity Toolkit 2022
Example 3: Trans women and MSM
Dr. Summers is a researcher at Greendale Community College
studying HIV. He is particularly interested in understanding HIV
vulnerabilities among marginalized and precarious people –
people who are undocumented, incarcerated and/or unhoused.
He has noticed a trend in his field, where researchers explore HIV
among men who have sex with men (MSM) and trans women,
as though there is a commonality between these two groups
that justifies their being discussed as one group. Dr. Summers is
aware that his work has contributed to this trend.
Decision
Dr. Summers returns to data that his lab collected two years ago,
and the manuscripts that were developed where MSM and trans
women were collapsed together as a single group. He realizes
that this decision was likely the result of too few trans women
in the sample such that the team could not reach meaningful
conclusions. He also reflects on his own false assumption –
he had assumed that the trans women were all people with
penises, who were having sex with other people with penises
and, as such, that their sexual behaviours and practices meant
that they would be indistinguishable from the cisgender men in
the sample who have sex with other cisgender men. The more he
digs into his own data, the more he realizes the error behind his
assumptions around gender and sexuality – not only are some of
the trans women in his sample having sex with women (some of
whom may have penises and many of whom likely do not), but
some of the men in the sample are trans, who have a variety of
body parts that they use sexually in a variety of ways!
Remember
We may find approaches to collapsing data in the literature that
are repeatedly replicated – but that doesn’t mean that they
are justified. Some of these approaches – like the conflation
of trans women and MSM in the HIV literature – are based on
assumptions about who these people are, what body parts they
have, and how they use those body parts for the purposes of
sex or reproduction. When we start to dig into these common
approaches to collapsing data and find that they not only do not
make sense, but may cause harm, our research and communities
will benefit from new directions in grouping our participants.
Gender & Sex in Methods & Measurement: Tool #3 18
A note on qualitative research, recruitment
& sampling
Much of this tool focuses on quantitative research with relatively
large sample sizes, and the challenges associated with recruiting
participants, collecting, coding, aggregating and collapsing data.
Sampling in qualitative research is likely to be non-probabilistic,
including purposeful, selective, quota, snowball or convenience
based. Here, the focus is on recruiting a relatively small number
of participants who can offer insights into the topic, phenomenon
or experience that is being studied. But many of the same
principles hold, including how themes are analyzed within groups
and subgroups, how to treat insights that are derived from
social locations that have only one participant represented, etc.
Importantly, people who are marginalized and minoritized based
on their genders, sexes and sexualities are also underrepresented
in qualitative research and failure to include participants of all
types in qualitative research is no less serious.
19 CGSHE Research Equity Toolkit 2022
Additional readings
Anderssen, N., & Malterud, K. (2017). Oversampling as a
methodological strategy for the study of self-reported health
among lesbian, gay and bisexual populations. Scandinavian
Journal of Public Health, 45(6), 637-646.
Bauer, G. R., & Scheim, A. I. (2013). Sampling bias in transgender
studies. The Lancet: Infectious Diseases, 13(10), 832.
Beischel, W. J., Schudson, Z. C., Hoskin, R. A., & van Anders, S.
M. (2022). The gender/sex 3x3: Measuring and categorizing
gender/sex beyond binaries. Psychology of Sexual Orientation
and Gender Diversity. Advance online publication.
Henderson, E. R., Blosnich, J. R., Herman, J. L., & Meyer, I. H.
(2019). Considerations on sampling in transgender health
disparities research. LGBT Health, 6(6), 267-270.
Hottes, T. S., Ferlatte, O., & Gilbert, M. (2015). Misclassification and
undersampling of sexual minorities in population surveys.
American Journal of Public Health, 105(1), e5.
Krause, H. (2020, Jan 17). Keeping data inclusivity without diluting
your results. We All Count. https://weallcount.com/2020/01/17/
keeping-data-inclusivity-without-diluting-your-results/
Nohr, E. A., & Olsen, J. (2013). Commentary: Epidemiologists have
debated representativeness for more than 40 years – has the
time come to move on? International Journal of Epidemiology,
42(4), 1016-1017.
Reisner, S. L., Deutsch, M. D., Bhasin, S., Bockting, W., Brown, G.
R., Feldman, J., Garofalo, R., Kreukels, B., Radix, A., Safer, J. D.,
Tangpricha, V., T’Sjoen, G., & Goodman, M. (2017). Advancing
methods for U.S. transgender health research. Current Opinion
in Endocrinology, Diabetes and Obesity, 23(2), 198-207.
Salway, T. J., Morgan, J., Ferlatte, O., Hawkins, B., Lachowsky, N.
J., & Gilbert, M. (2019). A systematic review of characteristics
of nonprobability community venue samples of sexual
minority individuals and associated methods for assessing
selection bias. LGBT Health, 6(5), 205-215.
Sotto-Santiago, S. (2019). Time to reconsider the world minority
in academic medicine. Journal of Best Practices in Health
Professions Diversity, 12(1), 72-78.
Surmiak, A. D. (2018). Confidentiality in qualitative research
involving vulnerable participants: Researchers’ perspectives.
Forum: Qualitative Social Research, 19(3).
Gender & Sex in Methods & Measurement: Tool #3 20
Co-Authors/Advisory Team Suggested Citation (APA)
Dr. A.J. Lowik is the lead author. Co-authors include Lowik, A., Cameron, J. J., Dame, J., Ford, J.,
Dr. Jessica Cameron, Jessy Dame, Dr. Jae Ford, Lex Pulice-Farrow, L., Salway, T., van Anders, S., &
Pulice-Farrow, Dr. Travis Salway, Dr. Sari van Anders. Shannon, K. (2022). Gender and Sex in Methods
& Measurement - Research Equity Toolkit. “Tool
Funding
#3: Sampling Plans and Data Analyses,” Centre for
The Project Lead is Dr. A.J. Lowik, Gender and Sexual Health Equity, University of
CGSHE Gender Equity Advisor at UBC. British Columbia.
This work is funded by a CIHR Sex and Gender Note: The advisory authors are listed
Science Chair in Gender-Transformative Sexual alphabetically, each having contributed equally
Health (PI: Dr. Kate Shannon). to this collaborative project; the exceptions to the
alphabetical ordering are the first author (who is
Design & Development
the project lead and primary author) and the last
Layout & design by Shivangi Sikri & Dr. Kate Milberry, author (who is the senior author/PI of CIHR Chair
CGSHE-UBC Comms Team. that funded this project).
Advisory & resource support by Carlie McPhee,
CGSHE Project Coordinator for Sexual Health.
