Survey Research Methods (2023) © 2023Author(s)

Vol. 17, No. 3, pp. 219-222

doi:10.18148/srm/2023.v17i3.8317
European Survey Research Association CC BY 4.0

Recent Methodological Advances in Panel Data Collection, Analysis,

and Application
Sabine Zinn1,2 and Tobias Wolbring3
1
DIW Berlin; German Socio Economic Panel Study
2
Humboldt University Berlin
3
FAU Erlangen-Nürnberg; School of Business, Economics and Society

Panel studies have become an indispensable part of today’s research world especially when
addressing causal questions and tracking changes over time. Three conditions are essential for
effective panel data analysis: 1) having a sufficiently long time series with a substantial number
of observations, 2) ensuring measurement consistency over time, and 3) using a meaningful
model for selecting elements from the target population. To meet these conditions, survey
research provides appropriate tools (e.g., effective motivational strategies to encourage panel
participation or statistical techniques to assess selection and measurement bias). However, it
is crucial for researchers and data analysts to not only use these resources, but also remain
vigilant regarding potential pitfalls. In addition, new data collection methods are emerging
that require researchers to assess their capabilities. This special issue addresses these demands
by presenting research on incentive systems and their effects, measurement problems in panel
studies, and new applications of panel data.

Keywords: panel; data collection, panel analysis

The list of panel studies has grown considerably in recent Youth (JLPS-Y), the African Cape Area Panel Study (CAPS)
times, and this expansion is warranted for several reasons. on health issues, and the Australian Election Study (AES), to
Due to the widely acknowledged challenges associated with mention just very few.
cross-sectional analyses when addressing causal questions For panel studies to yield valuable and high-quality find-
and the constraints of randomized experiments, scholars in- ings, three essential conditions must be satisfied. Firstly, a
creasingly rely on panel data for causal inference. Addition- sufficiently long time series of a substantial number of obser-
ally, panel data represents the sole practical resource for ex- vations is necessary to map changes both within and between
ploring changes within individual entities over time, ensur- entities. Secondly, it is imperative that the measurements re-
ing temporal order of cause and effects and offering a valu- main consistent over time, ensuring that the same variables
able solution to the issue of ecological fallacy in the study of are assessed consistently for the observed entities across dif-
social dynamics. ferent time points. Thirdly, to create broad statements about
The selection of entities to observe in panel data analysis the population, the underlying sample must originate from a
is contingent upon the specific research inquiry. In the realm quantifiable and well-controlled data generation process.
of social sciences, these entities typically encompass indi-
To attain the first condition, effective procedures for re-
viduals, households, or businesses. Nowadays, worldwide
cruiting and maintaining the observational units within the
panel studies encompass an extensive array of diverse sub-
panel are necessary, but also getting reliable and valid re-
jects. For example, there are large-scale and long-running
sponses is mandatory. In essence, this entails the imple-
general population household panels like the Panel Study of
mentation of motivation strategies and the maintenance of a
Income Dynamics (PSID) in the U.S., Understanding Society
seamless survey process. Common methods of motivation
in the U.K., and the Socio-Economic Panel (SOEP) in Ger-
are providing information and incentives and maintaining
many. But there exists also a great variety of topic-specific
contact. That is, respondents receive information about the
panel studies such as the German National Education Panel
study and its objectives commonly through letters, brochures
Study (NEPS), the Japanese Life Course Panel Survey of the
(sent via postal mail or electronically), and web pages. In-
centives foster high survey participation, especially when
providing unconditional monetary incentives shortly before
Contact information: Sabine Zinn, Deutsches Institut für the survey (Pforr et al., 2015). Also staying in touch with
Wirtschaftsforschung, Sozio-oekonomisches Panel, Mohrenstrasse respondents between survey waves is advantageous in this
58, 10117 Berlin, Germany (E-mail: szinn@diw.de). respect, as it helps to uphold their commitment to the study
220 SABINE ZINN AND TOBIAS WOLBRING

and ensures that contact information remains up to date. The third essential requirement for effective panel data
A seamless survey process requires questionnaires that are analysis is having a meaningful model for the selection of
understandable, i.e., not too complex concerning cognition elements from the target population. The statistical theory of
and visualization, and, at best, entertaining as well as survey sampling makes this a mandatory condition (Kish, 1995). A
environments without disturbance and inconvenience. That straightforward method to meet this requirement is to use a
way, respondents can answer truthy and without feeling un- random sample drawn according to a well-defined sampling
comfortable, thus minimizing the risk of misreporting, satis- design. Such a design enables the calculation of inclusion
ficing, item-nonresponse and break offs. Instruments to reach probabilities, which are used to determine design weights for
this include preloads (e.g., answers from previous waves are extrapolation purposes.
given as a starting point), short questionnaires, and targeted In the course of a panel study, it is common to experience
survey modes (e.g., self-administered surveys for sensitive attrition with participants dropping out over time. Typically,
questions and interviewer-based modes for complex ques- this attrition is quantifiable based on the initial gross amount
tions such as inquiries on household income). of survey entities, as specified in the sample design. Data
It is crucial to acknowledge potential mode and inter- from the panel itself (pre-wave information), as well as con-
viewer effects may introduce bias in target statistics when textual details about both respondents and non-respondents.
dealing with panel data, especially when combining modes The latter is available, for example, through interviewer ob-
for cost-efficiency. For instance, in a scenario where servations or external data sources such as small-scale re-
both Computer-Assisted Personal Interviews (CAPI) and gional data.
Computer-Assisted Web Interviews (CAWI) are used simul- However, when the data-generating process is unknown
taneously, there is a significant likelihood that each mode (e.g., in non-probability samples), it becomes very difficult
will yield varying attitude estimates (see Groves et al., 2011, to carry out this correction effectively. There are adjust-
for reference). This is because selection and measurement ment procedures such as reweighting claiming to make non-
may function differently across modes (e.g., Campanelli et probability samples useful for generalization to the popula-
al., 2015; Martin & Lynn, 2011; Vannieuwenhuyze et al., tion level (e.g., Liu et al., 2022). However, they rely on
2010). This circumstance also makes it difficult to satisfy the assumptions that are frequently quite demanding (Kohler,
second condition: invariance of measurements over time. 2019; Kohler et al., 2019) or require an extensive amount of
benchmark information sourced from random samples, pop-
Meeting this requirement is essential when analysing
ulation registries, or census data. Ideally, these benchmark
panel data as it guarantees the consistency of constructs.
data would be available on a longitudinal basis, which is sel-
Correcting measurement errors is possible when they are
dom the case for population registries and census data. As
identified or can be modelled (see, for example, Nakamura,
a result, well-constructed and well-maintained panel surveys
1990). However, addressing this issue necessitates aware-
often remain the only viable data source for tracking societal
ness and the use of suitable methodologies, such as mea-
changes on a micro, meso and macro level with acceptable
surement models. In general, measurement invariance serves
data quality.
as a quality benchmark and is one of the minimum criteria
when designing new questions and item sets (e.g., Leitgöb et Hence, survey research needs to consistently introduce
al., 2023; Vandenberg & Lance, 2000). Nonetheless, many and enhance effective techniques for choosing panel samples
studies do not automatically adhere to this standard (see, for across diverse settings (such as households, individuals, and
instance, Rutkowski & Svetina, 2014). There is also limited businesses) and in various domains (including general pop-
effort dedicated to regularly evaluating existing measurement ulation surveys, health assessments, and studies of migrant
instruments for their suitability and limited awareness in ap- communities). Moreover, there is an ongoing and pressing
plied panel research for this important precondition. Addi- requirement for methods to sustain panel stability over multi-
tionally, issues of comparability can arise when translating ple survey waves and concepts for regularly refreshing (prob-
questions into different languages. A direct translation does ability) panel samples.
not guarantee that respondents will interpret the questions in In this context, this special issue explores scientific in-
the same way. Question comprehension and response pat- quiries related to panel data within the dynamic interaction
terns can be influenced by culture (e.g., Dong & Dumas, between methodological rigor and practical data needs. The
2020; Emerson et al., 2017). Therefore, translations should following eight papers published in this special issue advance
also provide evidence of measurement invariance, which is knowledge on the collection and analysis of panel data in im-
often overlooked (ibid.). The likely reason is the contempo- portant ways:
rary need for swift data collection and analysis, sometimes A first set of studies addresses the issue of suitable in-
at the expense of data quality and result reliability. Survey centive schemes in panel studies and highlights the effective-
methodology research has a role in highlighting this shortfall ness of prepaid incentives. Becker (2023) delves into this
(Meitinger et al., 2020). topic theoretically, emphasizing the concept of reciprocity in
 RECENT METHODOLOGICAL ADVANCES IN PANEL DATA COLLECTION, ANALYSIS, AND APPLICATION 221

unconditional prepaid incentives, and provides empirical ev- Becker, R. (2023). The researcher, the incentive, the panelists
idence for important heterogeneity in panelists’ preference and their response: The role of strong reciprocity for
for strong reciprocity. the panelists’ survey participation. Survey Research
Beste et al. (2023), on the other hand, experiment with Methods, 17(3), 223–242. https://doi.org/10.18148/
various machine learning methods to assess their utility in srm/2023.v17i3.7975
predicting fieldwork outcomes based on prior wave data, Beste, J., Frodermann, C., Trappmann, M., & Unger, S.
leading to the development of an adaptive incentive scheme (2023). Case prioritization in a panel survey based
that they test through experimentation. on predicting hard to survey households by ma-
Another group of papers in the special issue deals with chine learning algorithms. Survey Research Meth-
response behaviour and measurement issues. Kraemer et al. ods, 17(3), 243–268. https://doi.org/10.18148/srm/
(2023) investigate satisficing behaviour across different panel 2023.v17i3.7988
waves, utilizing a six-wave experimental approach. They Campanelli, P., Blake, M., Mackie, M., & Hope, S. (2015).
detect satisficing behaviour within individual waves but not Mixed modes and measurement error: Using cogni-
consistently across waves. tive interviewing to explore the results of a mixed
Rettig and Struminskaya (2023) also address the problem modes experiment [ISER Working Paper Series,
of memory effects in panel studies. They do find such effects, (No. 2015-18)]. https://www.econstor.eu/bitstream/
but only on a small scale. Consequently, they conclude that 10419/126482/1/836342755.pdf
the potential for measurement errors due to memory effects Cornesse, C., Blom, A., Marie-Sohnius, L., González
across panel waves is minimal (especially after four months Ocanto, M., Rettig, T., & Ungefucht, M. (2023).
or longer). Experimental evidence on panel conditioning ef-
Cornesse et al. (2023) explore the impact of significantly fects when increasing the surveying frequency in
increasing survey frequency in an ongoing panel. They a probability-based online panel. Survey Research
present an experimental study conducted during the initial Methods, 17(3), 323–339. https://doi.org/10.18148/
pandemic period where respondents were queried weekly. srm/2023.v17i3.7990
They identify conditioning effects solely on questions related Dong, Y., & Dumas, D. (2020). Are personality measures
to COVID-19. valid for different populations? A systematic review
Paccagnella and Guidolin (2023) study the application of of measurement invariance across cultures, gender,
anchoring vignettes to address measurement invariance be- and age. Personality and Individual Differences,
tween groups. They investigate both priming effects and 160, 109956. https://doi.org/10.1016/j.paid.2020.
panel conditioning effects finding evidence of such effects 109956
in questions measuring customer satisfaction with a service. Emerson, S. D., Guhn, M., & Gadermann, A. M. (2017).
Finally, two papers in this special issue contribute to Measurement invariance of the satisfaction with life
the use of panel data in specific substantive research areas. scale: Reviewing three decades of research. Quality
Kopycka et al. (2023) describe an innovative use of cross- of Life Research, 26, 2251–2264. https://doi.org/10.
national panel data to create a new index for assessing em- 1007/s11136-017-1552-2
ployment precarity. They validate this index by measuring Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski,
adverse labour market experiences in both Germany and the J. M., Singer, E., & Tourangeau, R. (2011). Survey
U.S. using data from established panel studies. methodology. Wiley.
Lastly, Barth and Blasius (2023) present a panel study fo- Kish, L. (1995). Survey sampling. Wiley.
cused on metropolitan dwellings and their role in understand- Kohler, U. (2019). Possible uses of nonprobability sampling
ing neighbourhood development. The primary emphasis of for the social sciences. Survey Methods: Insights
their study lies in analysing rent development and its mea- from the Field. https : / / doi . org / 10 . 13094 / SMIF -
surement. 2019-00014
Kohler, U., Kreuter, F., & Stuart, E. A. (2019). Nonprobabil-
References ity sampling and causal analysis. Annual Review of
Statistics and its Applications, 6, 149–172. https://
Barth, A., & Blasius, J. (2023). Assessing rental price dy-
doi.org/10.1146/annurev-statistics-030718-104951
namics in two gentrified neighbourhoods in cologne
Kopycka, K., Kiersztyn, A., Sawiński, Z., Bieńkowski, S.,
by means of a dwelling panel. Survey Research
& Sovpenchuk, V. (2023). Use of panel surveys to
Methods, 17(3), 395–410. https://doi.org/10.18148/
measure employment precarity in a cross-national
srm/2023.v17i3.7987
framework. Survey Research Methods, 17(3), 353–
393. https://doi.org/10.18148/srm/2023.v17i3.7989
222 SABINE ZINN AND TOBIAS WOLBRING

Kraemer, F., Silber, H., Struminskaya, B., Bernd Weiß, M., Survey Research Methods, 17(3), 341–352. https :
Bosnjak, Koßmann, J., & Sand, M. (2023). Satisfic- //doi.org/10.18148/srm/2023.v17i3.7993
ing response behavior across time: Assessing neg- Pforr, K., Blohm, M., Blom, A. G., Erdel, B., Felderer, B.,
ative panel conditioning using an experimental de- Fräßdorf, M., Hajek, K., Helmschrott, S., Kleinert,
sign with six repetitions. Survey Research Methods, C., Koch, A., Krieger, U., Kroh, M., Martin, S.,
17(3), 269–300. https : / / doi . org / 10 . 18148 / srm / Saßenroth, D., Schmiedeberg, C., Trüdinger, E.-M.,
2023.v17i3.7986 & Rammstedt, B. (2015). Are incentive effects on
Leitgöb, H., Seddig, D., Asparouhov, T., Behr, D., Davidov, response rates and nonresponse bias in large-scale,
E., De Roover, K., Jak, S., Meitinger, K., Menold, face-to-face surveys generalizable to Germany? Ev-
N., Muthén, B., Rudnev, M., Schmidt, P., & van de idence from ten experiments. Public Opinion Quar-
Schoot, R. (2023). Measurement invariance in the terly, 79(3), 740–768. https://doi.org/10.1093/poq/
social sciences: Historical development, method- nfv014
ological challenges, state of the art, and future per- Rettig, T., & Struminskaya, B. (2023). Memory effects in on-
spectives. Social Science Research, 110, 102805. line panel surveys: Investigating respondents’ abil-
https://doi.org/10.1016/j.ssresearch.2022.102805 ity to recall responses from a previous panel wave.
Liu, A. C., Scholtus, S., & De Waal, T. (2022). Correcting se- Survey Research Methods, 17(3), 301–322. https :
lection bias in big data by pseudo-weighting. Jour- //doi.org/10.18148/srm/2023.v17i3.7991
nal of Survey Statistics and Methodology, smac029. Rutkowski, L., & Svetina, D. (2014). Assessing the hypoth-
https://doi.org/10.1093/jssam/smac029 esis of measurement invariance in the context of
Martin, P., & Lynn, P. (2011). The effects of mixed mode large-scale international surveys. Educational and
survey designs on simple and complex analyses Psychological Measurement, 74(1), 31–57. https://
[ISER Working Paper Series, No. 2011-28]. https: doi.org/10.1177/0013164413498257
//www.europeansocialsurvey.org/sites/default/files/ Vandenberg, R. J., & Lance, C. E. (2000). A review and
2023 - 06 / The % 5C % 20effect % 5C % 20of % 5C % synthesis of the measurement invariance literature:
20mixed % 5C % 20mode % 5C % 20survey % 5C % Suggestions, practices, and recommendations for
20designs.pdf organizational research. Organizational Research
Meitinger, K., Davidov, E., Schmidt, P., & Braun, M. (2020). Methods, 3(1), 4–70. https : / / doi . org / 10 . 1177 /
Measurement invariance: Testing for it and explain- 109442810031002
ing why it is absent. Survey Research Methods, Vannieuwenhuyze, J., Loosveldt, G., & Molenberghs, G.
14(4), 345–349. https : / / doi . org / 10 . 18148 / srm / (2010). A method for evaluating mode effects in
2020.v14i4.7655 mixed-mode surveys. Public Opinion Quarterly,
Nakamura, T. (1990). Corrected score function for errors-in- 74(5), 1027–1045. https : / / doi . org / 10 . 1093 / poq /
variables models: Methodology and application to nfq059
generalized linear models. Biometrika, 77(1), 127–
137.
Paccagnella, O., & Guidolin, M. (2023). Question order
and panel conditioning analysing self-reported data.