Andrew Baker, Brantly Callaway, Scott Cunningham, Andrew Goodman-Bacon, and Pedro H. C. Sant’Anna

Abstract from the paper: Difference-in-differences (DiD) is arguably the most popular quasi-experimental research design. Its canonical form, with two groups and two periods, is well-understood. However, empirical practices can be ad hoc when researchers go beyond that simple case. This article provides an organizing framework for discussing different types of DiD designs and their associated DiD estimators. It discusses covariates, weights, handling multiple periods, and staggered treatments. The organizational framework, however, applies to other extensions of DiD methods as well.

Replication Instructions

Overview: To replicate the analysis and results, users have two options: (1) use R, or (2) use Stata. Both approaches will ultimately produce the same figures and tables. The simplest approach is to use one environment throughout, but advanced users could mix (for example, use R to rebuild data and Stata for analysis). We outline both methods below. In all cases, start by unpacking the replication archive and setting the working directory to the top-level folder containing this README.

Option 1: Replication using R

1. Install R (version 4.x). It will also be convenient to use RStudio. 
2. Open the provided DiD_JEL.Rproj RStudio project, which will set your working directory to the replication package folder. Alternatively, set your R working directory manually to the folder containing the scripts/ and data/ directories.
3. Open scripts/R/00_master_did_jel.R in R (or RStudio) and run it. This master script will:
   • Load required libraries. We use renv to create a fully reproducible package. In the master code it will call renv::restore() which will download and install the packages from the renv.lock file into the local environment. 
   • Execute the code 0_make_data.R, which will read in the raw data and produce the combined dataset county_mortality_data.csv. (This step will also attempt to fetch data from the Census API for poverty and income – an internet connection is required for that step. If offline, the script will use the included final data from the archive, which already contains those variables. You can do this by commenting out the code from the master file (line 16 of scripts/R/00_master_did_jel.R)
   • Execute the analysis scripts 1_Adoption_Table.R through 4_GxT.R in sequence. These will perform all calculations and generate intermediate results, figures, and tables. As they run, you will see output in the R console and figures may briefly display.
   • Execute 5_honestdid.R to run the sensitivity analysis and produce any additional outputs (e.g., adjusted estimates or plots for the appendix).
4. Once 00_master_did_jel.R completes, check the figures/ and tables/ directories. They should now contain the replication results. You can open the PDF figures to compare with the paper’s figures, and open any table files (or the console output in R) to compare with the published tables. All numbers and graphs should match the article and its appendix.

Note: The R code is written to use relative paths via the here package, so as long as the working directory is correctly set (by opening the RProj or using setwd to the project root), no manual path edits are required. The entire R pipeline should run automatically. Total runtime in R is expected to be around 10 minutes (depending on your system).

Option 2: Replication using Stata

1. Install Stata (version 15 or higher recommended; the do-files were tested on Stata 17). Ensure you have an internet connection for the first run so that Stata can install any needed user-written commands.
2. Start Stata and change the working directory to the replication package folder. For example, in Stata’s command window, type:
   cd "C:\path\to\Difference-in-Differences-replication"
3. Open the file scripts/Stata/00_stata_master_did_jel.do in Stata’s do-file editor. Edit the line near the top that defines the global root directory: global rootdir "your pathname here"
   Replace "your pathname here" with the full path to the replication folder on your system (the folder containing the data, scripts, etc.). Save the changes.
4. In Stata, run the master do-file. You can do this by clicking “Do” in the editor or by typing do scripts/Stata/00_stata_master_did_jel.do in the Stata command prompt. The master script will then:
   • Install any missing Stata packages from SSC (you will see Stata connecting to net sites to install commands like csdid, drdid, etc.). This happens only once; subsequent runs will skip installations if the packages are already present.
   • Execute 0_stata_Make_data.do, which will load the provided data/county_mortality_data.csv and set up the analysis dataset in memory.
   • Execute the analysis scripts 1 through 5 in order. You will see output in the Stata Results window: for example, the adoption table being printed, regression results, and graph generation. The do-files will save graphs to figures/ (as PDF files) and tables to tables/ as they go.
5. When the master script finishes, verify the outputs against the paper’s tables and figures for confirmation.
6. Check the figures/ folder for newly created figure PDFs. Open them to ensure they correspond to the figures in the paper (titles, axes, and plotted values should match). For example, you should find figures illustrating the mortality trends for expansion vs. non-expansion states, event-study plots for multiple years, etc., as described in the paper.
7. Check the tables/ folder for any output tables. If LaTeX tables were saved (e.g., .tex files), you can compile them or open them in a text editor to verify the numbers align with the published tables.

Troubleshooting: If Stata stops with an error, read the error message and refer to the corresponding do-file. Common issues might include not having write permission in the directory (ensure that Stata can create files in figures/ and tables/) or package installation problems (if a network issue prevented downloading a package, try installing it manually via ssc install [pkg] and re-run). Make sure the $rootdir path was set correctly; if it’s incorrect, Stata will not find the data or scripts. If replication still fails, one can fall back to using R, as the R code might be more self-contained for data assembly.

Results Verification: Both approaches should yield the same substantive results. Differences will be due either to the use of bootstrap standard errors or cosmetic details of software graphing functionality (ie. histograms), but these should be small. The figures and tables generated by the R code can be directly compared to the figures and tables in the published article’s main text and appendix.

This replication package includes all data used in the study. All datasets are from public sources and are provided in the data/ directory of the archive. The authors certify that they have legitimate access to these data and permission to use them, and that they have the rights to redistribute the included data for replication purposes. Details on each data source are provided below. (All data are publicly available, so no special access or confidentiality restrictions apply.)

• CDC Mortality and Population Data: County-level mortality counts and population figures for various subgroups were obtained from publicly available CDC sources. Files in data/cdc/ include mortality.csv (total deaths of adults 20-64 by county-year) and population files broken down by demographic groups (female_pop.csv, white_pop.csv, hispanic_pop.csv, etc.). These data were originally sourced from the CDC WONDER database and are in the public domain.  Source = https://wonder.cdc.gov/deaths-by-underlying-cause.html. 
• BLS Unemployment Data: County-level unemployment rates and labor force data for years 2009–2019 were obtained from the U.S. Bureau of Labor Statistics Local Area Unemployment Statistics (LAU). The files laucnty09.xlsx through laucnty19.xlsx (located in data/bls/) contain annual county unemployment statistics for each year 2009–2019. These Excel files are directly from BLS (publicly available government data). Source = https://www.bls.gov/lau/tables.htm.
• Medicaid Expansion Status: The file expansion_status.csv in data/kff/ was compiled from the Kaiser Family Foundation’s data on state Medicaid expansion decisions under the Affordable Care Act. It lists each U.S. state’s expansion status and the year of expansion (if applicable). This information is publicly available through KFF and included here for convenience. Source = https://www.kff.org/status-of-state-medicaid-expansion-decisions/.
• Socioeconomic Covariates: County-level poverty rates and median household incomes for the relevant years were retrieved from the U.S. Census Bureau (American Community Survey). These variables are incorporated into the final dataset. The replication code uses the Census API to obtain these measures; however, the final compiled dataset (described below) already contains these variables for each county-year. Source = https://www2.census.gov/programs-surveys/saipe/datasets/

The above sources were merged to create a single analysis dataset of county-level observations used in the paper’s empirical analysis. This compiled dataset is provided in two formats for convenience:

• data/county_mortality_data.csv – a CSV file containing the merged panel dataset of 3,143 U.S. counties over multiple years (2010–2019). It includes variables such as county identifiers, year, adult mortality counts, population (ages 20–64), mortality rates, demographic population shares, unemployment rate, poverty rate, median income, and an indicator of Medicaid expansion status with the year of expansion.
• data/did_jel_aca_replication_data.dta – the same merged dataset in Stata .dta format. This is provided for users replicating with Stata, to avoid any import issues. It contains the identical information as the CSV file above. (Note: In the scripts, the CSV version is used as input for the data construction step, and this Stata file is provided as a ready-to-use copy of the final data.)

Rights and Licenses: All data in this package are publicly available and either in the public domain or provided under terms that allow redistribution for research purposes. The authors have verified that including these data in the replication archive does not violate any usage agreements. If users reuse the data, please cite the original data sources (e.g., CDC, BLS, KFF) as appropriate. No sensitive or restricted data are used in this study.

Supported Platforms: The replication materials can be run on any modern Windows, macOS, or Linux system. The code and data are not platform-dependent. The analysis was originally run on a 64-bit machine with 64 GB RAM and an Intel Xeon 1.8GHz processor, but such high specifications are not strictly required. A typical desktop or laptop (e.g., 8–16 GB RAM, multi-core CPU) should be sufficient to reproduce the results, though runtime may vary.

Software Required: Two software environments are provided for replication:

• Stata: The Stata replication code was tested on Stata 16 and 17. It should run on Stata 15 or higher, as it uses standard commands and user-written packages compatible with these versions. Stata is required if using the .do files for replication.
• R: The analysis can also be replicated in R (version 4.1 or higher recommended). An R Project file (DiD_JEL.Rproj) is included to facilitate using RStudio. The R scripts rely on several R packages (listed below).

R Packages Needed: To replicate in R, the following R packages (with the versions used in the original replication) are required:

• tidyverse (data manipulation and plotting; v2.0.0)
• readxl (reading Excel files; v1.4.5)
• here (for file paths, v1.0.2)
• kableExtra and modelsummary (for formatting tables; v1.4.0 and v2.5.0)
• did (Difference-in-Differences estimation package by Callaway & Sant’Anna, v2.3.0)
• HonestDiD (for robustness checks, v0.2.6)
• fixest (for fixed effects regressions, v0.13.2)
• ggthemes and patchwork (for plotting themes and combining plots, v5.2.0 and v.1.3.2)

All these packages are available on CRAN or GitHub. 

Stata User-Written Packages: The Stata do-files will automatically attempt to install required user-contributed commands (from SSC) if they are not already present. Specifically, the master do-file uses:

• csdid (Callaway and Sant’Anna’s staggered DID estimation command, v.1.81)
• drdid (Doubly-Robust DID commands by Sant’Anna et al., v1.91)
• honestdid (Honest DiD sensitivity analysis, by Rambachan & Roth, wrapped for Stata)
• regsave and estout (for saving regression results and outputting tables)
• coefplot and grc1leg2 (for plotting coefficients and combining graphs)

An active internet connection in Stata is required to install these on the fly. If internet access is not available, the user should install these packages beforehand or uncomment any provided alternative steps. Once installed, the do-files will run offline. No other proprietary tools are needed beyond Stata itself.

Memory and Runtime: The full replication (all analyses and generating all figures and tables) completes in approximately 10–15 minutes on a modern computer. Peak memory usage is moderate; a machine with at least 4 GB of free RAM is recommended. Most of the runtime is spent running estimations. There is no significant randomness in the code (any random number generation for, e.g., bootstrap confidence intervals uses set seeds), so results should be exactly reproducible across runs and platforms.

The replication package is organized into the following files and folders:

README.html – (this document) A detailed guide to the replication package, including how to run the code and information on data and software.

LICENSE – License file for the code (MIT License). This covers the authors’ code in the repository. (Data sources are public domain or similarly open; see Data Availability above.)

DiD_JEL.Rproj – R Project file for easy use with RStudio. Opening this project will set the working directory to the replication package root, which helps the R scripts find the data and output folders automatically.

Data Directory – data/: Contains all datasets used or produced (poverty and median income read directly from census.gov in /scripts/R/0_data_processing.R)

data/bls/ – raw unemployment data from BLS. Files laucnty09.xlsx ... laucnty19.xlsx are yearly spreadsheets of county unemployment and labor force data (for years 2009 through 2019 respectively).

data/cdc/ – raw mortality and population data from CDC sources. This includes mortality.csv (county-level deaths of age 20–64, by year), and population breakdowns by demographic group: female_pop.csv, white_pop.csv, hispanic_pop.csv, total_pop.csv (total population aged 20–64). These were used to compute mortality rates and demographic shares.

data/kff/ – contains expansion_status.csv, which lists each state’s Medicaid expansion status and year. Used to assign treatment (expansion) timing to each state’s counties.

data/county_mortality_data.csv – the merged analysis dataset (in CSV format) combining all of the above sources plus additional covariates (poverty rate, median income). Each row is a county-year observation. This is the main input data for analysis scripts.

data/did_jel_aca_replication_data.dta – the same merged dataset in Stata format. Provided for convenience to Stata users (so they do not need to manually import the CSV). The Stata code uses the CSV, but this file can be opened directly in Stata to inspect the data.

The following files are included in the archive under data/.

Scripts Directory – scripts/: Contains all code to reproduce the results. Two subfolders are provided for the two software options:

• scripts/R/ – R scripts for replication. These are numbered in the order they should be run:
  • 0_make_data.R – Reads the raw data (BLS, CDC, KFF) and constructs the merged dataset. It also retrieves the Census poverty and income data via API and adds those. The final combined dataset is written to data/county_mortality_data.csv.
  • 1_Adoption_Table>R – Produces a summary table of Medicaid expansion adoption timing (e.g., how many states or counties expanded in 2014, 2015, etc., versus not at all). This corresponds to a descriptive table in the paper.
  • 2_2x2.R – Performs the basic 2x2 Difference-in-Differences analysis (two groups, two periods) using the example of Medicaid expansion vs. non-expansion pre- vs. post-2014. It generates results for the simplest DiD scenario.
  • 3_2XT.R – Extends the analysis to a 2 x T setting (two groups, multiple periods). This script examines multiple periods of data (pre- and post-treatment) for treated vs. control groups, producing dynamic event-study style figures.
  • 4_GxT.R – Generalizes to a G x T setting (multiple groups with staggered treatment timing across T periods). This script uses methods appropriate for staggered adoption (multiple treatment groups entering at different times) and produces the corresponding results (aggregate treatment effect estimates and/or figures). It leverages the “building block” approach described in the paper to compute aggregate effects from 2x2 comparisons.
  • 5_honestdid.R – Conducts robustness and sensitivity checks, specifically the “Honest DiD” bias-adjustment and sensitivity analysis (following the approach of Rambachan and Roth). Computes how sensitive the results are to deviations from the parallel trends assumption and may output results or plots for an appendix.
  • 00_master_did_jel.R – A master R script that sequentially runs all the above R scripts in the correct order. Running this one file will execute the entire analysis pipeline from data construction to final output. (Note: Before running, ensure the required R packages are installed as described above.)
• scripts/Stata/ – Stata do-files for replication. The key files are:
  • 00_stata_master_did_jel.do – Master do-file that sets up the environment and runs all analysis do-files. **Important:** This file contains a line to define the global path $rootdir. The user should edit this line to point to the folder where the replication files are located on their system (or alternatively, launch Stata in that directory and define global rootdir accordingly). Once the path is set, running this master script will call all other scripts in order. It also installs any needed Stata packages (via SSC) at the beginning. The master script estimates that the full run takes about 11 minutes on a high-end machine (as noted in comments).
  • 0_stata_Make_data.do – Constructs the merged dataset for analysis (similar to the R script 0_data_processing.R). It reads in the county_mortality_data.csv provided in the data folder (which contains all needed variables) and performs any additional cleaning or variable transformations in Stata. The output is kept in memory for use by subsequent do-files (and could be saved as a Stata data file if needed).
  • 1_stata_adoption_table.do – Replicates the generation of the adoption summary table using Stata. It produces counts of states/counties by expansion status and year, matching the results from the R script for Table 1 (Medicaid expansion timing breakdown).
  • 2_stata_2x2.do – Runs the 2x2 DiD analysis in Stata, analogous to the R script. Computes the difference-in-differences estimate for the two-period, two-group case and stores or displays the results.
  • 3_stata_2xT.do – Runs the multiple-period (event study) analysis in Stata, computing period-by-period treatment effects or dynamic effects for the treated vs. control comparison, and aggregates them if needed. Results should match those from the R version.
  • 4_stata_GxT.do – Runs the staggered adoption (multiple group, multiple period) analysis in Stata. This uses the provided user-written commands (like csdid or drdid) to estimate the proper aggregations of 2x2 comparisons across groups and time, reproducing the main empirical results in the paper.
  • 5_stata_honestdid.do – Performs the Honest DiD sensitivity analysis in Stata, paralleling the R script. Uses the honestdid package to calculate how robust the findings are to potential violations of the parallel trends assumption, and outputs the results (e.g., bias-adjusted estimates or confidence intervals).

Output Directories – figures/ and tables/: These folders are placeholders for the results generated by the code.

• figures/ – After running the replication scripts, this will contain the figures (graphs) produced by the analysis, in PDF or PNG format. Each figure corresponds to one in the paper or appendix (e.g., event-study plots of mortality rates, etc.). The R scripts use ggsave() to save figures here, and the Stata scripts also output graphs (using graph export) to this folder with similar names.
• tables/ – This will contain output tables (as LaTeX .tex files, CSVs, or text) generated by the code, such as regression result tables or summary statistics tables. For example, the adoption summary table and regression tables are saved here. These files can be opened or compiled (if LaTeX) to verify that they match the tables in the published paper.

Note: Initially, figures/ and tables/ are empty. They will be populated when the code is run. The names and formats of outputs are chosen to correspond with those referenced in the paper (for instance, adoptions.tex or Figure3.pdf, etc.). 

**List of Programs and their Outputs ** Each exhibit below lists the output file and the script (with the line where it is written/exported).

R outputs

Stata outputs

Markdown Documentation – markdown/: This folder contains supplementary documentation in Quarto/Markdown format (and rendered HTML) for the code appendix:

• markdown/Code_Appendix.qmd and Code_Appendix.html – An appendix documenting the R and Stata code.

These markdown files serve as a readable “code appendix” and do not need to be executed by replicators.

.gitignore and .DS_Store – Standard housekeeping files (the former for git version control, the latter are incidental Mac system files). They have no bearing on replication and can be ignored.

Licenses and Permissions
Code License: The authors have released the code in this replication package under the MIT License (see the LICENSE file for full terms). This means that users are free to use, modify, and distribute the code, provided proper attribution is given. In short, the code is open-source. If you reuse substantial portions of the code, please cite the paper and give credit to the authors.
Data Usage: All data included are from public sources as noted. Government datasets (CDC, BLS, Census) are in the public domain and free to use. The Kaiser Family Foundation data on expansion status is publicly reported information. By including these files, we are redistributing them for replication purposes. To the best of our knowledge, this redistribution is allowed. Users should also cite the original data sources in any new work. No proprietary or confidential data are included, so no additional data licenses are required.
Authorizations: We reaffirm that we have permission to share all materials in this archive. If there are any questions about data permissions or if any data provider has concerns, please contact the authors (see the email above) or the AEA Data Editor’s office.
Acknowledgments
We thank the American Economic Association Data Editor and staff for their guidance on preparing this replication package. We also thank the providers of the public data used in this study – specifically the CDC, BLS, Census Bureau, and Kaiser Family Foundation – for making their data readily accessible. Their efforts in data collection and dissemination made this research possible.
The authors are grateful to colleagues and seminar participants who provided feedback on earlier versions of the paper and its companion code. Additionally, we acknowledge the contributors of the open-source software and packages utilized (including R and Stata communities, and the developers of the did, csdid, and related packages). Their tools greatly facilitated our analysis.
Any errors in the replication code or documentation are the sole responsibility of the authors. Please report any issues or questions about the replication package to the contact email provided, and we will be happy to assist.