Speech Accent Archive Scraper

Utilities to pull Speech Accent Archive samples and run phoneme-level audio summaries.

• python3 -m venv .venv && source .venv/bin/activate
• pip install -r requirements.txt

• python scripts/scrape_accent_archive.py --output-dir data --download-audio --convert-wav --write-transcripts --limit 50
• Outputs:
  • data/metadata.jsonl and data/metadata.csv with per-speaker metadata
  • data/transcripts/*.txt (canonical paragraph per speaker id)
  • data/transcripts/demographics.jsonl and data/transcripts/demographics.csv with per-speaker demographic fields
  • data/audio/*.mp3 (raw downloads) and data/wav/*.wav (mono 16 kHz)
• Flags:
  • --languages afrikaans german ... to restrict scraping
  • --limit 100 to stop after a fixed number of speakers
  • --write-transcripts to emit transcript and demographic files alongside audio

• Install Montreal Forced Aligner (MFA): pip install montreal-forced-aligner (or use the official installer).
• Download English resources:
  mfa model download dictionary english_mfa
mfa model download acoustic english_mfa
• Create one transcript file per wav (identical text) or let the scraper do it with --write-transcripts.
• Align wavs using the matching dictionary/acoustic model:
  mfa align data/wav data/transcripts english_mfa english_mfa data/alignments
• MFA will emit *.TextGrid files per utterance in data/alignments/.

• Create/activate a conda env (Python 3.11) with MFA:
  conda create -n aligner -c conda-forge python=3.11 montreal-forced-aligner -y
conda activate aligner
• Pin numba/librosa once (keeps MFA from crashing):
  conda install -n aligner -c conda-forge "numba<0.58" "librosa<0.10" -y
• Run alignment (models must be downloaded once; caches are local/writable):\

  PATH=/Users/christinewang/miniconda3/envs/aligner/bin:$PATH \
  MFA_ROOT_DIR=$(pwd)/.mfa \
  XDG_CACHE_HOME=$(pwd)/.cache \
  MPLCONFIGDIR=$(pwd)/.mplconfig \
  NUMBA_CACHE_DIR=$(pwd)/.numba_cache \
  NUMBA_DISABLE_JIT=1 NUMBA_DISABLE_CACHING=1 \
  mfa align --clean --num_jobs 1 --single_speaker data/wav data/transcripts english_mfa english_mfa data/alignments
  

• You only need to install/pin packages once per conda env; for future runs just conda activate aligner and re-run the mfa align ... command.

• Run after alignment:
  python scripts/phoneme_features.py --textgrid-dir data/alignments --audio-dir data/wav --output data/phoneme_features.csv
• Produces one row per phoneme with start/end times, duration, RMS energy, spectral centroid, and estimated pitch.

Please call Stella. Ask her to bring these things with her from the store: Six spoons of fresh snow peas, five thick slabs of blue cheese, and maybe a snack for her brother Bob. We also need a small plastic snake and a big toy frog for the kids. She can scoop these things into three red bags, and we will go meet her Wednesday at the train station.

• The scraper hits https://accent.gmu.edu pages in sequence; be respectful with limits if you expand crawling.
• All scraped assets stay under the data/ directory; nothing is committed by default.