Skip to content

Refactoring: split build/* into app/, tool/, src/, old/, and more? #45

@gwern

Description

@gwern

The current /static/build/ directory is swollen with obsolete or experimental scripts, Haskell executables, PHP executables, tools, etc. It needs to be heavily refactored for coherence, and to avoid problems like rebuilding Haskell modules repeatedly (only partially ameliorated by c455b54 adding app/ for some of the Haskell executables).

A first possible pass to split into is:

  • app/: for Haskell binaries which get compiled & installed, and depend on the Gwern.net library
  • tool/: for all the little scripts like italicizer.py
  • src/: the Haskell Gwern.net library
  • old/: for one-offs or experiments like text2epositive.py (mirrors js/old/)
  • ?

(NOTE: To move files, they must be moved with git mv of course, and all references to the old name updated, usually using gwsed /static/build/old /static/build/new/new, but also redirects set up in nginx/redirect/move.conf from the original file name to the new one and also for the HTML syntax-highlighted previews at $FILE.html as well.)

A GPT-5 Pro suggested ontology is as follows:

static/build/
  gwernnet.cabal
  cabal.project
  app/                 # Haskell Main modules only
  src/                 # Haskell library modules only

  tool/                # user-facing non-Haskell tools/scripts
    publish/
    ingest/
    qa/
    media/
    text/
    annotation/
    ai/

  lib/                 # sourced/included non-Haskell libraries
    sh/
    php/

  asset/               # PHP/CSS/font/icon asset-build pipeline
  editor/              # markdown.el, editor glue
  dev/                 # hooks, local developer machinery
  old/                 # obsolete, experiments, one-offs

Umbrella issue: Refactor static/build/

  - Phase 1: move Haskell into app/ + src/   [done/mostly done]
  - Phase 2: classify obsolete scripts into old/
  - Phase 3: move shell/python/php tools into tool/$category/
  - Phase 4: move shared sourced code into lib/
  - Phase 5: redirect audit + grep audit
Post-Cabal refinement: Haskell should probably remain simple:
`app/` for executable `Main` modules and `src/` for all importable Gwern.net
library modules. The remaining taxonomy should primarily classify shell/Python/PHP
tools, asset-build PHP, editor/dev glue, and obsolete scripts.

Proposed additional top-level buckets:

- `asset/`: PHP/CSS/font/icon asset-build pipeline
- `lib/`: sourced/included non-Haskell support code (`bash.sh`, shared PHP)
- `tool/{publish,ingest,qa,media,text,annotation,ai}/`: manually-run utilities
- `editor/`: editor integration
- `dev/`: hooks/local developer support
- `old/`: obsolete, one-offs, experiments

This is likely an umbrella issue rather than a true good-first-issue because moves
require reference updates plus nginx redirects for both original paths and `.html`
syntax-highlighted previews.

Metadata

Metadata

Assignees

Labels

AIIssue which can probably be resolved by agentic LLM AI coding, rather than scarce humans.BackendOriginal content, Hakyll/Haskell/scripts, Markdown/HTML etc. Usually not CSS/JS. Owner: Gwern.enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions