CMIP Linked Data Utilities Library

CMIP-LD is a Python library for working with CMIP (Coupled Model Intercomparison Project) Linked Data vocabularies. It provides tools to fetch, resolve, validate, and generate documentation for JSON-LD controlled vocabularies used across CMIP and related climate science projects.

• 🔗 Prefix Resolution - Resolve short prefixes (e.g., universal:frequency) to full URLs
• 📥 Data Fetching - Retrieve and expand JSON-LD documents with automatic dereferencing
• 📝 Documentation Generation - Auto-generate README files for vocabulary directories
• ✅ Validation - Validate JSON files against schemas and contexts
• 🔄 CI/CD Actions - GitHub Actions for automated vocabulary processing

git clone https://github.com/wcrp-cmip/CMIP-LD.git
cd CMIP-LD
pip install -e .

• Python 3.8+
• jsonld-recursive - JSON-LD processing
• requests - HTTP requests
• Optional: esgvoc - For Pydantic model integration

import cmipld

# Fetch and resolve a vocabulary term
data = cmipld.get("universal:frequency/mon")
print(data)

# Expand a JSON-LD document
expanded = cmipld.expand("universal:frequency")

import cmipld

# Get the full URL for a prefix
url = cmipld.mapping['universal']
# → 'https://wcrp-cmip.github.io/WCRP-universe/'

# Resolve a prefixed URI
full_url = cmipld.resolve_prefix("universal:frequency/mon")

from esgvoc.api import search

# Search for terms
results = search.find("frequency", term="mon")
print(results)

CMIP-LD/
├── cmipld/                    # Main Python package
│   ├── __init__.py            # Package initialization & client setup
│   ├── locations.py           # Prefix mappings and URL resolution
│   ├── prefix_mappings.json   # Prefix → repository mappings
│   ├── generate/              # Documentation generation tools
│   │   ├── create_readme.py   # Generate READMEs for vocab directories
│   │   ├── generate_summary.py
│   │   └── validate_json.py
│   └── utils/                 # Utility functions
│       ├── git/               # Git integration
│       ├── extract/           # Data extraction tools
│       └── ...
├── actions/                   # GitHub Actions for CI/CD
├── static/                    # Static assets (viewer, images)
├── notebooks/                 # Example Jupyter notebooks
└── scripts/                   # Standalone utility scripts

The create_readme.py script generates standardized documentation for vocabulary directories containing JSON-LD files:

python -m cmipld.generate.create_readme /path/to/src-data/universe

Features:

• Only processes directories with a _context file
• Extracts schema from Pydantic models (via esgvoc) or JSON keys
• Generates usage examples for cmipld, esgvoc, and direct HTTP
• Creates collapsible file listings
• Analyzes external dependencies

python scripts/collect_vocab_docs.py /path/to/src-data --output docs/vocabularies

This collects all vocabulary READMEs into a single folder for rendering with MkDocs.

CMIP-LD provides reusable GitHub Actions for vocabulary repositories:

See CONTRIBUTING.md for guidelines.

• esgvoc - ESGF Vocabulary API with Pydantic models
• jsonld-recursive - JSON-LD recursive resolution
• WCRP-universe - Universal vocabularies

Apache 2.0 - See LICENSE for details.

Developed by WCRP-CMIP for the climate science community.