cog-tiler

A Python tool for creating Cloud-Optimized GeoTIFF (COG) tile sets from large raster datasets, with intelligent data filtering and dual-format spatial indexes.

• ** Smart Data Filtering**: Automatically skip tiles with no/minimal data (configurable threshold)
• ** CRS Management**: Built-in validation and reprojection for common projections (Albers, EASE-Grid, Web Mercator)
• ** COG Output**: Cloud-optimized GeoTIFFs with internal overviews
• ** Dual Indexes**: GeoJSON (EPSG:4326) for web + GeoPackage (native CRS) for GIS
• ** Parallel Processing**: Multi-worker tile generation for faster processing
• ** Interactive Mode**: Friendly prompts with helpful defaults
• ** Auto Documentation**: Generates processing metadata and README files

pip install numpy rasterio shapely fiona

The script requires GDAL command-line utilities:

• gdalwarp - for reprojection
• gdal_translate - for COG conversion
• ogr2ogr - for index conversion

Installation:

# Ubuntu/Debian
sudo apt-get install gdal-bin python3-gdal

# macOS (Homebrew)
brew install gdal

# Windows (OSGeo4W or Conda)
conda install -c conda-forge gdal

# Verify installation
gdalwarp --version

Simply run the script without arguments for guided setup:

python cog-tiler.py

You'll be prompted for:

1. Source GeoTIFF file path
2. Regional scope (CONUS, global, local, generic)
3. Target projection (EPSG code)
4. Output resolution (meters/pixel)
5. Tile size (auto-suggested based on resolution)
6. Output directories
7. Data filtering options
8. Processing parameters

For automation or scripting:

python cog-tiler.py \
  --src input.tif \
  --out_dir ./tiles \
  --index_base ./indexes/tile_index \
  --regional-scope conus \
  --target-epsg 5070 \
  --dst_res 30 \
  --mapscale_m 15000 \
  --workers 8

1. Prepare your source data

   • Large GeoTIFF raster
   • Known CRS (or will be detected)

2. Run the script

   python cog-tiler.py

3. Follow the prompts

   • File paths
   • Regional scope and CRS
   • Tile parameters
   • Data filtering (recommended for sparse datasets)

4. Output files created

   • COG tiles in ./tiles/
   • Web index: tile_index_4326.geojson
   • GIS index: tile_index_5070.gpkg
   • Documentation: PROCESSING_README.md

For datasets with large areas of nodata (ocean, desert, missing data), enable filtering:

Filter out tiles with no/minimal data? [y/N]: y
Minimum valid data percentage per tile [10.0]: 10

Recommended thresholds:

• 0.1% - Very permissive (include almost any data)
• 5% - Moderate filtering
• 10% - Standard (recommended default)
• 25% - Strict (only densely populated tiles)

Benefits:

• Reduces storage by 50-90% for sparse datasets
• Faster processing and indexing
• Smaller index files for web applications

python cog-tiler.py \
  --src conus_raster.tif \
  --regional-scope conus \
  --target-epsg 5070 \
  --dst_res 30 \
  --mapscale_m 15000

Creates 500×500 pixel tiles (15km × 15km at 30m resolution)

# Interactive mode recommended for filtering setup
python cog-tiler.py
# Then enable filtering and set threshold to 10%

python cog-tiler.py --src input.tif --only_index

Useful for previewing tile layout before processing.

• conus - Continental US (expects EPSG:5070)
• global - Worldwide (expects EPSG:6931 EASE-Grid)
• local - Custom regional CRS
• generic - No CRS assertion

• EPSG:5070 - NAD83 Conus Albers (US Continental)
• EPSG:6931 - WGS84 NSIDC EASE-Grid 2.0 (Global)
• EPSG:3857 - Web Mercator (Web mapping)
• EPSG:4326 - WGS84 Geographic (Lat/Lon)

• NEAREST - Categorical data (land cover, masks)
• BILINEAR - Continuous data (elevation, probability) [default]
• CUBIC - Smoother continuous data
• AVERAGE - Safe downsampling
• LANCZOS - High quality for imagery

Common values: 5km, 10km, 15km, 30km, 61.44km, 122.88km

The script validates that mapscale ÷ resolution = whole number.

project/
├── tiles/
│   ├── dataset_c00000_r00000.tif
│   ├── dataset_c00000_r00001.tif
│   └── ...
├── indexes/
│   ├── tile_index_4326.geojson    # Web-friendly (WGS84)
│   └── tile_index_5070.gpkg       # GIS-native (Albers)
└── PROCESSING_README.md            # Auto-generated docs

Each tile feature includes:

• tile_id - Unique identifier (e.g., c00012_r00034)
• col, row - Grid coordinates
• mapscale_m - Tile size in meters
• dst_res - Pixel resolution
• width_px, height_px - Tile dimensions
• href - File path or URL
• regional_scope - Dataset scope
• processing_date - ISO timestamp

HREF prefix: https://cdn.example.org/data/tiles/

Generates web-ready URLs in the GeoJSON index.

--workers 16  # Use 16 CPU cores

Recommended: Set to number of logical CPU cores for best performance.

--tmp_dir /mnt/fast-ssd/tmp

Use fast SSD storage for intermediate files.

Install GDAL command-line tools (see Prerequisites section).

Your source file CRS doesn't match the expected CRS for the regional scope. Solutions:

• Reproject source first: gdalwarp -t_srs EPSG:5070 input.tif output.tif
• Use --force-crs flag (not recommended)
• Choose local or generic regional scope

Choose mapscale and resolution that divide evenly. Examples:

• ✅ 15,000m ÷ 30m = 500 pixels
• ✅ 122,880m ÷ 30m = 4,096 pixels
• ❌ 15,000m ÷ 40m = 375.0 (not clean)

Adjust --mapscale_m parameter. The script suggests optimal values based on resolution.

• Increase --workers
• Use SSD for --tmp_dir
• Enable data filtering to skip empty tiles
• Consider lower resolution for initial testing

1. Start with index-only: Use --only_index to preview tile layout
2. Enable filtering: Can reduce processing by 50-90% for sparse data
3. Use appropriate resolution: 30m for analysis, 100m+ for visualization
4. Optimize workers: Typically 1-2× CPU cores
5. Fast temp directory: SSD dramatically speeds up COG conversion

Written by: Alister A. Fenix for Vibrant Planet Data Commons

This project is licensed under the MIT License - see the LICENSE file for details.

Why MIT? This tool is designed to enable open science and maximize adoption across research institutions, agencies, and the broader geospatial community.

Issues and pull requests welcome!