Skip to content

aiddata/geo-cnn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

356 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Using Convolutional Neural Networks to Predict Survey Estimates and other Outcome Measures

Stage 0 - Settings and Data Preparation

Files:

  • push.sh - shell script to push local data to HPC (script args define dir on HPC and settings JSON to use)
  • settings_builder.py - class to manage settings for different scripts
  • settings/ - folder containing settings JSONs for jobs
  • scripts/ - folder containing one off scripts for preparing survey data and anything else not a core utility used in following stages

Notes:

Stage 1 - Training CNN

Files:

  • s1_jobscripts - jobscript
  • main.py - primary script called by jobscript
  • data_prep.py - class/functions to generate sample data for training CNN
  • create_grid.py - class to generate point grid
  • runscript.py - core class/functions for running PyTorch CNN
  • load_data.py - classes for reading Landsat imagery for CNN training data and extracting NTL imagery values for training data labels
  • load_survey_data.py - class to load survey data for use with CNN predictions
  • resnet.py - modified ResNet class (based on PyTorch ResNet class)
  • vgg.py - modified VGG class (based on PyTorch VGG class) - NOT FUNCTION

Notes:

Stage 2 - Training Secondary Models

Files:

  • s2_jobscript - jobscript

  • second_stage_model.py - primary script called by jobscript

  • model_prep.py - functions and classes for building second stage models

  • merge_outputs.py - script to merge second stage model metrics

Stage 3 - Generating Predictive Surface

Files:

  • build_surface_grid.py

S3A - Creating Point Grid

S3B - Generating CNN Features for Surface Grid

S3C - Generating Secondary Model Values for Surface Grid

S4 - Building Raster Surface

Files:

  • s4_main.py -

Notes:

  • x

Usage Notes

Stage 1 Training

Stage 1 Predict

Stage 2 Train and Predict

  • running with full set of cnn features (or more; ntl, etc.) will make stage 2 take significantly longer. If testing, start with PCA instead of full feature set.
  • when using MLP Classifier model, single variable models (e.g., only NTL) will not converge

Stage 3

Stage 4

About

Code to use PyTorch to train a CNN to predict various outcome measures based on satellite imagery.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages