Skip to content

GSA/datagov-harvester

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2,565 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

datagov-harvester

Test Suite Count Coverage
Unit Unit Test Count Unit Test Coverage
Integration Unit Test Count Unit Test Coverage
Playwright Playwright Test Count Playwright Test Coverage
Functional Functional Test Count Unit Test Coverage

This repository holds the source code the Data.gov Harvester 2.0, which includes three applications applications:

  • datagov-harvest-runner: This is a python application, chiefly composed of files in the harvester directory.

  • datagov-harvest-admin: This is a Flask app which manages the configuration of harvest sources, organizations, and the creation of harvest jobs.

  • datagov-harvest-proxy: This is an nginx app which owns the public route and proxies traffic to the internal Flask app route.

There is further documentation in the developer quickstart.

Documentation

Additional background for team members on Google Drive (not publicly accessible):

Contributing

See CONTRIBUTING for additional information.

Public domain

This project is in the worldwide public domain. As stated in CONTRIBUTING:

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

About

Main repo for Datagov Harvester 2.0. Contains the code for Flask API and Harvesting Logic

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors