Skip to content

1kbgz/ccflow-db

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ccflow-db

ccflow models for database access

Build Status codecov License PyPI

Overview

ccflow-db provides public, domain-neutral database callable models and publishers for ccflow workflows. It owns connection configuration, SQL execution, tabular reads, bulk writes, transaction handling, upsert/merge patterns, database-backed cache adapters, and Hydra config groups exposed through the lerna plugin entry point.

It should stay generic across domains and database backends. Domain schemas, provider-specific assumptions, and application deployment conventions belong outside this package.

Current Status

  • Implemented: package scaffold, version metadata, SQLiteConfig, parameterized SQLiteQueryModel, SQLiteKeyExistsModel, SQLiteTableWriteModel for append, replace, and primary-key upsert writes, byte-oriented SQLiteCacheStore for use with ccflow-etl cache format models, and the cache=sqlite config group.
  • Partial: SQLite uses the Python standard library as the first local backend; broad SQLAlchemy-style engine/session management and backend extras remain future work.
  • Missing: dataframe reads, chunked writes, dry-run SQL rendering, merge helpers beyond SQLite upsert, and non-SQLite integration tests.

Dependency Contract

  • Depends on ccflow for callable model and publisher interfaces.
  • Depends on ccflow-etl for cache format/result contracts.
  • Uses sqlite3 for the first local backend and may add a broader abstraction layer such as SQLAlchemy plus optional backend extras later.
  • Must not depend on finance packages or application-specific packages.

Test Convention

Default tests should use SQLite or in-memory database fixtures. Backend-specific tests such as Postgres should be opt-in and skipped unless the required environment variables are set.

Note

This library was generated using copier from the Base Python Project Template repository.

About

ccflow models for DB access

Resources

Code of conduct

Stars

Watchers

Forks

Releases

Contributors

Languages