Skip to content

Collection of tools, including "smart" iterable classes that internally adjust to fulfill various functions, methods/functions that allow using sufficiently shortened syntax for complex dictionary operations, and quick conversion tools for translating between json and sql databases as needed.

Notifications You must be signed in to change notification settings

Keenan-M-Stone/cachex

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

cachex

A lightweight Python toolkit containing:

  • Smart iterable classes for cache-like structures
  • Utility functions for manipulating dictionaries and JSON
  • Simple translation tools between JSON files and SQLite databases
parakyt 7/10/23
Document Version 0.1.0
Software Version 0.1.0

🚀 Installation:

git clone https://github.com/Keenan-M-Stone/cachex.git
pip install ./cachex

📦 Modules Overview

cachex/
│
├── cachex/
│   ├── __init__.py
│   ├── cachex.py
│   ├── dict_util.py
│   └── rosetta.py
│
├── setup.py
└── README.md

cachex.cachex

Implements specialized iterable containers that behave like caches or ranked collections.

Classes:

  • CAutoDeque: Maintains a deque that moves recently accessed items to the front and drops the oldest when full.
  • CAutoCounter: A counted set-like container with optional size limits.
  • CRankedList: Automatically sorts elements by their frequency count.
  • CLeaderBoard: Keeps a history and reorders elements dynamically based on access frequency.

Of these, the most useful for use cases explored was CRankedList.

Example:

from cachex import CRankedList

rl = CRankedList(maxsize=5)
for item in ["apple", "banana", "apple", "cherry"]:
    rl.put(item)
print(list(rl))  # ['apple', 'banana', 'cherry']

cachex.dict_util

Convenience functions for working with nested dictionaries and JSON-like data structures.

The most useful of the tools provided herein proved to be nest and drill functions. These and other tools which had a less broad application, but still had demonstrable uses are featured in the examples file cachex/dict_utils_demo.py.

🧩 Functions

drill(dictionary, default_out, *keys):
Safely traverse a nested dictionary structure. Returns the nested value if found, or default_out if any key is missing.

safe_put(dictionary, *args):
Safely insert a value into a nested dictionary, creating intermediate dictionaries as needed. Example: safe_put(d, "a", "b", 5) sets d["a"]["b"] = 5.

deep_merge(dict1, dict2):
Recursively merges two dictionaries without overwriting existing entries. Warns when duplicate keys are found.

key_smith(dictionary, current_keys=()):
Returns a list of tuples representing the full key paths to each leaf value in a nested dictionary.

checkKey(dictionary, key):
Returns True if key exists in the dictionary, otherwise False.

nest(vListKeys, oValues):
Builds a dictionary mapping each key in a list or range to a common value.

pretty_print(dictionary, indent=0):
Prints nested dictionaries in a JSON-like format with indentation for easy reading.

pretty_log(dictionary, indent=0, szRet=""):
Returns a string representation of a nested dictionary, formatted like pretty_print.

apply_nest(nested_dict, keys, func):
Applies a function to a nested dictionary value located by a list/tuple of keys. Returns True if successful, False if keys are missing or invalid.

clean_dict(d):
Recursively removes empty dictionaries and None entries from a nested dictionary.

intersection(lst1, lst2):
Returns the intersection between two iterables or dictionaries. Works with lists, sets, tuples, and dicts. For dicts, returns a dict containing only common keys.

filter_nested_dicts(main_dict, status_value):
Returns a filtered version of a nested dictionary, preserving only branches containing entries with status == status_value.

get_first_index_by_value(iterable, value):
Returns the index of the first occurrence of value in an iterable, or -1 if not found.

replace_first(iterable, val, new_value):
Returns a new iterable where the first occurrence of val is replaced with new_value. Works with lists, tuples, and sets.

is_subset(iterable1, iterable2):
Checks whether all items in iterable1 exist in iterable2.

dict_to_arg_list(args_dict):
Converts a dictionary into an ArgsNamespace object, where each key becomes an attribute. Example: dict_to_arg_list({"x": 5}).x == 5.

generate_combinations(iterables):
Returns all possible combinations (Cartesian product) from multiple iterables using itertools.product.

🧱 Classes

ArgsNamespace:
Simple wrapper for converting a dictionary into an object with attribute access. Example:

CSlottedDict:
Base class for slotted classes that can convert themselves to a dictionary. Useful for lightweight, memory-efficient data objects. * .to_dict() — Returns a standard Python dictionary of slot attributes. * Intended for use with other classes that define __slots__. * Example: args = ArgsNamespace({"a": 1, "b": 2}); print(args.a) # 1

cachex.rosetta

A conversion bridge between JSON and SQLite database files.

Functions:

  • to_database(conn, data): Write a dictionary into an SQLite database.
  • to_json(conn): Read data from a database into a dictionary.
  • convert_file(source, destination): Convert between .json and .db formats.

Example:

from cachex import convert_file

# Convert JSON to SQLite
convert_file("data.json", "data.db")

# Convert SQLite to JSON
convert_file("data.db", "data.json")

🏆 Acknowledgements

These tools were developed originally as part of collaboration with the NOAA verification team. Special thanks to John Wagner for inspiring the work and the Walrus group for assisting with testing and development.

About

Collection of tools, including "smart" iterable classes that internally adjust to fulfill various functions, methods/functions that allow using sufficiently shortened syntax for complex dictionary operations, and quick conversion tools for translating between json and sql databases as needed.

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages