A lightweight Python toolkit containing:
- Smart iterable classes for cache-like structures
- Utility functions for manipulating dictionaries and JSON
- Simple translation tools between JSON files and SQLite databases
| parakyt | 7/10/23 |
|---|---|
| Document Version | 0.1.0 |
| Software Version | 0.1.0 |
git clone https://github.com/Keenan-M-Stone/cachex.git
pip install ./cachexcachex/
│
├── cachex/
│ ├── __init__.py
│ ├── cachex.py
│ ├── dict_util.py
│ └── rosetta.py
│
├── setup.py
└── README.md
Implements specialized iterable containers that behave like caches or ranked collections.
Classes:
CAutoDeque: Maintains a deque that moves recently accessed items to the front and drops the oldest when full.CAutoCounter: A counted set-like container with optional size limits.CRankedList: Automatically sorts elements by their frequency count.CLeaderBoard: Keeps a history and reorders elements dynamically based on access frequency.
Of these, the most useful for use cases explored was CRankedList.
Example:
from cachex import CRankedList
rl = CRankedList(maxsize=5)
for item in ["apple", "banana", "apple", "cherry"]:
rl.put(item)
print(list(rl)) # ['apple', 'banana', 'cherry']Convenience functions for working with nested dictionaries and JSON-like data structures.
The most useful of the tools provided herein proved to be nest and drill functions.
These and other tools which had a less broad application, but still had demonstrable
uses are featured in the examples file cachex/dict_utils_demo.py.
drill(dictionary, default_out, *keys):
Safely traverse a nested dictionary structure. Returns the nested value if found, or default_out if any key is missing.
safe_put(dictionary, *args):
Safely insert a value into a nested dictionary, creating intermediate dictionaries as needed.
Example: safe_put(d, "a", "b", 5) sets d["a"]["b"] = 5.
deep_merge(dict1, dict2):
Recursively merges two dictionaries without overwriting existing entries. Warns when duplicate keys are found.
key_smith(dictionary, current_keys=()):
Returns a list of tuples representing the full key paths to each leaf value in a nested dictionary.
checkKey(dictionary, key):
Returns True if key exists in the dictionary, otherwise False.
nest(vListKeys, oValues):
Builds a dictionary mapping each key in a list or range to a common value.
pretty_print(dictionary, indent=0):
Prints nested dictionaries in a JSON-like format with indentation for easy reading.
pretty_log(dictionary, indent=0, szRet=""):
Returns a string representation of a nested dictionary, formatted like pretty_print.
apply_nest(nested_dict, keys, func):
Applies a function to a nested dictionary value located by a list/tuple of keys.
Returns True if successful, False if keys are missing or invalid.
clean_dict(d):
Recursively removes empty dictionaries and None entries from a nested dictionary.
intersection(lst1, lst2):
Returns the intersection between two iterables or dictionaries.
Works with lists, sets, tuples, and dicts.
For dicts, returns a dict containing only common keys.
filter_nested_dicts(main_dict, status_value):
Returns a filtered version of a nested dictionary, preserving only branches containing entries with status == status_value.
get_first_index_by_value(iterable, value):
Returns the index of the first occurrence of value in an iterable, or -1 if not found.
replace_first(iterable, val, new_value):
Returns a new iterable where the first occurrence of val is replaced with new_value.
Works with lists, tuples, and sets.
is_subset(iterable1, iterable2):
Checks whether all items in iterable1 exist in iterable2.
dict_to_arg_list(args_dict):
Converts a dictionary into an ArgsNamespace object, where each key becomes an attribute.
Example: dict_to_arg_list({"x": 5}).x == 5.
generate_combinations(iterables):
Returns all possible combinations (Cartesian product) from multiple iterables using itertools.product.
ArgsNamespace:
Simple wrapper for converting a dictionary into an object with attribute access.
Example:
CSlottedDict:
Base class for slotted classes that can convert themselves to a dictionary.
Useful for lightweight, memory-efficient data objects.
* .to_dict() — Returns a standard Python dictionary of slot attributes.
* Intended for use with other classes that define __slots__.
* Example: args = ArgsNamespace({"a": 1, "b": 2}); print(args.a) # 1
A conversion bridge between JSON and SQLite database files.
Functions:
to_database(conn, data): Write a dictionary into an SQLite database.to_json(conn): Read data from a database into a dictionary.convert_file(source, destination): Convert between .json and .db formats.
Example:
from cachex import convert_file
# Convert JSON to SQLite
convert_file("data.json", "data.db")
# Convert SQLite to JSON
convert_file("data.db", "data.json")These tools were developed originally as part of collaboration with the NOAA verification team. Special thanks to John Wagner for inspiring the work and the Walrus group for assisting with testing and development.