Skip to content

Latest commit

 

History

History
295 lines (211 loc) · 24.8 KB

File metadata and controls

295 lines (211 loc) · 24.8 KB

Unreleased

Bug Fixes

Project-Wide Limit Allocation

This is a user-visible change to the output of ProjectDirectory.hours_estimate(), commit_history() and file_change_history() whenever limit is used: they now return the number of rows that was asked for, which is generally more than before.

  • FIXED: ProjectDirectory.hours_estimate(), ProjectDirectory.commit_history() and ProjectDirectory.file_change_history() divided a project-wide limit with int(limit / len(self.repo_dirs)) and handed the same truncated value to every repository. Three defects followed: any limit below the repository count floored to zero and silently returned an empty frame (commit_history(limit=1) over two repositories returned no rows at all), the remainder was discarded (limit=3 returned 2 rows, limit=5 returned 4), and the divisor was repo_dirs, which is not filtered by ignore_repos when the project is built from Repository instances, so an ignored repository still consumed a share of the limit. All three now use the same quotient-plus-remainder allocation revs() already used: the first limit % len(repos) repositories receive one extra commit, and the shares are taken over the ignore_repos-filtered repos.
  • FIXED: Those three methods raised ZeroDivisionError on a project with no repositories, as did ProjectDirectory.revs(num_datapoints=N). They now return the same empty DataFrame an all-failing project returns.

Project Revision Limits

  • FIXED: ProjectDirectory.revs(limit=N) now distributes remainder revisions to repositories in project order instead of discarding them, so populated projects return up to the requested limit for values smaller than or not evenly divisible by the repository count.

Punchcard Normalization

  • FIXED: Repository.punchcard() and ProjectDirectory.punchcard() now preserve finite zeros when normalizing an all-zero metric instead of producing NaN values through division by zero.

Revision Sampling (num_datapoints)

This is a user-visible change to the output of revs, cumulative_blame and parallel_cumulative_blame whenever num_datapoints is used: they now return the number of rows that was asked for, which is generally fewer than before.

  • FIXED: Repository.revs(num_datapoints=N) returned more than N revisions. It derived skip = int(commit_count / N) and then kept every skipth row, so integer truncation inflated the result — on a 10-commit history num_datapoints=4 returned 5 rows and num_datapoints=6 returned all 10. It now selects exactly min(N, commit_count) evenly distributed revisions, keeping the existing newest-first order and always including both the newest and the oldest revision. Repository.cumulative_blame() and Repository.parallel_cumulative_blame() delegate num_datapoints to revs(), so they no longer perform (and report) more blame work than requested.
  • CHANGED: A non-positive num_datapoints now raises ValueError from Repository.revs() and ProjectDirectory.revs(). Previously num_datapoints=0 raised ZeroDivisionError from the internal skip calculation and negative values produced an arbitrary slice.
  • FIXED: ProjectDirectory.revs(num_datapoints=N) divides N across the member repositories, which floored to zero datapoints per repository whenever N was smaller than the repository count. Each repository now contributes at least one revision.
  • Explicit limit and skip arguments are unaffected; num_datapoints still applies only when neither is supplied.

pandas 3 Compatibility

  • CHANGED: The pandas>=2.0.0,<3.0.0 requirement is now pandas>=2.0.0. pandas 3 has been out for a while, so the upper cap meant pip install git-pandas either downgraded pandas or failed to resolve in any environment that already wanted pandas 3. pandas 3 requires Python >= 3.11, so resolvers on this package's declared 3.10 floor keep selecting pandas 2.x without needing an environment marker. CI now runs the suite against both majors.
  • FIXED: Repository.hours_estimate() and ProjectDirectory.hours_estimate() raised TypeError: unsupported operand type(s) for *: 'datetime.datetime' and 'float' on pandas 3. The method derived per-commit timestamps with commits.index.values.tolist(), which yields integer nanoseconds on pandas 2 but datetime.datetime objects on pandas 3. It now uses (index - index[0]).total_seconds(), which is independent of the index's datetime64 resolution — relevant because pandas 3 preserves the second resolution of to_datetime(..., unit="s") rather than always upcasting to nanoseconds. Estimated hours are unchanged on pandas 2.
  • FIXED: plot_lifeline() raised AttributeError: 'StringArray' object has no attribute 'sort' on pandas 3. Series.unique() returns a NumPy array on pandas 2 but a StringArray under pandas 3's default str dtype, and only the former supports in-place .sort(). The unique filenames are now sorted with sorted().

Optional Redis & Coverage Dependencies

  • CHANGED (user-visible): redis and coverage are no longer installed by pip install git-pandas. Neither is used by the core library — redis is imported behind a guard in gitpandas.cache and coverage is imported lazily inside Repository.coverage() — so a plain install no longer pulls in a Redis client and a test-coverage tool. They are now optional extras: install git-pandas[redis] to use RedisDFCache, and git-pandas[coverage] to use Repository.coverage() / Repository.file_change_rates(coverage=True). Both are included in the all and dev extras. Anyone relying on RedisDFCache after a bare pip install git-pandas must now install git-pandas[redis]; the failure mode is the existing explicit ImportError("Need redis installed to use redis cache").
  • FIXED: Repository.coverage() swallowed a missing coverage package. The import coverage sat inside a try whose final handler is a broad except Exception, so on a repository that does have a .coverage file the result was an empty DataFrame — indistinguishable from "no coverage data". The import now happens outside that block and raises an ImportError naming the git-pandas[coverage] extra. Repository.file_change_rates(coverage=True) re-raises it rather than returning an empty frame. Genuine "no coverage data" cases still return the empty DataFrame as before.

Cumulative Blame Output Shape

This is a user-visible change to the output of cumulative_blame and parallel_cumulative_blame. Callers that index those frames positionally (iloc[0], columns[0]) will see different data.

  • FIXED: Repository.cumulative_blame() and Repository.parallel_cumulative_blame() returned the string repository label column (and any labels_to_add columns) mixed in with the per-contributor LOC counts, so df.sum(axis=1) — total LOC over time — raised TypeError: can only concatenate str (not "int") to str. Label columns are now dropped, so the returned columns are exactly the contributor names, matching the columns ProjectDirectory.cumulative_blame() already returned.
  • FIXED: All three cumulative blame methods (Repository.cumulative_blame(), Repository.parallel_cumulative_blame(), ProjectDirectory.cumulative_blame()) returned a reverse-chronological (monotonically decreasing) DatetimeIndex, because revs() yields newest-first and nothing sorted. That made label-based date slicing raise KeyError: Value based partial slicing on non-monotonic DatetimeIndexes..., silently negated .diff(), and made .iloc[-1] the oldest row. The index is now sorted ascending.
  • ENHANCED: The zero-column filter in both Repository methods now does a numeric sum instead of an O(n) string concatenation over the label column.
  • CHANGED: When several revisions share one timestamp, ProjectDirectory.cumulative_blame() now consistently keeps the newest of them (index.duplicated(keep="last") on the ascending index). Previously a single-repository project kept the newest and a multi-repository project kept the oldest, because only the latter was index-sorted by the merge.

v2.5.0

New Features

Remote Operations & Cache Warming

  • NEW: Repository.safe_fetch_remote() - Safely fetch changes from remote repositories without modifying working directory
    • Read-only operation with comprehensive error handling
    • Support for dry-run preview and remote validation
    • Configurable remote names and pruning options
  • NEW: Repository.warm_cache() - Pre-populate repository cache for improved performance
    • Configurable method selection with intelligent parameter handling
    • Performance metrics and cache entry tracking
    • Significant performance improvements (1.5-10x speedup demonstrated)
  • NEW: ProjectDirectory.bulk_fetch_and_warm() - Efficiently process multiple repositories
    • Parallel processing support when joblib is available
    • Error isolation (failures in one repo don't affect others)
    • Comprehensive summary statistics and progress tracking

Enhanced Caching System

  • NEW: CacheEntry class with metadata tracking (timestamps, age calculation)
  • ENHANCED: Thread-safe cache operations with proper locking mechanisms
  • ENHANCED: Cache key consistency improvements using || delimiter format
  • ENHANCED: Cache timestamp and metadata access methods (get_cache_info(), list_cached_keys())

Documentation & Examples

  • NEW: Comprehensive remote operations documentation (docs/source/remote_operations.rst)
  • NEW: Cache warming and remote fetch example (examples/remote_fetch_and_cache_warming.py)
  • NEW: Cache timestamp usage example (examples/cache_timestamps.py)
  • NEW: Release analytics example (examples/release_analytics.py)

Bug Fixes

Commits In Tags History Walking

  • FIXED: Repository.commits_in_tags() never walked history. It returned exactly one row per tag — the tagged commit — rather than attributing every commit to the release that shipped it, as documented. The backwards walk now runs, so each commit is attributed to the first tag that contains it, stopping at the previous tag, at the start/end bounds, or at a root commit. The walk uses an explicit worklist, so a release spanning more commits than the interpreter's recursion limit no longer raises RecursionError.
  • FIXED: Tag lookup used positional Series.__getitem__, which emitted a pandas FutureWarning and would become a KeyError on a future pandas release.

Note: This is a user-visible output change. commits_in_tags() — and ProjectDirectory.commits_in_tags(), which aggregates it — now return one row per commit instead of one row per tag. Code that counted rows to count releases needs updating; count distinct values of the tag column instead.

Project Default Branch Detection

  • FIXED: ProjectDirectory and GitHubProfile now let each repository auto-detect main or master by default, so mixed-branch projects include every repository in branch-based history metrics. Passing an explicit default_branch continues to force that branch across all repositories.

Note: Repositories with neither a main nor master branch are now skipped with a warning during project initialization. Previously they were included but returned empty branch-based history.

Cumulative Blame

  • FIXED: Repository.cumulative_blame() and parallel_cumulative_blame() no longer duplicate rows when multiple commits share a timestamp.

File Change History

  • FIXED: Repository.file_change_history() now reports actual per-file insertions and deletions, including root commits, which restores non-zero churn metrics in file_change_rates().

Project Punchcard Aggregation

  • FIXED: ProjectDirectory.punchcard() now returns a well-formed empty DataFrame when no repository yields commit data and avoids pandas aggregation FutureWarnings.

File Ownership

  • FIXED: Repository.file_owner() always read the commit's committer regardless of the committer flag, so committer=False returned the top committer under a column labelled author. It now selects the identity to match the flag, matching Repository.blame(). This also fixes Repository.file_detail(committer=False) and ProjectDirectory.file_detail(committer=False), whose file_owner column reported the committer on rebased, cherry-picked, squash-merged, or web-UI-committed history.

MCP Server Serialization

  • FIXED: serialize_pandas_object() dropped every non-datetime DataFrame index during orient="records" conversion, so index-carried identity disappeared from tool results (e.g. a blame-shaped frame serialized as [{"loc": 7}] without the committer/author it belongs to). Named index levels, including MultiIndex levels such as file and (tag_date, commit_date), are now materialized as columns.
  • FIXED: Serialization mutated the DataFrame it was given — rewriting a DatetimeIndex into formatted strings and replacing datetime columns in place — which corrupted shared cached frames. Serialization now works on a copy and leaves its input untouched.
  • FIXED: Serializing a frame that keeps a column and index level of the same name (e.g. file_change_history()'s date) raised ValueError: cannot insert date, already exists.

GitHub Profile Discovery

  • FIXED: GitHubProfile now follows GitHub API pagination links and requests up to 100 repositories per page. If any page fails, discovery returns an empty profile instead of silently analyzing partial results.

Hours Estimation

  • FIXED: Repository.hours_estimate() now includes the first-commit allowance, increasing estimates by single_commit_hours per contributor and giving single-commit contributors a non-zero estimate.

Project Blame Aggregation

  • FIXED: ProjectDirectory.blame() now preserves committer/author and file grouping keys when combining multiple repositories. Contributors are aggregated by name across repositories, blame(by="file") no longer raises KeyError, and project-level bus factors are calculated from contributors instead of row numbers.

Project File Blame Aggregation

  • FIXED: ProjectDirectory.blame(by="file") grouped on (committer/author, file) only. Because file is a repository-relative path, files sharing a path across repositories (README.md, setup.py, __init__.py, ...) were silently summed into a single row reporting a line count that matched neither repository, and the originating repository was discarded entirely. The repository is now part of the grouping, so the result is indexed by (committer/author, file, repository) — matching the identification that file_detail(), file_change_rates(), and bus_factor(by="file") already carry. blame(by="repository") is unchanged; aggregating a contributor across repositories there remains correct.

Note: This adds an index level to blame(by="file") output. Code that indexes that result by (committer, file) needs updating.

Cumulative Blame Cache Mutation

  • FIXED: Repository.cumulative_blame() reshaped the revs() frame it received in place — adding a column per committer, deleting rev, and replacing the index with the commit dates. Because a cache backend hands out the stored DataFrame by reference, a later revs() call returned that wrecked frame, and a second cumulative_blame() call raised ValueError: Internal Error: self.revs() returned DataFrame without 'rev' column. (parallel_cumulative_blame() swallowed the same failure and returned an empty DataFrame). It now works on a copy.
  • FIXED: ProjectDirectory.cumulative_blame() renamed each member repository's cached cumulative_blame() columns in place while suffixing them with the repository name, so a later per-repository call returned Alice__repo1 instead of Alice. The suffixing now happens on copies.

Project Bus Factor Glob Filtering

  • FIXED: ProjectDirectory.bus_factor(by="repository") passed the caller's include_globs into each repository's ignore_globs argument. The caller's ignore_globs was silently discarded and include_globs doubled as an exclusion list, so the bus factor was computed over the wrong set of files — typically excluding exactly the files the caller asked to include. (by="file" was already correct.)

Project Empty-Result Guards

  • FIXED: ProjectDirectory.blame() and file_detail() raised AttributeError: 'NoneType' object has no attribute 'reset_index' when the project contained no repositories, or when every repository raised GitCommandError (repositories with no commits, or a branch that doesn't exist). Both now return a well-formed empty DataFrame matching their normal output shape, as sibling methods such as file_change_history() already did.
  • FIXED: ProjectDirectory.commits_in_tags() raised ValueError: No objects to concatenate when no repository yielded tagged commits. It now returns an empty DataFrame carrying the usual commit_sha, tag, and repository columns and the (tag_date, commit_date) index.

Punchcard Cache Mutation

  • FIXED: Repository.punchcard() wrote day_of_week and hour_of_day columns straight into the commit_history() frame it was handed. Because a cache backend returns the stored DataFrame by reference, every later commit_history() call then returned those two extra columns. The same in-place write affected ProjectDirectory.commit_history(), file_detail(), and revs(), which added a repository column to borrowed per-repository frames. All of these now copy before writing.

Cache Warming

  • FIXED: Repository.warm_cache() silently substituted limit=100 for commit_history and file_change_rates, so it populated a cache key that no ordinary call would ever hit — the default warm_cache() gave those two methods no speedup whatsoever. The injected limit is gone, so warming populates the keys callers actually use.
  • FIXED: hours_estimate(), punchcard(), cumulative_blame(), and parallel_cumulative_blame() resolved branch=None to the default branch before delegating to commit_history()/revs(). The inner call therefore keyed on "master" while a user's own commit_history() keyed on None, storing several copies of the same history and defeating cache reuse between them. The caller's branch value is now passed through unchanged; resolution happens only where a branch name is genuinely needed.

Cached Method Introspection

  • FIXED: @multicache returned a bare wrapper, so every decorated Repository method lost its name, docstring, and signature. help(), inspect.signature(), and any reflection over the API saw (self, *args, **kwargs). The decorator now applies functools.wraps, which restores introspection and — because the MCP server generates its tool schemas by reflecting over Repository — restores the real parameter names and defaults in every MCP tool schema.

Cache Correctness

  • FIXED: Repository.invalidate_cache() now targets the method-first cache key layout, processes every key when combined with a pattern, and returns accurate removal counts for in-memory and Redis backends.
  • FIXED: @multicache built cache keys from kwargs only, so any argument passed positionally was invisible to the key and collapsed to None. Keys are now resolved against the decorated method's signature (inspect.signature().bind() + apply_defaults()), so positional and keyword calls key identically. This fixes:
    • Repository(working_dir=<master-only repo>, cache_backend=...) raising ValueError: Could not detect default branch — the internal has_branch("main") / has_branch("master") probes shared one key.
    • file_detail() reporting the first file's owner for every file — the internal file_owner(rev, file_path, ...) calls shared one key.
  • FIXED: blame()'s key_list misspelled ignore_globs as ignore_blobs, so ignore_globs never contributed to the cache key and all variants collided. blame(ignore_globs=[...]) and bus_factor(ignore_globs=[...]) now return the same results with and without a cache backend.
  • FIXED: skip_broken was missing from the cache key of file_change_history, file_change_rates, revs, cumulative_blame, parallel_cumulative_blame, and tags, so skip_broken=True and skip_broken=False shared an entry.
  • FIXED: Cache key parts are now joined with || instead of _, which could occur inside the values themselves (e.g. get_file_content(path="docs", rev="release_2") collided with path="docs_release", rev="2").
  • NEW: @multicache validates key_list against the decorated method's signature at decoration time and raises ValueError on a name that isn't a parameter, turning the typo class of bug into a loud failure.

Note: The cache key format has changed. Stale EphemeralCache/DiskCache entries simply miss and recompute, but RedisDFCache users sharing a cache across versions should flush it (or use a new key prefix) to avoid retaining entries under the old format.

Testing & Quality

  • NEW: 38 comprehensive tests for remote operations and cache warming
  • NEW: Thread safety tests for cache operations
  • NEW: Edge case and error handling test coverage
  • NEW: Regression coverage for every fix above, including cached-vs-uncached parity, multi-repository aggregation against repositories that share file paths, and cache-immutability checks that fail if a method writes into a frame it was handed
  • NEW: Exact rev-to-rev regression tests pinning churn, file, elapsed-time, author, and committer values, and covering include_globs/ignore_globs through release_tag_summary()
  • IMPROVED: Overall test coverage and reliability
  • FIXED: Various minor bugs and future warnings

Backward Compatibility

  • No API was removed, renamed, or given a new required argument; every existing call still runs.
  • Three changes are user-visible in the data returned, and are called out in full above:
    • commits_in_tags() now returns one row per commit rather than one row per tag.
    • blame(by="file") on a ProjectDirectory gains a repository index level.
    • The cache key format changed; RedisDFCache users sharing a cache across versions should flush it.
  • Beyond those, results change only where they were previously wrong — the bug fixes above return corrected numbers for churn, hours, ownership, bus factor, and blame. Analyses pinned to specific values from an earlier release should expect them to move.
  • Existing cache backends work seamlessly with new features.

v2.4.0

  • Significant caching bugfixes and updates
  • Added a DiskCache that persists across runs
  • Added release analytics

v2.3.0

  • Updated coverage file parsing to use coverage.py API instead of direct file parsing
  • Added coverage>=5.0.0 as a core dependency
  • Added a basic MCP server
  • Added methods to Repository for getting files in repo, getting content of a file, and getting diffs of a revision

v2.2.1

  • Docs CI bugfix

v2.2.0

  • Support for default branch setting instead of assuming master, will infer if not passed
  • Better handling of ignore repos in project directory setup
  • Added a branch exists helper in repository
  • Docs corrections

v2.1.0

  • Imrpoved test suite
  • Many bugfixes
  • Updates for pandas v2

v2.0.0

  • Fully transitioned to ignore_globs and include_globs style syntax
  • Parallelized cumulative blame support with joblib threading backend
  • Added threading parallelism to many project directory functions.
  • Added a chaching module for optional redis or memory backed caching of certain resultsets

v1.2.0

  • Added ignore_globs option alongside all methods with ignore_dir and extensions, will be the only method for filtering files in v2.0.0

v1.1.0

  • _repo_name changed to repo_name in project directories (old method left with deprecation warning)
  • repo_name property added to repositories

v1.0.3

  • Support for estimating time spent developing on projects.

v1.0.2

  • bugfix in ignore_dir option for root level directories

v1.0.1

  • file details function

v1.0.0

  • Stable API
  • Punchcard dataframe added
  • Plotting helpers added to library under utilities module
  • Added github.com profile object

v0.0.6

  • Added file owner utility
  • Added lifelines example
  • Added rev to file change history table
  • Added file-wise blame using by='file' parameter
  • Bus Factor returns a dataframe
  • Now supporting python 2.7+ and 3.3+

v0.0.5

  • Added file change rates table with risk metrics
  • Added basic functionality with coverage files
  • Added limited time window based dataset functionality
  • Expanded docs

v0.0.4

  • Added cumulative blame and revision history

v0.0.3

  • Added approximate bus factor analysis

v0.0.2

  • Added blame

v0.0.1

  • Initial release, basic interface to commit history and descriptors