Skip to content

JSONL and SQLite emit raw stored values where XLSX normalizes them, so the three formats disagree #342

Description

@RyanDFIR

The XLSX render normalizes path display, with Profile showing the full profile path and Source
Item showing the path relative to it. JSONL and SQLite emit whatever the parser happened to
store, so the three output formats disagree about the same record.

Concretely, XLSX shows something like IndexedDB\chrome-extension_..._0.indexeddb.leveldb while
SQLite and JSONL carry an absolute path, because the IndexedDB parser was changed from storing a
bare folder name to storing the absolute path. Same record, three different strings. Anything
diffing across formats, or built against the older bare-folder value, breaks silently.

Paths are the known offender, but this should be a field-by-field pass across the three writers
rather than a path fix, because the same divergence is likely in timestamp formats,
friendly-versus-raw enum values, and None versus empty string.

Decide the canonical representation once and apply it uniformly. Likely: store absolute in the
item and let each writer derive what it displays. Normalizing inside one writer leaves them free
to drift apart again. JSONL goes through the encoder, so any normalization has to live there or
upstream so it cannot drift from the XLSX and SQLite logic.

One structural thing to settle while in there: SQLite already has both source_item and
source_path columns across different tables, and that split is itself part of the
inconsistency.

This is a breaking output change, so it wants versioning or a note, and it should be checked
against the tests and any external consumers.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions