Skip to content

Improve PDF benchmark parsing - #22

Open
ThomAub wants to merge 3 commits into
mainfrom
codex/pdf-benchmark-quality
Open

Improve PDF benchmark parsing#22
ThomAub wants to merge 3 commits into
mainfrom
codex/pdf-benchmark-quality

Conversation

@ThomAub

@ThomAub ThomAub commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a Hugging Face GDP PDF corpus setup script under benchmark/
  • make benchmark scripts recurse local corpus files
  • skip bad rectangle-table detections with empty first rows so heuristic table extraction can recover punch-list style PDFs
  • render direct PDF markdown from the parsed PDF result to preserve quality while avoiding the extra IR round trip

Verification

  • cargo test -p officemd_pdf
  • cargo test -p officemd_cli
  • cargo build --release -p officemd_cli
  • git diff --check -- benchmark/run.sh benchmark/compare.sh benchmark/setup-gdp-pdf-corpus.py crates/officemd_pdf/src/lib.rs crates/officemd_pdf/src/pdf_inspector/markdown/mod.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3705a6e5b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +188 to +190
if let Some(pages) = pages {
options = options.pages(pages.iter().copied());
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid padding filtered PDF output with unselected pages

When a page filter is passed here, process_pdf_mem_with_options still reports the full document page_count, and extract_ir_with_options then calls fill_missing_pages(..., diagnostics.page_count). For a long PDF, officemd stream file.pdf --pages 1-3 extracts only pages 1-3 but pads the IR back out to every page in the document, so the renderer emits empty ## Page sections for all unselected pages. Keep the requested page set through normalization, or skip full-document padding when a page filter is active.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant