Skip to content
voidrabPublic

About

PDF/A processing and more for Go

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

1,028 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

gopdfrab

codecov Go Reference Mentioned in Awesome Go

PDF/A processing for go!

Verify and convert PDF documents with a small, predictable open source library.

  • PDF/A-1b verification and conversion, passing the full Isartor and veraPDF conformance suites
  • PDF/A-2b verification and conversion, passing the veraPDF PDF/A-2b suite
  • PDF structural verification and repair against the Arlington object model
  • Decryption of encrypted PDFs (RC4 40/128, AES-128, AES-256)
  • Page rendering, text with its position, and links, for viewers
  • A CLI and two WebAssembly builds

The API is stable as of 1.0; CHANGELOG.md states the versioning policy. PDF/A-2u, PDF/A-3 and PDF/A-4 come next.

Command-line tool

go install github.com/voidrab/gopdfrab/cmd/gopdfrab@latest

gopdfrab verify docs/                     # verify every PDF under a directory
gopdfrab verify --json report.pdf         # machine-readable output
gopdfrab convert in.pdf out.pdf           # rewrite towards PDF/A-1b
gopdfrab convert --profile pdfa2b in.pdf  # or towards PDF/A-2b
gopdfrab convert --dpi 300 in.pdf         # tune the raster fallback

Subcommands are verify, convert, version and help. Exit codes are 0 conformant, 1 non-conformant, 2 error. verify walks directories recursively and prints a layout scripts can parse:

profile: pdfa1b

FAIL  issues=974 distinct=36  docs/L34a.pdf
  6.1.2/5     count=1    pages=doc        objs=1    "ViewerPreferences entry /PrintScaling requires PDF 1.5 or later"
  6.3.4/1     count=3    pages=doc,1      objs=3    "font Courier-Bold is not embedded"
PASS  docs/ok.pdf
ERROR docs/broken.pdf
  error "failed to parse structure: pdf: not a PDF document: no %PDF- header"

summary: pass=1 fail=1 error=1 files=3

An issue line groups every report of one check and message: clause/subclause, report count, pages (doc for document-level) and distinct objects. --detail lists one line per page and object instead; --json reports every occurrence. convert prints the profile, input, output and iteration count, then any rasterized pages, dropped features and lost objects, and last result: conformant or the residual issues.

Flag Meaning
--profile pdfa1b (default), legacy1b, pdfa2b, or pdf
--password password for an encrypted input; visible in ps, so prefer $GOPDFRAB_PASSWORD
--max-decoded-mb cap a single stream's decoded output in MB (0 = default 256)
--max-resident-mb cap a document's rebuildable caches in MB (0 = default 64)
--json emit machine-readable JSON
--detail list every page and object an issue was found on
--dpi convert only: raster fallback resolution (0 = default 150)
--max-iterations convert only: verify/fix loop bound (0 = default 4)
-o convert only: output path (default: the input with a .pdfa.pdf/.fixed.pdf suffix)

Library

go get github.com/voidrab/gopdfrab

Verifying

res, err := gopdfrab.Verify(path, gopdfrab.PDFA1B)      // open, verify, close
res, err := gopdfrab.VerifyBytes(data, gopdfrab.PDFA1B) // in-memory PDF
if err != nil {
    log.Fatal(err)
}
if !res.Valid {
    for i, issue := range res.Issues {
        fmt.Printf("#%d: %v\n", i+1, issue)
    }
}

VerifyAll(paths, profile) verifies a batch concurrently and returns one FileResult per path, each with Path, Result and Err.

To work with one file repeatedly, open it as a Document:

doc, err := gopdfrab.Open(path) // or OpenBytes(data)
if err != nil {
    log.Fatal(err)
}
defer doc.Close()

res, err := doc.Verify(gopdfrab.PDFA1B)
ok, err := doc.IsPDFA()                      // Verify(PDFA1B).Valid
ok, err := doc.IsPDF()                       // VerifyObjectModel().Valid
part, level, err := doc.ClaimedConformance() // what the file claims, e.g. "1", "B"
n, err := doc.PageCount()
version, err := doc.Version()                // header version, e.g. "1.7"
info, err := doc.Metadata()                  // Info dictionary entries
xmp, err := doc.XMPMetadata()                // raw XMP packet, as UTF-8

A program that never converts can import github.com/voidrab/gopdfrab/verify instead: the same verification API, without linking the converter (the WebAssembly module is less than half the size).

Inspecting issues

Each PDFError exposes the Check that flagged it, its page and its messages, and Result groups them:

for _, issue := range res.Issues {
    c := issue.Check()
    fmt.Println(c.Clause(), c.Subclause(), c.Name(), c.Description())
    fmt.Println(issue.Page(), issue.Messages())
}

fmt.Println(res.Summary()) // one line per Check
res.Checks()               // distinct Checks violated, sorted by clause
res.IssuesByCheck()        // map[Check][]PDFError
res.IssuesOnPage(1)        // 0 = document-level

Result, PDFError and Check marshal to a stable JSON shape.

Converting

Convert produces a PDF/A conformant rewrite. It runs pre-emptive fixups, then a verify/fix loop, and rasterizes a page only when no in-place repair works.

cr, err := gopdfrab.Convert(path, gopdfrab.PDFA1B) // or ConvertBytes, doc.Convert
if err != nil {
    log.Fatal(err)
}
defer cr.Close() // a large output spills to a temp file

if err := cr.Save("out.pdf"); err != nil { // atomic; WriteTo(w) streams, Output() returns bytes
    log.Fatal(err)
}
fmt.Println(cr.Iterations, cr.Result.Valid)
for _, iss := range cr.Residual() { // issues nothing could repair
    clause, sub := iss.Clause() // numbered as the target level numbers it
    fmt.Println(clause, sub, iss.Page(), iss.Messages())
}

ConvertAll(paths, profile) converts a batch concurrently. For a batch too large to hold every output at once, ConvertEach calls a callback on each result as it completes and closes it afterwards; returning an error stops the batch.

What a conversion lost

Conforming to PDF/A is not the same as looking like the input, so a result reports loss separately from cr.Result.Valid:

  • cr.RasterizedPages lists pages rebuilt as an image, and cr.RasterDrops what the rasterizer could not draw on them.
  • cr.LostObjects lists content that could not be carried over at all: an unresolvable object, an undecodable stream, an embedded file or annotation the level forbids, a layer the document hides, a digital signature, which rewriting the file invalidates at every level.
  • Options.CheckFidelity renders input and output with the same rasterizer and fills cr.Fidelity with a per-page comparison; Blanked() flags a page that lost its content.
cr, _ := gopdfrab.ConvertContext(ctx, path, gopdfrab.PDFA1B,
    gopdfrab.Options{CheckFidelity: true})
for _, pf := range cr.Fidelity {
    if pf.Blanked() {
        log.Printf("page %d lost its content during conversion", pf.Page)
    }
}

Options, cancellation and limits

Every entry point has a …Context counterpart taking a context.Context and an Options value (VerifyContext, VerifyBytesContext, VerifyAllContext, ConvertContext, ConvertBytesContext, ConvertAllContext, ConvertEachContext); the zero Options is the default.

cr, err := gopdfrab.ConvertContext(ctx, path, gopdfrab.PDFA1B, gopdfrab.Options{
    Password:      []byte("secret"), // decrypt an encrypted input
    RasterDPI:     300,              // raster fallback resolution (default 150)
    MaxIterations: 8,                // verify/fix loop bound (default 4)
    Workers:       4,                // batch concurrency (default runtime.NumCPU)
})

Password applies when the file is opened; for a Document, use OpenWithPassword or OpenBytesWithPassword. Cancellation is checked between fix iterations and raster passes, and batch forms stop dispatching new files.

A single stream decodes to at most 256 MB by default, a guard against decompression bombs. The caps are process-wide; set them once at startup with gopdfrab.SetLimits(gopdfrab.Limits{MaxDecodedStreamBytes: 512 << 20}). A zero field resets a cap; CurrentLimits and DefaultLimits report the values.

Errors and damaged files

Failures can be matched with errors.Is:

Error Meaning
gopdfrab.ErrNotPDF the input is not a PDF (no %PDF- header)
gopdfrab.ErrDamaged the cross-reference or trailer structure could not be parsed
gopdfrab.ErrEncrypted an encryption scheme gopdfrab does not implement
gopdfrab.ErrPasswordRequired a correct password is required to open the file
gopdfrab.ErrUnresolvableGraph Convert could not resolve the object graph, so no output was produced
gopdfrab.ErrInternal a defect in gopdfrab, recovered from a panic; please report it with the file

Damage short of that does not fail the document: it is reported as an issue and every other check still runs, and a conversion that had to null an object never reports itself valid.

Rendering pages

A Document also gives random access to its pages, for a viewer:

n, err := doc.Pages()       // walks the page tree; PageCount trusts /Count
page, err := doc.Page(0)    // zero-based, or ErrPageRange
w, h := page.Size()         // displayed size in points, rotation applied
img, err := page.Render(ctx, gopdfrab.RenderOptions{
    Scale:  2,                            // pixels per point: 144 dpi
    Region: image.Rect(0, 0, 800, 600),   // optional: render only this part
})
glyphs, err := page.Glyphs(ctx) // each character with its box, for selection and search
links, err := page.Links(ctx)   // URI links and links to pages, with their boxes

Positions are in points from the top-left corner of the displayed page. A render stops soon after its context is done. A document keeps decoded images, up to 64 MB, and glyph outlines between renders.

Substitute fonts

For fonts a document uses without embedding them, rendering draws and conversion embeds bundled substitutes: Liberation Sans, Serif and Mono and two Noto symbol faces, about 3.1 MB compressed. They are built in. A build with the gopdfrab_nofonts tag leaves them out, for a program that would rather fetch them only when a document needs one, as a WebAssembly build served over the network may:

// name is one of gopdfrab.FontFiles(), as in internal/fontsubst/fonts;
// the source returns its gzip-compressed bytes.
gopdfrab.SetFontSource(func(name string) ([]byte, error) { return fetch("fonts/" + name) })
err := gopdfrab.PreloadFonts() // optional: load them all now

A page rendered while a font cannot be loaded leaves out the glyphs it would draw; a conversion loads every font first and fails if one cannot be loaded.

Profiles and checks

PDFA1B, Legacy1B, PDFA2B and PDF are the ready-made profiles, named pdfa1b, legacy1b, pdfa2b and pdf as the CLI's --profile takes them (ProfileByName, Profiles, Profile.Name). Use Legacy1B for the verdicts of the older Isartor suite. A profile is immutable: AddCheck, RemoveCheck and Clear return a clone.

p := gopdfrab.PDFA1B.RemoveCheck(gopdfrab.Checks.Font.SimpleNotEmbedded)

p := gopdfrab.PDFA1B.Clear().AddCheck(
    gopdfrab.Checks.Transparency.ImageWithSoftMask,
    gopdfrab.Checks.Metadata.PDFAIdentifierMissing,
)

p := gopdfrab.NewProfile(gopdfrab.A1B).AddCheck(gopdfrab.Checks.Font.SimpleNotEmbedded)

res, err := doc.Verify(p)
Registry field Spec area
Checks.Structure 6.1.x file header, trailer, xref, object framing, limits
Checks.Colour 6.2.2 OutputIntent, 6.2.3.x device colours, 6.2.9–10
Checks.Image 6.2.4–6.2.7 image, form and PostScript XObjects
Checks.Transparency 6.2.8 transfer functions, 6.4 soft masks, blend modes, alpha
Checks.Font 6.3.x embedding, subsets, metrics, encoding
Checks.Annotation 6.5.x annotation types and dictionaries
Checks.Action 6.6.x action types and additional actions
Checks.Metadata 6.7.x XMP metadata, extension schemas, PDF/A identifier
Checks.Form 6.9 interactive forms
Checks.ObjectModel ISO 32000 object-model conformance, independent of PDF/A

An eleventh group, Checks.LogicalStructure (6.8.x), is PDF/A-1a only, so AllChecks() returns 204 checks over all eleven groups, 199 over the ten above. A clause names a check only within a conformance level, so lookups take one: CheckByClauseIn(gopdfrab.A1B, "6.3.4", 1), ChecksForClauseIn and check.ClauseIn(level). The table lists PDF/A-1 clauses; PDF/A-2 numbers many rules differently, and an issue gives the clause of the level it was reported under (PDFError.Level). The older CheckByClause and ChecksForClause are deprecated.

PDF object-model conformance

Checks.ObjectModel holds six checks derived from the Arlington PDF Model, the machine-readable ISO 32000 object model. They answer "is this even valid PDF", independent of PDF/A. The PDF profile enables only them, and ObjectModelOnly() builds an unshared copy of it.

res, err := gopdfrab.VerifyObjectModel(path) // or VerifyObjectModelBytes, doc.VerifyObjectModel
cr, err := gopdfrab.ConvertObjectModel(path) // or ConvertObjectModelBytes, doc.ConvertObjectModel

ConvertObjectModel applies every repair that is safe and preserves meaning, and reports anything else as a residual.

WebAssembly

wasm/ holds two syscall/js builds that run in the browser:

# verify only: about 6.2 MB
GOOS=js GOARCH=wasm go build -trimpath -ldflags="-s -w" -o gopdfrab-verify.wasm ./wasm/verify
# verify and convert: about 15.1 MB
GOOS=js GOARCH=wasm go build -trimpath -ldflags="-s -w" -o gopdfrab.wasm ./wasm

Each registers globals that take a Uint8Array and an optional profile name and return a Promise: gopdfrabVerify(bytes, profile?) and, in the full module only, gopdfrabConvert(bytes, profile?). The doc comments in wasm/main.go and wasm/internal/jsapi are the reference.

Performance

PDF/A-1b verification measured against the Java-based veraPDF and PDFBox Preflight on the combined Isartor and veraPDF corpora (773 files); benchmarks/README.md has the methodology. Startup and cold single-file numbers are dominated by JVM startup.

Benchmark vs veraPDF vs PDFBox Preflight
Startup time 149x faster 22x faster
Single file (cold, median file) 160x faster 50x faster
Batch throughput 20x faster 15x faster
Batch peak memory 11x smaller 15x smaller
Deployment footprint 9x smaller 3x smaller

Fuzzing

gopdfrab reads untrusted input, so internal/pdfgen generates broken PDFs from a seed to feed native Go fuzz targets for the whole pipeline and for semantic oracles (repeated runs match byte for byte; a conversion reported valid re-verifies as valid). Every seed replays on each go test run.

go test -run '^$' -fuzz=FuzzOpenBytes        -fuzztime=60s ./internal/pdf/
go test -run '^$' -fuzz=FuzzConvertRoundTrip -fuzztime=60s .

Security

To report a suspected vulnerability, follow SECURITY.md; please do not open a public issue for one.

Contributing

CONTRIBUTING.md covers the test suites, the test corpora and walkthroughs; ARCHITECTURE.md maps the codebase.

  • A PDF that fails and should not, or passes and should not, is the most useful thing to send. Please attach it to an issue.
  • Code changes: one concern per pull request, with a test.
  • New checks: mostly a data change in the check registry; see CONTRIBUTING.md.

Pull requests need a one-time signature on the Contributor Licence Agreement, so that the whole codebase can be released under both licences. You keep your copyright. A bot asks you to confirm it the first time you open a pull request.

Licensing

gopdfrab is dual-licensed under GNU AGPL 3.0 and a commercial licence. Get in touch about commercial licensing.

gopdfrab bundles third-party fonts and ICC profiles and embeds them into converted files; NOTICE gives each one's source and licence.

About

PDF/A processing and more for Go

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages