Skip to content

clarify manifest resources checks #1465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 3, 2023
Merged

clarify manifest resources checks #1465

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f81b423
feat: report as usage container resources not listed in manifest
rdeltour Dec 21, 2022
09244a4
feat: report as a usage when no reference were found to manifest items
rdeltour Dec 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/main/java/com/adobe/epubcheck/messages/DefaultSeverities.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ private void initialize()
// OPF
severities.put(MessageId.OPF_001, Severity.ERROR);
severities.put(MessageId.OPF_002, Severity.FATAL);
severities.put(MessageId.OPF_003, Severity.WARNING);
severities.put(MessageId.OPF_003, Severity.USAGE);
severities.put(MessageId.OPF_004, Severity.WARNING);
severities.put(MessageId.OPF_004a, Severity.ERROR);
severities.put(MessageId.OPF_004b, Severity.ERROR);
Expand Down Expand Up @@ -291,6 +291,7 @@ private void initialize()
severities.put(MessageId.OPF_095, Severity.ERROR);
severities.put(MessageId.OPF_096, Severity.ERROR);
severities.put(MessageId.OPF_096b, Severity.USAGE);
severities.put(MessageId.OPF_097, Severity.USAGE);

// PKG
severities.put(MessageId.PKG_001, Severity.WARNING);
Expand Down
1 change: 1 addition & 0 deletions src/main/java/com/adobe/epubcheck/messages/MessageId.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,7 @@ public enum MessageId implements Comparable<MessageId>
OPF_095("OPF-095"),
OPF_096("OPF-096"),
OPF_096b("OPF-096b"),
OPF_097("OPF-097"),

// Messages relating to the entire package
PKG_001("PKG-001"),
Expand Down
23 changes: 23 additions & 0 deletions src/main/java/com/adobe/epubcheck/opf/OPFChecker30.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,17 @@ protected void checkItem(OPFItem item, OPFHandler opfHandler)
{
report.message(MessageId.OPF_091, item.getLocation());
}

// Register media overlay usage
if (context.referenceRegistry.isPresent() && !Strings.isNullOrEmpty(item.getMediaOverlay()))
{
Optional<OPFItem> overlay = opfHandler.getItemById(item.getMediaOverlay());
if (overlay.isPresent())
{
context.referenceRegistry.get().registerReference(overlay.get().getURL(),
Reference.Type.MEDIA_OVERLAY, item.getLocation());
}
}
}

@Override
Expand Down Expand Up @@ -156,6 +167,18 @@ else if (!context.referenceRegistry.get().hasReferencesTo(item.getURL()))
}
}

// check that resources in the manifest are referenced (usage report)
// - search the reference registry
// - report if no reference (of a publication-resource type) is found
if (!(item.isInSpine() || item.isNav() || item.isNcx())
&& context.referenceRegistry.isPresent()
&& context.referenceRegistry.get().asList().stream()
.noneMatch(ref -> ref.targetResource.equals(item.getURL())
&& ref.type.isPublicationResourceReference()))
{
report.message(MessageId.OPF_097, item.getLocation(), item.getPath());
}

if (isBlessedItemType(mediatype, version))
{
// check whether media-overlay attribute needs to be specified
Expand Down
21 changes: 20 additions & 1 deletion src/main/java/org/w3c/epubcheck/core/references/Reference.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ public static enum Type
// Publication resources
GENERIC,
STYLESHEET,
MEDIA_OVERLAY,
HYPERLINK,
FONT,
IMAGE,
Expand All @@ -30,7 +31,25 @@ public static enum Type
SEARCH_KEY,
NAV_TOC_LINK,
NAV_PAGELIST_LINK,
OVERLAY_TEXT_LINK,
OVERLAY_TEXT_LINK;

public boolean isPublicationResourceReference()
{
switch (this)
{
case GENERIC:
case STYLESHEET:
case FONT:
case IMAGE:
case AUDIO:
case VIDEO:
case TRACK:
case MEDIA_OVERLAY:
return true;
default:
return false;
}
}
}

public final URL url;
Expand Down
3 changes: 2 additions & 1 deletion src/main/resources/com/adobe/epubcheck/messages/MessageBundle.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,8 @@ OPF_093=The "media-type" attribute is required for linked resources located in t
OPF_094=The "media-type" attribute is required for "%1$s" links.
OPF_095=The "media-type" attribute of "voicing" links must be an audio MIME type, but found "%1$s".
OPF_096=Non-linear content must be reachable, but found no hyperlink to "%1$s".
OPF_096b=No hyperlink was found to non-linear document "%1$s", please check that it can be reached from scripted content.
OPF_096b=No hyperlink was found to non-linear document "%1$s", please check that it can be reached from scripted content.
OPF_097=Resource "%1$s" is listed in the manifest, but no reference to it was found in content documents.

#Package
PKG_001=Validating the EPUB against version %1$s but detected version %2$s.
Expand Down
0 .../20-severity-tester/OPS/content_001.xhtml → .../20-severity-tester/OPS/content 001.xhtml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion src/test/resources/com/adobe/epubcheck/tools/20-severity-tester/OPS/toc.ncx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<navLabel>
<text>Loomings</text>
</navLabel>
<content src="content_001.xhtml"/>
<content src="content%20001.xhtml"/>
</navPoint>
</navMap>
</ncx>
Expand Down
0 ...s/20-warning-tester/OPS/content_001.xhtml → ...s/20-warning-tester/OPS/content 001.xhtml
View file
Open in desktop
File renamed without changes.
11 changes: 0 additions & 11 deletions src/test/resources/com/adobe/epubcheck/tools/20-warning-tester/OPS/content_002.xhtml
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion src/test/resources/com/adobe/epubcheck/tools/20-warning-tester/OPS/package.opf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<dc:identifier id="q">NOID</dc:identifier>
</metadata>
<manifest>
<item id="content_001" href="content_001.xhtml" media-type="application/xhtml+xml"/>
<item id="content_001" href="content%20001.xhtml" media-type="application/xhtml+xml"/>
<item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml" />
</manifest>
<spine toc="ncx">
Expand Down
2 changes: 1 addition & 1 deletion src/test/resources/com/adobe/epubcheck/tools/20-warning-tester/OPS/toc.ncx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<navLabel>
<text>Loomings</text>
</navLabel>
<content src="content_001.xhtml"/>
<content src="content%20001.xhtml"/>
</navPoint>
</navMap>
</ncx>
Expand Down
1 change: 1 addition & 0 deletions src/test/resources/com/adobe/epubcheck/tools/severity_override.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
ID Severity Message Suggestion
RSC-008 ERROR This is an overridden message This is an overridden suggestion.
PKG-010 ERROR
OPF-003 ERROR
NCX-004 SUPPRESSED
1 change: 0 additions & 1 deletion src/test/resources/epub-multiple-renditions/multiple-rendition-publication.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ Feature: EPUB Multiple-Rendition ▸ Full Publication Checks
Then warning RSC-019 is reported (for the missing metatada.xml file)
And warning RSC-017 is reported (for the missing selection attributes)
And the message contains 'At least one rendition selection attribute should be specified for each non-first rootfile element'
And warning OPF-003 is reported (for a mapping document without a link in the container.xml file)
And no other errors or warnings are reported


Expand Down
0 ...undeclared-warning/META-INF/container.xml → ...e-undeclared-usage/META-INF/container.xml
View file
Open in desktop
File renamed without changes.
0 ...-undeclared-warning/OPS/content_001.xhtml → ...ce-undeclared-usage/OPS/content_001.xhtml
View file
Open in desktop
File renamed without changes.
0 ...ndeclared-warning/OPS/not-in-manifest.txt → ...-undeclared-usage/OPS/not-in-manifest.txt
View file
Open in desktop
File renamed without changes.
0 ...source-undeclared-warning/OPS/package.opf → ...resource-undeclared-usage/OPS/package.opf
View file
Open in desktop
File renamed without changes.
0 ...t-resource-undeclared-warning/OPS/toc.ncx → ...est-resource-undeclared-usage/OPS/toc.ncx
View file
Open in desktop
File renamed without changes.
0 ...fest-resource-undeclared-warning/mimetype → ...nifest-resource-undeclared-usage/mimetype
View file
Open in desktop
File renamed without changes.
10 changes: 6 additions & 4 deletions src/test/resources/epub2/opf-publication.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,12 @@ Feature: EPUB 2 ▸ Open Packaging Format ▸ Full Publication Checks
Then warning OPF-035 is reported
And no other errors or warnings are reported

Scenario: Report a reference to a resource that is not listed in the manifest
When checking EPUB 'opf-manifest-resource-undeclared-warning'
Then warning OPF-003 is reported
And no other errors or warnings are reported
Scenario: Report (usage) a resource that is not listed in the manifest
Given the reporting level is set to usage
When checking EPUB 'opf-manifest-resource-undeclared-usage'
Then usage OPF-003 is reported
But no other usages are reported
And no errors or warnings are reported

Scenario: Verify that operating system files (`.DS_STORE`, `thumbs.db`) are ignored (issue 256)
When checking EPUB 'opf-manifest-os-files-ignore-valid'
Expand Down
11 changes: 11 additions & 0 deletions ...ckage-document/files/manifest-not-listing-container-resource-usage/EPUB/content_001.xhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html xmlns:epub="http://www.idpf.org/2007/ops" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="utf-8"/>
<title>Minimal EPUB</title>
</head>
<body>
<h1>Loomings</h1>
<p>Call me Ishmael.</p>
</body>
</html>
14 changes: 14 additions & 0 deletions ...b3/05-package-document/files/manifest-not-listing-container-resource-usage/EPUB/nav.xhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xml:lang="en" lang="en">
<head>
<meta charset="utf-8"/>
<title>Minimal Nav</title>
</head>
<body>
<nav epub:type="toc">
<ol>
<li><a href="content_001.xhtml">content 001</a></li>
</ol>
</nav>
</body>
</html>
Empty file added ...age-document/files/manifest-not-listing-container-resource-usage/EPUB/not-in-manifest.txt
View file
Open in desktop
Empty file.
16 changes: 16 additions & 0 deletions .../05-package-document/files/manifest-not-listing-container-resource-usage/EPUB/package.opf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="UTF-8"?>
<package xmlns="http://www.idpf.org/2007/opf" version="3.0" xml:lang="en" unique-identifier="q">
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:title id="title">Minimal EPUB 3.0</dc:title>
<dc:language>en</dc:language>
<dc:identifier id="q">NOID</dc:identifier>
<meta property="dcterms:modified">2017-06-14T00:00:01Z</meta>
</metadata>
<manifest>
<item id="content_001" href="content_001.xhtml" media-type="application/xhtml+xml"/>
<item id="nav" href="nav.xhtml" media-type="application/xhtml+xml" properties="nav"/>
</manifest>
<spine>
<itemref idref="content_001" />
</spine>
</package>
6 changes: 6 additions & 0 deletions ...ckage-document/files/manifest-not-listing-container-resource-usage/META-INF/container.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8" ?>
<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
<rootfiles>
<rootfile full-path="EPUB/package.opf" media-type="application/oebps-package+xml"/>
</rootfiles>
</container>
1 change: 1 addition & 0 deletions ...es/epub3/05-package-document/files/manifest-not-listing-container-resource-usage/mimetype
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
application/epub+zip
7 changes: 7 additions & 0 deletions src/test/resources/epub3/05-package-document/package-document.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,13 @@ Feature: EPUB 3 — Package document
When checking EPUB 'package-remote-font-undeclared-error'
Then error RSC-008 is reported
And no other errors or warnings are reported

Scenario: Report (usage) a container resource that is not listed in the manifest
Given the reporting level is set to usage
When checking EPUB 'manifest-not-listing-container-resource-usage'
Then usage OPF-003 is reported
But no other usages are reported
And no errors or warnings are reported


### 5.6.2 The item element
Expand Down
1 change: 0 additions & 1 deletion src/test/resources/epub3/06-content-document/content-document-css.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ Feature: EPUB 3 — Content Documents — CSS
Scenario: Report an attempt to `@import` a CSS file that is not declared in the manifest but is present in the container
When checking EPUB 'content-css-import-not-declared-error'
Then error RSC-008 is reported
And warning OPF-003 is reported
And no other errors or warnings are reported

@spec @xref:sec-container-iri @xref:sec-manifest-elem
Expand Down
1 change: 0 additions & 1 deletion src/test/resources/epub3/06-content-document/content-document-xhtml.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,6 @@ Feature: EPUB 3 — Content Documents — XHTML
Scenario: Report references to undeclared resources in `img srcset`
When checking EPUB 'content-xhtml-img-srcset-undeclared-error'
Then error RSC-008 is reported (undeclared resource in srcset)
And warning OPF-003 is reported (undeclared resource in container)
And no other errors or warnings are reported

Scenario: Allow an `img` element with a video resource
Expand Down
1 change: 1 addition & 0 deletions ...rlays/files/mediaoverlays-active-class-stylesheet-undeclared-valid/EPUB/content_001.xhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<head>
<meta charset="utf-8" />
<title>Minimal EPUB</title>
<link rel="stylesheet" href="styles.css"/>
</head>
<body>
<h1 id="c01">Loomings</h1>
Expand Down