Skip to content

Backend: Annotations: migrate HTML snippets to JSON format #56

Description

@gwern

Annotation popups rely on pregenerated static HTML snippets generated by LinkMetadata.hs's generateAnnotationBlock and stored in /metadata/annotation/$ESCAPED_URL.html. The HTML snippets look like fully preformatted HTML blocks which can be transcluded as-is; random recently added example:

<!-- $ cat ~/wiki/metadata/annotation/https%3A%2F%2Fpmc.ncbi.nlm.nih.gov%2Farticles%2FPMC4032098%2F.html -->
<p><a href="/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html" id="annotation-degutis-et-al-2014" class="link-modified-recently link-live link-annotated" data-url-archive="/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html" data-url-original="https://pmc.ncbi.nlm.nih.gov/articles/PMC4032098/" data-doi="10.1093/brain/awu062" data-id-ref="degutis-et-al-2014" data-link-icon="nlm-ncbi" data-link-icon-color="#20558a" data-link-icon-type="svg">“Holistic Face Training Enhances Face Processing in Developmental Prosopagnosia”</a></p>
<p> <span class="author">Joseph DeGutis, Sarah Cohan, <a href="https://en.wikipedia.org/wiki/Ken_Nakayama" class="link-live id-not" data-url-iframe="https://en.m.wikipedia.org/wiki/Ken_Nakayama#bodyContent" data-link-icon="wikipedia" data-link-icon-type="svg" title="Ken Nakayama">Ken Nakayama</a></span></p>
<p><span class="date cite-date" title="2014-03-29">2014</span></p>
<p><span class="link-tags"><a href="/doc/psychology/vision/index" class="prefetch-not link-tag link-page link-annotated icon-not" rel="tag" title="Link to psychology/vision tag index">sight</a></span></p>
<p><span class="filesize"><span class="filesize-bytes">2938017</span> <span class="filesize-percentage">70</span></span></p>
<blockquote>
<p>Recent case studies suggest that face recognition can be improved in individual developmental <a href="https://en.wikipedia.org/wiki/Prosopagnosics" id="_f2monRm2" class="link-live" data-url-iframe="https://en.m.wikipedia.org/wiki/Prosopagnosics#bodyContent" data-link-icon="wikipedia" data-link-icon-type="svg">prosopagnosics</a>. Using a 3-week online program targeting holistic face processing, DeGutis et al <span class="date-range">2014<sub><span title="2014 was 12 years ago.">12ya</span></sub></span> reveal perceptual improvements in 24 subjects. Those who reached more difficult levels of training showed the greatest improvements in holistic processing.</p>
<div class="horizontal-rule-nth-1">
<hr>
</div>
<p>Prosopagnosia has largely been regarded as an untreatable disorder. However, recent case studies using cognitive training have shown that it is possible to enhance face recognition abilities in individuals with developmental prosopagnosia.</p>
<p>Our goal was to determine if this approach could be effective in a larger population of developmental prosopagnosics.</p>
<p>We trained 24 developmental prosopagnosics using a 3-week online face-training program targeting holistic face processing. 12 subjects with developmental prosopagnosia were assessed before and after training, and the other 12 were assessed before and after a waiting period; they then performed the training and were then assessed again. The assessments included measures of front-view face discrimination, face discrimination with viewpoint changes, measures of holistic face processing, and a 5-day diary to quantify potential real-world improvements.</p>
<p>Compared with the waiting period, developmental prosopagnosics showed moderate but statistically-significant overall training-related improvements on measures of front-view face discrimination. Those who reached the more difficult levels of training (“better” trainees) showed the strongest improvements in front-view face discrimination and showed statistically-significantly increased holistic face processing to the point of being similar to that of unimpaired control subjects. Despite challenges in characterizing developmental prosopagnosics’ everyday face recognition and potential biases in self-report, results also showed modest but consistent self-reported diary improvements.</p>
<p>In summary, we demonstrate that by using cognitive training that targets holistic processing, it is possible to enhance face perception across a group of developmental prosopagnosics and further suggest that those who improved the most on the training task received the greatest benefits.</p>
<p><!-- <p>[<strong>Keywords</strong>: developmental prosopagnosia, computer-based cognitive remediation, configural/holistic processing]</p> --></p>
</blockquote>
<div class="aux-links-transclude-file">
<div class="collapse">
<p><strong>View HTML</strong>:</p>
<p><a href="/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html" class="id-not link-annotated-not include-content include-lazy" data-link-icon="nlm-ncbi" data-link-icon-color="#20558a" data-link-icon-type="svg">Holistic Face Training Enhances Face Processing in Developmental Prosopagnosia</a></p>
</div>
</div>

This, however, is a lie. The JS frontend has, for a long time, actually parsed out each individual labeled field, reformatted them on the fly, and then displayed them in various ways. The backend goes to all this effort to create a nice clean browseable HTML snippet (you can load them up in a browser and they'll render OK-ish), but it's all thrown away by the frontend.

It's a waste of code and effort (on both the backend and frontend), and also risky as sometimes the backend will add/change formatting which screws things up for the frontend.

So it would better to just drop the pretense that the snippets are anything but data exchange, and turn them into simple JSON blobs, as anyone would assume they actually are on the backend, because that is the usual design pattern for web dev. These are easier to read (at this point), serialize and parse, and validate or otherwise work with.

The above would instead be written out as /metadata/annotation/$ESCAPED_URL.json and look something like this:

{
  "title": "<a href=\"/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html\" id=\"annotation-degutis-et-al-2014\" class=\"link-modified-recently link-live link-annotated\" data-url-archive=\"/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html\" data-url-original=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC4032098/\" data-doi=\"10.1093/brain/awu062\" data-id-ref=\"degutis-et-al-2014\" data-link-icon=\"nlm-ncbi\" data-link-icon-color=\"#20558a\" data-link-icon-type=\"svg\">“Holistic Face Training Enhances Face Processing in Developmental Prosopagnosia”</a>",
  "authors": "Joseph DeGutis, Sarah Cohan, <a href=\"https://en.wikipedia.org/wiki/Ken_Nakayama\" class=\"link-live id-not\" data-url-iframe=\"https://en.m.wikipedia.org/wiki/Ken_Nakayama#bodyContent\" data-link-icon=\"wikipedia\" data-link-icon-type=\"svg\" title=\"Ken Nakayama\">Ken Nakayama</a>",
  "date": "<span class=\"date cite-date\" title=\"2014-03-29\">2014</span>",
  "tags": "<a href=\"/doc/psychology/vision/index\" class=\"prefetch-not link-tag link-page link-annotated icon-not\" rel=\"tag\" title=\"Link to psychology/vision tag index\">sight</a>",
  "fileSize": { "bytes": 2938017, "percentage": 70 },
  "abstract": "<p>Recent case studies suggest that face recognition can be improved in individual developmental <a href=\"https://en.wikipedia.org/wiki/Prosopagnosics\" id=\"_f2monRm2\" class=\"link-live\" data-url-iframe=\"https://en.m.wikipedia.org/wiki/Prosopagnosics#bodyContent\" data-link-icon=\"wikipedia\" data-link-icon-type=\"svg\">prosopagnosics</a>. Using a 3-week online program targeting holistic face processing, DeGutis et al <span class=\"date-range\">2014<sub><span title=\"2014 was 12 years ago.\">12ya</span></sub></span> reveal perceptual improvements in 24 subjects. Those who reached more difficult levels of training showed the greatest improvements in holistic processing.</p>\n<div class=\"horizontal-rule-nth-1\">\n<hr>\n</div>\n<p>Prosopagnosia has largely been regarded as an untreatable disorder. However, recent case studies using cognitive training have shown that it is possible to enhance face recognition abilities in individuals with developmental prosopagnosia.</p>\n<p>Our goal was to determine if this approach could be effective in a larger population of developmental prosopagnosics.</p>\n<p>We trained 24 developmental prosopagnosics using a 3-week online face-training program targeting holistic face processing. 12 subjects with developmental prosopagnosia were assessed before and after training, and the other 12 were assessed before and after a waiting period; they then performed the training and were then assessed again. The assessments included measures of front-view face discrimination, face discrimination with viewpoint changes, measures of holistic face processing, and a 5-day diary to quantify potential real-world improvements.</p>\n<p>Compared with the waiting period, developmental prosopagnosics showed moderate but statistically-significant overall training-related improvements on measures of front-view face discrimination. Those who reached the more difficult levels of training (“better” trainees) showed the strongest improvements in front-view face discrimination and showed statistically-significantly increased holistic face processing to the point of being similar to that of unimpaired control subjects. Despite challenges in characterizing developmental prosopagnosics’ everyday face recognition and potential biases in self-report, results also showed modest but consistent self-reported diary improvements.</p>\n<p>In summary, we demonstrate that by using cognitive training that targets holistic processing, it is possible to enhance face perception across a group of developmental prosopagnosics and further suggest that those who improved the most on the training task received the greatest benefits.</p>\n<p><!-- <p>[<strong>Keywords</strong>: developmental prosopagnosia, computer-based cognitive remediation, configural/holistic processing]</p> --></p>",
  "transcludeFile": "<div class=\"aux-links-transclude-file\">\n<div class=\"collapse\">\n<p><strong>View HTML</strong>:</p>\n<p><a href=\"/doc/www/pmc.ncbi.nlm.nih.gov/c6efaf561276752c9e3469c20e0f0da28e6f4175.html\" class=\"id-not link-annotated-not include-content include-lazy\" data-link-icon=\"nlm-ncbi\" data-link-icon-color=\"#20558a\" data-link-icon-type=\"svg\">Holistic Face Training Enhances Face Processing in Developmental Prosopagnosia</a></p>\n</div>\n</div>"
}

Metadata

Metadata

Assignees

Labels

AIIssue which can probably be resolved by agentic LLM AI coding, rather than scarce humans.BackendOriginal content, Hakyll/Haskell/scripts, Markdown/HTML etc. Usually not CSS/JS. Owner: Gwern.enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions