Repository navigation
Autoscroll before before archiving and take full-height screenshots #80
Description
Activity
- changed the title
[-]Support for full page screenshots instead of fixed dimensions[/-][+]Autoscroll before before archiving and take full-height screenshots[/+]on Mar 15, 2019 The code provided in this playwright issue solves the full-page screenshot problem for me
microsoft/playwright#620Here is the code I use to take a full page screenshot with playwright
const { chromium } = require('playwright'); (async () => { const browser = await chromium.launch({ channel: 'chrome' // or 'msedge', 'chrome-beta', 'msedge-beta', 'msedge-dev', etc. }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://apple.com/'); await scrollFullPage(page); await page.screenshot({ path: 'apple.png', fullPage : true }); await browser.close(); })(); async function scrollFullPage(page) { await page.evaluate(async () => { await new Promise(resolve => { let totalHeight = 0; const distance = 100; const timer = setInterval(() => { const scrollHeight = document.body.scrollHeight; window.scrollBy(0, distance); totalHeight += distance; if (totalHeight >= scrollHeight){ clearInterval(timer); resolve(); } }, 100); }); }); }`Is this feature natively available now or only via hacking in user scripts?
Ah fair enough, thanks! Seems like #51 encapsulates a whole ton of effort to make this happen, so thanks and good luck!
Reacted by Nick Sweeting, Taha Ahmed and Vasileios Anastasiouwill be great to have the ability to take full height screenshots! any update on this after 4 years?
My conclusion after a lot of work on this issue is that full-page screenshots up to ~8000px maximum height are ok, but many many pages are longer than that, and most common image formats actually don't support images that big. Even the formats that do (png) cause most image viewers to crash when you try to open them. You need to mess with Chrome's GPU memory settings to even get it to take more than 16,000px in one image, let alone the 90,000px+ that some long comment thread pages have.
Multiple screenshots are the better solution. My solution so far is one 4:3 screenshot at the top of the page, and then numbered 16:10 screenshots for like ~15 full-height scrolls down the page. Also works great for feeding it to vision and OCR models for analysis.
I built this ^ more advanced puppeteer based screenshot approach for a paying client last year, and it's still in active development. It's all in TS and ArchiveBox is all Python, so it takes time to bridge that gap, refactor, open source it, document it, package it, ship it, etc. for the public.
Reacted by Joker_, Kai Webber and 002kanidevnow has aninfiniscrollplugin out-of-the-box which scrolls the page up to N times and expands comments and detail blocks in the process. it doesnt implement the multiple screenshots approach yet but it's a step in the right direction.Reacted by fyachanThis is done and working well now in
abx-dlandarchivebox:dev: https://archivebox.github.io/abx-plugins/#infiniscroll
I've sumbitted a Chromium bug tracker feature request for adding a
--full-pageflag: https://bugs.chromium.org/p/chromium/issues/detail?id=854013Hopefully it's merged, allowing us to screenshot the full height of pages, instead of limiting them to the config settings defined by
DIMENSIONS.