Repository navigation
Formalize an HTML to block conversion tool #1061
Description
Activity
I agree with this. I'm using Gutenberg packages on the server with Miles to validate blocks and generally handle parts of the transformation. I know others are doing the same with the mentioned tools and packages.
What if an html to block conversion package was part of Gutenberg, and then ultimately part of core? That tackles a few things, wide distribution, the ability for products to use it on the server, and the ability for agents to use the package locally when working with WordPress.
Agents love building in HTML, they are excellent at it. Having a canonical conversion library I think is a great idea to make sure WordPress can work with whatever agents are creating.
I am happy to chip in thoughts and ideas based on my experiences.
Gutenberg already offers HTML to Blocks conversions, it's called "rawHandler" and it's part of the blocks package. Only requirement though is that the blocks need to be registered. I'd love if we improve that instead of inventing something new.
Gutenberg already offers HTML to Blocks conversions, it's called "rawHandler" and it's part of the blocks package. Only requirement though is that the blocks need to be registered. I'd love if we improve that instead of inventing something new.
@youknowriad , @swissspidy shared this GB PR in Slack, which is in line with that direction.
The problem is
rawHandlerthrows away the design which is really the whole point. You're right though that's a better way to think about it, especially if it could be extended to handle that.Reacted by Riad Benguella, annezazu and Chris HuberFormalizing a block conversion tool makes a lot of sense, but what do we mean when we say that?
rawHandleris a great home for a number of things those tools are doing and we should definitely consolidate some pieces intorawHandler. I had great success building the JS blocks-engine package on top of it, adding pieces that were missing or ill suited for Gutenberg itself.There are things that these other tools are doing that are more data liberation centric that muddies my mind on what this thread's suggestion actually is. Liberating data probably warrants a separate tool that uses
rawHandlerunder the hood and layers on the "other"stuff".blocks-engine does the following "other stuff"
- Fonts: Identifies Google Fonts sources and prepares font-handling plans
- Forms: Extracts fields, presentation, and behavior requirements for forms for downstream integration
- Website translation: Produces a site plan for converting a site—pages, routes, navigation, template parts, theme files, assets, etc
- Migrating site CSS: Analyzes source stylesheets, rewrites selectors for the converted markup, generates extra CSS for layout and appearance
- Images: Produces image inventories, references, and rewrite plans
- Source-site JS: Packages unsupported interactive regions with associated scripts and determines what JS should be preserved or dropped.
- Custom blocks: Dynamically generate block definitions for selected unsupported content
- Template parts: Determines how content / site structure should be converted to shared template parts
- Source bundles: Normalizes multi-file bundles, supports format conversion
- Reporting: Reports on success with evidence
blocks-engine, block-runner, and data liberation do the above (for the most part) plus other things like:
- Performant extraction: managing multiple workers to assess and extract data from source sites (static or hosted sites).
- Provider adapters: These tools are provider aware and tackle the effort of identifying elements that should never convert to blocks from HTML (cookie banners, advertisement popups, etc)
- Deduplication: Cross-page content and image deduplication
rawHandleris really good and can get better. If the suggestion of formalizing an HTML to block conversion tool is around pure conversion, that's awesome. It won't cause the above tools to go away—they'll just have the opportunity to simplify some things. If we're talking about a central tool to do a lot of those data liberation pieces, that's a bit more bigger thanrawHandleralone.Could we integrate this work also into https://github.com/WordPress/agent-skills? Its format supports both transformation instructions and scripts, making it a good fit for guiding agents through conversion and validation.
Building on the distinction between core conversion and broader site workflows, a shared skill could teach agents how to use the underlying tools, including
rawHandler, while keeping that guidance maintained by the WordPress project.Reacted by annezazu and Andy PeatlingOohh I think adding it to Agent Skills and including rawHandler is a great idea! Is someone game to take that on?
I'm game, @annezazu. I build Respira, whose HTML import converts agent-written HTML into each page builder's own format, blocks included, so I've hit most of the edges in this thread in production.
Plan for a first PR to WordPress/agent-skills, scoped to pure conversion as @borkweb framed it (the site-migration layer stays with blocks-engine and friends):
- A
wp-html-to-blocksskill that teaches agents to convert withrawHandleragainst the registered core blocks, rather than hand-writing block markup. - Keeping the design. Before conversion, map what
rawHandlerwould drop (colours, spacing, typography, layout) onto block supports and theme.json presets, so the design lands in block attributes instead of being thrown away. This is @apeatling's point, and it's the part agents get wrong most often. - Validation as a hard step. Parse, serialize and compare, in the spirit of Studio's
validate-blocks: the output either round-trips cleanly or the agent is told which block failed and why.core/htmlis a per-section fallback that gets reported, never the silent default. - A small script (Node,
@wordpress/blocksplus the core block library) that runs conversion and validation locally, so the skill works without a WordPress install, plus a few evals.
If #82013 lands improvements in
rawHandleritself, the skill points at them. I'll open the PR as a draft so it's easy to steer early.- A
Draft is up: WordPress/agent-skills#119. Six fixtures convert and validate cleanly, including a legacy page where the form and iframe are reported as
core/htmlfallbacks instead of slipping through. Three open questions are in the PR description (npm deps in a skill, whether the container step should go upstream as raw transforms, and thedata-block-attrsmarkers).Reacted by annezazuA skill is certainly a must-have, but IMHO as much as possible of those helper scripts should be in
@wordpress/*packages and work standalone. The skill is then just another entry point.Agreed, @swissspidy. The skill should be a thin entry point, and the conversion and validation logic should live where everything else can use it. Two pieces in the draft (WordPress/agent-skills#119) look like natural candidates to move:
- Validation: "parse, serialize, compare, and report which block failed and why" is useful well beyond agents (CI, importers, Studio's validate-blocks). It could be a standalone helper in
@wordpress/blocks, or a small package that runs headless in Node. - Containers: the step that builds Group/Columns/Buttons from
div/sectionmarkup. That's arguably raw transforms that belong in the core blocks themselves, alongside #82013.
If that direction sounds right, I'm happy to open a Gutenberg issue proposing the validation helper first, since it's the smaller and less opinionated piece, and keep the skill pointing at whatever lands.
- Validation: "parse, serialize, compare, and report which block failed and why" is useful well beyond agents (CI, importers, Studio's validate-blocks). It could be a standalone helper in
I am seeing a range of these kinds of tools https://www.accelerateplugin.com/block-runner/ & https://github.com/Automattic/blocks-engine & https://github.com/Automattic/studio/blob/trunk/apps/cli/ai/tools/validate-blocks.ts & https://github.com/a8cteam51/claude-code-plugins/tree/trunk/plugins/html-to-block-theme solving the same problem: converting HTML to blocks when working with AI.
It makes me wonder if we should just centralize a skill like this and keep it up to date from the WordPress project side as I imagine more and more people are going to re-invent the wheel here. Opening this issue for discussion and so folks who have worked on these sorts of problems can collectively chime in to create something shared and great.