Page MenuHomePhabricator

Deploy ParserMigration extension
Closed, ResolvedPublic

Description

I recently wrote the ParserMigration extension, which is an EditPage subclass which shows a side-by-side preview of edited article HTML in two different parser configurations, initially: tidied with tidy, and tidied with Html5Depurate.

Deployment of this extension will serve as a pilot for deployment of the Html5Depurate service. I am including deployment of the service under this tracking task.

Subtasks:

  • Write puppet module and role
  • Create labs instance for Html5Depurate in deployment-prep project
  • Add extension submodule and configuration to MW core, deploy to deployment-prep
  • Provision servers in eqiad and codfw for production deployment of the service (or use sca/scb as suggested by Antoine?)
  • Write pybal configuration and provision service IP
  • Test the monitoring provided by the puppet role, and add monitoring for the service IP
  • Enable MW extension in production

Please advise on any other necessary subtasks. In particular, is a security review needed? Is any approval needed before I deploy to deployment-prep?

Event Timeline

tstarling triaged this task as Medium priority.Jul 28 2016, 11:36 PM

The canonical location for the checklist to deploy a new extension is at https://www.mediawiki.org/wiki/Review_queue#Checklist

Looks like you have most things outlined in the description other than the security review (I don't think, given the purpose/use of the extension, that you'll need a product or design review, ftr). The security review is done before deploying to deployment-prep but the actual deploy there doesn't need prior approval other than that (and this, or another beta cluster-specific, task).

Setting up the service in deployment-prep should probably go in there somewhere after "deploy extension to deployment-prep" and before "enable extension in production" (the ordering of those in-between subtasks seems to be not too strict, correct?)

Change 344276 had a related patch set uploaded (by Tim Starling):
[operations/mediawiki-config@master] Deploy ParserMigration extension

https://gerrit.wikimedia.org/r/344276

Change 344276 merged by jenkins-bot:
[operations/mediawiki-config@master] Deploy ParserMigration extension

https://gerrit.wikimedia.org/r/344276

Mentioned in SAL (#wikimedia-operations) [2017-04-03T23:19:19Z] <thcipriani@tin> Started scap: SWAT: [[gerrit:344276|Deploy ParserMigration extension]] T141586 (l10nupdate only)

Mentioned in SAL (#wikimedia-operations) [2017-04-03T23:41:43Z] <thcipriani@tin> Finished scap: SWAT: [[gerrit:344276|Deploy ParserMigration extension]] T141586 (l10nupdate only) (duration: 22m 24s)

Mentioned in SAL (#wikimedia-operations) [2017-04-03T23:54:01Z] <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:344276|Deploy ParserMigration extension]] T141586 (for real) (duration: 00m 44s)

Any information about this tool? It has just appeared in prefs.

I imagine most of the information about this ParserMigration extension is in this task or in https://www.mediawiki.org/wiki/Html5Depurate.

It's not clear to me what the status of this task is. I think some of the subtask checkboxes in the task description can be checked. Maybe the whole task is now resolved/fixed?

Sorry, we are a little behind on this. The original plan was to deploy HTML5Depurate in production but we instead migrated to RemexHTML late last year. But, we will update the relevant pages. We are also going to send out more information on mailing list about ParserMigration and Linter next week. But, till that time, here is some more information from Scrum of Scrums update yesterday.

Hello and sorry for interrupting, @tstarling. I tried it today. It works, and I can see the difference in the whitespace. But I can't see any difference in self closed tags, as in

<div style="display:none;"/>text

or

text<b/>text

What's the problem? Thank you.

Hello and sorry for interrupting, @tstarling. I tried it today. It works, and I can see the difference in the whitespace. But I can't see any difference in self closed tags, as in

<div style="display:none;"/>text

or

text<b/>text

See https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy#Things_to_fix ... "Both the PHP parser as well as Parsoid right now have backward compatibility code to handle these. However, it is good to fix them up."

Thank you. Missed this. And sorry again.

What is the difference between the ParserMigration and the Linter?

What is the difference between the ParserMigration and the Linter?

The Linter finds for you broken pages.
ParserMigration duplicates for you the editing Preview window - how does it look now and how will it look in HTML5.

The Linter finds for you broken pages.
ParserMigration duplicates for you the editing Preview window - how does it look now and how will it look in HTML5.

Oh, i see, sorry. Is it possible to add ParserMigration to Linter interface? For example: It will be good to see ParserMigration on Special:LintErrors/self-closed-tag page.

Is it possible to add ParserMigration to Linter interface? For example: It will be good to see ParserMigration on Special:LintErrors/self-closed-tag page.

But why do you need this? It compares the current page versions. There is nothing editable in special pages. You can see the ParserMigration link on every non special page, including those listed in LintErrors.

The Linter finds for you broken pages.
ParserMigration duplicates for you the editing Preview window - how does it look now and how will it look in HTML5.

Oh, i see, sorry. Is it possible to add ParserMigration to Linter interface? For example: It will be good to see ParserMigration on Special:LintErrors/self-closed-tag page.

In the general case, ParserMigration compares output from the production version of the parser with output from a modified parser. Right now, it is comparing Tidy (HTML4 semantics) and RemexHTML (HTML5 semantics), but in the future, it might be something else. So, the link would only be useful temporarily.

Specifically for the self-closed-tag linter category, ParserMigration output will not show any difference because we have added a backward compatibility fix for that.

But, all that said, maybe there is some benefit to adding a temporary link in the Linter page to compare output via ParserMigration for some linter categories. We'll discuss that and see if this is the best way to do this. Thanks for the suggestion.