Summary: Links to RFC text with in-lined errata. HTML for v3 XML draft submissions
Release Date: Fri, September 20, 2019 at 4:18 PM UTC
Release Author: Henrik Levkowetz

• Added conversion of version 3 XML draft submissions to v3 HTML, in
  addition to text. Added a link to HTML versions (when present) to the
  document status page, and renamed the link to the htmlized version to
  htmlized. Converted the test-file for XML submission tests to v3.
  Added test code to check that HTML is generated for v3 XML submissions.

• Added class bg-danger to submission error messages on the submission
  upload result page. The big Meta-Data errors found pane with red
  background is so noticeable that class text-danger (which gives bold
  text, but no change of background) is insufficient to stand out clearly.
  Adding bg-danger gives field error messages red background, too.

• Added a link from the datatracker RFC pages to the new RFC renderings
  with inline errata on the RFC-Editor web page (where available).

• Updated rfc-editor sync code to recognise the new Tooling Issue queue
  state.

• Updated ietf.sync.rfceditor.parse_index() to use the new rfc-index
  schema (changed placement of the page-count element).

• Added code to catch a form error regularly caused by form spam and
  return a failure instead of triggering a server 500.

Coverage

Summary: IANA expert review tracking, new constraints labelling
Release Date: Sun, September 8, 2019 at 3:28 PM UTC
Release Author: Henrik Levkowetz

This release adds IANA expert review tracking for drafts, changes the
session request constraints labels, improves the handling of session slide
proposal uploads, and adds a number of smaller enhancements and issue
fixes. From the commit log:

• Merged in [16656] from rjsparks@nostrum.com:
  Added IANA expert review tracking.

• Merged in [16650], [16655] from rjsparks@nostrum.com:
  Removed the bootstrap project todo page. #2778 points to the old content
  so we can make new tickets for anything that is still relevant.

• Merged in [16649] from rjsparks@nostrum.com:
  Guard against attempts to use the review request view with a
  ReviewRequest id that does not match the document for the ReviewRequest.
  This would have exposed #2776 much earlier.

• Merged in [16647] from rjsparks@nostrum.com:
  Corrected a view that was looking at review assignment ids when it
  should have been looking at request review ids. Fixes #2776.

• Added a domain variable to context supplied to the agenda_package.txt
  template. Fixes issue #2793.

• Merged in [16646] from rjsparks@nostrum.com:
  Don't manually linewrap when looking at feedback - let the pasted styling
  do that. Fixes #2761.

• Merged in [16630] from rjsparks@nostrum.com:
  Rework how we are labeling constraints to provide a quick-fix for
  session requests at IETF106. Turned off several javascript functions that
  added/enforced old (possibly no longer correct) semantics for the session
  request forms. Fixes #2754.

• Merged in [16629] from rjsparks@nostrum.com:
  Removing unused template. I think it was an artifact of the Django 1.2
  to 1.6 conversion from 6 years ago.

• Merged in [16615] from peter@akayla.com:
  Spelling correction in a comment. No open issue.

• Merged in [16614] from peter@akayla.com:
  Improved so that new draft diffs (from the working group drafts page)
  work correctly for both -00 drafts and later revisions. Fixes issue #2697.

• Merged in [16604] from rjsparks@nostrum.com:
  Look at all scheduled sessions for a group when deciding to display a
  cancelled banner. Fixes #2764.

• The logic for displaying 'Email requested to be sent for earlier
  discuss' on IESG positions was wrong; it would count later discusses and
  discusses from other ADs as an earlier discuss. Moved any_email_sent()
  from ballot to position, and refined the criteria.

• Made sure Person.active_drafts() and .expired_drafts() don't return
  duplicate entries. Fixes issue #2784.

• Changed requirements to permit pyang 2.0 and greater.

• Added a migration to fix some m2m table columns incorrectly left as
  varchar(8) when we changed the size of name slugs to varchar(32).

• Merged in [16601] from rjsparks@nostrum.com:
  A management command to correct the state of several slide
  presentations uploaded during IETF105, and perform some missed powerpoint
  to pdf conversions.

• Merged in [16600] from rjsparks@nostrum.com:
  Fixed a subtle bug in the code that converts slides to PDF that was
  writing back into settings.

• Merged in [16599] from rjsparks@nostrum.com:
  Correct the uploaded_filename for slides that go through the
  approve-proposed-slides workflow.

• Merged in [16598] from rjsparks@nostrum.com:
  Add the ppt* to pdf conversion to the approve-suggested-slides
  workflow. Fixes #2762.

• Added textual information about patent application formats according to
  WIPO's Handbook on Industrial Property Information and Documentation, as
  input to future extension of the regex set.

• Added some new formats to the regex validator for patent numbers and
  patent application numbers. This is still not comprehensive.

• Adjusted migrations for Python2/3 compatibility.

• Updated database options settings to match our production database and
  dump files.

• Fixed an issue where the text file charset information for an upload would
  be overwritten on upload of multiple files for a draft.
  Fixes issue #2768.

• Added 30 minutes caching to agenda.ics

Coverage

Summary: IETF 105 code sprint
Release Date: Mon, August 5, 2019 at 4:43 PM UTC
Release Author: Henrik Levkowetz

This release contains datatracker bug fixes and enhancements from the
IETF-105 Code Sprint. This sprint saw several participants diving deeper
into the datatracker than before, accomplishing substantial tasks. Thanks
to everyone who contributed!

Notable improvements:

The document list pages produced by searches, group document pages, and
similar has been speeded up by a factor of more than 2 by doing detailed
timing measurements and one by one addressing the biggest time sinks. A
number of places where links to session slides were versioned have been
changed so as to always point to the latest version. A calendar display
component has been added, making it straightforward to display for instance
calendar pages for upcoming meetings. Ability to handle markdown uploads
and on-the-fly conversion for agendas and minutes has been added, as well as
the ability to provide WG closing notes, sent out in announcement and saved
for posterity on the group history page. The recently introduced Slide
proposal feature has been made more solid, as has the Python 2 / Python 3
compatibility.

Additional information, excerpted from the commit log:

• Merged in [16593] from rjsparks@nostrum.com:
  Added sorting of the documents in the agenda, minutes, bluesheets section
  of the session_details view. Fixes #2759.

• Merged in [16592] from rjsparks@nostrum.com:
  Changed how the session details page links to documents to not link to a
  specific version. Fixes #2758. See also #2756.

• Merged in [16590] from rjsparks@nostrum.com:
  Example of using webtest. The full testcase was not converted because the
  add_rule form is modified in-flight by javascript when a control value is
  changed.

• Merged in [16583] from rjsparks@nostrum.com:
  Restored the ability to remove documents while managing a personal list.
  Required updating code to match the recent change to Document's primary
  key. Fixes #2757.

• Merged in [16569] from rjsparks@nostrum.com:
  Added quotes around document name in the mailarchive search link on a
  document's main page. Fixes #2659.

• Merged in [16589] from rjsparks@nostrum.com:
  Added a calendar to the upcoming meetings page. Fixes #2719.

• Prevent login name case issues by generating apache permissions file
  entries for both uppercase and lowercase logins.

• Added a Py3 version of requirements.txt, excluding Trac which is still
  only available under Py2.7

• Did some preliminary work on converting bower management of web resources
  to yarn. New management command, replacing bower_install will be needed.

• Merged in [16562] from rcross@amsl.com:
  Added encoding='utf-8' parameter to io.open operations in tests that
  starting failing since adding unicode_literals.

• Merged in [16561] from rjsparks@nostrum.com:
  Added a prompt for login for approving slide suggestions instead of just
  presenting an error. Fixes #2750.

• Merged in [16559] from fenton@bluepopcorn.net:
  Added Closing Notes to history and about pages for groups. Fixes issue
  #2725.

• Merged in [16558] from rjsparks@nostrum.com:
  Made removed IPR disclosures visually distinctive on search results.
  Partially addresses #2687.

• Merged in [16557] from pusateri@bangj.com:
  Added the ability to upload agenda and minutes in markdown format.
  Partially addresses issue #1926. We're still displaying markdown as text.
  Fixes #2704.

• Fixed a number of additional Py2/3 compatibility issues.

• Added LookupError to the exceptions caught during text upload decoding,
  leading to upload form validation error.

• Merged in [16554] from rjsparks@nostrum.com:
  Rework how SlideSuggestion objects name their files. Improve the UI
  when there are multiple pending suggestions for updating the same
  presentation.

• Moved djangobwr into the local repository, due to consistent
  difficulties pulling it from github as an svn:external

• Enabled display of group pages also for groups of type adhoc and ise.

• Updated the PLAN to adjust priority on modeling BCPs correctly

• Added another check before building submitter email for mailtrigger
  entries, to ensure valid email addresses.

Coverage

Release Date: Thu, July 18, 2019 at 8:34 PM UTC
Release Author: Henrik Levkowetz

This bugfix release contains a number of fixes for issues with the big
Python 3 conversion release, many of which has already been patched in
production. Additionally, there are some fixes related to new slide upload
functionality and the test suite. From the commit log:

• Fixed some issues with the Py2/3 changes to the populate_yang_model_dirs
  management command.

• Added a check for submitter found before trying to get submitter name and
  email when generating email notification address lists.

• Shortened generated document titles a tad, to avoid line wrapping in
  the IESG agenda package (which sometimes breaks the relevant tests).

• Merged in [16515] from rjsparks@nostrum.com:
  Save files correctly when updates to an existing set of slides are
  proposed. Fixes #2748.

• Added information about versions (Datatracker, Python, and Django) at
  the beginning of test suite runs, to be able to clearly see which Python
  and Django versions were used during a test run.

• Replaced six.ensure_str with django's force_str, in order to be able
  to use earlier versions of six which more frequently are available on
  the development and production target platforms.

• Changed PDF page-counting code to read PDF files as binary.

• Replaced the email-sending logger.error() on bad Submission.submitter
  email address with syslog logging.

• Added an email gather handler for submission_submitter that tries an Alias
  name lookuup if the submitter doesn't look as if it contains an email
  address. This deals better with submission records that only has a name
  in the submitter field.

• Changed import path for settings_local.py to work with absolute_import
  under Py2, and also under Py3.

• Changed agenda file reading to read files as binary when looking for
  draft names listed in the agenda, in order to avoid issues with
  older uploads using latin-1 or other encodings where the lower
  127 character match ASCII.

Coverage

Summary: Python 3 compatible code
Release Date: Tue, July 16, 2019 at 10:21 PM UTC
Release Author: Henrik Levkowetz

This is a major release which contains conversion of the datatracker
Python code to run under Python-3.7 where it previously was specific
to Python-2.7. The code was first converted to Python-3.7, and then
additional work was done to make it compatible with both Python 2
and 3. The changes updated 458 files, and touched close to 4000
lines.

Making the code compatible with both Python 2 and 3 lets us continue
to use the converted codebase until we're ready to move the primary
webserver's mod_wsgi completely to Python 3. Otherwise, we would have
to merge all new bugfixes and sprint contributions to both trunk
(Python 2.7) and the new Python 3 branch, with doubled merge work
until we were ready to move the server to Python 3. (Testing, though,
will still need to be done under both 2 and 3 to ensure we don't loose
compatibility.)

Coverage

Release Date: Tue, July 16, 2019 at 2:14 PM UTC
Release Author: Henrik Levkowetz

This is a small bugfix release, to clear the slate before merging
in the Python 2/3 conversion. From the commit log:

• Added a validation step for SearchablePersonField, to avoid later
  server 500 errors on bad input.

• Merged in [16404] from rjsparks@nostrum.com:
  Only look for ietf/datatracker-env images when extracting the most
  recent build to tag as latest.

• Merged in [16359] from rcross@amsl.com:
  Fix registration import, use user.person if it exists.

• Merged in [16096] from rcross@amsl.com:
  Update admin permissions. Grant secretariat change permissions on
  dbtemplate so they can update proceedings pages.

• Turned off html autoescape in IPR email templates. This fixes
  inappropriate html escapes that occurred in various IPR-related
  emails.

• Added missing code to skip coverage measurement for skippable tests.

Coverage

Release Date: Thu, July 4, 2019 at 4:07 PM UTC
Release Author: Henrik Levkowetz

• Increased the length of the list Subscribed email field from 64 to 128,
  updated the import_mailman_listinfo management command, and added a
  migration for the model change.

Coverage

Release Date: Wed, July 3, 2019 at 8:27 PM UTC
Release Author: Henrik Levkowetz

This is a minor release that fixes some bugs and tweaks some settings.
From the commit log:

• Changed some tests to match production group type changes for some special
  groups.

• Made the bin/daily cron script quiteter.

• Various changes to make yang-related management commands quieter

• Added a fix for an issue in get_meeting_registration_data() which could
  happen if we tried to create a Person record for with a user of an
  existing Person.

• Changed the permitted length of patent_title from 127 to 255.

• Added additional guards against duplicate m2m entries.

• Added a utility to check copyright statements in specified files.

• Updated some functions and views in secr/srec/ to use GroupFeatures
  instead of hardcoded lists of group types.

• Made session requests also work for ad-hoc groups, which may not have a
  parent area.

Coverage

Release Date: Thu, June 20, 2019 at 1:16 PM UTC
Release Author: Henrik Levkowetz

This is a bugfix release that cleans up some remaining issues from the
Document/DocAlias primary key refactoring, and also fixes some other
unrelated issues. From the commit log:

• Fixed bad logic for the display of previous related IPR information in
  the IPR details template.

• Tweaked some test failure messages for better debugging.

• Merged in [16287] from rjsparks@nostrum.com:
  Send mail to a list when a new IETF WG -00 is submitted. Fixes #2730.

• Fixed some name versus pk issues lingering after the Document/DocAlias
  primary key refactoring. Fixes issue #2729.

• Added the option to order API resource lists by primary key, for stable
  ordering when paging through a list.

• Added generation of Meta.ordering for API ModelResources in the

• mkresources management command.

• The Document and DocAlias primary key refactoring caused the Tastypie
  API under /api/v1/ to change -- reverting that to use the old form, with
  name ask API key.

• Added a better error message for too long draft names during submission.

• Adjusted the test_runner to generate migration coverage data also when
  permitting mixed migrations.

• Added a guard against duplicate insertions in
  SearchRule.name_contains_index

• Fixed a docalias problem with generate-draft-aliases

• Merged in [16248] from rjsparks@nostrum.com:
  Removed the IETF chair as a recipient of new IPR disclosures if there
  is no group or active document to send them to. Fixes #2727.

• Added a management command to update the index table used by some
  community list search rules. Running this fixes the missing listings
  of drafts related to a group on group document pages and in searches.

• Improved the support for the mixed-release switch in the bin/mkrelease
  script.

Coverage

Summary: Refactoring of the primary index type for Document and DocAlias objects
Release Date: Sun, June 16, 2019 at 2:16 PM UTC
Release Author: Henrik Levkowetz

This release contains a series of 216 migration steps that changes the
Document and DocAlias primary keys from character strings to integers, and
makes corresponding code changes.

This was prompted by database limitations discovered when trying to make
DocAlias use a m2m document field; with 255 long strings as primary keys for
Document and DocAlias this violated the MySQL database limitations.

Changing the primary keys to integers should also improve efficiency.

Additionally, in order to make it possible to have DocAlias objects referring
to multiple documents, in order to reflect STD and BCP names that refer to
multiple RFCs, DocAlias has been refactored to use many-to-many tables instead
of foreign keys do toduments.

Due to the data migrations which create the new integer primary keys and adds
corresponding integer foreign keys matching the previous string foreign keys
in all tables having foreign keys to Document and DocAlias, some of these
migrations take a long time. The total set of migrations are expected to have
a runtime on the order of 2 hours.

This release is scheduled to be deployed on 16 Jun 2019 at 16:00 UTC, with
an expected datatracker downtime, for safety backup and application of the
migrations to run, of about 3 hours.

Coverage