Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update all dependencies #5382

Merged
merged 11 commits into from Feb 2, 2023
Merged

Conversation

escattone
Copy link
Contributor

@escattone escattone commented Jan 28, 2023

mozilla/sumo#1181

Summary

This PR provides an almost complete update of everything in the project with the exception of our node packages, which will be done in a separate PR:

  • Python (3.10 --> 3.11)
  • The foundational Python image used in Dockerfile (3.10-bullseye --> 3.11-bullseye)
  • Images versions within .circleci/config.yml
  • Python and other versions within .pre-commit-config.yaml that match pyproject.toml
  • OS and Python versions in .readthedocs.yml
  • All poetry dependencies, including dev group dependencies, to their latest versions, with the following exceptions and notes:
    • pip was updated in Dockerfile to its latest version, but removed as a poetry dependency, since it's already installed via the Dockerfile.
    • elasticsearch was updated to the latest version 7 (7.17.8 ). The upgrade to version 8 remains as a separate task that requires a matching upgrade to our ES server as well.
    • black was updated to 22.12.0, but not to its latest version 23.1.0, because version 23.1.0 introduces some formatting changes that will modify a significant number of our Python files. I thought it best to update black to version 23.1.0 as a separate PR after this PR is merged.
    • The "target version" of black was updated to the newly-supported py311.
    • Sphinx was updated to 5.3.0, not to the latest version 6.1.3, only because the sphinx-rtd-theme package does not yet support version 6 or above.
    • The latest version of django-ratelimit included a change to its import name from ratelimit to django_ratelimit.
    • The latest version of the django-timezone-field package no longer uses pytz.timezone for its TimeZoneField, but instead uses the built-in zoneinfo.ZoneInfo timezone. This meant a few minor changes, but fortunately does not require a database data migration for the Profile.timezone field, since the supported time zone strings are exactly the same.
    • pytz was completely removed. With packages like django-timezone-field and Django moving away from pytz, as well as the fact that Django version 5 will no longer support pytz time zones at all, I decided it was time to remove it altogether. After lots of reading and confusion, it was pretty easy in the end. I followed the migration guide provided by the creator of Python's built-in zoneinfo module, which boiled down to two things:
      • Replace pytz.timezone instances with zoneinfo.ZoneInfo instances
      • Replace pytz.timezone(...).localize(datetime_instance) calls with datetime_instance.replace(tzinfo=ZoneInfo(...)) calls. The new replace method on datetime.datetime instances includes a new fold keyword parameter which handles the possible "ambiguous" and "imaginary" times within time zones like US/Pacific that include daylight-savings-time (DST) changes (fold defaults to 0, so it makes that choice of how to interpret "ambiguous" and "imaginary" times for you by default).
    • The latest version of the django.utils.timezone.make_aware function no longer supports is_dst, so I removed it from the one place it was used. It's no longer needed due to the fact that "under the hood", Django uses the datetime.datetime.replace method which includes the new fold parameter mentioned above.
    • The latest bleach package includes two breaking changes:
      • Its ALLOWED_TAGS is now a frozenset instead of a list. In fact, the bleach.clean function prefers a set now instead of a list for its tags keyword argument, but you can still pass-in a list and bleach.clean will convert it to a set for you, so no worries there.
      • The bleach.clean function no longer accepts a styles keyword argument, which has been replaced by the css_sanitizer argument instead. In the end, for our purposes, it boils down to replacing styles=styles with css_sanitizer=CSSSanitizer(allowed_css_properties=styles) in the keyword arguments when calling bleach.clean.

TODO in Separate PR's

  • Update black to version 23.1.0 (and reformat all of our Python files) in a separate PR.
  • Update all of the node packages.
@escattone escattone changed the title update all non-dev deps except bleach & py-wikimarkup Jan 30, 2023
@escattone escattone changed the title update all deps Jan 30, 2023
@escattone escattone force-pushed the update-all-packages branch 4 times, most recently from 6858fc2 to 32d4deb Compare February 1, 2023 21:32
@escattone escattone marked this pull request as ready for review February 2, 2023 00:25
"an"<script>alert()</script>",
"an"<script>alert()</script>",
),
(
"an'<script>alert()</script>",
"an'&lt;script&gt;alert()&lt;/script&gt;",
"an'&amp;lt;script&amp;gt;alert()&amp;lt;/script&amp;gt;",
Copy link
Contributor Author

@escattone escattone Feb 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both of these test changes were due to an improvement made in the latest version of bleach. Our old version of bleach also provided secure, escaped results, but the latest version provides more accurately escaped results for display purposes.

Copy link
Collaborator

@akatsoulas akatsoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! r+

Squash before merging and maybe add a couple of issues for the standing items?
Nice work

@escattone escattone merged commit ba2f95a into mozilla:main Feb 2, 2023
2 checks passed
@escattone escattone deleted the update-all-packages branch February 2, 2023 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants