#Python 3.6 is the new 2.7 in terms of people not being willing to let go.
Python 3.6 is no longer supported upstream, so unless you're paying someone for Python support you are not getting security fixes (and haven't since last year). And if you're thinking of upgrading to 3.7, understand that you're so far behind at this point that 3.7 reaches EOL in about 8 months (June 2023), so you should probably target Python 3.8 at minimum if you're still on 3.6.
@brettcannon 3.6 seems like just a couple years ago and everyone freaking out about how cool f-strings are
@brettcannon We have an Ubuntu box with 3.5 as the system Python 3, migrating to a new box (22.04, 6 months old) that has Python 3.10, so hopefully that can tide us over. Meanwhile, I'm wanting 3.11 toys but not wanting to deal with maintaining my own python packages on them.
@brettcannon PHP: We force developers to update often.
Python: Hold my beer.
@cjcox We update one a year on a schedule. 3.6 (and 3.7) predate that and were on a roughly 18 month schedule, so folks can now plan their updates accordingly.
And I think "force" is a strong word here. We simply cannot support old versions indefinitely; we are a bunch of volunteers trying to keep old releases running for 5 years. If you want to stay on old versions, that's your prerogative, but we also don't need to support you just because you don't to upgrade.
@brettcannon Bad for companies though. If you think about it for a bit, you'll understand that. Your assumption that you're the only software product in the world is a bad assumption.
@cjcox It's not an assumption of being the only software product in the world, it's the fact that we can't support software forever on volunteer effort. Our support policy and release cadence for Python releases is stated upfront, so there's no surprises for anyone.
To be clear, you can update your install of Python every 5 years and still be on supported versions of Python, it's just a question of how much of a leap it will be for you to make that upgrade.
@cjcox Now if you want fewer releases of Python then that's fine and others have asked for that, but the general feedback we have received from the community is they like the cadence and predictability of annual releases
@brettcannon What is a release? Is there Python 3, or is there 3.1, 3.2, 3.3, 3.4? Of course, we've chose the latter. What's wrong? Nothing has to be wrong, but if 3.2 can't be used with 3.1, then there's the rub. And that's what you have today. Just something to think about.
@cjcox If by "can't be used with" as in "can't be used in the same process/app", then that's a price for using an interpreted programming language that changes. To avoid that would require either a compiled language with strong API/ABI guarantees (depending on whether you recompile the world or not) or a much smaller, simpler, interpreted language like a Lisp that's effectively done with its development.
To each their own, but Python has chosen to be developed like this for decades for reasons.
@brettcannon My point is that you can't say, programmed in "Python 3", because it's meaningless. Again, it's just something to think on. I think you're argument to my original point was the word "force" (which I still think is apropos), but I figured you'd be more upset about the comparison to PHP, but appears that would be accurate.
@brettcannon Just when we thought we finished the migration from 2.7 to 3.7 we realised that we’re behind again… Although probably easier to upgrade now, at this point it’s better to start thinking about a process that can constantly keep you up to date with the latest stable version, rather than wait until the alarm goes off
@christosliontos I know plenty of companies that upgrade every two years as they find annual too fast. But yes, people need a cadence, else they forget. But Python is like any other software that needs maintenance; I bet these same folks update their OSs.
@brettcannon @christosliontos one trick I really like is running CI tests against multiple versions, including the most recent release (even if production is running something older)
Provides lots of advance warnings of incompatibilities
I found the way GitHub does that with cutting edge Rails releases really inspiring:
@simon @christosliontos I have a desire/goal to create a GitHub Action that reads your `pyproject.toml` and then calculates all of the released Python versions that your `requires-python` happens to specify. It would let you opt-**out** of running alphas, betas, and/or RCs of CPython to help stay on top of things.
@brettcannon @simon This is great. It ‘kindly’ forces you to be proactive.
“Instead of waiting for the final release, we’d upgrade every week by pulling in the latest changes from Rails master and run all of our tests against that new version”
@brettcannon I think this is the muscular contraction timing problem you see in large corporations and enterprises. Each version jump represents an investment in terms of engineering effort that you have to convince the being counters is necessary. It is incredibly hard to get them to understand that this needs to be an ongoing aspect of life to be handled properly.
@feoh The way to pitch it is if you want to continue to get free bug and security fixes you need to upgrade your Python runtime just like you update your OS
@brettcannon oh, for sure. The problem is that I’ve had a fair bit of experience with bean counters and decision makers who see anything other than new features to drive adoption as very abstract, distant problems, that only need to actually be dealt with when they go critical. Is this an insane way to run a ship? Absolutely. Does it happen? All the time :(.
@feoh @brettcannon The bean counters see the problem as "it worked yesterday, why doesn't it still work today"; they don't understand that in reality it was subtly broken yesterday, and today we _know_ how it was broken.
@pjz @brettcannon that’s absolutely true. They have no sense of the great game of cat and mouse between exploit writers and software Maintainers. Do you think this would’ve gotten better given all the attention and publicity the exploit marketplace has gotten recently, but I guess biz dev types don’t pay attention to that :-)
@feoh @brettcannon @pjz did someone say this about birdsite elsewhere? Or did I just read you saying it in this thread, Paul, and assume that it was about birdsite?
Great suggestion, and in general, software that is unmaintained upstream such as Python 3.6, gets security fixes only in some cases. For example, the Ubuntu security teams continuously fix the security on Python 3.6 without changing the minor version. http://changelogs.ubuntu.com/changelogs/pool/main/p/python3.6/python3.6_3.6.9-1~18.04ubuntu1.8/changelog
@brettcannon thankfully that upgrade to 3.7 and 3.8 and 3.9 are practically painless.
@webology For pure Python code I would hope that is true in general as long as you pay attention to the deprecation warnings. Obviously things are trickier with C-based stuff simply because we don't have the same deprecation mechanisms available to us. We try our best, though.
@brettcannon I think tools like pyupgrade have a pretty good hit rate to fix a bunch of annoyances too.
I have lucked out on C-based projects and have mostly been able to avoid non-mainstream projects. Normally psycopg2 is about it, and they are well supported.
@brettcannon @chrisjrn I suspect Python 3.6 usage will self retire quicker than 2.7 did :-)
Mostly at present I’m seeing Python 3.6 because it’s packaged for CentOS 7 (ends support June 2024), and Ubuntu 18.04 (ends support ~April 2023). So just OS release upgrades should get people onto 3.8+ “soon”.
(But Python support durations and OS support durations being mismatched will be an ongoing challenge. Possibly mitigated by “no Python in default install” choices.)
@ewenmcneill @brettcannon @chrisjrn I agree totally just yesterday there was a post on our project's forum asking for python 2.7 support (on an unsupported version of the software of course), I have yet to see someone asking for old python 3 support.
@brettcannon learned this the hard way today trying to update python in a xenial docker image
@brettcannon Question, are there big (breaking) changes for minor revs? If not, why are distros so slow to update?
@akulbe Depends on your definition of "big". #Python predates #semver , so that minor part of Python's version number is more like a release count than saying, "only features".
You can look at https://docs.python.org/3/whatsnew/3.11.html#porting-to-python-3-11 for a list of some things that you would have to update in Python 3.11 which from a C level was somewhat "big".
@brettcannon I can understand 2.x to 3.x being hard but is 3.6 to 3.8 that different? Or is it more about the logistics of updating a fleet? I haven’t kept up.
@richburroughs @brettcannon we’re going through a big 3.6 to 3.9 migration currently and it’s not too bad. Mostly logistics for us.
@sethmlarson @brettcannon Maybe remove the upper bound <3 at https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html#migrating-as-a-package-maintainer ?
See @henryiii's https://iscinumpy.dev/post/bound-version-constraints/
@brettcannon @simon interestingly and sadly, our prod env will be in 3.5/3.6 for a long time, even though the sec ops have been mentioned this a couple of times but the devs simply don’t want to upgrade them.
@kenlai @brettcannon do the devs know that 3.11 has a 20% performance increase? I wonder if that might tempt them
@simon @brettcannon I don’t think the reason is tech-related since we’re having a big re-org now. The devs are afraid of changing anything since it may backfire.
@kenlai @brettcannon sounds like they might be operating without a comprehensive automated test suite (I've been there! Upgrades are a nightmare)
@simon @brettcannon btw, the network automation that I’m leading is fully on 3.11 now
@brettcannon @simon @kenlai We're used to #Python 2.7 being at 50% and all 3.x releases being some portion of the remaining 50% (usually 10-15% each). Now with 2.7 mostly gone I think we're seeing more healthy upgrade behavior across 3.x versions. From #urllib3's stats across all downloads:
- 12% 3.6
- 30% 3.7
- 22% 3.8
- 12% 3.9
- 7% 3.10
- 2% 3.11
Anecdotally from talking to others I believe multi-hop (eg 3.6->3.9) upgrades are being much more common, too!
@sethmlarson @brettcannon @simon @kenlai
Nicely graded #Python waves flowing in here:
@sethmlarson @brettcannon @simon @kenlai I assume this being related to the default Python versions being installed in the various Operating Systems.
@venthur @sethmlarson @simon @kenlai Probably which plays out with CI because that is not at all the distribution I've seen from other telemetry
@brettcannon I keep periodically going back to this release cycle chart to check on where I stand with different projects using various versions of Python.
https://python-release-cycle.glitch.me
@brettcannon and if you really care for your Self, you should move to Python 3.11.
@brettcannon - Changes in Python, The Language, are an argument for never using Python.
@julie78787 I'm sorry you feel that way, but can respect your decision to use a different programming language.
@brettcannon why not 3.10 at minimum?
@jeremyfee I've been around long enough to know there's no way that will take off in the community.
@brettcannon not suggesting to drop support for 3.8 and rather at a least support the latest release
@jeremyfee I can't remember the last time I hit a project that didn't support 3.10, and I was surprised when I hit one that didn't support Python 3.11 the other day. The community has done a great job in getting 3.11 support out!
@brettcannon I'm old enough to remember the 1.5.2 dead-enders.