Recent searches
Search options
Python web devs, what's your approach to package versioning in requirements.txt? To pin or not to pin?
@pamelafox pinned everything to == precise numbers a few months ago. Now using Dependabot with GitHub Actions to bring them all up to date. A colleague today merged a couple of dozen pull requests including two exclusions for specific versions which are not compatible with the rest (scientific packages for our data processing backend jobs). I expect to sleep more soundly going forwards...
@graham_knapp oh can dependabot do that? i only knew it as the "omg your repos on fire" alerter. Is there a workflow file for that?
@pamelafox It's in the GitHub repo settings, though we've set up a dedicated branch for it with an action to auto-merge from develp
@pamelafox I was a poetry sceptic but I've become a convert, I always appreciated the simplicity of requirements.txt but eventually had to admit it was time to switch to something more sophisticated.
@pamelafox PDM is great, I've stopped using requirements.txt. However, when I last did I used pip tools to have a requirements.in (unpinned) turned into a pinned requirements.txt.
@pamelafox Pin so you don't accidentally install newly released version that's buggy and incompatible with your codebase. Use something like dependabot or pyup.io that will create PR for updating version so it can go through CI first.
@mariatta Thanks! Going to try that approach with dependabot now.
@pamelafox I use pip-tools and pin the version number in my requirements.in file such that I get the latest bug fix version (eg requests==2.28.*). When I run pip-compile, it pins the latest bug fix version. It also depends on the library though. If I am only using the basic features (eg requests) there are probably not going to be too many breaking changes going from 2.28 to 2.35 etc. in which case set it to 2.*.*
@pamelafox if you don’t pin versions then are you actually versioning anything?
@pamelafox for projects; pip-tools or pdm (some team members don’t like pdm, so pip-tools is a good compromise). Always pinned. I usually set upper bounds in pyproject/requirements.in for semver to minor (eg Django<4.2.0), to avoid potential breaking changes, but any patches can be easily re-pinned for.
For shared packages; hatch for packaging, no pinning, minimal constraints- if anything, lower bounds only eg “Django>=4.2.0”. No upper bounds unless absolutely needed.
@pamelafox when given the choice between 2 things in Python I usually choose what Python Bytes podcast told me to choose : I call it "Okken's Razor" @brianokken
@pamelafox I tend to use Poetry these days, with versions in pyproject.toml restricted to the latest major. I’ve got GitHub alerts on security vulnerabilities and once a month I do update day to handle normal updates.
@pamelafox Broad rule of thumb:
- no pinning for libraries unless there are known incompatibilities. But then, requirements are in pyproject.toml anyway.
- pin for applications, but use dependabot or something to test updates.
- for both, try to keep updated and test
- you’re not testing enough, probably, test more.
@brianokken thank you! I'm going through all my repos and pinning/dependabot-ing today. For the repos where I have full test coverage, I'll add anthony's dependabot-bot to auto-merge.
@brianokken so lets say i get to 100% coverage on my pytest tests. what kind of testing do you like to do once an app is deployed to staging? selenium/playwright?
@pamelafox I’ll let you know when I get that far. All of my web apps are side projects. 
@pamelafox @brianokken I would usually go for a simpler end to end testing before selenium. Depending what your system is this could just be another pyest suite (for example, maybe you have an API to test?). If your front end is very JavaScript heavy then maybe something like selenium could help.
I don’t have experience with playwright but I’d like to try it