fosstodon.org is one of the many independent Mastodon servers you can use to participate in the fediverse.
Fosstodon is an invite only Mastodon instance that is open to those who are interested in technology; particularly free & open source software. If you wish to join, contact us for an invite.

Administered by:

Server stats:

11K
active users

Yann Büchau :nixos:

@SReyCoyrehourcq @khinsen It's true that the ecosystem moves quickly and one can not rely on an analysis to still work *exactly the same* in a couple of months when executed with the then-latest versions. But that's not just the case for Python! Relying on backwards-compatibility of any software stack is not enough and a very poor choice in the first place. As suggested in the README, using something like :guix: or :nixos: or at least containers is necessary.

@nobodyinperson @khinsen You're right. And honestly, i thing we have the same with R language and with many other languages that depend of some large ecosystem with more or less unstable packages : NodeJs, Rust, etc.
Doing dev work with #nix or #guix is not so easy, and "package yourself mantra" hit some very common limits in many real cases : lack of experience, lack of time, lack of interest, windows dev everywhere.. I'm using guix even if some of my dev tool are not compatible (Java/Scala).

@nobodyinperson @khinsen I'm trying to package Quarto in #guix because i think it's important, but when i see the ten thousand layers (chromium, deno, rust, etc.) behind this very common tool of datascience, and the fact that nobody really care of that in the Data Science World, i'm a little desesperate. We leave the simple Rmarkdown package behind us for this new huge beast.

@SReyCoyrehourcq @nobodyinperson @khinsen

Just a side note

I remember a humongous amount of work was done to bootstrap Java on guix

So there might be a way to run your Java based tool

Maybe you could have an environment with some dependencies and then run a binary in there

Just saying

@calispera @nobodyinperson @khinsen About bootstrap status of different language this website give very good information : bootstrappable.org/projects/ja

But you're right yes, that need some huge works from many peoples during many years. I'm interested by the bootstrapping of Scala (actually not done) but this is out of my current knowledge...

bootstrappable.orgFrom C++ to Java

@SReyCoyrehourcq Bootstrapping is a problem that hasn't been recognized as relevant in scientific computing yet. It's mostly cybersecurity specialists who care about it. I hope this will change.

The most frustrating situation is #Rust, in my opinion. There's a recent language that claims security as its reason for existence, and yet doesn't care much about bootstrapping.

@calispera @nobodyinperson

@khinsen @nobodyinperson @calispera @SReyCoyrehourcq

Well, bootstrapping #Haskell is also something. Blog post from Haskeller pov <joachim-breitner.de/blog/802-M>.

Pandoc seems required for almost 1400 Guix packages. For example r-bionet "Functional analysis of biological networks".

www.joachim-breitner.deMore thoughts on a bootstrappable GHC – Blog – Joachim Breitner's Homepage

@nobodyinperson I see two distinct situations: (1) archiving a computation for re-running it identically later. #Guix and #Nix are the right tools for that. (2) maintaining a tool in usable state over many years. That requires either a stable foundation or a lot of work.

ActivePapers is special because it is infrastructure for archivable computations. Even if I update it with every Python release, I'd force users to do the same with their code, which makes no sense.

@SReyCoyrehourcq

@khinsen @SReyCoyrehourcq Interesting, I didn't have that in mind, thanks!

@nobodyinperson In a resilient world, each software layer would try to absorb such issues and protect their dependees from them. ResilientPython (vaporware!) would adapt itself to new CPUs and offer an unchanged API for Python code. People could just update ResilientPython and continue to run their code.

Python reality is the opposite: it adds to the lower-level seismic chocs causing software collapse.

@SReyCoyrehourcq

@khinsen @nobodyinperson @SReyCoyrehourcq Yes, for instance one has to rely on archive.debian.org for old distros in the #Debian world and keep in mind that all archives certificates are expired for old releases, so an Acquire::Check-Valid-Until "false" is mandatory for apt. That would cause ansible/docker/whatever failures easily.

@nobodyinperson @SReyCoyrehourcq @khinsen I have code (including data analytics code) in C, Common Lisp, and scheme that runs exactly the same after 30+ years, Clojure after 15 years, R after 20+ years. The Python ecosystem is almost uniquely awful for backwards compatibility.

@jack @nobodyinperson @SReyCoyrehourcq @khinsen This is the main reason I avoided using #Python for a production code almost 20 years ago.
The other main reason was the availability at the time of MS-SQL DBD driver in #Perl (well it is a Sybase one, but it is perfectly the same).
People who use all that code base today still thank gosh.

@gisgeek @jack @nobodyinperson @SReyCoyrehourcq @khinsen about a year ago I installed Intel's Python (an otherwise wonderful piece of software) over the system Python and broke the bash shell app on Ubuntu. While this falls squarely under the FAFO category, it (along with the horrible performance of Pandas) was the turning point in my walking away from Python. None of my Python 2.x code runs (obviously)

@ChristosArgyrop @jack @nobodyinperson @SReyCoyrehourcq @khinsen

" There's something very important I forgot to tell you! Don't cross the streams… It would be bad…
Try to imagine all life as you know it stopping instantaneously and every molecule in your body exploding at the speed of light."

- Egon Spengler

Neve, never, never cross system Python and other stuff. 😎

@gisgeek @jack @nobodyinperson @SReyCoyrehourcq @khinsen Doing so is the stuff FAFO is made of (though I have done so with perl and lived go tell the story). Intel's Python now sits unused in a container

@gisgeek @jack @nobodyinperson @SReyCoyrehourcq @khinsen By the way if you do any serious numerical or hpc related work with Python, use Intel's version (inside a container or by changing the shebang at the top)

@ChristosArgyrop @jack @nobodyinperson @SReyCoyrehourcq @khinsen
Yep, in those cases the only good way is using a special execution environment and eventually a whole container, but if one has even to use non trivial extensions in other languages the task could easily become challenging. A lot of scholars nowadays have little confidence in the building of C/C++ libraries and mixed languages with tons of dependencies.

@gisgeek @jack @nobodyinperson @SReyCoyrehourcq @khinsen Second and third order dependencies are what is killing scientists now (certain R and Python packages come to mind). But the R community exercised ruthless control over what gets into and what remains at CRAN and Bioconductor, so things near always work. Python is everything goes until it breaks right now.