Random insight of the night: every couple years, someone stands up and bemoans the fact that programming is still primarily done through the medium of text. And surely with all the power of modern graphical systems there must be a better way. But consider:

* the most powerful tool we have as humans for handling abstract concepts is language
* our brains have several hundred millenia of optimizations for processing language
* we have about 5 millenia of experimenting with ways to represent language outside our heads, using media (paper, parchment, clay, cave walls) that don't prejudice any particular form of representation at least in two dimensions
* the most wildly successful and enduring scheme we have stuck with over all that time is linear strings of symbols. Which is text.

So it is no great surprise that text is well adapted to our latest adventure in encoding and manipulating abstract concepts.

@rafial Both accurate and also misses the fact that Excel is REGULARLY misused for scientific calculations and near-programming level things since its GUI is so intuitive for doing math on things.

Like, GUI programming is HERE, we just don't want to admit it due to how embarrassing it is.

@Canageek very good point. Excel is actually the most widely used programming environment by far.

@rafial Now what we need to do is make a cheap, easy to use version of it that is designed for what scientists are using it for it. Column labels, semantic labels, faster calculations, better dealing with mid-sized data (tens of thousands of data point range), etc

@Canageek I'm wondering, given your professional leanings if you can comment on the use of "notebook" style programming systems such as Jupyter and of course Mathematica. Do you have experience with those? And if so how do they address those needs?

@urusan @rafial Whomever wrote this is an idiot that has never worked in actual physical sciences where you have actual physical experiments, just FYI. Like 10% of my labs works involves computers at all. But sure, science now moves at the speed of software.

Follow

@Canageek @rafial The attention grabbing headline is pretty stupid, but there's a lot of interesting content in the article. It's especially relevant in the fields I've worked in.

@urusan @rafial It totally ignores the advantages of PDF though, like the fact there are a stack of independent implementations that can view it, which means we will still be able to read these files in 50 years, unlike whatever format they are using, and we can even print them out on paper to edit them (for example, how my boss and I do it, as he doesn't know LaTeX, which is what I write in).

Or the fact that like, 90% of scientists don't know how to program and are unlikely to learn.

@urusan @rafial In my lab right now... I can program a little, one post doc has spent some time doing python tutorials, and I think that is it, out of ten people.

Organic labs might have less then that, as there is a reputation at least that organic chemists and programming are like oil and water: if you are good at one, you are likely terrible at the other.

@Canageek @rafial Well, I don't personally think this kind of dynamic documentation is going to fully replace static documentation. Static webpages haven't been fully replaced by dynamic webpages either.

However, the growing notebook ecosystem does address issues that didn't have well-formed norms around them before.

In addition to the obvious CS/Math applications, there's a lot of areas where complicated statistical work gets done with code, and then how do you publish that?

@urusan @rafial You public the source files as ESI the way we do with input files in crystallography?

@urusan @rafial (I might be giving opinions different then normal due to spending two days trying to get this figure to compile and dealing with moving to lualatex from pdflatex as a result)

@Canageek @rafial Actually, this gets at the heart of what I'm talking about.

Jupyter handles the software dependencies for you as part of the kernel selection.

As these norms become better established over time, the tools to deal with these issues automatically will just be there.

Just dropping some source code on someone is putting the burden of replication on them and not on the platform.

@urusan @rafial Right but then you are stuck with a system 0 people understand instead of 1 person.

If I wanted to use GNUplot as it was built I could use the script a former grad student passed on, but I want it in LaTeX so the fonts and such match.

(Honestly, if I could go into campus it would be done, but the software we have that makes these graphs is trapped on one computer, and not worth me going into campus during a pandemic to get, at least not yet)

Honestly, if you are going to try and replicate any of my work you aren't going to go to my data: You are going to synthesize the compounds yourself and take the measurements on your equipment against your standards so that it doesn't turn out to be some dumb different between how I set up my experiments and how you do, or some defect in my hardware, etc etc.

@Canageek @urusan @rafial > 90% of scientists don't know how to program and are unlikely to learn

Do you think that's stable, or does it really mean "90% of old scientists"?

I probably listen to biased sources, but my impression is that basically any field doing any amount of statistics these days does it in Jupyter.

If that counts as "know how to program", of course.

@clacke @urusan @rafial Well yeah, but most scientists don't do statistics. Most chemists, most biologists, geologists, etc.

Like, there is a reason computational is a subfeild of every discipline.

I think it is going up, due to more stats and computations being used, but I also think we are way to reliant on stats these days and use them instead of getting good data.

@Canageek @clacke @rafial Getting good data is a noble goal, but you'll always have to cope with statistical uncertainty in science.

Even in computer science, where we theoretically control the underlying systems we're studying perfectly, there's often still statistical uncertainty to deal with.

I don't see how that would be any better in the real world where there's uncertainty in measurement.

That said, you're right that you want to get good enough data that your statistics are simple.

@urusan @clacke @rafial Right, I've been frustrated with this in science for decades. We should do half as many studies and put twice or more as much funding into each one so we have actually decent stats.

For example, lately you have to justify the minimum number of rats for ethics committees for any experiment. Fuck that, use 4 times as many so we can be confident in our work instead of justifying it to heck and back.

@urusan @clacke @rafial (This is part of why I picked a field where all the stats were baked into software and validated before my Dad was born, and the sample sizes for everything else is around 10^26 (ie the number of atoms in solution)

@Canageek @clacke @rafial That's kinda crazy.

Either killing rats is totally unethical and we shouldn't be doing it all or it doesn't matter how many you kill. There's no middle ground here.

@urusan @clacke @rafial Nope, its counted as bad, but justifiable so you have to minimize the number you use, at least as I understand it.

Likewise, academic human studies are typically very underfunded, which is why there is such a bias towards small sample sizes and all the participants being undergrads found on campus.

@Canageek @urusan @rafial A Swedish stand-up comedy musician has a routine "Ain't it Weird" about modern life. The very first segment goes:

> Listen, my grandpa was a mason, from Borlänge, Gösta was his name, construction worker, he used to say this:
> "In my day, we made an honor out of building houses as strong as possible, so they would last as long as possible. But now, they have computers that calculate how weak they can build a joint without it falling in on itself."
> *ding*
> Ain't it weird?

Peter Carlsson & Blå Grodorna: "Än'te könstut?" (Dalecarlian Swedish)

He's wrong, but he's not wrong.

youtube.com/watch?v=9s0reaWAjw…

@clacke @urusan @rafial I've heard from engineers that anyone can build a bridge. And engineer can build a bridge that only JUST carries the required load to make it not cost a fortune

Sign in to participate in the conversation
Fosstodon

Fosstodon is an English speaking Mastodon instance that is open to anyone who is interested in technology; particularly free & open source software.