Follow

why does nixos-rebuild switch --upgrade want to download literally _every single package_ i have again? even if there is no version upgrade???

this sort of stuff makes me want to just give up and install ubuntu...

@peanutbutter144 If that’s the case then something changed for sure.

If almost everything is affected then some core package was updated (maybe a security fix in SSL) and now that change must be propagated through the system to make it reproducible.

That’s the trade‑off.

@kmicu hmm, ok. i guess ill leave it updating overnight tonight and see how it goes.

apart from this one issue, ive really liked nixos. but every now and then i wish i had chosen to use a more popular / lower maintenance distro...

@peanutbutter144 if downloading the whole system more or less every month is the deal breaker then I recommend switching to a traditional/mutable distro cuz that ‘we need to re‑download everything almost each month' part will never change in Guix/Nix (or any other reproducible system).

I think those projects should more prominently display and talk about their trade‑offs like much bigger disk usage, a lot more frequent compilation, and much bigger network bandwidth requirement.

@kmicu ok. i knew about the disk usage, thats mentioned somewhere easy to find on nixos.org but i didnt know about the amount of recompiling and network usage until i experienced it myself.

so i agree that nixos and guix should make that more prominent.

if the recompiles and full system re-download are as often as once a month, i might consider intalling a traditional distro instead, thats a bit top often for me i think...

@kmicu @peanutbutter144 I can't help to think https://github.com/NixOS/rfcs/pull/17 would make things better on that front.

It would probably require to drop the 100s of TB of cache we are currently sitting on :(

@Ninjatrappeur @peanutbutter144 that feature improves the situation in a sense that we get a little bit less compilation, downloads, and disk usage but those are still order of magnitude bigger than on a mutable distro.

By design Guix/Nix cannot minimize those resources as efficiently as mutable package managers—they can do this cuz they are allowed to lose information, to be not reproducible, and to brick our systems on an update ;)

@kmicu @peanutbutter144 I do agree about that.

That said, having to rebuild the whole (g)nixpkgs for every bash/systemd/(guix init system?) bump seems pretty bad to me.

What I was trying clumsily to point out was that even by staying reproducible, we have room for improvement in the optimization space :)

@Ninjatrappeur @kmicu it turns out updating my entire system didn't actually take that long, i am back to loving nixos!

@Ninjatrappeur @peanutbutter144 @kmicu

even without an intensional store, couldn't the binary cache servers use intensional hashes?
or maybe they already do that?

@grainloom @kmicu @peanutbutter144 Depending on how the NAR format itself can be backward compatible, it could be a really good point.

I have to think a bit more about that/try out some stuff.

I'll send you a detailed answer later during the weekend.
0/ @grainloom I actually totally forgot this idea and went back to it this afternoon :)

Actually my mental model was bogus. The NAR files (the serialized packages stored to the cache) are actually content-adressed. This is already implemented.

It means my comment about massive re-downloads was irrelevant.

Some side effects could break the icontent-adressed nature of the cached NAR (if the NAR includes some link to some other store paths - which are still input-adressed - for instance), but I guess that is pretty marginal. Actually not so sure, we probably should compute the frequency of this corner case in nixpkgs.

Anyways, thanks for the comment! It indirectly made me correct my mental model about the nix cache address system! For once, things are better than expected, yay to that!


CC @kmicu
@peanutbutter144
@grainloom I just realized something.

A corrolary to this would be: we should refrain ourselves as much as possible to include any store paths in our derivations.

Meaning the use of `subtituteInPlace` and the various "#!${pkgs.bash}` should be considered as dark patterns since it will make all the downstream packages to be not only rebuilt for every dependency changed but also re-downloaded by the clients for a single string change :O
Sign in to participate in the conversation
Fosstodon

Fosstodon is an English speaking Mastodon instance that is open to anyone who is interested in technology; particularly free & open source software.