Recent searches
Search options
Hey #rstats fam: what do we need to have a transparent and developer friendly package system? Posted on r-universe b/c that is what I think is the closest:
@josi for sake of Data Science, will you offer the apologies? r-universe is not an alternative to CRAN!
@job_n no it is not hence the issue. I think CRAN folk have had many opportunities to apologize for their behavior and haven't in the past. How long should you look past bullying "for the sake of data science"?
@josi UNDERSTANDABLE, but .....
@josi not exactly an answer to the discussion, but 3 years ago we had similar discussions and it lead to a working group within the R consortium: https://github.com/RConsortium/r-repositories-wg/ you might be interested on Monday's meeting as @landau and Charlie Gao are going to present how they think r-universe can be used for that
@Lluis_Revilla @josi @landau this looks really interesting. Any chance I could listen in?
@AlexAxthelm @Lluis_Revilla @landau ^ that would be nice. I read the minutes every once and a blue moon but they don't really seem like much happens
@josi Great, I wasn't sure if you knew about it. I'd like to know what are the "external" expectations of such a group: What would you expect to happen? Who should do that and when and how? @AlexAxthelm
@Lluis_Revilla @AlexAxthelm i think at minimum a proposal with steps to make the package submission process 1) clearer and 2) transparent.
From what i can tell based on https://github.com/RConsortium/r-repositories-wg/blob/main/Documents/Proposal%20to%20CRAN.md
there isn't actually anything the folks at CRAN have agreed to.
@josi @AlexAxthelm sadly, I think those expectations are not realistic for such working group 
We are not involved in the CRAN submission process or review team. If you have an idea on how to improve those within those constrains I'm all ears.
However, recently there was a survey about the CRAN submission process (they asked about that). I haven't heard anything about the answers but I hope the CRAN cookbook will take good shape based on that. Help from the community is well received.
@Lluis_Revilla @josi I’d say I’m mostly interested is seeing where the state of the conversation is around package installation from non-CRAN sources, and how to ensure interoperability now that installing from a 3rd party source is pretty easy
@AlexAxthelm @Lluis_Revilla @josi Pretty easy from conda forge, assuming the packages you're looking for exist there, that is. But you'll install things with mamba instead of install.packages().
@AlexAxthelm I am not sure I follow: you can install from a simple .tar.gz or other formats with install.packages or R CMD install. Do you mean that R should also include other repositories by default?
I'll think more about the interoperability part.
@josi
@Lluis_Revilla @josi is guess I used the wrong word there. I think a better word might be tracibility?
That is, in a world where there are multiple valid sources (especially when there are multiple sources for a single package), do we have the right tools to determine where my local copy was installed from.
It would also be really nice if I could have multiple local copies and switch between them easily (dev/release), but that seems like a harder nut to crack.
@AlexAxthelm I think knowing where a package was installed from could be added easily but I would need to look into it. I got into several issues with renv, which main purpose is that: it didn't trace packages well even if using pack. So it might not be as easy as it seems...
To have multiple copies of packages and switch between them install them in separate libraries using .libPaths or the corresponding arg. This is how devtools::dev_mode() works. The drawback is that unloading is not "safe".
@Lluis_Revilla @AlexAxthelm i think thats the problem. 
CRAN as it is today, is an existential threat to the growth of R and i think pretending otherwise is just putting your head in the sand
@josi @Lluis_Revilla @AlexAxthelm 
I’m one of the CRAN cookbook writers! We just reviewed the survey results in a recent team meeting and will take some time to carefully consider how we share the summarized results with the whole community. Our goal is to hopefully improve some things in the process (especially for new maintainers) as we continue writing these common submission issue recipes.
@jasminedaly @Lluis_Revilla @AlexAxthelm thanks Jasmine! My worry is that changes will "blame" the developers and tell them what they need to do better rather than improving CRAN itself.
E.g. in this reply by Uwe here they say that if you follow the CRAN policies everything should be fine. From experience I know that this is not always true.
The policies and ad-hoc checks should be part of the pre-checks and cannot be checked by the developer.
@josi @jasminedaly @Lluis_Revilla I think one of the biggest changes that would benefit the CRAN submission process for developers would be if there were a way to run the CRAN checks *prior to submitting*
There’s a lot of ways that could look, from being able to upload and check without it being a submission (and saying yeah, let’s submit for human review _after_ auto checks come back clean) to a docker image that closely emulates to CRAN environment.
@AlexAxthelm @josi @jasminedaly there are two. One for windows and one for mac: https://win-builder.r-project.org/upload.aspx
https://mac.r-project.org/macbuilder/submit.html
They do not check for usban and other compilers but they cover most checks
@josi I have been "working" on that for several years. It often comes to my mind this adverb: "Alone a youth runs fast, with an elder slow, but together they go far." that's why I often invite people and talk about that working group. Take care! @AlexAxthelm
@AlexAxthelm I am not sure how to handle people interested in it (I tried to "add" them in different ways over the years with no clear guidelines or best practices ). But it is open to everybody interested, so here is the link for this regular meeting: https://zoom.us/j/91379577358 @josi
@Lluis_Revilla @AlexAxthelm nifty! perhaps the schedule can be put on the GitHub page? If i join i promise to be as quiet as a mouse!
@josi I am not the chair of the working group. I don't feel comfortable doing that without consent of the chair and the rest of the regular people.
@AlexAxthelm
@josi I’d also suggest taking a look at https://github.com/r-hub/repos. It’s more similar to the Julia registry style of package hosting.
It currently hosts a package index of packages mirrored from CRAN, but there’s nothing stopping a service from repurposing this machinery for non-CRAN package sources - for example, using a CI job to submit a new package release directly to this style of a repository.
@josi I've been leaning more to conda forge lately as cran has been less reliable in recent months.
@josi I mean, unfriendly opinion is (1) sustained funding (2) install.packages("x") needs to find x in r-universe
I think the design of r-universe precludes (2), which I personally think means it will always, at best, be an alternative option
@josi I fundamentally believe the median R user doesn't understand where they're installing packages from (or that they're downloading packages at all)
Anything that isn't equivalently black-box as a simple "install packages()" is doomed to only be used by the nerds
@josi because this has been seen by more than the 1 or 2 people I expected, let me be extra clear:
I think it's an objectively good thing that most R users don't need to think that much about where packages come from
& as a religious user of rig and pak I'm a big fan of dependency management tools for nerds 
@MikeMahoney218 @josi Can you explain why you think that (2) is impossible by design? I think it could be straightforward if install.packages() supported it. We “just” need to say goodbye to the idea of a flat package namespace.
@MikeMahoney218 @josi Ah, okay. I thought you meant there was something in the design of R-Universe preventing integration into base R.
@klmr @josi I think requiring people to track which universe they're trying to install from isn't a great interface for the average user, especially once you get into dependencies
Also concerned about the possibility for people to accidentally get forked versions, old releases, malicious squatters... all the issues with installing from gh directly rather than a central repo
But the biggest one is that I think people know they want X, and they're less clear on who really "owns" it
@MikeMahoney218 @josi I don’t think having namespaced packages is actually a big problem in practice, even for beginners, and it avoids a host of other problems. What it requires is syntactic support: we need to normalise referring to packages as ns/pkg, as we already do routinely for e.g. GitHub repos:
install.packages('r-lib/pak')
@klmr @MikeMahoney218 @josi I think that if something like R-universe’s namespacing were adopted by CRAN, that would solve a lot of problems in a way that’s familiar to a lot of people.
For example, dockerhub has user/org level namespacing but also a global “official” namespace (that is not perfect but does imply a higher level of trust)
@josi as long as a replacement stays as user (I.e. the non package developers)friendly as CRAN , as opposed to say python packaging