Fosstodon

Snakemake Release RobotBeep, Beep - I am your friendly <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> release announcement bot.There is a new release of the Snakemake executor for <a href="https://fediscience.org/tags/SLURM" class="mention hashtag" rel="nofollow noopener" target="_blank">#SLURM</a> on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#HPC</a> systems. Its version now is 1.6.0!Give us some time, and you will automatically find the plugin on <a href="https://fediscience.org/tags/Bioconda" class="mention hashtag" rel="nofollow noopener" target="_blank">#Bioconda</a> and <a href="https://fediscience.org/tags/Pypi" class="mention hashtag" rel="nofollow noopener" target="_blank">#Pypi</a>.If you want to discuss the release, you will find the maintainers here on Mastodon! <a href="https://fediscience.org/@rupdecat" class="u-url mention" rel="nofollow noopener" target="_blank">@rupdecat</a> and <a href="https://fosstodon.org/@johanneskoester" class="u-url mention" rel="nofollow noopener" target="_blank">@johanneskoester</a>If you discover any issues, please report them on <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues</a>.See <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v1.6.0" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v1.6.0</a> for details. Here is the header of the changelog:𝑅𝑒𝑙𝑒𝑎𝑠𝑒 𝑁𝑜𝑡𝑒𝑠 (𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑦 𝑎𝑏𝑏𝑟𝑖𝑔𝑒𝑑): 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐬* added github action to label long pending issues as 'stale' (<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/239" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/239</a>) ([6d7c50a](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/6d7c50ae492c9557b6cc39119c572c5b5ef1b341" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/6d7c50ae492c9557b6cc39119c572c5b5ef1b341</a>)) * treat sbatch errors as job errors instead of workflow errors (<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/322" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/322</a>) ([5e38507](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/5e3850762c73abf1fff8ba9e1e8c18379251041f" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/5e3850762c73abf1fff8ba9e1e8c18379251041f</a>)) * using the current version of the announcement bot for Mastodon (<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/333" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/333</a>) ([03e0e24](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/03e0e2426edbe2957be93f02f3ae3834f710faff" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/03e0e2426edbe2957be93f02f3ae3834f710faff</a>))𝐁𝐮𝐠 𝐅𝐢𝐱𝐞𝐬* allow unse...

Charles Soeder<a href="https://mstdn.science/tags/snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#snakemake</a> suddenly started objecting to this idiom at random for some reason? wtf

Johannes KösterThere have been 7 new <a href="https://fosstodon.org/tags/Snakemake" class="mention hashtag" rel="tag">#Snakemake</a> releases since my last post in May. The most important new feature is --omit-flags, which allows you to deactivate individual input/output flags like "pipe" for the entire workflow (e.g. for debugging). Apart from that, many bugs have been fixed.<a href="https://snakemake.github.io" target="_blank" rel="nofollow noopener" translate="no">https://snakemake.github.io</a>

Eli Roberson (he/him)In recent years, we pivoted away from bare metal installs and manually analyzing data files. Instead, we mostly use <a href="https://genomic.social/tags/Docker" class="mention hashtag" rel="nofollow noopener" target="_blank">#Docker</a> containers and <a href="https://genomic.social/tags/SnakeMake" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakeMake</a> workflow management.Our cluster runs containers. It's easy. Locally we use singularity. Also easy. But local servers use singularity defined in the rule. On cluster, a separate yaml is required to specify images...Which means I've scripted scanning the SnakeMake files, pulling image and rule names, and writing the yaml automagically.

Christian MeestersThe <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#isc25</a> is over and I half-recovered from the weekend, too. Time to continue my thread summing up the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> !To me, an important contribution was from Michael Jahn from the Charpentier Lab: A complete re-design of the workflow catalogue. Have a look: <a href="https://snakemake.github.io/snakemake-workflow-catalog/" rel="nofollow noopener" translate="no" target="_blank">https://snakemake.github.io/snakemake-workflow-catalog/</a> - findability of ready-to-use workflows has greatly improved! Also, the description on how to contribute is now easy to find.A detailed description has been published in the <a href="https://fediscience.org/tags/researchequals" class="mention hashtag" rel="nofollow noopener" target="_blank">#researchequals</a> collection <a href="https://www.researchequals.com/collections/hm1w-cg" rel="nofollow noopener" translate="no" target="_blank">https://www.researchequals.com/collections/hm1w-cg</a> under <a href="https://doi.org/10.5281/zenodo.15574642" rel="nofollow noopener" translate="no" target="_blank">https://doi.org/10.5281/zenodo.15574642</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleResearch</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Snakemake Release RobotBeep, Beep - I am your friendly <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> release announcement bot.There is a new release of the Snakemake executor for <a href="https://fediscience.org/tags/SLURM" class="mention hashtag" rel="nofollow noopener" target="_blank">#SLURM</a> on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#HPC</a> systems. Its version now is 1.4.0!Give us some time, and you will automatically find the plugin on <a href="https://fediscience.org/tags/Bioconda" class="mention hashtag" rel="nofollow noopener" target="_blank">#Bioconda</a> and <a href="https://fediscience.org/tags/Pypi" class="mention hashtag" rel="nofollow noopener" target="_blank">#Pypi</a>.If you want to discuss the release, you will find the maintainers here on Mastodon! <a href="https://fediscience.org/@rupdecat" class="u-url mention" rel="nofollow noopener" target="_blank">@rupdecat</a> and <a href="https://fosstodon.org/@johanneskoester" class="u-url mention" rel="nofollow noopener" target="_blank">@johanneskoester</a>If you discover any issues, please report them on <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues</a>.See <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v1.4.0" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v1.4.0</a> for details. Here is the header of the changelog:Release Notes (possibly abbriged): ### Features* adding reservation flag ([#323](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/323" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/323</a>)) ([d4e0a0f](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/d4e0a0f4160f5336c5bf36be8f097b8a01f77718" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/d4e0a0f4160f5336c5bf36be8f097b8a01f77718</a>))### Documentation* review and edit new docs ([#237](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/237" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/issues/237</a>)) ([ec82a70](<a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/ec82a70b95652afcbc21d44a4dbca14d78dc8936" rel="nofollow noopener" translate="no" target="_blank">https://github.com/snakemake/snakemake-executor-plugin-slurm/commit/ec82a70b95652afcbc21d44a4dbca14d78dc8936</a>))

Christian MeestersReturning from the <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#isc25</a> I will continue this thread with something applicable everywhere, not just on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#HPC</a> clusters:Workflow runs can crash. There are a number of possible reasons. Snakemake offers a `--rerun-incomple` flag (or short `--ri`) which lets a user resume a workflow.This contribution from Filipe G. Viera describes a small fix to stabilize the feature. Not only will incomplete files be removed after a crash, now it is ensured that all metadata with them are deleted too, before resuming: <a href="https://zenodo.org/records/15490098" rel="nofollow noopener" translate="no" target="_blank">https://zenodo.org/records/15490098</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian MeestersToday tooting from the <a href="https://fediscience.org/tags/ISC25" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISC25</a> - the International Supercomputing Conference. What better opportunity to brag about something I've done to facilitate using GPUs with Snakemake?Here is my contribution, simpler job configuration for GPU jobs:<a href="https://doi.org/10.5281/zenodo.15551797" rel="nofollow noopener" translate="no" target="_blank">https://doi.org/10.5281/zenodo.15551797</a>Not alone though: Without valuable input of <a href="https://mstdn.science/@dryak" class="u-url mention" rel="nofollow noopener" target="_blank">@dryak</a> . Without him, I would have overlooked something crucial.And when we talk about reproducible AI, my take is that we ought to consider workflow managers, too. Something which protocols what you have done with little effort.<a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian MeestersThis morning, I am travelling to the <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#isc25</a> and hit a minor bug on <a href="https://fediscience.org/tags/researchequals" class="mention hashtag" rel="nofollow noopener" target="_blank">#researchequals</a>. Hence, no updates in the collection.But still a few to describe without adding the latest contributions:For instance, this one (<a href="https://zenodo.org/records/15490064" rel="nofollow noopener" translate="no" target="_blank">https://zenodo.org/records/15490064</a>) by Filipe G. Vieira: a helper function to extract checksums from files to compare with checksums Snakemake was already able to calculate. Really handy!<a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian MeestersBefore I continue uploading - and I do have a couple of more contributions to add to the <a href="https://fediscience.org/tags/ResearchEquals" class="mention hashtag" rel="nofollow noopener" target="_blank">#ResearchEquals</a> collection - first another contribution by Johanna Elena Schmitz and Jens Zentgraf made at the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> One difficulty when dealing with a different scientific question: Do I need to re-invent the wheel (read: write a Workflow from scratch?) just to address my slightly different question?Snakemake already allowed to incorporate "alien" workflows, even <a href="https://fediscience.org/tags/Nextflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#Nextflow</a> workflows, into desired workflows. The new contribution allows for a more dynamic contribution - with very little changes.Check it out: <a href="https://zenodo.org/records/15489694" rel="nofollow noopener" translate="no" target="_blank">https://zenodo.org/records/15489694</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian Meesters<a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> in a <a href="https://fediscience.org/tags/Tmux" class="mention hashtag" rel="nofollow noopener" target="_blank">#Tmux</a> session on our <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#HPC</a> cluster. It is time to start into the weekend! All I have to do is detach the session and leave.

Christian MeestersLet's take a look at another contribution of Johanna Elena Schmitz and Jens Zentgraf from the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> Snakemake users probably know that `$ snakemake [args] --report`will generate a self-contained HTML report. Including all plots and <a href="https://fediscience.org/tags/metadata" class="mention hashtag" rel="nofollow noopener" target="_blank">#metadata</a> a researcher's heart longs for.Now, why trigger this manually? If the workflow runs successfully, now we can write (or configure):`$ snakemake [args] --report-after-run`and Snakemake will autogenerate the same report.For details see <a href="https://doi.org/10.5281/zenodo.15489764" rel="nofollow noopener" translate="no" target="_blank">https://doi.org/10.5281/zenodo.15489764</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian MeestersOne important feature implemented in the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> : Snakemake will calculate file checksums to detect changes. If a file changes, the rule producing it needs to be re-executed when a workflow it re-triggered. But what if a file is too big for reasonable checksum calculation? You do not what to wait forever, after all.This contribution describes the implementation of a threshold users may set: <a href="https://doi.org/10.5281/zenodo.15489401" rel="nofollow noopener" translate="no" target="_blank">https://doi.org/10.5281/zenodo.15489401</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Christian MeestersOne important bug fix during the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> : the config replacement. Now, users can overwrite existing configurations entirely with `--replace-workflow-config`.Details: <a href="https://zenodo.org/records/15479268" rel="nofollow noopener" translate="no" target="_blank">https://zenodo.org/records/15479268</a>More at <a href="https://www.researchequals.com/collections/hm1w-cg/" rel="nofollow noopener" translate="no" target="_blank">https://www.researchequals.com/collections/hm1w-cg/</a><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/openscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#openscience</a>

Jan van der LaanWe have defined a workflow in <a href="https://datasci.social/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a>. As the input data format has changed in latest years we had to rewrite the first script in the workflow. However, this triggers a rerun of the workflow in all previous years. As this is over a week of computation time we would like to avoid that. Is there a way to let <a href="https://datasci.social/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> know that this change does not affect older years? I seems that `touch <output of step1>` would work. Is this the best solution?

Christian MeestersDid you know? During the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#SnakemakeHackathon2025</a> we had a staggering 194 work items!It took a while, but now we are gathering contribution reports and present them online as a ResearchEquals (<a href="https://fediscience.org/@ResearchEquals@akademienl.social" rel="nofollow noopener" translate="no" target="_blank">https://fediscience.org/@ResearchEquals@akademienl.social</a>) collection:<a href="https://www.researchequals.com/collections/hm1w-cg" rel="nofollow noopener" translate="no" target="_blank">https://www.researchequals.com/collections/hm1w-cg</a>The first 10 are online and I will post some highlights in the coming weeks.<a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#Snakemake</a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleComputing</a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#ReproducibleResearch</a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenScience</a>

Johannes KösterWhoops, there have been 19 new <a href="https://fosstodon.org/tags/Snakemake" class="mention hashtag" rel="tag">#Snakemake</a> releases since my last update. Apart from a lot of little bug fixes, we had several usability improvements for resource and temp handling and support for more checksum algorithms when validating output files. <a href="https://snakemake.readthedocs.io/en/stable/project_info/history.html" target="_blank" rel="nofollow noopener" translate="no">https://snakemake.readthedocs.io/en/stable/project_info/history.html</a>

Recent searches

Search options

Administered by:

Server stats:

#snakemake