Fosstodon

1 post1 participant0 posts today

Michael Sumnerwow, <a href="https://rstats.me/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> is good - just read-from-url filter/select/unique super straightforward<a href="https://rstats.me/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#python</a>

narkodeI am transforming a project from <a href="https://nerdculture.de/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> to <a href="https://nerdculture.de/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> and I already like it! I don't want to talk bad about <a href="https://nerdculture.de/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a>, it did a lot for the data science community. But, beside of being blazingly fast, the syntax of <a href="https://nerdculture.de/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> is much more logic and consistent!

Towards Data ScienceStruggling with slow GIS processing? Tony Albanese's article demonstrates a lightning-fast method for generating transects using <a href="https://hachyderm.io/tags/Geopandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Geopandas</a> and <a href="https://hachyderm.io/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a>, reducing processing time from days to seconds. <a href="https://towardsdatascience.com/harnessing-polars-and-geopandas-to-generate-millions-of-transects-in-seconds-d37b176a0b57/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://towardsdatascience.com/harnessing-polars-and-geopandas-to-generate-millions-of-transects-in-seconds-d37b176a0b57/</a>

:rss: DevelopersIOAWS LambdaでDuckDBとAWS Data Wrangler、Polarsの処理性能を比較してみた <a href="https://dev.classmethod.jp/articles/comparing-csv-to-parquet-conversion-speed-in-aws-lambda/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://dev.classmethod.jp/articles/comparing-csv-to-parquet-conversion-speed-in-aws-lambda/</a><a href="https://rss-mstdn.studiofreesia.com/tags/dev_classmethod" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dev_classmethod</a> <a href="https://rss-mstdn.studiofreesia.com/tags/AWS_Lambda" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AWS_Lambda</a> <a href="https://rss-mstdn.studiofreesia.com/tags/awswrangler" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#awswrangler</a> <a href="https://rss-mstdn.studiofreesia.com/tags/DuckDB" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DuckDB</a> <a href="https://rss-mstdn.studiofreesia.com/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> <a href="https://rss-mstdn.studiofreesia.com/tags/AWS_CDK" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AWS_CDK</a>

HabrPolars для обработки JSON и ParquetПривет, Хабр! Сегодня рассмотрим тему обработки временных рядов с помощью Polars. Начну с того, что в Pandas для агрегации временных рядов принято использовать метод resample() . Он удобен и привычен, но имеет свои ограничения по производительности и гибкости. Polars, в свою очередь, имеет метод groupby_dynamic() , который позволяет группировать данные по динамическим временным интервалам.<a href="https://habr.com/ru/companies/otus/articles/892812/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://habr.com/ru/companies/otus/articles/892812/</a><a href="https://zhub.link/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> <a href="https://zhub.link/tags/%D0%B2%D1%80%D0%B5%D0%BC%D0%B5%D0%BD%D0%BD%D1%8B%D0%B5_%D1%80%D1%8F%D0%B4%D1%8B" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#временные_ряды</a> <a href="https://zhub.link/tags/%D0%BE%D0%B1%D1%80%D0%B0%D0%B1%D0%BE%D1%82%D0%BA%D0%B0_%D0%B2%D1%80%D0%B5%D0%BC%D0%B5%D0%BD%D0%BD%D1%8B%D1%85_%D1%80%D1%8F%D0%B4%D0%BE%D0%B2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#обработка_временных_рядов</a> <a href="https://zhub.link/tags/%D0%B0%D0%BD%D0%B0%D0%BB%D0%B8%D1%82%D0%B8%D0%BA%D0%B0" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#аналитика</a>

PyCharm BlogPolars vs. pandas – Python のデータフレームライブラリを徹底比較 <a href="https://techhub.social/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://techhub.social/tags/Pycharm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Pycharm</a> <a href="https://techhub.social/tags/Datascience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Datascience</a> <a href="https://techhub.social/tags/Pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Pandas</a> <a href="https://techhub.social/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a><a href="https://blog.jetbrains.com/pycharm/2025/02/polars-vs-pandas" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://blog.jetbrains.com/pycharm/2025/02/polars-vs-pandas</a>

CuducosDo I know any <a href="https://tech.lgbt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> contributor or maintainer? I would love to help with that bug/exploration, but I am not sure where to start!<a href="https://github.com/pola-rs/polars/issues/21851" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/pola-rs/polars/issues/21851</a> <a href="https://tech.lgbt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://tech.lgbt/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a>

EuroSciPyWorking on core array computing libraries that power <a href="https://fosstodon.org/tags/scientificPython" class="mention hashtag" rel="tag">#scientificPython</a>? <a href="https://fosstodon.org/tags/EuroSciPy2025" class="mention hashtag" rel="tag">#EuroSciPy2025</a> wants your proposals on optimized array operations, vectorization techniques, and numerical foundations. Submit your groundbreaking work!<a href="https://pretalx.com/euroscipy-2025/cfp" target="_blank" rel="nofollow noopener noreferrer" translate="no">https://pretalx.com/euroscipy-2025/cfp</a><a href="https://fosstodon.org/tags/ScientificComputing" class="mention hashtag" rel="tag">#ScientificComputing</a> <a href="https://fosstodon.org/tags/Python" class="mention hashtag" rel="tag">#Python</a> <a href="https://fosstodon.org/tags/EuroSciPy" class="mention hashtag" rel="tag">#EuroSciPy</a> <a href="https://fosstodon.org/tags/numpy" class="mention hashtag" rel="tag">#numpy</a> <a href="https://fosstodon.org/tags/scipy" class="mention hashtag" rel="tag">#scipy</a> <a href="https://fosstodon.org/tags/pandas" class="mention hashtag" rel="tag">#pandas</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="tag">#polars</a>

Data SciencePolars is a lightning fast DataFrame library/in-memory query engine with parallel execution and cache efficiency. And now you can use is with the tidyverse syntax: <a href="https://www.tidypolars.etiennebacher.com/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.tidypolars.etiennebacher.com/</a> <a href="https://genomic.social/tags/rstats" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rstats</a> <a href="https://genomic.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> <a href="https://genomic.social/tags/optimisation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#optimisation</a>

Mainmatter🌊 Kisters provides real-time flood warnings—but traffic spikes made scaling costly. We helped them build a Rust-based system using Polars to process data efficiently across servers, edge, and browsers (WASM)—keeping performance high and costs low.Read more ➡️ <a href="https://mainmatter.com/cases/kisters/?utm_source=linkedin" target="_blank" rel="nofollow noopener noreferrer" translate="no">https://mainmatter.com/cases/kisters/?utm_source=linkedin</a> <a href="https://fosstodon.org/tags/rustlang" class="mention hashtag" rel="tag">#rustlang</a> <a href="https://fosstodon.org/tags/WASM" class="mention hashtag" rel="tag">#WASM</a> <a href="https://fosstodon.org/tags/edgecomputing" class="mention hashtag" rel="tag">#edgecomputing</a> <a href="https://fosstodon.org/tags/cloudoptimization" class="mention hashtag" rel="tag">#cloudoptimization</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="tag">#polars</a>

WolfAt last night’s <a href="https://hachyderm.io/@mug" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@mug</a> meeting we looked at a lot of different solutions to <a href="https://hachyderm.io/tags/adventofcode" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adventofcode</a> day 1 in many different languages. Two that were very interesting to me were <a href="https://hachyderm.io/tags/Zig" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Zig</a> and <a href="https://hachyderm.io/tags/haskell" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#haskell</a>. The way these two languages worked was really quite fascinating. After seeing real code in these two languages, I can tell they are not for me; but they were interesting and illuminating nonetheless. There was a solution entirely in <a href="https://hachyderm.io/tags/SQL" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SQL</a>. Another in <a href="https://hachyderm.io/tags/vim9script" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vim9script</a>. Another in <a href="https://hachyderm.io/tags/swiftlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swiftlang</a> <a href="https://hachyderm.io/tags/swift" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swift</a> (I don’t think that one’s in the repo yet). I wrote several implementations myself. The one I felt most proud of is <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> with the core written in <a href="https://hachyderm.io/tags/rustlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rustlang</a> <a href="https://hachyderm.io/tags/rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#rust</a> tied together with <a href="https://hachyderm.io/tags/PyO3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PyO3</a>. The one I felt was maybe the best tool for the job was entirely based on <a href="https://hachyderm.io/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a>. As I said in a previous post, I tried to solve it in <a href="https://hachyderm.io/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>, but the API exposed by Polars at least as far as I could tell, made it no better than simple lists in Python. I need to get deeper knowledge here. The repo lives here: <a href="https://github.com/MichiganUnixUserGroup/MUG-2025-03-11-Advent-of-Code" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/MichiganUnixUserGroup/MUG-2025-03-11-Advent-of-Code</a>.

WolfI solved a small problem trying to do it using only <a href="https://hachyderm.io/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> (it was <a href="https://hachyderm.io/tags/adventofcode" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adventofcode</a> day 1). I know Pandas pretty well. The solution to both parts together turned out to be very short. Tried to do the same thing in <a href="https://hachyderm.io/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>. Way harder. Didn’t get to leverage DataFrames. Basically just used Series and they were just glorified lists. This was my first experience with Polars. So my ignorance was certainly a factor. But wow. Pandas just did the job better. I’m sure Polars is still growing. I really want to like it!

Eric R. ScottThere are now so many "backends" for {dplyr}—duckdb with {duckplyr}, polars with {tidypolars}, various database engines with {dbplyr}, {data.table} with {dtplyr}. Is there a blog post or flow chart somewhere with pros and cons of each? Like, comparisons of memory requirements, speed, and how likely they are to "just work"?<a href="https://fosstodon.org/tags/rstats" class="mention hashtag" rel="tag">#rstats</a> <a href="https://fosstodon.org/tags/duckdb" class="mention hashtag" rel="tag">#duckdb</a> <a href="https://fosstodon.org/tags/dplyr" class="mention hashtag" rel="tag">#dplyr</a> <a href="https://fosstodon.org/tags/tidyverse" class="mention hashtag" rel="tag">#tidyverse</a> <a href="https://fosstodon.org/tags/polars" class="mention hashtag" rel="tag">#polars</a>

Hacker NewsThe Best Way to Use Text Embeddings Portably Is with Parquet and Polars — <a href="https://minimaxir.com/2025/02/embeddings-parquet/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://minimaxir.com/2025/02/embeddings-parquet/</a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HackerNews</a> <a href="https://mastodon.social/tags/TextEmbeddings" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TextEmbeddings</a> <a href="https://mastodon.social/tags/Parquet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Parquet</a> <a href="https://mastodon.social/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> <a href="https://mastodon.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DataScience</a> <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MachineLearning</a>

benrutterWas noodling around with Gleam a while back, needed a dataframe library and set up a half-baked interface for polars.I can't beleive how easy the js ffi made it! Language FFI is such a hidden superpower, more of this please from bew languages! 😍<a href="https://mastodon.green/tags/programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#programming</a> <a href="https://mastodon.green/tags/gleam" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#gleam</a> <a href="https://mastodon.green/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a>

Alexandre B A Villares 🐍"Plotando estatísticas básicas com <a href="https://ciberlandia.pt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> e <a href="https://ciberlandia.pt/tags/Matplotlib" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Matplotlib</a> - <a href="https://ciberlandia.pt/tags/NLP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NLP</a> 04 " <a href="https://bolha.us/@dunossauro" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@dunossauro</a> <a href="https://ciberlandia.pt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a><a href="https://www.youtube.com/watch?v=4HpSFIekqDw" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.youtube.com/watch?v=4HpSFIekqDw</a> Hoje eu aprendi uma ideia ótima do Dunossauro que é imaginar que o ax do fig, ax do matplotlib (axis/eixo) é como uma haste onde penduramos as coisas! Como é fundamental o trabalho dele pra nossa comunidade.Update: inicialmente achei que era uma tradução corrente mas ele me explicou que não.

Jim GardnerDoes anyone have advice for unit testing a Polars Python expression? Say I have:def my_func(a, b): return (pl.col(a) / pl.col(b))How do you unit test? We want to compare the result of this function to an expectation. At the moment we’re wrapping the result in str(), which gives us a stringified version of the expression. Then we assert that it is equal to the stringified expression that we would expect.It works, just feels a little bit fragile. Unfortunately you can’t compare expressions directly with the equality operator ==. And polars.testing doesn’t have asserts for it either.(We are of course also going to have integration tests, where the expression actually gets used on a df fixture)<a href="https://bird.makeup/users/jeroenhjanssens" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@jeroenhjanssens</a> <a href="https://hachyderm.io/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> <a href="https://hachyderm.io/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://hachyderm.io/tags/PyData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PyData</a>

Isa BernardiniUne auteure de <a href="https://mastodon.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> à suivre assurément. J’en ai lu plusieurs d’elle, dont « Block 46 ». Voici son « dernier » sorti. Je ne gagne absolument rien à faire sa pub, je suis juste une personne qui aime ce qu’elle <a href="https://mastodon.social/tags/%C3%A9crit" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#écrit</a> et représente. Une bien belle personne.

Markus SuhrJust caught up with the recent Delta Lake webinar, > Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & RustSome interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. <a href="https://gruene.social/tags/ApacheAirflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ApacheAirflow</a> tasks<a href="https://youtu.be/BR9oFD0QMAs" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://youtu.be/BR9oFD0QMAs</a><a href="https://gruene.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dataengineering</a> <a href="https://gruene.social/tags/datascience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#datascience</a> <a href="https://gruene.social/tags/duckdb" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#duckdb</a> <a href="https://gruene.social/tags/daft" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#daft</a> <a href="https://gruene.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#polars</a> <a href="https://gruene.social/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pandas</a> <a href="https://gruene.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#python</a> <a href="https://gruene.social/tags/spark" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#spark</a> <a href="https://gruene.social/tags/deltalake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#deltalake</a> <a href="https://gruene.social/tags/databricks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#databricks</a> <a href="https://gruene.social/tags/airflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#airflow</a> <a href="https://gruene.social/tags/bigdata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bigdata</a> <a href="https://gruene.social/tags/smalldata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#smalldata</a>

Alexandre B A Villares 🐍"Estáticas básicas de texto com <a href="https://ciberlandia.pt/tags/Polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Polars</a> e <a href="https://ciberlandia.pt/tags/Spacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Spacy</a> - <a href="https://ciberlandia.pt/tags/NLP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NLP</a> 03 " <a href="https://ciberlandia.pt/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://www.youtube.com/watch?v=FgsdmxILSFo" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.youtube.com/watch?v=FgsdmxILSFo</a>

Recent searches

Search options

Administered by:

Server stats:

#polars