Fosstodon

Sara Zan📢 Don't overlook this in the wave of releases! <a href="https://mastodon.social/tags/MistralAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#MistralAI</a> has a new coding LLM: it's <a href="https://mastodon.social/tags/Devstral" class="mention hashtag" rel="nofollow noopener" target="_blank">#Devstral</a>, an open model perfect for on-prem, private and local deployments 🐈📰 Have a look at the announcement: <a href="https://mistral.ai/news/devstral" rel="nofollow noopener" translate="no" target="_blank">https://mistral.ai/news/devstral</a><a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://mastodon.social/tags/GenAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#GenAI</a> <a href="https://mastodon.social/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMs</a> <a href="https://mastodon.social/tags/Devstral" class="mention hashtag" rel="nofollow noopener" target="_blank">#Devstral</a> <a href="https://mastodon.social/tags/SWEBench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEBench</a>

Sara Zan🧠 Another flagship model released! <a href="https://mastodon.social/tags/Anthropic" class="mention hashtag" rel="nofollow noopener" target="_blank">#Anthropic</a> just unveiled Claude Opus 4 and Claude Sonnet 4, and they are at the top of the leaderboard for coding 💻📰 Check out the announcement: <a href="https://www.anthropic.com/news/claude-4" rel="nofollow noopener" translate="no" target="_blank">https://www.anthropic.com/news/claude-4</a><a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://mastodon.social/tags/GenAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#GenAI</a> <a href="https://mastodon.social/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMs</a> <a href="https://mastodon.social/tags/Claude" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude</a> <a href="https://mastodon.social/tags/Claude4" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude4</a> <a href="https://mastodon.social/tags/SweBench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SweBench</a>

N-gated Hacker News🎉🥳 OMG, Refact.ai scored a groundbreaking 69.8 on <a href="https://mastodon.social/tags/SWEbench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEbench</a> and now it's charging you in coins! 💰🔧 Apparently, solving 349 out of 500 tasks makes it the reigning champion of open-source AI agents. Who knew moving from request limits to coin tossing was the future of tech? 🤪👨‍💻 <a href="https://refact.ai/blog/2025/open-source-sota-on-swe-bench-verified-refact-ai/" rel="nofollow noopener" translate="no" target="_blank">https://refact.ai/blog/2025/open-source-sota-on-swe-bench-verified-refact-ai/</a> <a href="https://mastodon.social/tags/RefactAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#RefactAI</a> <a href="https://mastodon.social/tags/openSourceAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#openSourceAI</a> <a href="https://mastodon.social/tags/techInnovation" class="mention hashtag" rel="nofollow noopener" target="_blank">#techInnovation</a> <a href="https://mastodon.social/tags/coinTossing" class="mention hashtag" rel="nofollow noopener" target="_blank">#coinTossing</a> <a href="https://mastodon.social/tags/HackerNews" class="mention hashtag" rel="nofollow noopener" target="_blank">#HackerNews</a> <a href="https://mastodon.social/tags/ngated" class="mention hashtag" rel="nofollow noopener" target="_blank">#ngated</a>

michabbb<a href="https://social.vivaldi.net/tags/Devstral" class="mention hashtag" rel="nofollow noopener" target="_blank">#Devstral</a>: New <a href="https://social.vivaldi.net/tags/opensource" class="mention hashtag" rel="nofollow noopener" target="_blank">#opensource</a> Model for Coding Agents by <a href="https://social.vivaldi.net/tags/MistralAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#MistralAI</a> & <a href="https://social.vivaldi.net/tags/AllHandsAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AllHandsAI</a> 🧠• 🏆 <a href="https://social.vivaldi.net/tags/Devstral" class="mention hashtag" rel="nofollow noopener" target="_blank">#Devstral</a> achieves 46.8% on <a href="https://social.vivaldi.net/tags/SWEBench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEBench</a> Verified, outperforming previous <a href="https://social.vivaldi.net/tags/opensource" class="mention hashtag" rel="nofollow noopener" target="_blank">#opensource</a> models by over 6% points and surpassing <a href="https://social.vivaldi.net/tags/GPT4" class="mention hashtag" rel="nofollow noopener" target="_blank">#GPT4</a> mini by 20%🧵👇<a href="https://social.vivaldi.net/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://social.vivaldi.net/tags/coding" class="mention hashtag" rel="nofollow noopener" target="_blank">#coding</a>

HabrКак мы собираем SWE-bench на других языкахСовременная разработка ПО — это плавильный котел языков: Java, C#, JS/TS, Go, Kotlin… список можно продолжать. Но когда дело доходит до оценки ИИ-агентов, способных помогать в написании и исправлении кода, мы часто упираемся в ограничения. Популярный бенчмарк SWE-bench, например, долгое время поддерживал только Python. Чтобы преодолеть разрыв между реальностью разработки и возможностями оценки ИИ, наша команда в<a href="https://habr.com/ru/companies/doubletapp/articles/901032/" rel="nofollow noopener" translate="no" target="_blank">https://habr.com/ru/companies/doubletapp/articles/901032/</a><a href="https://zhub.link/tags/swebench" class="mention hashtag" rel="nofollow noopener" target="_blank">#swebench</a> <a href="https://zhub.link/tags/%D0%B8%D0%B8" class="mention hashtag" rel="nofollow noopener" target="_blank">#ии</a> <a href="https://zhub.link/tags/%D0%BD%D0%B5%D0%B9%D1%80%D0%BE%D1%81%D0%B5%D1%82%D0%B8" class="mention hashtag" rel="nofollow noopener" target="_blank">#нейросети</a> <a href="https://zhub.link/tags/ml" class="mention hashtag" rel="nofollow noopener" target="_blank">#ml</a> <a href="https://zhub.link/tags/%D0%BC%D0%B0%D1%88%D0%B8%D0%BD%D0%BD%D0%BE%D0%B5_%D0%BE%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D0%B5" class="mention hashtag" rel="nofollow noopener" target="_blank">#машинное_обучение</a> <a href="https://zhub.link/tags/%D0%B8%D1%81%D0%BA%D1%83%D1%81%D1%81%D1%82%D0%B2%D0%B5%D0%BD%D0%BD%D1%8B%D0%B9_%D0%B8%D0%BD%D1%82%D0%B5%D0%BB%D0%BB%D0%B5%D0%BA%D1%82" class="mention hashtag" rel="nofollow noopener" target="_blank">#искусственный_интеллект</a> <a href="https://zhub.link/tags/github" class="mention hashtag" rel="nofollow noopener" target="_blank">#github</a> <a href="https://zhub.link/tags/open_source" class="mention hashtag" rel="nofollow noopener" target="_blank">#open_source</a>

Habr[Перевод] Сравнение бенчмарков LLM для разработки программного обеспеченияВ этой статье мы сравним различные бенчмарки, которые помогают ранжировать крупные языковые модели для задач разработки программного обеспечения.<a href="https://habr.com/ru/articles/857754/" rel="nofollow noopener" translate="no" target="_blank">https://habr.com/ru/articles/857754/</a><a href="https://zhub.link/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLM</a> <a href="https://zhub.link/tags/%D0%B1%D0%B5%D0%BD%D1%87%D0%BC%D0%B0%D1%80%D0%BA%D0%B8" class="mention hashtag" rel="nofollow noopener" target="_blank">#бенчмарки</a> <a href="https://zhub.link/tags/%D0%B1%D0%B5%D0%BD%D1%87%D0%BC%D0%B0%D1%80%D0%BA%D0%B8%D0%BD%D0%B3" class="mention hashtag" rel="nofollow noopener" target="_blank">#бенчмаркинг</a> <a href="https://zhub.link/tags/HumanEval" class="mention hashtag" rel="nofollow noopener" target="_blank">#HumanEval</a> <a href="https://zhub.link/tags/DevQualityEval" class="mention hashtag" rel="nofollow noopener" target="_blank">#DevQualityEval</a> <a href="https://zhub.link/tags/CodeXGLUE" class="mention hashtag" rel="nofollow noopener" target="_blank">#CodeXGLUE</a> <a href="https://zhub.link/tags/Aider" class="mention hashtag" rel="nofollow noopener" target="_blank">#Aider</a> <a href="https://zhub.link/tags/SWEbench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEbench</a> <a href="https://zhub.link/tags/ClassEval" class="mention hashtag" rel="nofollow noopener" target="_blank">#ClassEval</a> <a href="https://zhub.link/tags/BigCodeBench" class="mention hashtag" rel="nofollow noopener" target="_blank">#BigCodeBench</a>

michabbb🚀 <a href="https://social.vivaldi.net/tags/Claude35Sonnet" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude35Sonnet</a> is now rolling out on <a href="https://social.vivaldi.net/tags/GitHubCopilot" class="mention hashtag" rel="nofollow noopener" target="_blank">#GitHubCopilot</a>, bringing advanced coding capabilities directly to <a href="https://social.vivaldi.net/tags/VisualStudioCode" class="mention hashtag" rel="nofollow noopener" target="_blank">#VisualStudioCode</a> and <a href="https://GitHub.com" rel="nofollow noopener" translate="no" target="_blank">https://GitHub.com</a>• 🏆 Performance highlights: - Highest score among public models on <a href="https://social.vivaldi.net/tags/SWEbench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEbench</a> Verified - 93.7% accuracy on <a href="https://social.vivaldi.net/tags/HumanEval" class="mention hashtag" rel="nofollow noopener" target="_blank">#HumanEval</a> for <a href="https://social.vivaldi.net/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#Python</a> function writing• 💻 Key features: - Production-ready code generation - Inline debugging assistance - Automated test suite creation - Contextual code explanations• ⚙️ Technical details: - Runs via <a href="https://social.vivaldi.net/tags/AmazonBedrock" class="mention hashtag" rel="nofollow noopener" target="_blank">#AmazonBedrock</a> - Cross-region inference for enhanced reliability - Available to all <a href="https://social.vivaldi.net/tags/GitHub" class="mention hashtag" rel="nofollow noopener" target="_blank">#GitHub</a> Copilot Chat users and organizationsSource: <a href="https://www.anthropic.com/news/github-copilot" rel="nofollow noopener" translate="no" target="_blank">https://www.anthropic.com/news/github-copilot</a>

michabbb🚀 <a href="https://social.vivaldi.net/tags/Anthropic" class="mention hashtag" rel="nofollow noopener" target="_blank">#Anthropic</a> announces major updates to their <a href="https://social.vivaldi.net/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> model lineup:💻 Upgraded <a href="https://social.vivaldi.net/tags/Claude35Sonnet" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude35Sonnet</a> shows significant improvements: • Achieves 49% on <a href="https://social.vivaldi.net/tags/SWEbench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEbench</a> Verified coding benchmark • Leads in software engineering capabilities • Maintains same price and speed as predecessor • Tested by US and UK <a href="https://social.vivaldi.net/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> Safety Institutes🔄 New <a href="https://social.vivaldi.net/tags/Claude35Haiku" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude35Haiku</a> introduction: • Matches <a href="https://social.vivaldi.net/tags/Claude3Opus" class="mention hashtag" rel="nofollow noopener" target="_blank">#Claude3Opus</a> performance at lower cost • Scores 40.6% on SWEbench Verified • Optimized for user-facing products • Available across multiple cloud platforms🖱️ Pioneering <a href="https://social.vivaldi.net/tags/ComputerUse" class="mention hashtag" rel="nofollow noopener" target="_blank">#ComputerUse</a> beta feature: • Allows AI to navigate interfaces like humans • Scores 22% on <a href="https://social.vivaldi.net/tags/OSWorld" class="mention hashtag" rel="nofollow noopener" target="_blank">#OSWorld</a> benchmark • Currently in experimental phase • Supported by new safety classifiers⚡ Enterprise adoption: • <a href="https://social.vivaldi.net/tags/GitLab" class="mention hashtag" rel="nofollow noopener" target="_blank">#GitLab</a> reports 10% improvement in DevSecOps tasks • <a href="https://social.vivaldi.net/tags/Replit" class="mention hashtag" rel="nofollow noopener" target="_blank">#Replit</a> leverages computer use for app evaluation • <a href="https://social.vivaldi.net/tags/Cognition" class="mention hashtag" rel="nofollow noopener" target="_blank">#Cognition</a> notes enhanced problem-solving capabilities<a href="https://www.anthropic.com/news/3-5-models-and-computer-use" rel="nofollow noopener" translate="no" target="_blank">https://www.anthropic.com/news/3-5-models-and-computer-use</a>

marmelabHow do AI software engineering agents work?🤔🤖Find the answer, along with valuable insights from the creators of SWE-bench & SWE-agent, in this article⬇️<a href="https://newsletter.pragmaticengineer.com/p/ai-coding-agents?r=3sbses&utm_campaign=post&utm_medium=web" rel="nofollow noopener" translate="no" target="_blank">https://newsletter.pragmaticengineer.com/p/ai-coding-agents?r=3sbses&utm_campaign=post&utm_medium=web</a> Great read! 👏 <a href="https://mastodon.online/@gergelyorosz" class="u-url mention" rel="nofollow noopener" target="_blank">@gergelyorosz</a>, <a href="https://mastodon.social/@elin" class="u-url mention" rel="nofollow noopener" target="_blank">@elin</a> Nilsson <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://mastodon.social/tags/SWEbench" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEbench</a> <a href="https://mastodon.social/tags/SWEagent" class="mention hashtag" rel="nofollow noopener" target="_blank">#SWEagent</a> <a href="https://mastodon.social/tags/softwareengineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#softwareengineering</a>

Recent searches

Search options

Administered by:

Server stats:

#swebench