Fosstodon

0 posts0 participants0 posts today

LavX NewsUnlocking Cost-Efficient AI: AIBrix and the Future of Large Language Model DeploymentAs open-source large language models like LLaMA and Mistral gain traction, AIBrix emerges as a pivotal tool for organizations looking to optimize AI deployments. This innovative cloud-native infrastru...<a href="https://news.lavx.hu/article/unlocking-cost-efficient-ai-aibrix-and-the-future-of-large-language-model-deployment" rel="nofollow noopener noreferrer" target="_blank">https://news.lavx.hu/article/unlocking-cost-efficient-ai-aibrix-and-the-future-of-large-language-model-deployment</a><a href="https://mastodon.cloud/tags/news" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#news</a> <a href="https://mastodon.cloud/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a> <a href="https://mastodon.cloud/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Kubernetes</a> <a href="https://mastodon.cloud/tags/AIBrix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIBrix</a> <a href="https://mastodon.cloud/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a>

Reshmi AravindExplore how vLLM optimizes the Llama Stack for efficient and scalable LLM inference in this deep dive from the vLLM team.<a href="https://blog.vllm.ai/2025/01/27/intro-to-llama-stack-with-vllm.html" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://blog.vllm.ai/2025/01/27/intro-to-llama-stack-with-vllm.html</a>Excellent ,<a href="https://bsky.brid.gy/r/https://bsky.app/profile/terrytangyuan.xyz" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@terrytangyuan.xyz</a> !<a href="https://mstdn.social/tags/redhat" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#redhat</a> <a href="https://mstdn.social/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#opensource</a> <a href="https://mstdn.social/tags/VLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#VLLM</a>

HabrПостроение инфраструктуры для работы с языковыми моделями: опыт X5 TechПривет, Хабр! Я Мичил Егоров, руководитель команды разработки продуктов искусственного интеллекта в X5 Tech. В последнее время языковые модели (LLM) стали неотъемлемой частью многих бизнес-процессов, начиная от чат-ботов и заканчивая автоматической обработкой отзывов клиентов. Однако, чтобы эффективно использовать такие модели, необходима мощная и гибкая инфраструктура. За последний год команда X5 Tech значительно выросла, проверила множество гипотез и протестировала различные модели. Основные кейсы использования включают чат-боты, суфлёры для модераторов, автоматическое резюмирование и обработку отзывов клиентов. В этой статье расскажу, как команда X5 Tech построила инфраструктуру для работы с языковыми моделями, какие вызовы преодолели и какие решения были приняты.<a href="https://habr.com/ru/companies/X5Tech/articles/880288/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://habr.com/ru/companies/X5Tech/articles/880288/</a><a href="https://zhub.link/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> <a href="https://zhub.link/tags/%D1%81_%D0%BD%D1%83%D0%BB%D1%8F" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#с_нуля</a> <a href="https://zhub.link/tags/%D0%BF%D0%BE%D1%81%D1%82%D1%80%D0%BE%D0%B5%D0%BD%D0%B8%D0%B5_%D0%B8%D0%BD%D1%84%D1%80%D0%B0%D1%81%D1%82%D1%80%D1%83%D0%BA%D1%82%D1%83%D1%80%D1%8B" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#построение_инфраструктуры</a> <a href="https://zhub.link/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#opensource</a> <a href="https://zhub.link/tags/%D0%97%D0%B0%D0%BA%D1%80%D1%8B%D1%82%D1%8B%D0%B5_%D1%80%D0%B5%D1%88%D0%B5%D0%BD%D0%B8%D1%8F" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Закрытые_решения</a> <a href="https://zhub.link/tags/%D0%BC%D0%B0%D1%81%D0%BA%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5_%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D1%85" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#маскирование_данных</a> <a href="https://zhub.link/tags/clearml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#clearml</a> <a href="https://zhub.link/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a> <a href="https://zhub.link/tags/%D0%BB%D0%BE%D0%B3%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#логирование</a> <a href="https://zhub.link/tags/%D0%BC%D0%BE%D0%BD%D0%B8%D1%82%D0%BE%D1%80%D0%B8%D0%BD%D0%B3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#мониторинг</a>

Adam :redhat: :ansible: :bash:Introducing vLLM Inference Provider in Llama Stack<a href="https://blog.vllm.ai/2025/01/27/intro-to-llama-stack-with-vllm.html" target="_blank" rel="nofollow noopener noreferrer" translate="no">https://blog.vllm.ai/2025/01/27/intro-to-llama-stack-with-vllm.html</a><a href="https://fosstodon.org/tags/artificialintelligence" class="mention hashtag" rel="tag">#artificialintelligence</a> <a href="https://fosstodon.org/tags/AI" class="mention hashtag" rel="tag">#AI</a> <a href="https://fosstodon.org/tags/vLLM" class="mention hashtag" rel="tag">#vLLM</a> <a href="https://fosstodon.org/tags/llamastack" class="mention hashtag" rel="tag">#llamastack</a> <a href="https://fosstodon.org/tags/opensource" class="mention hashtag" rel="tag">#opensource</a> <a href="https://fosstodon.org/tags/RedHat" class="mention hashtag" rel="tag">#RedHat</a>

Adam :redhat: :ansible: :bash:Achieve better large language model inference with fewer GPUs"we achieved approximately 55-65% of the throughput on a server config that is approximately 15% of the cost"<a href="https://www.redhat.com/en/blog/achieve-better-large-language-model-inference-fewer-gpus" target="_blank" rel="nofollow noopener noreferrer" translate="no">https://www.redhat.com/en/blog/achieve-better-large-language-model-inference-fewer-gpus</a><a href="https://fosstodon.org/tags/OpenShiftAI" class="mention hashtag" rel="tag">#OpenShiftAI</a> <a href="https://fosstodon.org/tags/RedHat" class="mention hashtag" rel="tag">#RedHat</a> <a href="https://fosstodon.org/tags/OpenShift" class="mention hashtag" rel="tag">#OpenShift</a> <a href="https://fosstodon.org/tags/AI" class="mention hashtag" rel="tag">#AI</a> <a href="https://fosstodon.org/tags/Kubernetes" class="mention hashtag" rel="tag">#Kubernetes</a> <a href="https://fosstodon.org/tags/vllm" class="mention hashtag" rel="tag">#vllm</a> <a href="https://fosstodon.org/tags/kubeflow" class="mention hashtag" rel="tag">#kubeflow</a> <a href="https://fosstodon.org/tags/kserve" class="mention hashtag" rel="tag">#kserve</a>

michabbb🎯 <a href="https://social.vivaldi.net/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> Language Model Platform Launch 🔧 Leverages <a href="https://social.vivaldi.net/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a> technology with custom <a href="https://social.vivaldi.net/tags/GPU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GPU</a> scheduler for running various <a href="https://social.vivaldi.net/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> models 🤖 Supports major models: <a href="https://social.vivaldi.net/tags/Llama3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Llama3</a> (405B/70B/8B), <a href="https://social.vivaldi.net/tags/Qwen2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2</a> 72B, <a href="https://social.vivaldi.net/tags/Mixtral" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Mixtral</a>, <a href="https://social.vivaldi.net/tags/Gemma2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Gemma2</a>, <a href="https://social.vivaldi.net/tags/Jamba15" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Jamba15</a>, <a href="https://social.vivaldi.net/tags/Phi3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Phi3</a><a href="https://glhf.chat/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://glhf.chat/</a>

jordanJust occurred to me that I could use a micro <a href="https://mastodon.jordanwages.com/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a> like <a href="https://mastodon.jordanwages.com/tags/omnivision" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#omnivision</a> to automatically add <a href="https://mastodon.jordanwages.com/tags/alttext" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#alttext</a> to my image posts without having to manually input them. I typically use <a href="https://mastodon.jordanwages.com/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ChatGPT</a> to generate my alt-text anyway, and rarely have to manually edit the content. Since I own this server, I could even do it at the database level, directly modifying the content.Worth it? Potential pitfalls? Waste of effort?<a href="https://huggingface.co/NexaAIDev/omnivision-968M#how-to-use-on-device" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://huggingface.co/NexaAIDev/omnivision-968M#how-to-use-on-device</a><a href="https://mastodon.jordanwages.com/tags/mastoadmin" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mastoadmin</a> <a href="https://mastodon.jordanwages.com/tags/mastobot" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mastobot</a> <a href="https://mastodon.jordanwages.com/tags/bot" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bot</a> <a href="https://mastodon.jordanwages.com/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://mastodon.jordanwages.com/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llm</a>

michabbbNew Cloud Platform for Large Language Model Deployment 🚀🔧 Run any <a href="https://social.vivaldi.net/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#opensource</a> <a href="https://social.vivaldi.net/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> supported by <a href="https://social.vivaldi.net/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a> on autoscaling <a href="https://social.vivaldi.net/tags/GPU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GPU</a> clusters, supporting models up to 640GB VRAM🤖 Compatible with major models: <a href="https://social.vivaldi.net/tags/Llama3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Llama3</a> 405B/70B/8B, <a href="https://social.vivaldi.net/tags/Qwen2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2</a> 72B, <a href="https://social.vivaldi.net/tags/Mixtral" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Mixtral</a> 8x22B, <a href="https://social.vivaldi.net/tags/Gemma2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Gemma2</a> 27B, <a href="https://social.vivaldi.net/tags/Phi3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Phi3</a>, and more💻 Features include: - <a href="https://social.vivaldi.net/tags/OpenAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenAI</a> compatible <a href="https://social.vivaldi.net/tags/API" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#API</a> - Custom-built <a href="https://social.vivaldi.net/tags/GPU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GPU</a> scheduler - Support for full-weight and 4-bit AWQ repos - Multi-tenant architecture for cost efficiency🆓 Currently free during beta phase, promising competitive pricing post-launch<a href="https://glhf.chat/landing/home" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://glhf.chat/landing/home</a>

Brian Cunnie-- Could NOT find Python (missing: Python_INCLUDE_DIRS Interpreter Development.Module Development.SABIModule)→sudo apt install -y python3.10-dev<a href="https://ruby.social/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://ruby.social/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a>

Judith van StegerenMany companies are currently scrambling for ML infra engineers. They need people that know how to manage AI infrastructure, and that can seriously speed up training and inference with specialized tooling like vLLM, Triton, TensorRT, Torchtune, etc.<a href="https://fosstodon.org/tags/inference" class="mention hashtag" rel="tag">#inference</a> <a href="https://fosstodon.org/tags/training" class="mention hashtag" rel="tag">#training</a> <a href="https://fosstodon.org/tags/genai" class="mention hashtag" rel="tag">#genai</a> <a href="https://fosstodon.org/tags/triton" class="mention hashtag" rel="tag">#triton</a> <a href="https://fosstodon.org/tags/vllm" class="mention hashtag" rel="tag">#vllm</a> <a href="https://fosstodon.org/tags/pytorch" class="mention hashtag" rel="tag">#pytorch</a> <a href="https://fosstodon.org/tags/torchtune" class="mention hashtag" rel="tag">#torchtune</a> <a href="https://fosstodon.org/tags/tensorrt" class="mention hashtag" rel="tag">#tensorrt</a> <a href="https://fosstodon.org/tags/nvidia" class="mention hashtag" rel="tag">#nvidia</a>

Redd T. PandaWe've swapped our <a href="https://unfufadoo.net/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a> to <a href="https://unfufadoo.net/tags/MistralAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MistralAI</a> <a href="https://unfufadoo.net/tags/Pixtral" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Pixtral</a> and its awesome, we are using this <a href="https://unfufadoo.net/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://unfufadoo.net/tags/model" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#model</a> for our <a href="https://unfufadoo.net/tags/alttext" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#alttext</a> posts for our <a href="https://unfufadoo.net/tags/memes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#memes</a>, <a href="https://unfufadoo.net/tags/images" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#images</a> and <a href="https://unfufadoo.net/tags/photos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#photos</a>. <a href="https://unfufadoo.net/tags/mastodon" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mastodon</a> <a href="https://unfufadoo.net/tags/Fediverse" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Fediverse</a> <a href="https://unfufadoo.net/tags/community" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#community</a>

michabbb🚀 <a href="https://social.vivaldi.net/tags/Qwen2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2</a>.5: New <a href="https://social.vivaldi.net/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> model family released by Qwen Team<a href="https://social.vivaldi.net/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> variants: 0.5B to 72B parameters, support 29+ languages including English, Chinese, French, Spanish Specialized models: <a href="https://social.vivaldi.net/tags/Qwen2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2</a>.5Coder for coding, <a href="https://social.vivaldi.net/tags/Qwen2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2</a>.5Math for mathematics 128K token context length, can generate up to 8K tokens <a href="https://social.vivaldi.net/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> under Apache 2.0 license (except 3B and 72B variants)💡 Key improvements:Enhanced knowledge (85+ on <a href="https://social.vivaldi.net/tags/MMLU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MMLU</a>) Better coding skills (85+ on <a href="https://social.vivaldi.net/tags/HumanEval" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HumanEval</a>) Improved math capabilities (80+ on <a href="https://social.vivaldi.net/tags/MATH" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MATH</a>) Stronger instruction following and long text generation Better handling of structured data and outputs (e.g., <a href="https://social.vivaldi.net/tags/JSON" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#JSON</a>)🔬 Performance highlights:<a href="https://social.vivaldi.net/tags/Qwen2572B" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Qwen2572B</a> competitive with leading models like <a href="https://social.vivaldi.net/tags/Llama3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Llama3</a> and <a href="https://social.vivaldi.net/tags/MistralAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MistralAI</a> Smaller models (e.g., 3B) show impressive efficiency <a href="https://social.vivaldi.net/tags/QwenPlus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#QwenPlus</a> API model competes with <a href="https://social.vivaldi.net/tags/GPT4" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GPT4</a> and <a href="https://social.vivaldi.net/tags/Claude" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Claude</a> on some benchmarks🛠️ Available via <a href="https://social.vivaldi.net/tags/HuggingFace" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HuggingFace</a>, <a href="https://social.vivaldi.net/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a>, and other deployment options 📊 Comprehensive benchmarks and comparisons provided in the blog post <a href="https://qwenlm.github.io/blog/qwen2.5/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://qwenlm.github.io/blog/qwen2.5/</a>

Tao of MaciGPU Compute and LLMs on the AceMagic AM18As part of my forays into LLMs and GPU compute “on the small” I’ve been playing around with the AceMagic AM18 in a few unusual ways. If you missed my initial impressions, you might want to check them out first.(...)<a href="https://botsin.space/tags/am18" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#am18</a> <a href="https://botsin.space/tags/amd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#amd</a> <a href="https://botsin.space/tags/amdgpu" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#amdgpu</a> <a href="https://botsin.space/tags/gpu" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#gpu</a> <a href="https://botsin.space/tags/hardware" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#hardware</a> <a href="https://botsin.space/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#homelab</a> <a href="https://botsin.space/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llm</a> <a href="https://botsin.space/tags/ml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ml</a> <a href="https://botsin.space/tags/ollama" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ollama</a> <a href="https://botsin.space/tags/proxmox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#proxmox</a> <a href="https://botsin.space/tags/review" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#review</a> <a href="https://botsin.space/tags/ryzen" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ryzen</a> <a href="https://botsin.space/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a> <a href="https://botsin.space/tags/zluda" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#zluda</a><a href="https://taoofmac.com/space/blog/2024/04/13/2100" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://taoofmac.com/space/blog/2024/04/13/2100</a>

HabrКак ускорить LLM-генерацию текста в 20 раз на больших наборах данныхВсем привет, я Алан, разработчик-исследователь в MTS AI. В команде фундаментальных исследований мы занимаемся исследованием LLM, реализацией DPO и валидацией наших собственных языковых моделей. В рамках этих задач у нас возникла потребность в генерации большого количества данных с помощью LLM. Такая генерация обычно занимает много времени. Однако за последний год, с ростом популярности LLM, стали появляться различные инструменты для развертывания таких моделей. Одной из самых эффективных библиотек для инференса языковых моделей является библиотека vLLM. В статье показывается, как с помощью асинхронных запросов и встроенных особенностей vLLM можно увеличить скорость генерации примерно в 20 раз. Приятного чтения!<a href="https://habr.com/ru/companies/mts_ai/articles/791594/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://habr.com/ru/companies/mts_ai/articles/791594/</a><a href="https://zhub.link/tags/%D0%B8%D1%81%D0%BA%D1%83%D1%81%D1%81%D1%82%D0%B2%D0%B5%D0%BD%D0%BD%D1%8B%D0%B9_%D0%B8%D0%BD%D1%82%D0%B5%D0%BB%D0%BB%D0%B5%D0%BA%D1%82" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#искусственный_интеллект</a> <a href="https://zhub.link/tags/nlp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nlp</a> <a href="https://zhub.link/tags/%D1%8F%D0%B7%D1%8B%D0%BA%D0%BE%D0%B2%D1%8B%D0%B5_%D0%BC%D0%BE%D0%B4%D0%B5%D0%BB%D0%B8" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#языковые_модели</a> <a href="https://zhub.link/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a> <a href="https://zhub.link/tags/%D0%BE%D0%B1%D1%80%D0%B0%D0%B1%D0%BE%D1%82%D0%BA%D0%B0_%D0%B5%D1%81%D1%82%D0%B5%D1%81%D1%82%D0%B2%D0%B5%D0%BD%D0%BD%D0%BE%D0%B3%D0%BE_%D1%8F%D0%B7%D1%8B%D0%BA%D0%B0" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#обработка_естественного_языка</a> <a href="https://zhub.link/tags/%D0%BD%D0%B5%D0%B9%D1%80%D0%BE%D1%81%D0%B5%D1%82%D0%B8" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#нейросети</a> <a href="https://zhub.link/tags/%D0%BD%D0%B5%D0%B9%D1%80%D0%BE%D0%BD%D0%BD%D1%8B%D0%B5_%D1%81%D0%B5%D1%82%D0%B8" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#нейронные_сети</a> <a href="https://zhub.link/tags/%D0%B8%D0%BD%D1%84%D0%B5%D1%80%D0%B5%D0%BD%D1%81" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#инференс</a> <a href="https://zhub.link/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llm</a>

Karim LalaniSelf-hosting LLMs — Deployment Considerations for LangChain and LangServe applications<a href="https://medium.com/@klcoder/self-hosting-llms-deployment-considerations-for-langchain-and-langserve-applications-b9ae689c744a?sk=95c1b2d147764f0aafad932ef825ea6f" rel="nofollow noopener noreferrer" target="_blank">https://medium.com/@klcoder/self-hosting-llms-deployment-considerations-for-langchain-and-langserve-applications-b9ae689c744a?sk=95c1b2d147764f0aafad932ef825ea6f</a><a href="https://me.dm/tags/llm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llm</a> <a href="https://me.dm/tags/genai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#genai</a> <a href="https://me.dm/tags/langchain" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#langchain</a> <a href="https://me.dm/tags/ollama" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ollama</a> <a href="https://me.dm/tags/vllm" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vllm</a> <a href="https://me.dm/tags/llamacpp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#llamacpp</a>

michabbbServing <a href="https://social.vivaldi.net/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> 24x Faster On the <a href="https://social.vivaldi.net/tags/Cloud" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cloud</a> with <a href="https://social.vivaldi.net/tags/vLLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vLLM</a> and <a href="https://social.vivaldi.net/tags/SkyPilot" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SkyPilot</a> <a href="https://blog.skypilot.co/serving-llm-24x-faster-on-the-cloud-with-vllm-and-skypilot/" rel="nofollow noopener noreferrer" target="_blank">https://blog.skypilot.co/serving-llm-24x-faster-on-the-cloud-with-vllm-and-skypilot/</a><a href="https://social.vivaldi.net/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a>

Recent searches

Search options

Administered by:

Server stats:

#vllm