Login

Recent searches

No recent searches

Search options

Only available when logged in.
fosstodon.org is one of the many independent Mastodon servers you can use to participate in the fediverse.
Fosstodon is an invite only Mastodon instance that is open to those who are interested in technology; particularly free & open source software. If you wish to join, contact us for an invite.

Administered by:

Server stats:

9.9K
active users

fosstodon.org: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#MI250X

0 posts0 participants0 posts today
Benjamin Carr, Ph.D. 👨🏻‍💻🧬<p><a href="https://hachyderm.io/tags/Frontier" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Frontier</span></a> trained a <a href="https://hachyderm.io/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ChatGPT</span></a>-sized <a href="https://hachyderm.io/tags/largelanguagemodel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>largelanguagemodel</span></a> with only 3,000 of its 37,888 <a href="https://hachyderm.io/tags/Radeon" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Radeon</span></a> <a href="https://hachyderm.io/tags/GPU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GPU</span></a> — the world's fastest <a href="https://hachyderm.io/tags/supercomputer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>supercomputer</span></a> blasts through one trillion parameter model with only 8 percent of its <a href="https://hachyderm.io/tags/MI250X" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MI250X</span></a> GPUs<br>The paper notes that the key challenge in training such a large <a href="https://hachyderm.io/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLM</span></a> is the amount of memory required, which was 14 terabytes at minimum. <a href="https://www.tomshardware.com/tech-industry/supercomputers/frontier-trained-a-chatgpt-sized-large-language-model-with-only-3000-of-its-37888-radeon-gpus-the-worlds-fastest-supercomputer-blasts-through-one-trillion-parameter-model-with-only-8-percent-of-its-mi250x-gpus" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">tomshardware.com/tech-industry</span><span class="invisible">/supercomputers/frontier-trained-a-chatgpt-sized-large-language-model-with-only-3000-of-its-37888-radeon-gpus-the-worlds-fastest-supercomputer-blasts-through-one-trillion-parameter-model-with-only-8-percent-of-its-mi250x-gpus</span></a></p>
HPC Guru<p>1 trillion parameter <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLM</span></a> run on 3000 of the 37,000 AMD Instinct <a href="https://mastodon.social/tags/MI250X" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MI250X</span></a> <a href="https://mastodon.social/tags/GPUs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GPUs</span></a> on the <a href="https://mastodon.social/tags/Frontier" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Frontier</span></a> <a href="https://mastodon.social/tags/supercomputer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>supercomputer</span></a> at ORNL<br> <br>This is beyond <a href="https://mastodon.social/tags/GPU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GPU</span></a>-rich 🙂</p><p><a href="https://arxiv.org/abs/2312.12705" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="">arxiv.org/abs/2312.12705</span><span class="invisible"></span></a></p><p><a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AI</span></a> <a href="https://mastodon.social/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a></p>
ExploreLive feeds
About