Español
English
Français
Português
Deutsch
Italiano
The generative AI revolution has brought with it a pressing need for specialized hardware capable of handling the intensive demands of running and training AI models locally. While standard PCs might seem like a viable option at first glance, they quickly reveal their limitations when confronted with the sheer scale of modern AI models.
A typical PC or laptop struggles to even load large language models (LLMs) exceeding 13 billion parameters. This pales in comparison to the frontier models, which are rumored to boast over a trillion parameters. Even high-end workstation PCs find it challenging to effectively manage LLMs with more than 70 billion parameters. This performance gap highlights the need for purpose-built AI workstations.
One company stepping up to address this challenge is Tenstorrent with their QuietBox 2. At first glance, the QuietBox 2 resembles a standard PC workstation. However, beneath the surface lies a completely different beast. The QuietBox 2 incorporates four of Tenstorrent’s proprietary Blackhole AI accelerators. These accelerators, coupled with 128 GB of GDDR6 memory (a specialized memory type commonly found in GPUs), and 256 GB of DDR5 system memory, deliver a combined total of 384 GB of memory. This substantial memory capacity allows the QuietBox 2 to load substantial models such as OpenAI’s GPT-OSS-120B.
Beyond just loading models, the QuietBox 2 is designed for efficient model execution. It can run mid-sized models like Meta’s Llama 3.1 70B at impressive speeds, reportedly achieving nearly 500 tokens per second. To put that into perspective, this speed is significantly faster than what most standard PCs can achieve, enabling a much smoother and more responsive AI development experience.
The emergence of AI workstations like the QuietBox 2 signifies a crucial step forward in making AI development and deployment more accessible. By providing the necessary hardware to run these models locally, it empowers researchers, developers, and businesses to explore the full potential of generative AI without relying solely on cloud-based solutions. As AI models continue to grow in complexity and size, expect to see even more innovation in the AI workstation space, pushing the boundaries of what's possible with local AI processing. The future of AI development may well be on your desktop.