Thanks to AI PCs, the specifications of all computers are likely to increase

key takeaways

  • AI PCs require adequate memory and storage for local processing.
  • Newer computers with specialized NPUs are necessary for AI features like Microsoft's Copilot.
  • 16GB of RAM is the minimum for an AI PC, and 32GB is recommended for those interested in AI features.

The era of the “AI PC” is upon us, and soon you'll have the option to run some AI features locally without an internet connection, even on relatively modest systems. However, what you may not know is that despite having AI accelerators, these computers will still require a lot of memory and storage to provide local AI features.

AI PC has local processing power

Microsoft's Copilot AI feature is one of the main reasons why a new generation of computers with specialized NPUs (neural processing units) are hitting the market. That's one reason why the new Surface Pro devices have ARM Snapdragon processors that are usually found in phones. Right now, the minimum requirement for a Copilot AI PC is an NPU with 40 TOPS (trillions of operations per second) and that's what's packed into these new chips.

Copilot AI logo on keyboard key.

There are many reasons to want local AI features. It's fast, you're not competing for resources with other users, and you don't have to pay beyond the cost of the hardware. It's certainly exciting to have your laptop running something as smart as ChatGPT and its contemporaries!

Even “small” LLMs are huge

The thing is that these AI “models” are huge. The smallest and most efficient LLMs (Large Language Models) can require 1-3GB of memory, even though they are not that smart. As the model gains parameters and becomes more capable, it needs more space. The super-smart LLM models you access via the web can take up hundreds of gigabytes of RAM when it comes to their size. With hundreds of billions or in some cases even over a trillion parameters, these AI models are not meant to be run on small laptops!

Then we have LLMs small enough to run on smartphones, like the phi-3-mini which has a capacity of 2.4GB. This is in line with what these AIs run on laptops, although perhaps not as weak!

Models need to live in RAM

AI chip hologram with circuits around it.
Lucas Gouveia/How-To Geek

AI chip hologram with circuits around it

While an AI model taking up a few GB of space may not seem like a big deal, you have to understand that the entire model needs to be in RAM for the technology to perform well. Those NPUs are specialized parallel processors that crunch the numbers on these virtual neural networks with billions of “parameters” at the same time.

If the model is not in RAM, the NPU cannot be fed fast enough to give you fast results. This will affect the speed of tasks like LLM answering, or real-time translation, or image generation. So if you want AI features to be available at the press of a button or by simply talking to your computer, the model essentially needs to reserve as much RAM as it needs.

Sure, all these new PCs have ultra-fast SSDs that can swap data in and out of RAM quickly, but aside from the extra wear and tear on your SSD, it’s still not fast enough to keep everything running smoothly, and don’t forget that you still want all your other apps running at the same time! If your PC has to dump your browser and other apps from RAM to disk every time you want the AI ​​to do something, it’s not going to be a great experience.

16GB is the absolute minimum

While 8GB of RAM is still perfectly fine for a modern office desktop or laptop used for general productivity and media consumption, Windows computers with AI features won't be able to function in such a small space. That's why Microsoft has made 16GB of RAM the absolute minimum for Copilot PCs.

While this is a decent amount for a normal non-AI PC, I imagine that for a PC that runs local AI assistants all the time in the background, this would be more like an 8GB PC plus AI, unless you disable the AI ​​part of the equation.

Some AI PCs won’t have upgradeable RAM

If your new AI Windows PC uses an integrated system-on-a-chip or has its RAM permanently soldered on, as is the case with many thin and light laptops, you're stuck with the amount of RAM that came with the system. That means you'll need only as much RAM as you need at the start, or you'll need a new computer if you got your calculations wrong.

A while back, I looked at whether 32GB of RAM is enough to aim for, and my conclusion was that this amount of memory is still way too much for average users. However, considering the size of future AI models, I think 32GB should be what anyone who cares about these AI features should aim for.

If you care about AI, get more memory

Whether or not you care about local AI features is, of course, the last thing that really matters here. If you don't, it's awesome that these Copilot PCs are pushing the minimum limit of RAM to 16GB. Just disable or ignore the AI ​​part of the equation and you'll be fine. In fact, unless you have a compelling alternative reason, skip these AI PCs altogether and stick with a more mainstream option.

On the other hand, if you're excited about the native AI features and see many ways to use these tools in the future, don't be satisfied with the minimum requirement unless you have the option to add more RAM later.

Leave a Comment

“The Untold Story: Yung Miami’s Response to Jimmy Butler’s Advances During an NBA Playoff Game” “Unveiling the Secrets: 15 Astonishing Facts About the PGA Championship”