Last week, I decided to pull the trigger and buy a mini PC for local AI testing. While everyone has been rushing (or was recently at least) to buy older, second-hand M4 Mac Minis relatively cheaply, I chose a different route.
Firstly, the pricing of second-hand M4 Mac Minis with “only” 16GB RAM cannot be justified, in my opinion.
Secondly, I find it unacceptable to be locked into Apple’s ecosystem, which I genuinely don’t enjoy.
And finally, the Mac Mini isn’t upgradable. If you later realise that 16GB or even 32GB of RAM isn’t enough, you’re out of options.
I was working with a limited budget, so after some research, I decided to buy a mini PC powered by an AMD Ryzen 7 255H with a Radeon 780M iGPU from GMKtec (model K12).
My budget allowed me to source a new barebones GMKtec K12 on eBay for £250.
- 256GB M.2 NVMe drive: £27
- 2×16GB RAM: £150
(Both sourced second-hand on eBay)
Total: £427
(For comparison, M4 Mac Minis with 16GB RAM go for around £500 on eBay.)
The GMKtec comes with Windows 11 Pro, or you can install Linux, which is well supported on this machine. I went with Ubuntu (planning a dual-boot setup later).
First impressions are great. Boot time is a matter of seconds, and running Hermes Agent with Qwen3.6-35B-A3B (GGUF) is, I’d say, satisfactory. 22 tokens per second isn’t huge, but it’s enough to do some interesting things.
So far, I’ve mainly used it for Obsidian (notes and tasks), as well as basic tasks like checking news and weather forecasts.
Next, I want to connect it with n8n and see how well it handles automation.
What I also ended up with and what a friend called a “Steam Deck in a box” is a relatively capable gaming PC, at least for older titles.
Update
I quickly realised that 32GB RAM isn’t enough, so I ordered 2×48GB sticks from eBay. That brings the total to 96GB RAM.
Will it help massively? Probably not.
The real bottleneck here is the CPU and integrated GPU. However, the extra RAM should allow Hermes to run with memory compression enabled, which I previously had to disable.
Right now, the system struggles whenever it hits the context limit. A 32K context technically works, but once it gets there, everything slows down dramatically.
I don’t expect more RAM to fully solve this, but it’s worth testing how far I can push the K12. Cooling might also become an issue.
The Best Model for This Setup
There’s some hope on the horizon for those of us running local AI on “potato” machines.
The fastest model I’ve tested so far, which also handles tool calling well, is Qwen3.6-35B-A3B-UD-Q3_K_M.gguf from Unsloth.
It’s a sparse MoE model. While the full model is large, only a small portion is activated per token during inference. According to Qwen, it’s effectively 35B total / ~3B active.
I’m seeing speeds of around 22.5 tokens per second via llama.cpp.
So far, this is the fastest model that works reliably with Hermes Agent (from Nous Research) for tasks like checking weather, sending emails, etc. Smaller models like Qwen 3.5 4B or even 9B struggled with tool use and were slower in practice.
Conclusion
This experiment might end up being a bit of a failure. I may realise that this 1+ year-old hardware simply isn’t enough, and that using APIs for more powerful models is the more practical option.
That said, I still want to experiment further. Especially with OpenClaw, which might offer better performance for certain workflows.
I’m also curious to see what next-gen CPUs from AMD, Intel, and others will bring to local AI inference. I’ve heard promising things, although pricing is still a concern.
At this stage, I don’t need an AI agent for coding or business use (yet). I’m mainly trying to figure out how to save time in my daily routines.