How much RAM do you think an on-device AI model gets to use before your Spotify stream stutters and your camera app dies in the background? Whatever number you just guessed, it’s probably generous. That’s the uncomfortable math behind a TechCrunch report on AI’s memory crunch coming for Android apps, and it’s the kind of story that gets buried under louder headlines about billion-dollar acquisitions.
I review AI tools for a living. Most of what crosses my desk is a wrapper around someone else’s API with a waitlist attached. On-device AI is different because it can’t fake it. Either the model fits in memory alongside everything else your phone is doing, or it doesn’t. There’s no growth-hacking your way past a hardware ceiling.
Why memory is the constraint nobody wants to talk about
The pitch for on-device inference is genuinely good. No round trip to a server. No per-token bill. Your data stays on your hardware. Anyone who has watched a cloud AI feature go from free to $20 a month understands the appeal of a model that runs locally and costs the developer nothing per call.
But phones are shared spaces. Android has to keep the OS alive, the browser tabs warm, the messaging app resident, the music playing. Every megabyte an AI model claims is a megabyte something else loses. On mid-range devices, which is most devices on the planet, that budget is tight before you add a language model to the mix.
This is why I’ve been skeptical of demo videos shot on flagship hardware. A model that behaves beautifully on a top-tier phone with 16GB of RAM tells you almost nothing about how it behaves on the device your actual users own. The demo is a marketing artifact. The memory ceiling is the product.
MacPaw’s move is the interesting one
Against that backdrop, MacPaw partnering with Liquid AI to offer on-device inference to developers building for its app store reads less like a feature announcement and more like a bet on where the constraint gets solved. Giving developers an on-device inference path as infrastructure rather than something each team builds alone is a sensible response to a hard problem. Whether it holds up under real workloads is a separate question, and I’d want to see it tested on modest hardware before saying anything stronger.
What I like about it is the honesty of the framing. Nobody is claiming the model is smarter than a frontier system in a data center. The claim is that it runs where you are. That’s a narrower promise, which makes it a more credible one.
The rest of the Android news tells the same story
Look at what else landed this week and a pattern shows up:
- Mesh, Automattic’s CRM, arriving on Android. Another app competing for the same finite memory pool.
- Google rolling out its age-assurance technology to Android developers worldwide. More platform-level machinery running underneath everything else.
- Reports that NVIDIA is about to buy Hugging Face, which would put the dominant AI hardware company and the dominant model distribution hub under one roof.
That last one deserves a beat of attention, and I’ll flag clearly that it’s currently a report rather than a confirmed deal. If it happens, the company selling the chips would also own the place developers go to find models. Every conversation about which model fits on which device would run through infrastructure owned by a single vendor. I’m not going to pretend I know how that plays out. I do know that vertical integration tends to be great for the integrator and complicated for everyone else.
What this means if you build things
The practical takeaway is unglamorous. On-device AI is an engineering discipline, not a feature you bolt on. Model size, quantization, when to load and when to release, what happens when Android decides your process is expendable — those decisions determine whether users experience your AI feature as magic or as the reason their phone got worse.
The tools that win here won’t be the ones with the most impressive benchmark scores. They’ll be the ones that behave predictably on a three-year-old mid-range phone with a browser open and music playing. That’s a boring thing to optimize for, which is exactly why most teams won’t.
My read is that the next year of mobile AI gets decided by memory management rather than model quality. Not a thrilling storyline. Probably the accurate one. I’ll be watching which developers ship features that survive contact with real devices, and which ones quietly pull them after the one-star reviews about battery drain start rolling in.
🕒 Published: