I Tried Local AI on HP’s OmniBook Ultra: A Private, Powerful Beast With Zero Token Fees

Every time you send a prompt to ChatGPT, Claude, or Copilot, your data travels across the public internet to a massive data center owned by a tech conglomerate. For most casual searches, this arrangement feels harmless. However, when you deal with confidential business documents, proprietary source code, or sensitive personal data, sending raw text to the cloud introduces genuine privacy risks. On top of that, power users and software developers quickly discover that cloud API tokens, monthly subscriptions, and usage tiers add up to a heavy recurring expense.

Enter the era of local AI. Instead of renting compute cycles from external servers, your personal laptop handles the math directly on its own silicon. The HP OmniBook Ultra, powered by AMD’s next-generation Ryzen AI 300 series processors, promises to transform how we think about on-device machine learning. I spent considerable time testing this machine, pushing open-weight language models, coding assistants, and local image generators through their paces. Here is an in-depth breakdown of what it feels like to run local AI on a modern, ultra-portable laptop without paying a single penny in token fees.

Why Run AI Locally? The Shift Away from the Cloud

The tech industry spent the last decade convincing everyone that cloud computing was the answer to every problem. While cloud infrastructure offers immense scale, generative AI has exposed several major weaknesses in the purely cloud-dependent model.

The first pain point is privacy. When an organization feeds internal financial records or legal contracts into a third-party model, they risk data leaks or inadvertent training on proprietary intellectual property. Many enterprise teams have completely banned cloud AI services for this reason. Running an open-source model locally on your own hardware means the data never leaves your physical device. There are no tracking logs, no remote telemetry, and no third parties storing your prompt history.

The second pain point is cost. Modern API access for models like GPT-4o or Claude 3.5 Sonnet is billed per million tokens. For light use, it seems cheap. For developers processing gigabytes of text, testing automated agent workflows, or running constant automated code reviews, token fees can easily balloon into hundreds or thousands of dollars per month. A machine capable of running these tasks locally acts as a fixed-cost investment. Once you buy the laptop, your inference cost is essentially just the electricity it pulls from the wall.

Finally, there is latency and connectivity. Cloud services experience outages, rate limits, and network lag. When you run a quantized model locally, you can work on an airplane, in a remote cabin, or during a total broadband outage. The response starts generating immediately without waiting in a cloud server queue.

Inside the HP OmniBook Ultra: The Hardware Powering the AI

To run large language models on an ultra-thin laptop, you need specialized hardware. The HP OmniBook Ultra relies on the AMD Ryzen AI 9 HX 375 processor, built on AMD’s Zen 5 architecture. This chip represents a massive leap forward for mobile computing, combining a capable central processing unit, an advanced integrated graphics card, and a dedicated Neural Processing Unit.

Here is what makes this hardware setup unique for local machine learning:

• The AMD XDNA 2 NPU: The dedicated neural processing unit delivers up to 55 TOPS (Trillion Operations Per Second) of INT8 performance. This allows background AI tasks, audio cleanup, camera framing, and specialized model operations to execute with minimal battery impact.
• Radeon 890M Graphics (RDNA 3.5): For running large language models via unified memory, the integrated GPU is the real workhorse. It features 16 compute units that handle matrix multiplication much faster than a standard CPU.
• Fast LPDDR5X Memory: Local AI performance lives and dies by memory bandwidth. The OmniBook Ultra is configured with 32GB of high-speed LPDDR5X RAM running at 7500 MT/s. Because the CPU and GPU share this memory pool, you can allocate a large portion directly to VRAM, allowing 8-billion and 14-billion parameter models to load completely into memory.
• Thermal Management: Despite measuring less than an inch thick, HP engineered a dual-fan cooling system that keeps the chip from aggressive thermal throttling during long inference runs.

Setting Up Local AI on the OmniBook Ultra

Setting up local AI used to require deep Linux expertise, complex Python environments, and painful CUDA driver configurations. Today, the open-source ecosystem has evolved to the point where anyone can download and run an AI model in under five minutes.

To evaluate the HP OmniBook Ultra, I used three primary software environments:

1. LM Studio: A polished, user-friendly desktop application that allows you to search, download, and run quantized GGUF models directly from Hugging Face. LM Studio leverages Vulkan and DirectML backends to offload computations to the Radeon integrated GPU.
2. Ollama: A lightweight command-line tool designed for developers. It makes running models feel as simple as running Docker containers. Ollama is ideal for hooking local models into development environments, code editors, and internal API wrappers.
3. HP AI Companion: HP’s built-in proprietary software suite designed to leverage the on-device NPU for document analysis, system optimization, and basic knowledge retrieval without requiring complex setup.

Setting up LM Studio on the OmniBook Ultra was remarkably smooth. Within minutes, I downloaded popular open-weight models including Meta’s Llama 3.1 8B, Mistral 7B Instruct, Microsoft’s Phi-3.5 Mini, and Qwen 2.5 7B. The software automatically identified the AMD hardware, allocated memory layers to the GPU, and began processing prompts instantly.

Real-World Testing: How Fast Does It Actually Run?

When running local language models, performance is measured in tokens per second (t/s). A reading speed of roughly 5 to 7 tokens per second matches normal human speech, while 15 to 20 tokens per second feels brisk and comfortable for interactive reading. Anything above 25 tokens per second feels lightning fast.

I tested multiple model families and quantization levels on the HP OmniBook Ultra. Quantization compresses the model weights (usually from 16-bit floating points down to 4-bit or 8-bit integers) so they consume less memory and process faster with negligible loss in reasoning ability.

1. Everyday Text Generation: Llama 3.1 8B (Q4_K_M)

Meta’s Llama 3.1 8B is currently the gold standard for compact, general-purpose intelligence. Running at 4-bit quantization, the model fits neatly into about 5GB of memory. On the OmniBook Ultra, text generation averaged between 18 and 22 tokens per second. Long-form article drafting, creative writing, and email generation felt completely fluid. The words appeared faster than I could read them, delivering an experience indistinguishable from a premium cloud subscription.

2. Complex Reasoning: Qwen 2.5 14B (Q4_K_M)

Pushing the hardware harder, I loaded Alibaba’s Qwen 2.5 14B model. A 14-billion parameter model requires around 9GB of memory and places significantly higher demands on memory bandwidth. The OmniBook Ultra handled it with surprising composure, averaging between 10 and 13 tokens per second. While generation was noticeably slower than the 8B model, the answers showed far deeper logical reasoning and superior handling of multi-step problem solving.

3. Coding Assistance: DeepSeek-Coder 6.7B

For programmers, running a private coding assistant is one of the most compelling reasons to own an AI laptop. I connected Ollama to Visual Studio Code using the Continue extension, pointing it at a locally hosted DeepSeek-Coder model. Auto-completion, code refactoring, and bug detection functioned entirely offline. The system provided code explanations at 24 tokens per second. Most importantly, proprietary codebases never left the internal NVMe drive, keeping company secrets strictly secure.

4. Document Analysis and Summarization

One of the best tests of local inference is feeding large, messy PDF files into the model context window. I loaded a 65-page technical documentation PDF into LM Studio and asked the model to extract specific security compliance criteria. Thanks to the expanded context handling of modern models, the OmniBook Ultra digested the entire document, processed the prompt evaluation phase in roughly 12 seconds, and produced an accurate, structured summary without hallucinating key details.

Local AI vs. Cloud AI: A Realistic Comparison

To put this machine in perspective, it helps to compare the experience of running local AI on the HP OmniBook Ultra directly against using paid cloud platforms like ChatGPT Plus or Anthropic Claude Pro.

Battery Life, Heat, and Fan Noise

Running intensive matrix calculations locally is one of the most demanding tasks you can assign to a laptop. Under normal web browsing and word processing, the HP OmniBook Ultra runs whisper-quiet, with battery life easily stretching past 14 hours thanks to the efficiency of the Zen 5 processor cores.

However, when you push the GPU and NPU with sustained local AI generation, the physics change:

Power Consumption: Sustained inference pulls between 28 and 35 watts from the battery. If you were to run continuous model generation non-stop, the battery would drain in roughly three to four hours. In real-world usage, however, AI interaction is bursty. You ask a question, read the answer for a minute, and ask another. Under realistic intermittent usage, the OmniBook Ultra easily delivers 7 to 9 hours of productive AI-assisted work away from a wall outlet.

Fan Acoustics and Heat: The dual fans ramp up smoothly rather than whining abruptly. Even under sustained 10-minute generation loops, the noise remains a low, gentle whoosh that easily blends into an office background. The aluminum chassis gets warm near the top hinge, but the palm rests and keyboard remain comfortable to touch.

The HP AI Companion: Useful Tool or Bloatware?

HP includes its proprietary software called the HP AI Companion on this laptop. Usually, pre-installed software from laptop manufacturers feels like unwanted bloatware. In this case, HP did a respectable job creating a genuinely functional utility.

The companion app offers two main tabs:

• Discover: This section acts as a system utility that monitors your hardware performance, optimizes power profiles, and provides tips on using generative tools.
• Ask: This is a local chat interface that leverages local processing. You can drop personal folders, text documents, or spreadsheets into the app. It indexes your documents locally using vector embeddings and lets you chat with your documents using on-device models.

While advanced developers will still prefer open-source power tools like LM Studio or AnythingLLM, HP’s AI Companion provides an approachable, clean entry point for business users who want immediate document analysis without understanding what a GGUF file or quantization parameter is.

Where Local AI Still Struggles

While the HP OmniBook Ultra proves that local AI is practical today, it is essential to remain realistic about the current boundaries of edge computing. A thin-and-light laptop cannot replace a $300,000 server rack packed with enterprise GPUs.

The most obvious limitation is model parameter capacity. The OmniBook Ultra handles 7B and 8B models effortlessly, and it runs 14B models respectably well. However, attempting to load a 70-billion parameter model like Llama 3.1 70B causes the system to run out of memory. If a model cannot fit entirely inside the 32GB memory pool alongside the Windows operating system, it spills over into swap storage on the SSD, slowing generation down to an unusable crawl of less than one token per second.

The second limitation is breadth of general knowledge. Compact 8B models are exceptionally smart at summarization, pattern recognition, formatting, and coding. Yet, they lack the encyclopedic fact-retrieval depth found in cloud giants like GPT-4o. If you ask niche trivia questions or obscure historical facts, smaller models are significantly more prone to subtle hallucinations unless grounded with local retrieval documents.

Who Is the HP OmniBook Ultra For?

The HP OmniBook Ultra is not just another standard laptop refresh. It is targeted at specific users who will find immense value in its unique hardware strengths:

• Software Developers: Programmers who want private code autocompletion and code generation without leaking proprietary repositories to cloud services.
• Legal, Medical, and Financial Professionals: Anyone handling sensitive records, private transcripts, or client confidentiality where cloud AI uploads are legally prohibited.
• Frequent Travelers and Digital Nomads: Professionals who work on trains, planes, or in locations with spotty, unreliable internet access and need uninterrupted access to generative tools.
• Tech Enthusiasts and AI Tinkerers: Users who dislike subscriptions, want full control over their models, and enjoy tweaking system prompts, temperatures, and quantizations on bare metal.

The Verdict: A Glimpse into the Future of Personal Computing

Testing local AI on the HP OmniBook Ultra makes one thing clear: the future of personal computing is hybrid. For massive research queries requiring sprawling web searches, cloud models will always have their place. But for everyday writing, coding, summarization, and private document processing, running AI locally on your laptop is faster, cheaper, and fundamentally more secure.

The OmniBook Ultra delivers on the promise of the next-generation AI PC. It takes edge machine learning out of the realm of bulky, screaming desktop gaming rigs and packs it into a slim, premium chassis that you can carry in a messenger bag. If you want to break free from monthly AI subscriptions, keep your sensitive data entirely in your own hands, and experience the raw speed of dedicated silicon, the HP OmniBook Ultra is a private, capable powerhouse that proves local AI has finally arrived.