A lawyer records an hour with a client. A clinic holds years of notes, images and consultation audio. A bank keeps its customer-service calls. Searching all of that, or turning it into text, is work AI does well, and many of those organizations cannot hand it to an outside vendor because their policy says the data does not leave.

That rule had a price. They settled for lower quality. Two October releases change the trade, with limits worth reading.

Two pieces that fit on your machine

First, EmbeddingGemma 2, released by Google on October 6. It has 740 million parameters under Apache 2.0, built in modules: 270 million for text and code, 170 million for images and video, 300 million for audio. Everything lands in one shared vector space, so a query like “chest X-ray, shadow in the lower lobe” can pull back the image, the note and the audio clip that discuss the same thing.

Google’s memory figures, taken on a Pixel 11 Pro, are about 191 MB with text only and about 567 MB for the full model. That is a phone. The question moved from “do I need a server?” to “which pocket?”

Second, Parakeet Ultra from Moondream, an open 600-million-parameter model that transcribes 25 languages. On the FLEURS set it cuts word error from 11.62% to 9.55% compared with the NVIDIA model it was trained from.

Running on top of it, Nigel Cannings reported an hour of audio transcribed in 0.73 seconds on an NVIDIA DGX Spark. That is roughly 4,903 times real time, at the same accuracy. It is his measurement, with code he optimized, and I have not reproduced it. Numbers like this are often taken with many requests in flight, so it does not mean one of your recordings comes back in under a second.

What changes and what does not

What changes is that the data can stay on your network. Recordings, case files and contracts get indexed and transcribed on a machine you control.

What does not change is that regulatory compliance is its own question. Owning the model does not put you in good standing by itself. And four gaps remain.

Gap	Why it matters
Something to write the answer	EmbeddingGemma 2 does not generate text. It finds the document and leaves the summary to another model, which also has to run locally
Protecting the vectors	A study presented at EMNLP 2023 exactly recovered 92% of 32-token texts from their vectors, including full names in clinical notes
Testing on your data	Google reports strong multilingual performance in general. Your clinical or legal vocabulary is a separate test
Permissions and logs	Who queried what, and what got recorded

On the first row, the writing model is the heavy part. I covered that in the Colibrì post, where a giant model ran on a laptop at one token every 10 to 20 seconds, and in the Alex Finn one on why RAM was not the bottleneck. For drafting over sensitive data, the realistic choice today is a mid-size model. Searching and transcribing no longer needs a big one.

The second row is the one I think people underrate. Plenty of teams treat the vector database as a harmless index. Morris, Kuleshov, Shmatikov and Rush show it is not. Give it the same encryption and the same access rules as the documents themselves.

Where to start

Do not buy hardware first. Take a handful of real documents, already anonymized, and an hour of test audio. Run both pieces on a single machine and watch three things: whether search finds what your operations people know it should find, how many words of the transcript need hand correction, and how long it really takes.

That kind of test is part of what gets settled in AI Maestro for law firms, and the same logic holds for clinics and financial services. Stage 1 runs on the tools the company’s policy already approved, even if that policy bans cloud models, so nobody has to buy anything new to find out whether this works.

Try local search on your own documents