Embeddings

My workstation is an AMD Strix Halo. It has an NPU, and the NPU can run an embedding model – embed-gemma-300m-FLM, served by Lemonade alongside the same model on the integrated GPU. Same weights, two runtimes. Moving embeddings to the NPU is more an experiment in using available hardware than anything else. I’ve experimented with Whisper transcription on the NPU successfully with (dicta)[]; it was acceptable, slower than the iGPU, slightly worse, but functional enough that I use it and the GPU stays free for more demanding tasks. This seemed like a natural follow up when I saw it in (Lemonade)[]’s list of NPU models.