Salah Eddine Bekhouche PRO

Thanks, good point. I checked the example again: all four models fit in dedicated VRAM (~15.8 of 24 GB after the image), auto_evict was off, and the chat speed did not change after the image. I added a note to the post with this and the shared-memory case. I also opened an issue for the spill: https://github.com/lemonade-sdk/lemonade/issues/4092

Thanks, good question. I tested it on Windows (RX 7900 XTX, Vulkan, 2026.45.3):

• Each model type has its own LRU slot, so an image request does not unload the chat model. A second LLM only evicts the first LLM.
• There is no global admission check before a load. If a load fails, Lemonade unloads all other models and retries once.
• There is also an opt-in global policy (auto_evict) that evicts idle models or the most "disposable" one under VRAM pressure. Pinning protects a model.

One thing I found while testing: on Windows the load does not fail when VRAM is full, the driver spills to shared memory. After an SDXL generation the chat model stayed loaded but dropped from ~103 to ~19 tok/s until I reloaded it. So "not cold" does not always mean "fast". In the four-model example everything fit, so this did not happen there.