Run on <4gb vram

#86
by Hackerdada - opened

https://github.com/Jit-Roy/WeeLLM
You can run it now on <4gb Vram. Kaggle code is also attached with this repo.

Thanks for sharing WeeLLM! I’m interested in using it for film restoration on Apple Silicon.

Does WeeLLM currently support Apple M1–M5 Macs through Metal/MPS, or is the implementation CUDA-only?
If Apple support is planned, what are the main technical obstacles?

This implementation is CUDA only for now.
And I have never worked on Apple chips, So I have no idea regarding that.
I am just a student and have solely developed WeeLLM.

Sign up or log in to comment