Repository navigation
Will there be exmperimental 4gb VRAM support with smaller model quants? #160
Marczellito
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I know i'm asking for a lot, don't treat it as a request feature, but i'm wondering - are there any plans for support of graphics cards with VRAM smaller than 8 GB? That would probably also require smaller quants of models, as even LFM2.5-24B-A2B takes about 4 GB of VRAM even before any expert.
Than you for your work!
All reactions