LocalLLaMA

2585 readers

7 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago

MODERATORS

submitted 1 week ago by Disonantezko@lemmy.sdf.org to c/localllama@sh.itjust.works

6 comments fedilink hide all child comments

I have an GTX 1660 Super (6 GB)

Right now I have ollama with:

Do you recommend any other local models to play with my GPU?

you are viewing a single comment's thread
view the rest of the comments

[–] possiblylinux127@lemmy.zip 2 points 1 week ago (1 children)

Mistral

I personally run models on my laptop. I have 48 GB of ram and a i5-12500U. It runs a little slow but usable

[–] Disonantezko@lemmy.sdf.org 2 points 1 week ago (1 children)

My gear is an old:

I7-4790 16GB RAM

How many tokens by second?

[–] possiblylinux127@lemmy.zip 1 points 1 week ago

The biggest bottleneck is going to be memory. I would just stick with GPU only since your GPU memory has the most bandwidth.