this post was submitted on 18 Jul 2024
479 points (98.4% liked)

Memes

45595 readers
1155 users here now

Rules:

  1. Be civil and nice.
  2. Try not to excessively repost, as a rule of thumb, wait at least 2 months to do it if you have to.

founded 5 years ago
MODERATORS
gary_host_laptop@lemmy.ml
sexy_peach@feddit.de
cyclohexane@lemmy.ml
cypherpunks@lemmy.ml
479
Offline version of Chat GPT (lemmy.ml)
submitted 3 months ago by lidd1ejimmy@lemmy.ml to c/memes@lemmy.ml
27 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] ignotum@lemmy.world 8 points 3 months ago (3 children)

70b model taking 1.5GB? So 0.02 bit per parameter?

Are you sure you're not thinking of a heavily quantised and compressed 7b model or something? Ollama llama3 70b is 40GB from what i can find, that's a lot of DVDs

[–] PixelatedSaturn@lemmy.world 9 points 3 months ago

Ah yes probably the Smaler version, your right. Still, a very good llm better than gpt 3

[–] 9point6@lemmy.world 7 points 3 months ago

Less than half of a BDXL though! The dream still breathes

[–] Steve@startrek.website 5 points 3 months ago

For some reason, triple layer writable blu-ray exists. 100GB each

https://www.verbatim.com/prod/optical-media/blu-ray/bd-r-xl-tl/bd-r-xl-tl/