this post was submitted on 19 Jan 2025
424 points (97.7% liked)
Technology
61456 readers
4218 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This is dumb. Moore's law may be mostly dead, but chips are still progressing at an absurd pace. In 6 years you'll be able to run the o1 model on a raspberry Pi with no internet access.
Maybe, but i never mentioned years into the future. Of course technology will improve. The hardware will get better and more effcient, and the algorithms and techniques will improve.
But as it stands now, i still think what i said is true. We obviously don’t have exact numbers, so i can only speculate.
Having lots of memory is a big part of inference, so I was going to reply to you that prices of memory stopped going down at a similar historical rate, but i found this, which is interesting
https://ourworldindata.org/grapher/historical-cost-of-computer-memory-and-storage?time=2020..latest
The cost when down by about 0.1x from 2000 to 2010. 2010-2020 it was only about 0.23x. 2020-2023 shows roughly another halving of the price, which is still a pretty good rate.
The available memory is still only one part. The speed of the memory and the compute connected to it also plays a big part in how these current systems work.
Nvidias latest gen looks to be 30% faster after 2 years of development with about the same power usage increase. So no reduction in Joules per GOP, just a speed increase.
In 6 years they might go 2x the speed of today but need double the watts (to deliver the same energy in half the time).
lmaoooo