this post was submitted on 07 Aug 2024
826 points (96.7% liked)
Technology
59342 readers
5211 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
From what I've heard the general idea is to run AI search on your browsing history, which is a very useful feature. I'm not deep into AI tech at all but to me it looks like that would involve local finetuning, ingesting all that history during inference sounds like a bad idea. It also wouldn't be necessary to generate stuff, only answer "Can you find that article about how nature makes blue feathers" and it's going to spit out previously-read links that match that kind of thing. Also, tl;dr-bot it.
Oh and there's already AI, as in ML, in firefox, in the form of machine translation. Language detection seems to be built-in, translating requires downloading a model per language pair, 16M parameters. Trained on workstations with 8GPUs. Which is all to say: You don't need gigantic GPU farms if you aren't training gazillion parameter models on the whole internet.
It shoudn't be finetuning, if anything it should be RAG with an embeddings model + regular inference.
This is kinda cool, but it still doesn't seem to justify bogging down a machine with a huge LLM. And I am speaking as a massive local LLM enthusiast who uses them every day.