this post was submitted on 19 Aug 2024
526 points (98.2% liked)
Technology
59300 readers
4481 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
That's technically true, but it's as misleading as saying they get their search results from Yandex. Their results are aggregated from several search engines, not just Bing. They also have their own web crawler, DuckDuckBot, which absolutely respects RobotRules.
Edit: I'm told my information is out of date. No more Yandex because of Uncle Sam. Yahoo is just Bing now, so that index doesn't count anymore. The bulk of the rest of their sources are largely inconsequential specialized search engines. Their sources page states that they "largely source from Bing".
Fair point.
Where is your evidence for that? It used to be Bing and Yandex, but now it's just Bing. They use other non search engine APIs and do a small amount of crawling AFAIK. Details of who uses what here: https://seirdy.one/posts/2021/03/10/search-engines-with-own-indexes/
I read it on their sources page at some point, but it looks like that page has changed since last I looked.
There's an article from this spring's 2600 magazine that claims it's all Bing results. I haven't dug through their Python code and I'm definitely no expert anyway, but I'd also prefer not to post an article from a small independent magazine like that to let the people on Lemmy who do know more than me take a look.