Wired: ‘Perplexity Is a Bullshit Machine’
Via The Verge, a well-researched article on what exactly Perplexity’s system is doing when it ignores robots.txt
and doesn’t announce itself while scraping content it’s not supposed to see. See also, Robb Knight’s detailed exploration of how they work.
In my most generous mind, I can see where someone providing a service that leverages several different means of gathering information might see an LLM and web scraping as distinct activities. That said, the opacity and move-fast-break-things nature of most AI startups have already left a bad taste in people’s mouths, and this is just another case of it.