If you’ve used ChatGPT Search or Perplexity, you know that being able to search the web and see citations inline greatly improves these AI chatbots. Results are better when they involve timely information, and web search may reduce so-called hallucinations (i.e. when a generative AI outputs incorrect information).
That’s why French startup Linkup is building an API that lets developers access web content from premium, trusted sources and hand the results to a large language model (LLM) to enrich its answers. Many AI developers call this workflow Retrieval-Augmented Generation (or RAG).
More importantly, the future of scraping bots is uncertain. If there’s no pre-existing financial agreement between content publishers and the entities scraping web pages, these bots are lifting content from the open web without paying, and many people aren’t happy about that deal — which
Continue Reading on TechCrunch
This preview shows approximately 15% of the article. Read the full story on the publisher's website to support quality journalism.