After Google’s Gemini was publicly released, it took only twenty-four hours for someone to notice that chats were appearing in Google search results. Google quickly responded to this apparent leak, but the reason behind it is not as sinister as initially thought.

Gemini provides a way to create a publicly viewable version of a private chat through a link at the bottom of each chat. These chat pages were crawled and indexed by Google, despite the presence of a robots.txt file in the root of the Gemini subdomain. This file controls crawler activity on websites but did not prevent the indexing of chat pages.

Although the robots.txt file had been in place since February 8th, indicating that Google had taken measures to block web crawlers, it seems that public links to the chat pages may have caused them to be indexed. One such public link was discovered, suggesting that this was a likely cause.

Despite efforts to prevent indexing, both Google and Bing indexed pages from the Gemini subdomain. However, they later began dropping these pages from search results. This could be due to the low quality of the chat pages, making them unsuitable for displaying in search results.

In summary, while robots.txt can block crawler activity, it does not guarantee that content will not be discovered, crawled, or indexed. The dropping of Gemini chat pages from search results may be attributed to their low quality and lack of relevance for users.