Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
A new report claims that ChatGPT Search can be manipulated with hidden text that contains instructions that tell ChatGPT Search how to respond to an answer Tests also showed that ChatGPT can be manipulated without instructions, just with hidden text.
ChatGPT search can be manipulated with hidden text
A report by The Guardian describes how they used hidden text on a fake website to trick ChatGPT Search into showing them the website’s hidden text answer. Text is hidden when the font matches the background color of the page, such as a white font on a white background.
They then asked ChatGPT Search to visit the website and answer a question based on the text on the page. ChatGPT Search crawled the page, indexed the hidden content and used it in the response.
They first evaluated ChatGPT using a non-exploitative control page on a fake review website to test ChatGPT’s response. He read the reviews and returned a normal response.
The Guardian researchers then sent ChatGPT Search to a fake website that was instructed to give a positive review, and ChatGPT Search followed the instructions and returned positive reviews.
The researchers did a third test with positive reviews written in hidden text but no instructions, and ChatGPT Search again returned positive reviews.
Like The Guardian explained it:
“…when the hidden text included instructions for ChatGPT to return a favorable review, the response was always overwhelmingly positive. This was the case even when the site had negative reviews – the hidden text could be used to override the actual review score.
The simple inclusion of hidden text by third parties without instructions can also be used to ensure a positive rating, with one test involving extremely positive fake reviews that affected the summary returned by ChatGPT.”
The above test is similar test of ChatGPT that university computer science professor did in March 2023 where he tricked ChatGPT into saying he was a time travel expert.
What these tests prove is that ChatGPT’s training data and ChatGPT Search Bot input hidden text, but they can also be manipulated in the following instructions. The Guardian quotes a security expert as saying that OpenAI was aware of the exploit and that it could be fixed by the time the article is published.
Why can AI search engines be manipulated?
One loophole in AI search is a technology called RAG (Retrieval Augmented Generation), a technique that can retrieve information from a search engine so that AI can use it to generate answers to questions from up-to-date and (presumably) authoritative sources. How do AI search engines determine credible websites? Perplexity AI, for example, uses a modified version of PageRank to identify trustworthy websites to cite in its AI search engine.
ChatGPT Search is based on Bing, but also has its own indexing tool that can retrieve information in real time. It’s probably not unreasonable to guess that if a website is included in Bing’s search index then it’s probably included in ChatGPT Search, which should protect ChatGPT Search from being affected by hidden text. Pages with hidden text would likely be excluded from Bing’s search index. Additionally, it may be possible to mask a website to show different content to the ChatGPT Search Bot (up-to-date list of OpenAI Search Crawler bots is available here).
Other ways to manipulate AI search engines
That said, there are other ways that researchers discovered last year that may still be effective (Read: Researchers discover how SEO for AI search). In this research paper last year, researchers tested nine influencing strategies AI search engines:
Nine Strategies for Manipulating AI Search Engines
- Authoritative: Changing the writing style to be more persuasive in authoritative claims
- Keyword Optimization: Adding more keywords from search queries
- Adding statistics: Modifying existing content to include statistics instead of interpretive information.
- Cite sources (citing reliable sources)
- Adding Citations: Adding quotes and citations from high-quality sources
- Easy to understand: Makes content easier to understand
- Fluency Optimization works to make content more articulate
- Unique words: Adding words that are less used, are rare and unique, but without changing the meaning of the content
- Technical terms: This strategy adds both unique and technical terms wherever it makes sense and without changing the meaning of the content
The researchers found that the first three strategies were the best. In particular, adding keywords to web pages helped a lot.
Can ChatGPT search be manipulated?
I happened to hear claims made at a recent search conference that Google AI reviews can be manipulated to show products from certain big brands in response to search queries. I have not verified that this is true, but the claim was made by a reliable and authoritative source. Regarding ChatGPT search, I’ve noticed some interesting things about which sites it chooses to publish information and under what circumstances, which could be a way to influence rankings. So it’s no surprise that there are ranking holes in ChatGPT search. AI Search looks a lot like the early days of traditional search.
Featured Image Shutterstock/Antonello Marangi