[ad_1]
As more publishers strike content licensing deals with OpenAI, the maker of ChatGPT, a He studies Put out this week by Tu Center for Digital Journalism – Considering how an AI-powered chatbot produces citations (i.e. sources) for publishers’ content – makes for interesting, or worrying, reading.
In short, the results suggest that publishers remain at the mercy of a generative AI tool’s tendency to invent or distort information, regardless of whether they allow OpenAI to crawl their content or not.
The research, conducted at Columbia Journalism School, examined citations produced by ChatGPT after it was asked to source sample citations taken from a mix of publishers — some of whom had signed deals with OpenAI, others who had not.
The Center obtained collective citations from 10 stories produced by a total of 20 randomly selected publishers (i.e. 200 different citations in total) – including content from The New York Times (which is currently suing OpenAI in a copyright claim); The Washington Post (not affiliated with the maker of ChatGPT); The Financial Times (which signed a licensing deal); And others.
“We selected citations that, if pasted into Google or Bing, would display the source article among the top three results and evaluated whether OpenAI’s new search tool would correctly identify which article was the source of each citation,” wrote Tow researchers Claudia Jazwinska and Aisvarya Chandrasekar. In a Blog post They explain their approach and summarize their findings.
“What we found was not promising for news publishers,” they continue. “Although OpenAI emphasizes its ability to provide users with timely answers with links to relevant web resources, the company makes no explicit commitment to ensuring the accuracy of those citations. This is a notable omission for publishers who expect their content to be referenced and represented faithfully.” .
“Our tests found that no publisher – regardless of their degree of affiliation with OpenAI – was spared inaccurate representations of their content in ChatGPT,” they added.
Unreliable sources
The researchers say they found “several” instances in which publishers’ content was inaccurately cited by ChatGPT — and they also found what they called a “spectrum of accuracy in the responses.” So, while they found “some” citations that were completely correct (i.e. ChatGPT accurately returned the publisher, date, and URL of the citation it was shared with), there were “many” citations that were completely wrong; And “some” fell somewhere in between.
In short, ChatGPT citations appear to be an unreliable mixed bag. The researchers also found very few cases in which the chatbot did not show complete confidence in its (incorrect) answers.
Some quotes were obtained from publishers who blocked OpenAI’s search crawlers. In those cases, the researchers say they expected there would be problems producing valid citations. But they found that this scenario raised another problem – that the bot “rarely” “admitted to being unable to provide an answer.” Instead, I resorted to simplification in order to establish some sources (albeit incorrect ones).
“In total, ChatGPT returned partially or completely incorrect answers on 153 occasions, although it only admitted to being unable to accurately respond to a query seven times,” the researchers said. “Only in these seven outputs did the chatbot use qualifying words and phrases such as ‘it seems’, ‘it is possible’, ‘it may be’ or phrases such as ‘I was unable to locate the exact article’.”
They compare this unhappy situation to a standard Internet search where a search engine like Google or Bing typically either sets an exact quote, directs the user to the website(s) where they found it, or states that it found no results with an exact match.
“ChatGPT’s lack of transparency about its trustworthiness of an answer can make it difficult for users to evaluate the veracity of a claim and understand which parts of the answer they can and cannot trust,” they say.
For publishers, there may also be reputational risks stemming from incorrect citations, they point out, as well as the commercial risk of directing readers elsewhere.
Data out of context
The study also highlights another issue. He points out that ChatGPT could essentially serve as a reward for plagiarism. The researchers recount a case in which ChatGPT incorrectly cited a website that plagiarized a piece of “deeply reported” New York Times journalism, that is, by copying and pasting text without attribution, as the source of a New York Times story — and speculated that, in this case, a bot may have created This incorrect response was to fill the information gap that resulted from his inability to crawl the New York Times website.
“This raises serious questions about OpenAI’s ability to filter and verify the quality and veracity of its data sources, especially when dealing with unlicensed or stolen content,” they point out.
In other findings likely to worry publishers who signed deals with OpenAI, the study found that ChatGPT citations weren’t always reliable in their cases either — so letting its crawlers in didn’t seem to guarantee accuracy either.
The researchers say the main issue is that OpenAI technology treats journalism as “decontextualized content,” with seemingly little regard for the circumstances of its original production.
Another issue the study points out is the variability of ChatGPT responses. The researchers tested asking the same query multiple times to the bot, and found that it “usually returned a different answer each time.” Although this is typical of GenAI tools, overall, in the context of citation, this inconsistency is clearly suboptimal if you are looking for accuracy.
While Tow’s study is small in scope — the researchers admit that “more rigorous” testing is needed — it is nonetheless notable given the high-profile deals that major publishers are busy striking with OpenAI.
If media companies hoped that these arrangements would lead to special treatment for their content compared to competitors, at least in terms of producing accurate sources, this study suggests that OpenAI has not yet provided any such consistency.
While publishers who do not have licensing deals but also Didn’t do that They blocked OpenAI’s crawlers entirely – perhaps in the hope of capturing at least some traffic when ChatGPT returns content about their stories – which makes for depressing reading as well, since the citations may not be accurate in their cases either.
In other words, there is no guaranteed “visibility” for publishers in the OpenAI search engine even when they allow its crawlers in.
A complete ban on crawlers does not mean that publishers can save themselves the risk of reputational damage by avoiding any mention of their stories in ChatGPT. The study found that the bot still incorrectly attributed articles to The New York Times despite the ongoing lawsuit, for example.
“An agency of little meaning”
The researchers concluded that as it stands, publishers have “little ability to influence” what happens to their content when ChatGPT gets its hands on it (directly or indirectly).
The blog post includes a response from OpenAI to the research findings — which accuses the researchers of conducting “unusual testing of our product.”
“We support publishers and creators by helping 250 million ChatGPT users weekly discover high-quality content through summaries, citations, clear links, and attribution,” OpenAI also told them, adding: “We’ve collaborated with partners to improve the accuracy of inline citations and respect publisher preferences, including… By enabling how they appear in search by managing OAI-SearchBot in their robots.txt file we will continue to improve search results.”
[ad_2]