Readers perceive automated news articles as having inferior word choice and numerical presentation compared to those written by humans.
News articles created through traditional methods appear to be easier to understand than those generated automatically. This conclusion comes from a recent study conducted by LMU, published in the journal Journalism: Theory, Practice, and Criticism.
The research group at the Department of Media and Communication (IfKW) evaluated over 3,000 online news readers in the UK. Each participant reviewed one out of 24 articles, split evenly between those generated using automation and those crafted by journalists. “In general, readers found the 12 automated pieces to be considerably less clear,” notes lead researcher Sina Thäsler-Kordonouri. This finding remained true even though journalists had edited the AI-generated articles before they went live.
Issues with numbers and vocabulary
The survey revealed that one of the main reasons for readers’ dissatisfaction was the choice of words in the AI-generated texts. Readers expressed concerns that these articles included overly complex, inappropriate, or unusual terminology. Additionally, there was a notable decline in satisfaction regarding how the automated pieces dealt with numbers and statistical information.
The shortcomings identified by readers in how automated articles managed numbers and vocabulary partly account for the greater difficulty in understanding them, according to the study’s authors. Nonetheless, readers rated both types of articles equally in terms of the overall ‘character’ of writing and the coherence of narrative structure.
Increased human involvement needed
Professor Neil Thurman, the project’s lead, recommends that “when developing or editing automated news articles, it is crucial for journalists and technology specialists to minimize the use of numbers, clarify challenging words, and incorporate more descriptive language that helps readers visualize the story.”
This research is the first to assess the differences in comprehensibility between automated and manual news articles while investigating the reasons behind these differences. “Our findings highlight the necessity of ensuring human participation in the automated generation of data-driven news content, as well as the need to enhance it,” states Sina Thäsler-Kordonouri.
