How Generative AI distorts human sentiment

Using Generative AI and Large Language Models (LLMs) can alter the sentiment of the original text, making the outcome unreliable, researchers have found.

The paper, Who’s Speaking, Machine or Man? How Generative AI Distorts Human Sentiment, has been published by leading academics at the Gillmore Centre for Financial Technology at Warwick Business School.

They conducted a comprehensive analysis of 50,000 tweets, utilising ChatGPT to rephrase the original text.

Employing the Valence Aware Dictionary for Sentiment Reasoning (VADER) methodology to compare the original tweets with their GPT-4 rephrased counterparts, the researchers discovered that LLMs changed the linguistic features of the text.

In most cases, they shifted the sentiment towards increased neutrality, effectively transitioning the text away from both positive and negative orientations.

Ashkan Eshghi, Houlden Fellow at the Gillmore Centre for Financial Technology, said: “Our findings reveal a notable shift towards neutral sentiment in LLM-rephrased content compared to the original human-generated text. This shift affects both positive and negative sentiments, ultimately reducing the variation in content sentiment.

“While LLMs do tend to move positive sentiments closer to neutrality, the shift in negative sentiments towards a neutral position is more pronounced. This overall shift towards positivity can significantly impact the application of LLMs in sentiment analysis.”

Ram Gopal, Director of the Gillmore Centre for Financial Technology, said: “Extensive literature on the multifaceted use of user-generated content already exists, ranging from predicting stock prices to evaluating service quality, but we have found that the substantial use of LLMs introduces a significant concern: potential bias.

“This bias arises from the application of LLMs for tasks such as paraphrasing, rewriting, and even content creation, resulting in sentiments that may diverge from those the individual would have expressed without LLMs being used.”

The research by the academic team at the Gillmore Centre  - launched thanks to a £3 million donation from Clive Gillmore, founder and Group CEO of Mondrian Investment Partners and an alumnus of WBS, to look at the impact of emerging technologies such as artificial intelligence on financial activities - goes on to propose a mitigation method aimed at reducing bias and enhancing the reliability of user-generated content.

This involves predicting or estimating the sentiment of original tweets by analysing the sentiments of their rephrased counterparts.

However, further investigation is needed to see if other linguistic features would change if AI were used, such as emotion, sentence structure, or ratio of specific words in a sentence.

Yi Ding, Assistant Professor of Information Systems at the Gillmore Centre, said: “We have seen that there are around 180 million OpenAI monthly active users worldwide, and more businesses are jumping aboard the AI hype train, harnessing its usage as a business tool.

“Conducting this study looking at the use of Generative AI alongside human sentiment will play a critical role in LLM future developments, ultimately enhancing output, helping remove biases and improving efficiency for anyone who uses it.”

The academics plan to employ other predictive models to infer the authentic human sentiments and propose further mitigation approaches in subsequent work.

Ram Gopal is Professor of Information Systems Management and Director of the Gillmore Centre for Financial Technology. He lectures on Digital Transformation on the Executive MBA and Global Online MBA.

Ashkan Eshghi is a Houlden Fellow and teaches Artificial Intelligence for Business on the Undergraduate programme.

Yi Ding is Assistant Professor of Information Systems and teaches on the Warwick Executive Diploma in Digital Leadership.

Learn more about how to innovate with new technology in the short Executive Education course Business Impacts of Artificial Intelligence.

For more articles on Finance and Markets sign up to the Core Insights newsletter here.