RuOpinionNE-2024: Extraction of Opinion Tuples from Russian News Texts
Natalia Loukachevitch, Natalia Tkachenko, Anna Lapanitsyna, Mikhail Tikhomirov, Nicolay Rusnachenko
2025-04-10
Summary
This paper talks about a competition where teams used AI to pull out detailed opinions from Russian news articles, like figuring out who’s talking, what they’re talking about, and how they feel about it.
What's the problem?
News articles mix facts and opinions in complicated ways, making it hard for AI to separate who’s saying what about whom and whether it’s positive or negative.
What's the solution?
Researchers set up a challenge using AI models to break down sentences into opinion parts (like who’s speaking, who they’re talking about, and their feelings), with the best results coming from fine-tuning large AI models.
Why it matters?
This helps analyze news bias, track public opinion, and improve tools that automatically summarize or fact-check articles, especially in tricky cases where politics or propaganda are involved.
Abstract
In this paper, we introduce the Dialogue Evaluation shared task on extraction of structured opinions from Russian news texts. The task of the contest is to extract opinion tuples for a given sentence; the tuples are composed of a sentiment holder, its target, an expression and sentiment from the holder to the target. In total, the task received more than 100 submissions. The participants experimented mainly with large language models in zero-shot, few-shot and fine-tuning formats. The best result on the test set was obtained with fine-tuning of a large language model. We also compared 30 prompts and 11 open source language models with 3-32 billion parameters in the 1-shot and 10-shot settings and found the best models and prompts.