{"id":107046,"date":"2024-12-15T12:00:00","date_gmt":"2024-12-15T12:00:00","guid":{"rendered":"https:\/\/industry-science.com\/?post_type=article&p=107046"},"modified":"2025-02-04T12:28:51","modified_gmt":"2025-02-04T11:28:51","slug":"social-media-language-models","status":"publish","type":"article","link":"https:\/\/industry-science.com\/en\/articles\/social-media-language-models\/","title":{"rendered":"Improving Social Media Moderation with Generative Language Models"},"content":{"rendered":"\n

The threat of disinformation in the digital age<\/h2>\n\n\n\n

In today’s digital age<\/a>, society faces new challenges posed by generative language models<\/a> and the global spread of disinformation on social media. The latter not only influences public opinion, but also poses a threat to democratic processes. Former Google CEO Eric Schmidt aptly remarked that the internet is the greatest anarchist experiment in history and the first that humanity has constructed but does not understand [1]. This quote illustrates the immense complexity and unpredictability of the internet, which has  revolutionized the exchange of information while simultaneously providing a platform for disinformation and hate speech.<\/p>\n\n\n\n

A significant example of the current challenges facing the global community is the choice of the term “post-truth” as the 2016 word of the year. In a post-factual era, where facts and logic often take a back seat to emotional and personal beliefs, objective facts have less influence on public opinion than ever before [2]. This development underlines the urgent need to develop and implement effective strategies to combat disinformation.<\/p>\n\n\n\n

Why disinformation must be tackled<\/h2>\n\n\n\n

The targeted spread of disinformation is a serious challenge that goes beyond simply bad journalism or unintentional misinformation. Disinformation is often deliberately used to strengthen political power or promote financial interests [3]. Various studies also show that disinformation and hate speech are purposely used to intensify social divisions.<\/p>\n\n\n\n

One example of this is the rise of hate crime against refugees and migrants, which is largely motivated by  the spread of disinformation about these groups on social media. At worst, disinformation can result in real violence [2]. A Spanish study shows that hate speech and disinformation are most often spread by politicians and radical groups [4]. In other cases, disinformation is deliberately used to undermine the credibility of established news services [5<\/a>].<\/p>\n\n\n\n

This development is particularly threatening for democracies, which rely on an open and constructive exchange of opinions. The latter is significantly disrupted by such manipulation. Disinformation not only serves as a tool for exerting influence, but also acts as a catalyst for hate crime, which jeopardizes social cohesion.<\/p>\n\n\n\n

Against this background, this paper aims to investigate whether a generative language model such as GPT-4o is able to effectively detect and correct cases of disinformation in social media moderation.<\/p>\n\n\n\n

The complexity of disinformation<\/h2>\n\n\n\n

Having emphasized the importance of combating disinformation as a key component of protecting democratic processes, it is important to develop a clear understanding of what disinformation actually entails. This enables both the identification of disinformation and the development of appropriate countermeasures.<\/p>\n\n\n\n

\"Sieben
Figure 1: Seven types of disinformation according to Wardle.<\/em><\/figcaption><\/figure>\n\n\n\n

Disinformation can be divided into different categories, as shown in the figure above. The seven main types of disinformation include:<\/p>\n\n\n\n

    \n
  1. Satire or parody: Content that has no malicious intent but has the potential to mislead people.<\/li>\n\n\n\n
  2. Misleading content: This describes information that is deliberately being misrepresented or taken out of context to cause harm or distort a particular topic or person.<\/li>\n\n\n\n
  3. Imposter content: This type of disinformation occurs when authentic sources are imitated or falsified to give the appearance of credibility.<\/li>\n\n\n\n
  4. Fabricated content: This is completely fabricated content created with the intent to deceive and cause harm.<\/li>\n\n\n\n
  5. False connection: These are misleading headlines, images or videos that do not reflect the actual content of the article or news item.<\/li>\n\n\n\n
  6. False context: Genuine material is placed in a false or distorted context in order to manipulate perception.<\/li>\n\n\n\n
  7. Manipulated content: In this form of disinformation, genuine image or video material is deliberately edited or manipulated to create a false impression. [3]<\/li>\n<\/ol>\n\n\n\n

    Disinformation is a complex phenomenon that can undermine public perception and trust in information in various ways. This poses a considerable challenge for generative language models and makes the research question particularly exciting: Can a generative language model successfully cope with the many facets of disinformation?<\/p>\n\n\n\n

    Insights into current research<\/h2>\n\n\n\n

    Research in the field of hate speech is extensive, but there is comparatively less work on uncovering disinformation. Nevertheless, some studies in this area are worth highlighting.<\/p>\n\n\n\n

    In one paper, researchers developed an automated framework for detecting disinformation in political speeches. Using various classification methods, the system extracts specific features, such as topic, location, profile and credibility of the speaker as well as contextual information. The trained language model, which is based on a Support Vector Machine, achieved a recognition accuracy of 74% on the “Liar”  dataset, consisting of about 12,000 manually labeled political statements [6]. German research institutes have also made significant contributions: The Fraunhofer Gesellschaft developed a tool to warn against fake news, while the Max Planck Institute developed a language model to classify Twitter posts according to German criminal law [7<\/a>].<\/p>\n\n\n\n

    In the field of generative language models, the study “Detecting Hate Speech with GPT-3” by Collins is worth mentioning. This study examined the ability of the GPT-3 language model to recognize and classify hate speech. In various learning scenarios (zero-shot, one-shot, few-shot), the accuracy of the language model varied between 55% and 67%, but could be increased to 85% in few-shot scenarios [8].<\/p>\n\n\n\n

    Data collection and classification of information sources<\/h2>\n\n\n\n

    In this experiment, a total of 50 data sets were collected, covering a variety of topics and sources. 40 of these 50 data setswere classified as potential cases of disinformation, while the remaining 10 data sets were categorized as correct information. The potential cases of disinformation were further divided into four categories: ten records contained satirical content, ten were marked as  false context, ten held fabricated content, and the remaining ten represented misinformation.<\/p>\n\n\n\n

    The data sets include both online messages and comments from social media users, such as Facebook and Twitter\/X. Only text content was considered. Visual or multimodal content such as images and videos were not considered, with the exception of cases where text was extracted from images.<\/p>\n\n\n\n

    Correctiv.org and Mimikama were used as reliable sources for the collection of fake news. Satirical content was obtained from recognized satirical sources such as “Der Postillon” and “Titanic Magazin”. The serious data sets came from various established news portals such as “Die Zeit”, “ZDF”, “Welt” and “Der Spiegel”. This selection ensures broad thematic coverage as well as highly relevant content.<\/p>\n\n\n\n

    The data sets cover a diverse range of topics, such as climate change, celebrities, crimes, curiosities, political events and scientific topics, such as health and technology. All selected topics are of current relevance, with the oldest contribution dating back to the beginning of 2024. The majority of the contributions focus on the months of May and June 2024, which ensures the topicality and relevance of the data.<\/p>\n\n\n\n

    Application of the GPT-4o language model<\/h2>\n\n\n\n

    The GPT-4o model is activated via the OpenAI platform, using a Python notebook. The Python script runs through the collected data sets iteratively and passes three key data points to the GPT-4o language model: The publication date of the article, the title (if available) and the content of the article. First, the language model uses this information to create a search query that returns five results via a search engine API.<\/p>\n\n\n\n

    The provider Tavily was selected for the search engine API, as the integration into a Python notebook proved to be particularly straightforward due to the library provided [9]. The corresponding URLs of these search results are then read out, using Python. The “Beautiful Soup” library is used to extract the text from the HTML content [10]. This content is then summarized by the GPT-4o language model and returned in a JSON array.<\/p>\n\n\n\n

    In the next step, the GPT-4o model is  activated once again to make an assessment based on the original contribution and the summarized information of the recorded web pages. This happens via a defined format, in which the language model indicates whether the content is correct, partially correct or incorrect. The language model also provides a reason for the rating and suggests corrections, if necessary. <\/p>\n\n\n\n

    The results are saved in an Excel spreadsheet and the author manually analyzes whether the classification is correct and the reasoning is comprehensible. Another aspect of the analysis is to check whether the language model has correctly captured the context of the post: whether a satirical post has been recognized as such or whether a serious post has incorrectly been classified as satire, for example.<\/p>\n\n\n\n

    The technical procedure of the experiment can be seen in the figure below.<\/p>\n\n\n\n

    \"Workflow
    Figure 2: Workflow for using GPT-4o to detect fake news: data input, search request, web scraping and content evaluation.<\/em><\/figcaption><\/figure>\n\n\n\n

    Evaluation and assessment<\/h2>\n\n\n\n

    The evaluation of the results followed a clear pattern: If the author was able to fully understand the language model’s reasoning, the data set does not receive a point deduction. This assumes that the language model has correctly recognized the content and interpreted the context of the post appropriately. Context <\/strong>refers to the language model’s ability to grasp the overall meaning and intention of the post.<\/p>\n\n\n\n

    The following example illustrates this approach: If the language model does not recognize the satirical intention behind a post, it manages to identify an apparent content error, yet does not take into account that the supposed disinformation is a deliberate stylistic choice.<\/p>\n\n\n\n

    Points are deducted as follows:<\/p>\n\n\n\n