Unsupervised NLP for quantifying sentiment deviation and framing in global climate discourse

1 Introduction

In recent decades, climate change has become one of the hottest topics around the world. It is associated with public opinions, political agendas, and international relations. Mass media is the tool that helps shape the perception of both the threat of climate change and its reality. Nevertheless, climate change coverage in news differs according to various regions and media organizations. The differences occur because of diverse editorial stances, culture, and politics (Painter et al., 2020). All of these may result in skewed perceptions of environmental facts (Feldman et al., 2021). Eventually, these framing differences may influence public understanding and participation in combating climate change. The current study proposes to create a systematic approach based on data analysis to detect the change in polarity and editorial frames of climate change coverage in the news from international media outlets such as BBC, CNN, Al Jazeera, and The Guardian. In particular, the investigation will concentrate on the consistent frames and shifts in sentiment caused by events, for example, climate summits and natural disasters.

Through measurement of sentiment shifts, this research intends to increase transparency within the domain of climate communication and understand the dynamics of climate narratives for the benefit of both scholars and policymakers (Mertens et al., 2022). The current literature points at the shortcomings associated with lexicon-based models applied to formal texts (Shaeri et al., 2025) and stresses the need for reliable context-aware models in environmental NLP (Ghiloufi et al., 2025; Gao et al., 2021; Kaji and Kitsuregawa, 2021; Rao et al., 2020). This research tries to fulfill that need through usage of transformer-based embeddings and scientific baselines.

There is an obvious lack of research in the field of the relationship between sentiment analysis, framing detection, and trends modeling of climate change media news. Even though there has been research done on individual components like sentiment analysis, topic classification, etc., (Zhou and Zafar, 2021; Duan et al., 2022; Battistelli et al., 2023; Koyama et al., 2022; Kulkarni et al., 2018; Liao et al., 2020; Lim et al., 2021; Recasens et al., 2013; Van den Berg and Boomgaarden, 2020; Zhang et al., 2019), few scholars have tried to integrate all these methods and see the development of the tone of the editorials over time or place (Han and Rho, 2023). Moreover, most studies are based on a one-time dataset collected by some website at a certain moment. This prevents scholars from following the real-time sentiment changes due to the global events (Leber et al., 2021).

In order to fill this gap, this paper introduces an advanced multi-layered analytical methodology. This methodology implies data collection, comprehensive text preprocessing, sentiment assessment by means of the Zero-Shot BART-large-MNLI transformer, topic detection using the BERTopic model, and changepoint detection using the PELT approach. Through these steps, the study contributes to the understanding of the ways sentiment change in climate news coverage occurs over time and space. The outcome of the analysis will be a valuable contribution to the body of knowledge of computational media analysis, offering a clear and replicable framework for creating a more balanced global climate dialogue. Quantifying sentiment change and detecting the shifts associated with major global events, the proposed framework provides a scalable approach to media framing and sentiment deviation analysis in environmental communication context.

2 Materials and methods

The methodology suggested in this work includes the utilization of the free and open source framework for the measurement of sentiment deviation and for the analysis of the frames created by editorial staff in regard to climate change issues covered by the news media. In order to implement all steps required for the implementation of the analysis and for measuring the sentiment deviation, several natural language processing techniques are used one after another. Six steps can be distinguished within the entire process: data extraction and preparation, utilities preparation, transformer-based sentiment analysis, topic modeling, statistical validation, and the analysis of structural breaks using changepoint technique.

2.1 Data collection

Data includes news articles obtained from multiple international organizations such as BBC, CNN, Al Jazeera, The Guardian, DW, and The New York Times. Articles from individual sources are stored in distinct.jsonl files that contain fields for storing titles, texts, and published times of these articles. For the inter-regional analysis, we use the hierarchy map reduction algorithm where articles are assigned to regions using source’s metadata defined through Equation 1.

R a = M region ( Source ( a ) ) (1)

In our case, Ra represents the area that corresponds to article a. The mapping is carried out through the use of two sequences of lookups: source-to-country function f and country-to-area function g, as shown in Equation 2.

R a = g ( f ( Source ( a ) ) ) (2)

This way, we can easily compare and compile the data of the sentiments in various regions worldwide.

2.1.1 Dataset description

Final dataset contains precisely 33,889 preprocessed articles from 22 international media sources, filtered from 90,555 raw extracted articles. The data collection has been performed through targeted web scraping using inclusion criteria limited only to explicit climate change terms. In order to assure high quality and validity of the data for the NLP approach, strict exclusion criteria have been applied in the preprocessing stage: non-English articles were deleted; exact duplicates were deleted based on the clean text of the article; and truncated articles with less than 100 characters were deleted to make sure that there would be enough text for context-based sentiment analysis. Although the extreme temporal limits of the dataset range between 1970 and 2025 years, most of the data dates back from the beginning of 2000s until nowadays, reflecting the emergence of digital journalism. The full details about articles quantity, temporal range, and regional distribution for each media source can be found in Table 1.

Table 1. Dataset overview detailing the total number of preprocessed articles, geographical region, and temporal coverage for the 22 international news sources analyzed in this study.

2.2 Data preprocessing

To ensure reliable data quality, data preprocessing was conducted, which entailed the transformation of texts into lowercase, stripping away of HTML tags while keeping necessary punctuation such as periods, commas, and negations required to provide contextual frames for transformer-based sentiment analysis, handling of white space, elimination of duplicate and truncated articles, elimination of articles from other languages except English, creation of a utilities module to facilitate consistent mapping, formulation of methods of handling raw texts, and construction of an intricate dictionary of large-scale climatic events that we would use to map our changepoints to in future parts of the project.

2.3 Sentiment analysis

For measuring the sentiment score of climate change stories in the online news world, although VADER (Valence Aware Dictionary and sEntiment Reasoner; Hutto and Gilbert, 2014) is commonly utilized, it is mainly designed for social media texts. In order to identify the underlying framing that is characteristic of professional journalism, we chose Zero-Shot BART-large-MNLI as our main tool for analysis (Devlin et al., 2019; Yin et al., 2019).

BART-MNLI assesses sequences against a set of candidate labels l = { positive , neutral , negative }, outputting a probability distribution P ( l ∣ a ) that represents the model’s confidence for each label l given an article a. To map these discrete classification probabilities onto a continuous sentiment scale ranging from −1 to +1, we assigned a predefined weight W ( l ) to each label, where W ( positive ) = 1, W ( neutral ) = 0, and W ( negative ) = − 1. The continuous sentiment score S ( a ) for an article is formulated as the weighted sum of these probabilities:

S ( a ) = l ∈ L ∑ P ( la ) · W ( l ) (3)

Given the assigned weights, this equation simplifies directly to the difference between the positive and negative confidence scores:

S ( a ) = P ( positive ∣ a ) − P ( negative ∣ a ) (4)

To rigorously validate the BART-MNLI model and address the limitations of lexicon-based methods, two independent human coders (authors of this study) manually annotated a random subset of 100 articles from the corpus. The annotation guidelines instructed the coders to classify the overarching editorial tone of each article text as ‘positive’, ‘negative’, or ‘neutral’ concerning climate change issues. Against this human gold standard, the BART-MNLI transformer achieved a remarkable Accuracy of 98.0%, an F1-score of 0.98, and high inter-annotator reliability (Cohen’s κ = 0.96). By contrast, the 54.00% directional agreement between BART and VADER across the wider corpus highlighted VADER’s inability to parse complex journalistic context and implicit framing. Consequently, VADER was discarded from the final analysis. All reported sentiment scores, figures, and changepoints in this study are derived exclusively from the validated BART-MNLI model (Figure 1).

2.4 Topic modeling using BERTopic

For the extraction of relevant themes, we employed the default configuration of BERTopic (Grootendorst, 2022; Baumer et al., 2015; Blei and Lafferty, 2006; Dieng et al., 2020; Li and Goldwasser, 2020). To ensure baseline reproducibility, the model utilized the default all-MiniLM-L6-v2 sentence-transformer embedding model. Dimensionality reduction was handled via UMAP (configured with n_neighbors = 15, n_components = 5, and a cosine distance metric), and density-based clustering was executed using HDBSCAN (min_cluster_size = 10, min_samples = 10). HDBSCAN inherently handles outlier documents by isolating them into a dedicated outlier class (Topic −1), preventing noisy text from distorting the thematic clusters. Topic interpretability and quality were assessed qualitatively by examining the top TF-IDF keyword representations for each cluster. Following cluster generation, the algorithm applies the process of class TF-IDF to identify the keywords essential for the creation of every topic. The formula used for the importance weight of the term ‘x’ in the class ‘c’ can be expressed as Equation 5.

W x , c = t f x , c ⋅ log ( 1 + A f x ) (5)

The notation “tfx, c” represents the frequency with which term x occurs in topic c, while “fx” denotes the frequency for each topic. The notation “a” represents the average word size for each topic class. The above model helps to have more understandable topics when assessed in the context of the global news media.

2.5 Relative sentiment deviation and temporal analysis

First, editorial framing is measured by the extent that a particular news article deviates from a regional consensus concerning a particular topic and within a particular time frame. Let S ( a i ) be the sentiment score of article i, which is derived from region R, topic T, and month of publication m. The regional baseline sentiment μ R , T , m is derived according to Equation 6.

μ R , T , m = 1 ∣ D R , T , m ∣ ∑ jϵ D R , T , m S ( a j ) (6)

The relative sentiment deviation score of article a i is directly derived by subtracting the sentiment baseline, according to Equation 7.

D R , T , m ( a i ) = S ( a i ) − μ R , T , m (7)

Because a purely region-normalized baseline risks masking systemic regional framing (e.g., an entire region being systematically pessimistic), we additionally benchmarked scores against an absolute scientific baseline proxy. Recognizing the difficulty of quantifying a perfectly objective tone without a dedicated secondary corpus analysis, we established S IPCC = 0.15 strictly as a conceptual, illustrative parameter representing a generally constructive scientific consensus tone (akin to official IPCC summaries). While this static proxy serves primarily to demonstrate the mathematical capability of the dual-baseline methodology, treating relative regional deviation and absolute scientific deviation as two distinct metrics provides a multidimensional view of media framing. Future applications of this framework can dynamically ingest actual IPCC reports to generate real-time evolving baselines.

To detect meaningful shifts in framing patterns over time, we apply the PELT (Pruned Exact Linear Time) changepoint detection algorithm (Adams and MacKay, 2007; Ahmed et al., 2020; Truong et al., 2020). To handle temporal noise and missing data, time-series data were aggregated at a weekly level, with missing weeks handled via linear interpolation prior to changepoint detection. Let y = { y 1 , , … , , y T } represent a sequence of deviation values. PELT identifies optimal segmentation points by minimizing a penalized cost function as shown in Equation 8. Here, C ( · ) denotes the kernel cost (Radial Basis Function [RBF] in our implementation), K is the number of changepoints, and β is a penalty term controlling sensitivity. To ensure robustness, a sensitivity analysis was conducted by varying the penalty parameter (β ∈ [ 1 , 5 ]). The analysis confirmed that major changepoint locations remained highly stable within a ± 1 week margin of uncertainty. Consequently, the parameter was empirically set to β = 3 to prevent overfitting and restrict the algorithm to identifying only major, sustained structural breaks in editorial framing, rather than minor weekly volatility.

min τ = ( ∑ k = 0 K C ( y t k + 1 : t k + 1 ) + βK ) (8)

Here, C ( · ) denotes the kernel cost (Radial Basis Function [RBF] in our implementation), K is the number of changepoints, and β is a penalty term controlling sensitivity. The penalty parameter was empirically set to β = 3 to prevent overfitting and restrict the algorithm to identifying only major, sustained structural breaks in editorial framing, rather than minor weekly volatility.

2.6 Statistical validation

For purposes of establishing the robustness of the trends observed through computational means and due to the natural variability associated with the processing of unstructured text data, statistical verification techniques were employed. Kruskal-Wallis H-test was performed to identify any statistically significant variations in sentiment framing among different geographic regions. The choice of this particular test was dictated by its lack of assumption regarding normality of sentiment scores distribution. In addition, to account for the uncertainty associated with media outlets’ framing practices, 95% confidence intervals were computed for each media outlet’s deviation score via non-parametric bootstrap technique (n = 1,000).

3 Results

The suggested NLP pipeline tries to measure the sentiment and editorial framing of climate stories from various angles around the world and across various media. These are displayed visually in a report form, illustrating various trends.

3.1 Regional sentiment patterns

Aggregating the sentiment scores from these various sources reveals stark regional differences in the reception of climate news. From Figure 2, it can be observed that the press in Asia, Africa, and Oceania tends to frame climate-change coverage most positively. In contrast, the Middle East is the only region with an aggregated negative sentiment, while Europe sits near neutral. North America and South America sit somewhere in between, with moderately positive framing. Such global divergences indicate that the framing of climate issues reflects considerations such as the allocation of resources, regional policy priorities, and distinctive media ecosystems in each region. It is important to note that the dataset features an outlet imbalance, with the NZ Herald accounting for a significant portion of the Oceanic data. To prevent outlet-specific volume from confounding the regional analysis, regional sentiment was computed analytically by first calculating the mean sentiment for each individual outlet, and subsequently averaging those source-level means to obtain the final, unweighted regional score. A Kruskal-Wallis H-test on these outlet-weighted means indicated that while descriptive regional differences exist, they do not reach strict statistical significance (h = 11.352 , p = 0.078), suggesting that editorial framing is more strongly driven by individual outlet identities than monolithic regional mandates.

3.2 Source-level sentiment trends

When examining the data collected for each individual media source, the variations in editorials according to region appear to align with particular institutional positions. As one can observe from Figure 3, media outlets such as China Daily (China), Stuff (New Zealand), and NZ Herald (New Zealand) use a very positive tone in the majority of cases. In contrast, news media sources like BBC (UK), Sky News Australia, and Fox News (USA) tend to use a negative tone.

3.3 Temporal sentiment dynamics

Figure 4 reveals that the temporal discourse concerning climate change is correlated with external events. Shifts from optimism to anxiety and vice versa correlate with international events like natural calamities, large-scale climate summits, and scientific breakthroughs. Because the media rarely adopts a centrist attitude, it is common for changes in climate-related discourses to be in tandem with significant events although not necessarily causation, but rather a correlational structural break. Furthermore, the stable trend and then volatility preceding the early 2000s (as shown in Figure 4) are mainly because of the scarcity of data in the preceding years due to the limited presence of digital journalism.

3.4 Topic distribution across sources

The BERTopic algorithm was able to create 31 distinct topics (including one outlier class). Some of the main themes identified through the process include separate topics related to ‘Amazon resource extraction’, ‘drought and agriculture’ and ‘international climate policy’. In addition, the use of BERTopic proves that the degree of specificity for the topics generated is not homogenous. As seen from the regional topic distribution shown in Figure 5, the agenda for Folha in Brazil is characterized by the extraction of Amazon resources and disharmony in politics, while the agenda for Sky News in Australia is dominated by climate policy discussions based on the political environment there. On the other hand, the agendas for African media such as the Daily Nation in Kenya are largely driven by the drought in the region and weak agricultural sector caused by insufficient rains. This suggests that thematic framing may be heavily influenced by selective localized information.

3.5 Deviation scores and structural breaks

Based on Equation 7, the report calculates the relative deviation from stage to stage; consequently, The Economist is identified as having the largest positive deviation (+0.1356), thereby making it a more optimistic source among its regional competitors and compared to the science-based value. On the other hand, the large negative deviations of BBC (−0.1546) and Fox News (−0.1402) show the inclination of these sources towards the negative frame. Moreover, the changepoint analysis based on Equation 8 proves that the shifts in the framing are not consistent but rather change over time. For instance, BBC was seen as showing several structural breaks in tone, for instance, in April 2022, April 2023, and August 2023. (Note: The full 95% bootstrapped CI for all 22 media sources is presented in the supplementary dataset summary).

4 Discussion

4.1 Comparative media ecosystems

The cross-regional comparison of climate coverage demonstrates different media ecosystems. Media outlets in Europe and North America have more pronounced polarization. They show different sentiment trends which intersect and then separate from each other again over time. It suggests the dynamic process of positioning which leads to greater sentiment divergence in those media ecosystems. At the same time, there is more synchronization between Asian and South American outlets. The content of their articles matches better the national policy debates and economy-related news. This suggests a more cohesive framing context in which climate is a less polarized and more centrally directed topic. These observed differences in media ecosystems are consistent with the potential influence of regional politics and institutional priorities on the volatility and directionality of climate stories. More divided regions demonstrate more sentiment shifts.

4.2 Individual source dynamics

Outside of these regional trends, there is a distinct trend for each news outlet of sentiment over time. Both BBC and The Guardian tend to alternate between an optimistic and cautious tone. This suggests a response to climate events that are taking place as well as adapting editorial priorities. In this case, one can see that these two institutions rely on flexible approaches to their storytelling instead of a consistent ideological position. China Daily demonstrates a consistently positive climate sentiment, which may reflect a communication style aligned with government-led climate initiatives and national policies. The Daily Nation shows sensitivity to drought-related issues and agricultural impacts. This suggests that the outlet’s framing may reflect local vulnerabilities to climate change, with sentiment potentially tracking the region’s immediate environmental challenges.

4.3 Interpretation of framing patterns

The presence of regional polarization patterns combined with individual outlet trajectories suggests that climate framing happens on multiple levels. Sentiment deviation cannot be seen as an indication of only ideology; it is likely to be caused by different policy concerns, economic interests, climate vulnerability of the region, or framing norms of the editorials. It should also be considered that framing evolves over time. As a result, the analysis of changepoints revealed structural changes in framing over time. Many of these points were connected with major climate events, meaning that frames can adapt to exogenous shocks and change accordingly. Nevertheless, this is only a correlation between the event and the framing and cannot prove anything concerning editorial intentionality.

It should be noted that negative sentiment deviation does not necessarily mean ideological bias. For the topic of climate change, negative polarity means nothing else but truthful reports about catastrophes, policy failures, or risks of climate change.

4.4 Implications for climate communication

The results provide insight into the ways climate narratives influence public perception. Polarized ecosystems could be associated with increased uncertainty or ideological polarization. On the contrary, cohesive ecosystems might enhance policy messages of centralization. The fact of existence of specific sentiment fingerprints of outlets under consideration emphasizes the importance of transparency and computational audit of climate communication worldwide. Combining the quantitative approach to measuring framing and time considerations allows understanding the mechanisms behind the formation and adaptation of climate narratives.

4.5 Limitations

However, despite the potential of the framework suggested above in measuring climate media framing, there are also several important shortcomings. First, the analysis of sentiment changes triggered by certain events presupposes the availability of a set list of key global climate events. In this way, it provides the possibility of structured analysis of certain changes, although not all of them can be related to certain events listed. Some of them might be connected to some changes in the editorial policy, local politics, or institutional dynamics, which are not reflected in the current set of events mapped. Second, in the process of development of our double-baseline approach, we used the scientific baseline proxy (IPCC) to overcome the problem of masking of regional systemic framing, however, it should be noted that the definition of the objective climate sentiment is still rather challenging. The scientific baseline proxy applied in this study represents a static estimate and does not reflect the complexity of the evolution of the scientific consensus in the last five decades. Third, the data used is mostly represented by English-language media. Fourth, although embeddings built by transformer-based architectures (BART-MNLI) are very efficient at recognizing contexts compared to lexicons such as VADER, there is a possibility that they will fail to identify certain frames, sarcasm, or ideologies embedded in context. Lastly, no attention is paid to the effect of media on audiences’ behaviors because the focus here is on media messages themselves, not their influence on public opinion. Nevertheless, the framework provides an effective foundation for the computational audit of climate communication which can be developed further to deal with mentioned weaknesses.

5 Conclusion

In this research, the scalable and unsupervised NLP pipeline for estimating the sentiment deviation and editorial framing of the climate change discussion on a global level was suggested. This pipeline utilizes transformer-based classification, topic modeling (Equation 5), calculation of framing in terms of relative deviations (Equation 7), and changepoint detection (Equation 8) in order to obtain an interpretable picture of the narrative tendencies of different media landscapes.

Climate change discussion occupies different space on a global level. The movement is not only geographically dependent but also on the specific editorial framing of the media outlet and attractions of worldwide and local events. In Asia/Oceania, the news articles are highly positively skewed, while the beat in Europe and Middle East is negatively framed. Clearly, the cross-poach between the media outlets can be observed, where institutional and regional identities appear to intersect with the progression of the narrative.

Finally, this project serves as an example of what computational sentiment audit can achieve when estimating the dynamic framing of climate change discussion.

6 Future work

In the future, there is scope for extending this framework in various ways. Firstly, we can incorporate structured databases of global events, including GDELT and similar datasets, to strengthen the linkage between the identified changepoints and actual events taking place. This process can be improved further by using automatic natural language inference tools, which will help in building more robust connections between events. Secondly, incorporating more language sources from different countries and regions into the data will give a more comprehensive picture of the climate narratives globally. In addition, the use of multilingual transformer models along with cross-lingual embeddings will allow us to compare the climate narrative discourse across different regions and languages (Kim et al., 2024), thus reducing the bias towards English. Thirdly, incorporating additional metadata related to author, article, and media outlet properties, including front page stories, political orientation, and popularity, can help in distinguishing framing from institutional biases. Lastly, future work might examine how the trends in the media framing and sentiment are associated with indicators of public opinions. For instance, social media activity and support for climate policy measures might indicate how the climate narratives have been affecting the public perception of the problem and society’s decisions.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Mary Stampone, University of New Hampshire, United States

Reviewed by: Maria Chatzimina, Foundation for Research and Technology Hellas (FORTH), Greece

Pouya Shaeri, Arizona State University, United States