unga-climate-classifier

This model is a fine-tuned version of microsoft/deberta-v3-base trained to classify climate-related sentences in English using a dataset of 5,600 annotated sentences from the United Nations General Assembly Corpus. It was developed to build the Executive Comparative Climate Attention (ECCA) indicator, introduced in a paper published in Global Environmental Politics.

How to use

from transformers import pipeline classifier = pipeline("text-classification", model="mljn/unga-climate-classifier")

text = "Climate change poses a fundamental threat to our future."

result = classifier(text)

print(result)

[{'label': 'climate', 'score': 0.9988275170326233}]

How to cite

If you use this model or the underlying dataset or indicator, please cite:

Emiliano Grossman, Malo Jan; Executive Climate Change Attention: Toward an Indicator of Comparative Climate Change Attention. Global Environmental Politics 2025; doi: https://doi.org/10.1162/glep.a.1

@article{grossman2025executive,
  title={Executive Climate Change Attention: Toward an Indicator of Comparative Climate Change Attention},
  author={Grossman, Emiliano and Jan, Malo},
  journal={Global Environmental Politics},
  pages={1--14},
  year={2025},
  publisher={MIT Press 255 Main Street, 9th Floor, Cambridge, Massachusetts 02142, USA~…}
}

Model evaluation

It achieves the following results on the evaluation set:

• Loss: 0.0807
• Accuracy: 0.975
• F1 Macro: 0.9710
• Accuracy Balanced: 0.9715
• F1 Micro: 0.975
• Precision Macro: 0.9705
• Recall Macro: 0.9715
• Precision Micro: 0.975
• Recall Micro: 0.975

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 80
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.06
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro	Accuracy Balanced	F1 Micro	Precision Macro	Recall Macro	Precision Micro	Recall Micro
No log	1.0	123	0.1057	0.9726	0.9675	0.9583	0.9726	0.9783	0.9583	0.9726	0.9726
No log	2.0	246	0.1102	0.9726	0.9683	0.9697	0.9726	0.9669	0.9697	0.9726	0.9726
No log	3.0	369	0.0894	0.9798	0.9763	0.9729	0.9798	0.9800	0.9729	0.9798	0.9798
No log	4.0	492	0.1098	0.9762	0.9723	0.9723	0.9762	0.9723	0.9723	0.9762	0.9762
0.1374	5.0	615	0.1026	0.9798	0.9763	0.9729	0.9798	0.9800	0.9729	0.9798	0.9798

Framework versions

• Transformers 4.36.2
• Pytorch 2.5.0+cu121
• Datasets 2.6.0
• Tokenizers 0.15.2