|
--- |
|
language: |
|
- de |
|
- es |
|
- en |
|
pipeline_tag: token-classification |
|
tags: |
|
- politics |
|
- communication |
|
- public sphere |
|
license: cc-by-4.0 |
|
--- |
|
# Model Card for Model ID |
|
|
|
This is a Named Entity Recognition model fine-tuned for public entities: |
|
- Politicians |
|
- Parties |
|
- Authorities |
|
- Media |
|
- Journalists |
|
|
|
## Model Details |
|
Public Entity Recognition (PER). PER is a domainspecific version of NER, that is trained for five entities types that are common to public discourse: politicians, parties, authorities, media, and journalists. PER can be used for preprocessing documents, in a pipeline with other classifiers or directly for analyzing information in texts. The taxonomy for PER is taken from the database of (German) public speakers (Schmidt et al., 2023) and aims at low-threshold integration into computational social science research. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
The performance for female entities (only applying to politicians and journalists) is slightly below that for male entities. This applies to entities that are referred to by name (Anna-Lena Baerbock/Olaf Scholz) or by profession (Innenministerin/Innenminister). |
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
|
|
|
## How to Get Started with the Model |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForTokenClassification |
|
from transformers import pipeline |
|
|
|
model_name = "Sami92/XLM-PER-L" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = AutoModelForTokenClassification.from_pretrained(model_name) |
|
|
|
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer) |
|
|
|
text = ''' |
|
Nach dem Treffen mit Außenministerin Baerbock betont Israels Premier die Eigenständigkeit seines Landes. |
|
Baerbock hatte zur Zurückhaltung aufgerufen. |
|
Nach seinem Treffen mit Außenministerin Annalena Baerbock und dem britischen Außenminister David Cameron dringt der israelische Ministerpräsident Benjamin Netanjahu auf die Unabhängigkeit seines Landes. |
|
''' |
|
|
|
entities = ner_pipeline(text) |
|
|
|
for entity in entities: |
|
print(f"Entity: {entity['word']}, Type: {entity['entity']}, Score: {entity['score']:.4f}") |
|
|
|
|
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The model was first fine-tuned on a weakly annotated dataset: German newspaper articles (total = 267,786) and German Wikipedia articles (total = 4,348). |
|
The weak annotation was based on the [database of public speakers](https://github.com/Leibniz-HBI/DBoeS-data/). |
|
In a second step the model was fine-tuned on a manually annotated dataset of 3090 sentences from similar sources. The test-split of this data was used for evaluation. |
|
|
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
- Learning Rate = 5e-6 |
|
- Scheduler = Reduce learning rate on plateau |
|
- Batch size = 8 |
|
- Epochs = 20 |
|
|
|
|
|
|
|
#### Metrics |
|
|
|
- type: f1 |
|
value: 0.82 |
|
- type: recall |
|
value: 0.80 |
|
- type: precision |
|
value: 0.85 |
|
|
|
|
|
## Model Card Authors [optional] |
|
```bibtex |
|
@article{nenno_bootstrapping_2024, |
|
title = {Bootstrapping public entities. {Domain}-specific {NER} for public speakers}, |
|
volume = {0}, |
|
issn = {1931-2458}, |
|
url = {https://www.tandfonline.com/doi/abs/10.1080/19312458.2024.2388098}, |
|
doi = {10.1080/19312458.2024.2388098}, |
|
number = {0}, |
|
urldate = {2024-08-13}, |
|
journal = {Communication Methods and Measures}, |
|
author = {Nenno, Sami}, |
|
year = {2024}, |
|
pages = {1--26}, |
|
} |
|
``` |
