Model Card for Model ID

This is a Named Entity Recognition model fine-tuned for public entities:

Politicians
Parties
Authorities
Media
Journalists

Model Details

Public Entity Recognition (PER). PER is a domainspecific version of NER, that is trained for five entities types that are common to public discourse: politicians, parties, authorities, media, and journalists. PER can be used for preprocessing documents, in a pipeline with other classifiers or directly for analyzing information in texts. The taxonomy for PER is taken from the database of (German) public speakers (Schmidt et al., 2023) and aims at low-threshold integration into computational social science research.

Bias, Risks, and Limitations

The performance for female entities (only applying to politicians and journalists) is slightly below that for male entities. This applies to entities that are referred to by name (Anna-Lena Baerbock/Olaf Scholz) or by profession (Innenministerin/Innenminister).

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

model_name = "Sami92/XLM-PER-L"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

text = '''
Nach dem Treffen mit Außenministerin Baerbock betont Israels Premier die Eigenständigkeit seines Landes.
Baerbock hatte zur Zurückhaltung aufgerufen.
Nach seinem Treffen mit Außenministerin Annalena Baerbock und dem britischen Außenminister David Cameron dringt der israelische Ministerpräsident Benjamin Netanjahu auf die Unabhängigkeit seines Landes.
'''

entities = ner_pipeline(text)

for entity in entities:
    print(f"Entity: {entity['word']}, Type: {entity['entity']}, Score: {entity['score']:.4f}")

Training Details

Training Data

The model was first fine-tuned on a weakly annotated dataset: German newspaper articles (total = 267,786) and German Wikipedia articles (total = 4,348). The weak annotation was based on the database of public speakers. In a second step the model was fine-tuned on a manually annotated dataset of 3090 sentences from similar sources. The test-split of this data was used for evaluation.

Training Hyperparameters

Learning Rate = 5e-6
Scheduler = Reduce learning rate on plateau
Batch size = 8
Epochs = 20

Metrics

type: f1 value: 0.82
type: recall value: 0.80
type: precision value: 0.85

Model Card Authors [optional]

@article{nenno_bootstrapping_2024,
    title = {Bootstrapping public entities. {Domain}-specific {NER} for public speakers},
    volume = {0},
    issn = {1931-2458},
    url = {https://www.tandfonline.com/doi/abs/10.1080/19312458.2024.2388098},
    doi = {10.1080/19312458.2024.2388098},
    number = {0},
    urldate = {2024-08-13},
    journal = {Communication Methods and Measures},
    author = {Nenno, Sami},
    year = {2024},
    pages = {1--26},
}