Instructions to use softcatala/fullstop-catalan-punctuation-prediction with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use softcatala/fullstop-catalan-punctuation-prediction with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="softcatala/fullstop-catalan-punctuation-prediction")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("softcatala/fullstop-catalan-punctuation-prediction") model = AutoModelForTokenClassification.from_pretrained("softcatala/fullstop-catalan-punctuation-prediction", device_map="auto") - Notebooks
- Google Colab
- Kaggle
This model predicts the punctuation of Catalan language.
The model restores the following punctuation markers: "." "," "?" "-" ":"
Based on the work https://github.com/oliverguhr/fullstop-deep-punctuation-prediction
Results
The performance differs for the single punctuation markers as hyphens and colons, in many cases, are optional and can be substituted by either a comma or a full stop. The model achieves the following F1 scores for Catalan language:
| Label | CA |
|---|---|
| 0 | 0.99 |
| . | 0.93 |
| , | 0.82 |
| ? | 0.76 |
| - | 0.89 |
| : | 0.64 |
| macro average | 0.84 |
License
Softcatalà's contributions to this model are licensed under the MIT License.
The punctuation prediction approach is based on FullStop, whose source code is licensed under MIT.
The pretrained checkpoint identified in config.json is
BSC-TeMU/roberta-base-ca,
developed by the Barcelona Supercomputing Center and licensed under
Apache License 2.0. This checkpoint was fine-tuned for
Catalan punctuation prediction. Its Apache-2.0 terms continue to apply to the
underlying pretrained model; the MIT license covers Softcatalà's contributions.
Contact
Jordi Mas jmas@softcatala.org
- Downloads last month
- 63