Commit
·
3f5b139
1
Parent(s):
89ce038
update model card README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,5 @@
|
|
1 |
---
|
2 |
license: mit
|
3 |
-
language: es
|
4 |
tags:
|
5 |
- generated_from_trainer
|
6 |
model-index:
|
@@ -8,74 +7,26 @@ model-index:
|
|
8 |
results: []
|
9 |
---
|
10 |
|
11 |
-
|
12 |
-
|
13 |
-
This model is a fine-tuned version of [flax-community/spanish-t5-small](https://huggingface.co/flax-community/spanish-t5-small) on the [Spanish Poetry Dataset](https://www.kaggle.com/andreamorgar/spanish-poetry-dataset/version/1) dataset.
|
14 |
-
|
15 |
-
The model was created during the [First Spanish Hackathon](https://somosnlp.org/hackathon) organized by [Somos NLP](https://somosnlp.org/).
|
16 |
-
|
17 |
-
The team who participated was composed by:
|
18 |
-
|
19 |
-
- 🇨🇺 [Alberto Carmona Barthelemy](https://huggingface.co/milyiyo)
|
20 |
-
- 🇪🇸 [Andrea Morales Garzón](https://huggingface.co/andreamorgar)
|
21 |
-
- 🇨🇴 [Jorge Henao](https://huggingface.co/jorge-henao)
|
22 |
-
- 🇮🇳 [Drishti Sharma](https://huggingface.co/DrishtiSharma)
|
23 |
|
|
|
24 |
|
|
|
25 |
It achieves the following results on the evaluation set:
|
26 |
-
- Loss: 2.
|
27 |
-
- Perplexity: 17.43
|
28 |
|
29 |
## Model description
|
30 |
|
31 |
-
|
32 |
-
|
33 |
-
Example:
|
34 |
|
35 |
-
|
36 |
-
poema:
|
37 |
-
estilo: Pablo Neruda &&
|
38 |
-
sentimiento: positivo &&
|
39 |
-
palabras: cielo, luna, mar &&
|
40 |
-
texto: Todos fueron a verle pasar
|
41 |
-
```
|
42 |
|
43 |
-
|
44 |
-
|
45 |
-
You can use this model directly with a pipeline for masked language modeling:
|
46 |
-
|
47 |
-
```python
|
48 |
-
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
49 |
-
model_name = 'hackathon-pln-es/poem-gen-spanish-t5-small'
|
50 |
-
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
51 |
-
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
|
52 |
-
|
53 |
-
author, sentiment, word, start_text = 'Pablo Neruda', 'positivo', 'cielo', 'Todos fueron a la plaza'
|
54 |
-
input_text = f"""poema: estilo: {author} && sentimiento: {sentiment} && palabras: {word} && texto: {start_text} """
|
55 |
-
inputs = tokenizer(input_text, return_tensors="pt")
|
56 |
-
|
57 |
-
outputs = model.generate(inputs["input_ids"],
|
58 |
-
do_sample = True,
|
59 |
-
max_length = 30,
|
60 |
-
repetition_penalty = 20.0,
|
61 |
-
top_k = 50,
|
62 |
-
top_p = 0.92)
|
63 |
-
detok_outputs = [tokenizer.decode(x, skip_special_tokens=True) for x in outputs]
|
64 |
-
res = detok_outputs[0]
|
65 |
-
```
|
66 |
|
67 |
## Training and evaluation data
|
68 |
|
69 |
-
|
70 |
-
For each poem we generate new examples:
|
71 |
-
- content: *line_i* , generated: *line_i+1*
|
72 |
-
- content: *concatenate(line_i, line_i+1)* , generated: *line_i+2*
|
73 |
-
- content: *concatenate(line_i, line_i+1, line_i+2)* , generated: *line_i+3*
|
74 |
-
|
75 |
-
The resulting dataset has the columns `author`, `content`, `title` and `generated`.
|
76 |
-
|
77 |
-
For each example we compute the sentiment of the generated column and the nouns. In the case of sentiment, we used the model `mrm8488/electricidad-small-finetuned-restaurant-sentiment-analysis` and for nouns extraction we used spaCy.
|
78 |
-
|
79 |
|
80 |
## Training procedure
|
81 |
|
@@ -94,14 +45,14 @@ The following hyperparameters were used during training:
|
|
94 |
|
95 |
| Training Loss | Epoch | Step | Validation Loss |
|
96 |
|:-------------:|:-----:|:------:|:---------------:|
|
97 |
-
|
|
98 |
-
| 2.
|
99 |
-
| 2.
|
100 |
-
| 2.
|
101 |
-
| 2.
|
102 |
-
| 2.
|
103 |
-
| 2.
|
104 |
-
| 2.
|
105 |
|
106 |
|
107 |
### Framework versions
|
|
|
1 |
---
|
2 |
license: mit
|
|
|
3 |
tags:
|
4 |
- generated_from_trainer
|
5 |
model-index:
|
|
|
7 |
results: []
|
8 |
---
|
9 |
|
10 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
11 |
+
should probably proofread and complete it, then remove this comment. -->
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
13 |
+
# poem-gen-spanish-t5-small
|
14 |
|
15 |
+
This model is a fine-tuned version of [hackathon-pln-es/poem-gen-spanish-t5-small](https://huggingface.co/hackathon-pln-es/poem-gen-spanish-t5-small) on the None dataset.
|
16 |
It achieves the following results on the evaluation set:
|
17 |
+
- Loss: 2.8723
|
|
|
18 |
|
19 |
## Model description
|
20 |
|
21 |
+
More information needed
|
|
|
|
|
22 |
|
23 |
+
## Intended uses & limitations
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
|
25 |
+
More information needed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
## Training and evaluation data
|
28 |
|
29 |
+
More information needed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
## Training procedure
|
32 |
|
|
|
45 |
|
46 |
| Training Loss | Epoch | Step | Validation Loss |
|
47 |
|:-------------:|:-----:|:------:|:---------------:|
|
48 |
+
| 2.7082 | 0.73 | 30000 | 2.8878 |
|
49 |
+
| 2.6251 | 1.46 | 60000 | 2.8940 |
|
50 |
+
| 2.5796 | 2.19 | 90000 | 2.8853 |
|
51 |
+
| 2.5556 | 2.93 | 120000 | 2.8749 |
|
52 |
+
| 2.527 | 3.66 | 150000 | 2.8850 |
|
53 |
+
| 2.5024 | 4.39 | 180000 | 2.8760 |
|
54 |
+
| 2.4887 | 5.12 | 210000 | 2.8749 |
|
55 |
+
| 2.4808 | 5.85 | 240000 | 2.8707 |
|
56 |
|
57 |
|
58 |
### Framework versions
|