Hugging Face's logo

KarthikaRajagopal
/
BERT_Transformer

Model card Files Community
KarthikaRajagopal commited on
Commit
1e6f0d8
·
verified ·
1 Parent(s): bc813d2

Upload BERT_Transformer.ipynb

Browse files
Files changed (1) hide show
  1. BERT_Transformer.ipynb +2574 -0
BERT_Transformer.ipynb ADDED
@@ -0,0 +1,2574 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": [],
7
+ "gpuType": "T4"
8
+ },
9
+ "kernelspec": {
10
+ "name": "python3",
11
+ "display_name": "Python 3"
12
+ },
13
+ "language_info": {
14
+ "name": "python"
15
+ },
16
+ "accelerator": "GPU",
17
+ "widgets": {
18
+ "application/vnd.jupyter.widget-state+json": {
19
+ "b3d817946b4a4461b7dc6ee1823c821a": {
20
+ "model_module": "@jupyter-widgets/controls",
21
+ "model_name": "HBoxModel",
22
+ "model_module_version": "1.5.0",
23
+ "state": {
24
+ "_dom_classes": [],
25
+ "_model_module": "@jupyter-widgets/controls",
26
+ "_model_module_version": "1.5.0",
27
+ "_model_name": "HBoxModel",
28
+ "_view_count": null,
29
+ "_view_module": "@jupyter-widgets/controls",
30
+ "_view_module_version": "1.5.0",
31
+ "_view_name": "HBoxView",
32
+ "box_style": "",
33
+ "children": [
34
+ "IPY_MODEL_7c94cf8a7691426490807ab423609448",
35
+ "IPY_MODEL_fefb56e8ebd64bd18c8bb33cc6c2f367",
36
+ "IPY_MODEL_101687acfd6741748a85fc43e9cff508"
37
+ ],
38
+ "layout": "IPY_MODEL_40c21d55b6d84018a903242d6ddc0ead"
39
+ }
40
+ },
41
+ "7c94cf8a7691426490807ab423609448": {
42
+ "model_module": "@jupyter-widgets/controls",
43
+ "model_name": "HTMLModel",
44
+ "model_module_version": "1.5.0",
45
+ "state": {
46
+ "_dom_classes": [],
47
+ "_model_module": "@jupyter-widgets/controls",
48
+ "_model_module_version": "1.5.0",
49
+ "_model_name": "HTMLModel",
50
+ "_view_count": null,
51
+ "_view_module": "@jupyter-widgets/controls",
52
+ "_view_module_version": "1.5.0",
53
+ "_view_name": "HTMLView",
54
+ "description": "",
55
+ "description_tooltip": null,
56
+ "layout": "IPY_MODEL_ad482254e6874b0184f3f356ef7a9543",
57
+ "placeholder": "​",
58
+ "style": "IPY_MODEL_a7dfb2bee2894d8ba161dbb6a617ee98",
59
+ "value": "config.json: 100%"
60
+ }
61
+ },
62
+ "fefb56e8ebd64bd18c8bb33cc6c2f367": {
63
+ "model_module": "@jupyter-widgets/controls",
64
+ "model_name": "FloatProgressModel",
65
+ "model_module_version": "1.5.0",
66
+ "state": {
67
+ "_dom_classes": [],
68
+ "_model_module": "@jupyter-widgets/controls",
69
+ "_model_module_version": "1.5.0",
70
+ "_model_name": "FloatProgressModel",
71
+ "_view_count": null,
72
+ "_view_module": "@jupyter-widgets/controls",
73
+ "_view_module_version": "1.5.0",
74
+ "_view_name": "ProgressView",
75
+ "bar_style": "success",
76
+ "description": "",
77
+ "description_tooltip": null,
78
+ "layout": "IPY_MODEL_3c578a1040474c0d940c4c15fe84fcc6",
79
+ "max": 570,
80
+ "min": 0,
81
+ "orientation": "horizontal",
82
+ "style": "IPY_MODEL_609e51e02f594d80946922a1b079642a",
83
+ "value": 570
84
+ }
85
+ },
86
+ "101687acfd6741748a85fc43e9cff508": {
87
+ "model_module": "@jupyter-widgets/controls",
88
+ "model_name": "HTMLModel",
89
+ "model_module_version": "1.5.0",
90
+ "state": {
91
+ "_dom_classes": [],
92
+ "_model_module": "@jupyter-widgets/controls",
93
+ "_model_module_version": "1.5.0",
94
+ "_model_name": "HTMLModel",
95
+ "_view_count": null,
96
+ "_view_module": "@jupyter-widgets/controls",
97
+ "_view_module_version": "1.5.0",
98
+ "_view_name": "HTMLView",
99
+ "description": "",
100
+ "description_tooltip": null,
101
+ "layout": "IPY_MODEL_1ed35d72c6964205b92ac7f48f767705",
102
+ "placeholder": "​",
103
+ "style": "IPY_MODEL_fad6c334971d4149b37eeee98d9b4d62",
104
+ "value": " 570/570 [00:00<00:00, 54.4kB/s]"
105
+ }
106
+ },
107
+ "40c21d55b6d84018a903242d6ddc0ead": {
108
+ "model_module": "@jupyter-widgets/base",
109
+ "model_name": "LayoutModel",
110
+ "model_module_version": "1.2.0",
111
+ "state": {
112
+ "_model_module": "@jupyter-widgets/base",
113
+ "_model_module_version": "1.2.0",
114
+ "_model_name": "LayoutModel",
115
+ "_view_count": null,
116
+ "_view_module": "@jupyter-widgets/base",
117
+ "_view_module_version": "1.2.0",
118
+ "_view_name": "LayoutView",
119
+ "align_content": null,
120
+ "align_items": null,
121
+ "align_self": null,
122
+ "border": null,
123
+ "bottom": null,
124
+ "display": null,
125
+ "flex": null,
126
+ "flex_flow": null,
127
+ "grid_area": null,
128
+ "grid_auto_columns": null,
129
+ "grid_auto_flow": null,
130
+ "grid_auto_rows": null,
131
+ "grid_column": null,
132
+ "grid_gap": null,
133
+ "grid_row": null,
134
+ "grid_template_areas": null,
135
+ "grid_template_columns": null,
136
+ "grid_template_rows": null,
137
+ "height": null,
138
+ "justify_content": null,
139
+ "justify_items": null,
140
+ "left": null,
141
+ "margin": null,
142
+ "max_height": null,
143
+ "max_width": null,
144
+ "min_height": null,
145
+ "min_width": null,
146
+ "object_fit": null,
147
+ "object_position": null,
148
+ "order": null,
149
+ "overflow": null,
150
+ "overflow_x": null,
151
+ "overflow_y": null,
152
+ "padding": null,
153
+ "right": null,
154
+ "top": null,
155
+ "visibility": null,
156
+ "width": null
157
+ }
158
+ },
159
+ "ad482254e6874b0184f3f356ef7a9543": {
160
+ "model_module": "@jupyter-widgets/base",
161
+ "model_name": "LayoutModel",
162
+ "model_module_version": "1.2.0",
163
+ "state": {
164
+ "_model_module": "@jupyter-widgets/base",
165
+ "_model_module_version": "1.2.0",
166
+ "_model_name": "LayoutModel",
167
+ "_view_count": null,
168
+ "_view_module": "@jupyter-widgets/base",
169
+ "_view_module_version": "1.2.0",
170
+ "_view_name": "LayoutView",
171
+ "align_content": null,
172
+ "align_items": null,
173
+ "align_self": null,
174
+ "border": null,
175
+ "bottom": null,
176
+ "display": null,
177
+ "flex": null,
178
+ "flex_flow": null,
179
+ "grid_area": null,
180
+ "grid_auto_columns": null,
181
+ "grid_auto_flow": null,
182
+ "grid_auto_rows": null,
183
+ "grid_column": null,
184
+ "grid_gap": null,
185
+ "grid_row": null,
186
+ "grid_template_areas": null,
187
+ "grid_template_columns": null,
188
+ "grid_template_rows": null,
189
+ "height": null,
190
+ "justify_content": null,
191
+ "justify_items": null,
192
+ "left": null,
193
+ "margin": null,
194
+ "max_height": null,
195
+ "max_width": null,
196
+ "min_height": null,
197
+ "min_width": null,
198
+ "object_fit": null,
199
+ "object_position": null,
200
+ "order": null,
201
+ "overflow": null,
202
+ "overflow_x": null,
203
+ "overflow_y": null,
204
+ "padding": null,
205
+ "right": null,
206
+ "top": null,
207
+ "visibility": null,
208
+ "width": null
209
+ }
210
+ },
211
+ "a7dfb2bee2894d8ba161dbb6a617ee98": {
212
+ "model_module": "@jupyter-widgets/controls",
213
+ "model_name": "DescriptionStyleModel",
214
+ "model_module_version": "1.5.0",
215
+ "state": {
216
+ "_model_module": "@jupyter-widgets/controls",
217
+ "_model_module_version": "1.5.0",
218
+ "_model_name": "DescriptionStyleModel",
219
+ "_view_count": null,
220
+ "_view_module": "@jupyter-widgets/base",
221
+ "_view_module_version": "1.2.0",
222
+ "_view_name": "StyleView",
223
+ "description_width": ""
224
+ }
225
+ },
226
+ "3c578a1040474c0d940c4c15fe84fcc6": {
227
+ "model_module": "@jupyter-widgets/base",
228
+ "model_name": "LayoutModel",
229
+ "model_module_version": "1.2.0",
230
+ "state": {
231
+ "_model_module": "@jupyter-widgets/base",
232
+ "_model_module_version": "1.2.0",
233
+ "_model_name": "LayoutModel",
234
+ "_view_count": null,
235
+ "_view_module": "@jupyter-widgets/base",
236
+ "_view_module_version": "1.2.0",
237
+ "_view_name": "LayoutView",
238
+ "align_content": null,
239
+ "align_items": null,
240
+ "align_self": null,
241
+ "border": null,
242
+ "bottom": null,
243
+ "display": null,
244
+ "flex": null,
245
+ "flex_flow": null,
246
+ "grid_area": null,
247
+ "grid_auto_columns": null,
248
+ "grid_auto_flow": null,
249
+ "grid_auto_rows": null,
250
+ "grid_column": null,
251
+ "grid_gap": null,
252
+ "grid_row": null,
253
+ "grid_template_areas": null,
254
+ "grid_template_columns": null,
255
+ "grid_template_rows": null,
256
+ "height": null,
257
+ "justify_content": null,
258
+ "justify_items": null,
259
+ "left": null,
260
+ "margin": null,
261
+ "max_height": null,
262
+ "max_width": null,
263
+ "min_height": null,
264
+ "min_width": null,
265
+ "object_fit": null,
266
+ "object_position": null,
267
+ "order": null,
268
+ "overflow": null,
269
+ "overflow_x": null,
270
+ "overflow_y": null,
271
+ "padding": null,
272
+ "right": null,
273
+ "top": null,
274
+ "visibility": null,
275
+ "width": null
276
+ }
277
+ },
278
+ "609e51e02f594d80946922a1b079642a": {
279
+ "model_module": "@jupyter-widgets/controls",
280
+ "model_name": "ProgressStyleModel",
281
+ "model_module_version": "1.5.0",
282
+ "state": {
283
+ "_model_module": "@jupyter-widgets/controls",
284
+ "_model_module_version": "1.5.0",
285
+ "_model_name": "ProgressStyleModel",
286
+ "_view_count": null,
287
+ "_view_module": "@jupyter-widgets/base",
288
+ "_view_module_version": "1.2.0",
289
+ "_view_name": "StyleView",
290
+ "bar_color": null,
291
+ "description_width": ""
292
+ }
293
+ },
294
+ "1ed35d72c6964205b92ac7f48f767705": {
295
+ "model_module": "@jupyter-widgets/base",
296
+ "model_name": "LayoutModel",
297
+ "model_module_version": "1.2.0",
298
+ "state": {
299
+ "_model_module": "@jupyter-widgets/base",
300
+ "_model_module_version": "1.2.0",
301
+ "_model_name": "LayoutModel",
302
+ "_view_count": null,
303
+ "_view_module": "@jupyter-widgets/base",
304
+ "_view_module_version": "1.2.0",
305
+ "_view_name": "LayoutView",
306
+ "align_content": null,
307
+ "align_items": null,
308
+ "align_self": null,
309
+ "border": null,
310
+ "bottom": null,
311
+ "display": null,
312
+ "flex": null,
313
+ "flex_flow": null,
314
+ "grid_area": null,
315
+ "grid_auto_columns": null,
316
+ "grid_auto_flow": null,
317
+ "grid_auto_rows": null,
318
+ "grid_column": null,
319
+ "grid_gap": null,
320
+ "grid_row": null,
321
+ "grid_template_areas": null,
322
+ "grid_template_columns": null,
323
+ "grid_template_rows": null,
324
+ "height": null,
325
+ "justify_content": null,
326
+ "justify_items": null,
327
+ "left": null,
328
+ "margin": null,
329
+ "max_height": null,
330
+ "max_width": null,
331
+ "min_height": null,
332
+ "min_width": null,
333
+ "object_fit": null,
334
+ "object_position": null,
335
+ "order": null,
336
+ "overflow": null,
337
+ "overflow_x": null,
338
+ "overflow_y": null,
339
+ "padding": null,
340
+ "right": null,
341
+ "top": null,
342
+ "visibility": null,
343
+ "width": null
344
+ }
345
+ },
346
+ "fad6c334971d4149b37eeee98d9b4d62": {
347
+ "model_module": "@jupyter-widgets/controls",
348
+ "model_name": "DescriptionStyleModel",
349
+ "model_module_version": "1.5.0",
350
+ "state": {
351
+ "_model_module": "@jupyter-widgets/controls",
352
+ "_model_module_version": "1.5.0",
353
+ "_model_name": "DescriptionStyleModel",
354
+ "_view_count": null,
355
+ "_view_module": "@jupyter-widgets/base",
356
+ "_view_module_version": "1.2.0",
357
+ "_view_name": "StyleView",
358
+ "description_width": ""
359
+ }
360
+ },
361
+ "11d40ffb85954661b268713060f3cef5": {
362
+ "model_module": "@jupyter-widgets/controls",
363
+ "model_name": "HBoxModel",
364
+ "model_module_version": "1.5.0",
365
+ "state": {
366
+ "_dom_classes": [],
367
+ "_model_module": "@jupyter-widgets/controls",
368
+ "_model_module_version": "1.5.0",
369
+ "_model_name": "HBoxModel",
370
+ "_view_count": null,
371
+ "_view_module": "@jupyter-widgets/controls",
372
+ "_view_module_version": "1.5.0",
373
+ "_view_name": "HBoxView",
374
+ "box_style": "",
375
+ "children": [
376
+ "IPY_MODEL_41c72ef64cd54336b0b8166e8961e47a",
377
+ "IPY_MODEL_ac0a83c40413408f89bda9ae635ca53a",
378
+ "IPY_MODEL_0af9f2620091468f86d293df3b87bf4b"
379
+ ],
380
+ "layout": "IPY_MODEL_9e2d5c239bbe4f1983e4777caaa7537a"
381
+ }
382
+ },
383
+ "41c72ef64cd54336b0b8166e8961e47a": {
384
+ "model_module": "@jupyter-widgets/controls",
385
+ "model_name": "HTMLModel",
386
+ "model_module_version": "1.5.0",
387
+ "state": {
388
+ "_dom_classes": [],
389
+ "_model_module": "@jupyter-widgets/controls",
390
+ "_model_module_version": "1.5.0",
391
+ "_model_name": "HTMLModel",
392
+ "_view_count": null,
393
+ "_view_module": "@jupyter-widgets/controls",
394
+ "_view_module_version": "1.5.0",
395
+ "_view_name": "HTMLView",
396
+ "description": "",
397
+ "description_tooltip": null,
398
+ "layout": "IPY_MODEL_2567eca5b8714bb7b3fffb063151a1fc",
399
+ "placeholder": "​",
400
+ "style": "IPY_MODEL_365b5e96a5cf405c905c957a51beb202",
401
+ "value": "model.safetensors: 100%"
402
+ }
403
+ },
404
+ "ac0a83c40413408f89bda9ae635ca53a": {
405
+ "model_module": "@jupyter-widgets/controls",
406
+ "model_name": "FloatProgressModel",
407
+ "model_module_version": "1.5.0",
408
+ "state": {
409
+ "_dom_classes": [],
410
+ "_model_module": "@jupyter-widgets/controls",
411
+ "_model_module_version": "1.5.0",
412
+ "_model_name": "FloatProgressModel",
413
+ "_view_count": null,
414
+ "_view_module": "@jupyter-widgets/controls",
415
+ "_view_module_version": "1.5.0",
416
+ "_view_name": "ProgressView",
417
+ "bar_style": "success",
418
+ "description": "",
419
+ "description_tooltip": null,
420
+ "layout": "IPY_MODEL_aeea1627a8ad4f93a38478ef5f31725c",
421
+ "max": 435755784,
422
+ "min": 0,
423
+ "orientation": "horizontal",
424
+ "style": "IPY_MODEL_15d2db319fde489abad449f9d778cd06",
425
+ "value": 435755784
426
+ }
427
+ },
428
+ "0af9f2620091468f86d293df3b87bf4b": {
429
+ "model_module": "@jupyter-widgets/controls",
430
+ "model_name": "HTMLModel",
431
+ "model_module_version": "1.5.0",
432
+ "state": {
433
+ "_dom_classes": [],
434
+ "_model_module": "@jupyter-widgets/controls",
435
+ "_model_module_version": "1.5.0",
436
+ "_model_name": "HTMLModel",
437
+ "_view_count": null,
438
+ "_view_module": "@jupyter-widgets/controls",
439
+ "_view_module_version": "1.5.0",
440
+ "_view_name": "HTMLView",
441
+ "description": "",
442
+ "description_tooltip": null,
443
+ "layout": "IPY_MODEL_0a96f9ca62174fb885b853c002809cff",
444
+ "placeholder": "​",
445
+ "style": "IPY_MODEL_f8ad6617b9234a98a6706bc613d9ef48",
446
+ "value": " 436M/436M [00:01<00:00, 245MB/s]"
447
+ }
448
+ },
449
+ "9e2d5c239bbe4f1983e4777caaa7537a": {
450
+ "model_module": "@jupyter-widgets/base",
451
+ "model_name": "LayoutModel",
452
+ "model_module_version": "1.2.0",
453
+ "state": {
454
+ "_model_module": "@jupyter-widgets/base",
455
+ "_model_module_version": "1.2.0",
456
+ "_model_name": "LayoutModel",
457
+ "_view_count": null,
458
+ "_view_module": "@jupyter-widgets/base",
459
+ "_view_module_version": "1.2.0",
460
+ "_view_name": "LayoutView",
461
+ "align_content": null,
462
+ "align_items": null,
463
+ "align_self": null,
464
+ "border": null,
465
+ "bottom": null,
466
+ "display": null,
467
+ "flex": null,
468
+ "flex_flow": null,
469
+ "grid_area": null,
470
+ "grid_auto_columns": null,
471
+ "grid_auto_flow": null,
472
+ "grid_auto_rows": null,
473
+ "grid_column": null,
474
+ "grid_gap": null,
475
+ "grid_row": null,
476
+ "grid_template_areas": null,
477
+ "grid_template_columns": null,
478
+ "grid_template_rows": null,
479
+ "height": null,
480
+ "justify_content": null,
481
+ "justify_items": null,
482
+ "left": null,
483
+ "margin": null,
484
+ "max_height": null,
485
+ "max_width": null,
486
+ "min_height": null,
487
+ "min_width": null,
488
+ "object_fit": null,
489
+ "object_position": null,
490
+ "order": null,
491
+ "overflow": null,
492
+ "overflow_x": null,
493
+ "overflow_y": null,
494
+ "padding": null,
495
+ "right": null,
496
+ "top": null,
497
+ "visibility": null,
498
+ "width": null
499
+ }
500
+ },
501
+ "2567eca5b8714bb7b3fffb063151a1fc": {
502
+ "model_module": "@jupyter-widgets/base",
503
+ "model_name": "LayoutModel",
504
+ "model_module_version": "1.2.0",
505
+ "state": {
506
+ "_model_module": "@jupyter-widgets/base",
507
+ "_model_module_version": "1.2.0",
508
+ "_model_name": "LayoutModel",
509
+ "_view_count": null,
510
+ "_view_module": "@jupyter-widgets/base",
511
+ "_view_module_version": "1.2.0",
512
+ "_view_name": "LayoutView",
513
+ "align_content": null,
514
+ "align_items": null,
515
+ "align_self": null,
516
+ "border": null,
517
+ "bottom": null,
518
+ "display": null,
519
+ "flex": null,
520
+ "flex_flow": null,
521
+ "grid_area": null,
522
+ "grid_auto_columns": null,
523
+ "grid_auto_flow": null,
524
+ "grid_auto_rows": null,
525
+ "grid_column": null,
526
+ "grid_gap": null,
527
+ "grid_row": null,
528
+ "grid_template_areas": null,
529
+ "grid_template_columns": null,
530
+ "grid_template_rows": null,
531
+ "height": null,
532
+ "justify_content": null,
533
+ "justify_items": null,
534
+ "left": null,
535
+ "margin": null,
536
+ "max_height": null,
537
+ "max_width": null,
538
+ "min_height": null,
539
+ "min_width": null,
540
+ "object_fit": null,
541
+ "object_position": null,
542
+ "order": null,
543
+ "overflow": null,
544
+ "overflow_x": null,
545
+ "overflow_y": null,
546
+ "padding": null,
547
+ "right": null,
548
+ "top": null,
549
+ "visibility": null,
550
+ "width": null
551
+ }
552
+ },
553
+ "365b5e96a5cf405c905c957a51beb202": {
554
+ "model_module": "@jupyter-widgets/controls",
555
+ "model_name": "DescriptionStyleModel",
556
+ "model_module_version": "1.5.0",
557
+ "state": {
558
+ "_model_module": "@jupyter-widgets/controls",
559
+ "_model_module_version": "1.5.0",
560
+ "_model_name": "DescriptionStyleModel",
561
+ "_view_count": null,
562
+ "_view_module": "@jupyter-widgets/base",
563
+ "_view_module_version": "1.2.0",
564
+ "_view_name": "StyleView",
565
+ "description_width": ""
566
+ }
567
+ },
568
+ "aeea1627a8ad4f93a38478ef5f31725c": {
569
+ "model_module": "@jupyter-widgets/base",
570
+ "model_name": "LayoutModel",
571
+ "model_module_version": "1.2.0",
572
+ "state": {
573
+ "_model_module": "@jupyter-widgets/base",
574
+ "_model_module_version": "1.2.0",
575
+ "_model_name": "LayoutModel",
576
+ "_view_count": null,
577
+ "_view_module": "@jupyter-widgets/base",
578
+ "_view_module_version": "1.2.0",
579
+ "_view_name": "LayoutView",
580
+ "align_content": null,
581
+ "align_items": null,
582
+ "align_self": null,
583
+ "border": null,
584
+ "bottom": null,
585
+ "display": null,
586
+ "flex": null,
587
+ "flex_flow": null,
588
+ "grid_area": null,
589
+ "grid_auto_columns": null,
590
+ "grid_auto_flow": null,
591
+ "grid_auto_rows": null,
592
+ "grid_column": null,
593
+ "grid_gap": null,
594
+ "grid_row": null,
595
+ "grid_template_areas": null,
596
+ "grid_template_columns": null,
597
+ "grid_template_rows": null,
598
+ "height": null,
599
+ "justify_content": null,
600
+ "justify_items": null,
601
+ "left": null,
602
+ "margin": null,
603
+ "max_height": null,
604
+ "max_width": null,
605
+ "min_height": null,
606
+ "min_width": null,
607
+ "object_fit": null,
608
+ "object_position": null,
609
+ "order": null,
610
+ "overflow": null,
611
+ "overflow_x": null,
612
+ "overflow_y": null,
613
+ "padding": null,
614
+ "right": null,
615
+ "top": null,
616
+ "visibility": null,
617
+ "width": null
618
+ }
619
+ },
620
+ "15d2db319fde489abad449f9d778cd06": {
621
+ "model_module": "@jupyter-widgets/controls",
622
+ "model_name": "ProgressStyleModel",
623
+ "model_module_version": "1.5.0",
624
+ "state": {
625
+ "_model_module": "@jupyter-widgets/controls",
626
+ "_model_module_version": "1.5.0",
627
+ "_model_name": "ProgressStyleModel",
628
+ "_view_count": null,
629
+ "_view_module": "@jupyter-widgets/base",
630
+ "_view_module_version": "1.2.0",
631
+ "_view_name": "StyleView",
632
+ "bar_color": null,
633
+ "description_width": ""
634
+ }
635
+ },
636
+ "0a96f9ca62174fb885b853c002809cff": {
637
+ "model_module": "@jupyter-widgets/base",
638
+ "model_name": "LayoutModel",
639
+ "model_module_version": "1.2.0",
640
+ "state": {
641
+ "_model_module": "@jupyter-widgets/base",
642
+ "_model_module_version": "1.2.0",
643
+ "_model_name": "LayoutModel",
644
+ "_view_count": null,
645
+ "_view_module": "@jupyter-widgets/base",
646
+ "_view_module_version": "1.2.0",
647
+ "_view_name": "LayoutView",
648
+ "align_content": null,
649
+ "align_items": null,
650
+ "align_self": null,
651
+ "border": null,
652
+ "bottom": null,
653
+ "display": null,
654
+ "flex": null,
655
+ "flex_flow": null,
656
+ "grid_area": null,
657
+ "grid_auto_columns": null,
658
+ "grid_auto_flow": null,
659
+ "grid_auto_rows": null,
660
+ "grid_column": null,
661
+ "grid_gap": null,
662
+ "grid_row": null,
663
+ "grid_template_areas": null,
664
+ "grid_template_columns": null,
665
+ "grid_template_rows": null,
666
+ "height": null,
667
+ "justify_content": null,
668
+ "justify_items": null,
669
+ "left": null,
670
+ "margin": null,
671
+ "max_height": null,
672
+ "max_width": null,
673
+ "min_height": null,
674
+ "min_width": null,
675
+ "object_fit": null,
676
+ "object_position": null,
677
+ "order": null,
678
+ "overflow": null,
679
+ "overflow_x": null,
680
+ "overflow_y": null,
681
+ "padding": null,
682
+ "right": null,
683
+ "top": null,
684
+ "visibility": null,
685
+ "width": null
686
+ }
687
+ },
688
+ "f8ad6617b9234a98a6706bc613d9ef48": {
689
+ "model_module": "@jupyter-widgets/controls",
690
+ "model_name": "DescriptionStyleModel",
691
+ "model_module_version": "1.5.0",
692
+ "state": {
693
+ "_model_module": "@jupyter-widgets/controls",
694
+ "_model_module_version": "1.5.0",
695
+ "_model_name": "DescriptionStyleModel",
696
+ "_view_count": null,
697
+ "_view_module": "@jupyter-widgets/base",
698
+ "_view_module_version": "1.2.0",
699
+ "_view_name": "StyleView",
700
+ "description_width": ""
701
+ }
702
+ },
703
+ "77ec6a31e7e54427b07c86503e812fa6": {
704
+ "model_module": "@jupyter-widgets/controls",
705
+ "model_name": "HBoxModel",
706
+ "model_module_version": "1.5.0",
707
+ "state": {
708
+ "_dom_classes": [],
709
+ "_model_module": "@jupyter-widgets/controls",
710
+ "_model_module_version": "1.5.0",
711
+ "_model_name": "HBoxModel",
712
+ "_view_count": null,
713
+ "_view_module": "@jupyter-widgets/controls",
714
+ "_view_module_version": "1.5.0",
715
+ "_view_name": "HBoxView",
716
+ "box_style": "",
717
+ "children": [
718
+ "IPY_MODEL_a868304889e541e4a96bdea7aa4d0ae6",
719
+ "IPY_MODEL_140baf68f4d4402e91519ab0900b5f2a",
720
+ "IPY_MODEL_91bc807afe7e4dff85b62ce17b6a3274"
721
+ ],
722
+ "layout": "IPY_MODEL_d2769b1c72034c72b35c61ecc3b24957"
723
+ }
724
+ },
725
+ "a868304889e541e4a96bdea7aa4d0ae6": {
726
+ "model_module": "@jupyter-widgets/controls",
727
+ "model_name": "HTMLModel",
728
+ "model_module_version": "1.5.0",
729
+ "state": {
730
+ "_dom_classes": [],
731
+ "_model_module": "@jupyter-widgets/controls",
732
+ "_model_module_version": "1.5.0",
733
+ "_model_name": "HTMLModel",
734
+ "_view_count": null,
735
+ "_view_module": "@jupyter-widgets/controls",
736
+ "_view_module_version": "1.5.0",
737
+ "_view_name": "HTMLView",
738
+ "description": "",
739
+ "description_tooltip": null,
740
+ "layout": "IPY_MODEL_7d42a14075894cc7a48f323879d95012",
741
+ "placeholder": "​",
742
+ "style": "IPY_MODEL_73b2babc82ea49cdbde79cba4ed8f576",
743
+ "value": "tokenizer_config.json: 100%"
744
+ }
745
+ },
746
+ "140baf68f4d4402e91519ab0900b5f2a": {
747
+ "model_module": "@jupyter-widgets/controls",
748
+ "model_name": "FloatProgressModel",
749
+ "model_module_version": "1.5.0",
750
+ "state": {
751
+ "_dom_classes": [],
752
+ "_model_module": "@jupyter-widgets/controls",
753
+ "_model_module_version": "1.5.0",
754
+ "_model_name": "FloatProgressModel",
755
+ "_view_count": null,
756
+ "_view_module": "@jupyter-widgets/controls",
757
+ "_view_module_version": "1.5.0",
758
+ "_view_name": "ProgressView",
759
+ "bar_style": "success",
760
+ "description": "",
761
+ "description_tooltip": null,
762
+ "layout": "IPY_MODEL_6c33a9324d5e49ee89731cda9c0c608e",
763
+ "max": 49,
764
+ "min": 0,
765
+ "orientation": "horizontal",
766
+ "style": "IPY_MODEL_10129c9a736c4c8badbd76ce47445de6",
767
+ "value": 49
768
+ }
769
+ },
770
+ "91bc807afe7e4dff85b62ce17b6a3274": {
771
+ "model_module": "@jupyter-widgets/controls",
772
+ "model_name": "HTMLModel",
773
+ "model_module_version": "1.5.0",
774
+ "state": {
775
+ "_dom_classes": [],
776
+ "_model_module": "@jupyter-widgets/controls",
777
+ "_model_module_version": "1.5.0",
778
+ "_model_name": "HTMLModel",
779
+ "_view_count": null,
780
+ "_view_module": "@jupyter-widgets/controls",
781
+ "_view_module_version": "1.5.0",
782
+ "_view_name": "HTMLView",
783
+ "description": "",
784
+ "description_tooltip": null,
785
+ "layout": "IPY_MODEL_3b3478a41eed44898b9d94c9d7d1b32b",
786
+ "placeholder": "​",
787
+ "style": "IPY_MODEL_29ef19cc3b6a4acaab0b829b602e6c32",
788
+ "value": " 49.0/49.0 [00:00<00:00, 1.46kB/s]"
789
+ }
790
+ },
791
+ "d2769b1c72034c72b35c61ecc3b24957": {
792
+ "model_module": "@jupyter-widgets/base",
793
+ "model_name": "LayoutModel",
794
+ "model_module_version": "1.2.0",
795
+ "state": {
796
+ "_model_module": "@jupyter-widgets/base",
797
+ "_model_module_version": "1.2.0",
798
+ "_model_name": "LayoutModel",
799
+ "_view_count": null,
800
+ "_view_module": "@jupyter-widgets/base",
801
+ "_view_module_version": "1.2.0",
802
+ "_view_name": "LayoutView",
803
+ "align_content": null,
804
+ "align_items": null,
805
+ "align_self": null,
806
+ "border": null,
807
+ "bottom": null,
808
+ "display": null,
809
+ "flex": null,
810
+ "flex_flow": null,
811
+ "grid_area": null,
812
+ "grid_auto_columns": null,
813
+ "grid_auto_flow": null,
814
+ "grid_auto_rows": null,
815
+ "grid_column": null,
816
+ "grid_gap": null,
817
+ "grid_row": null,
818
+ "grid_template_areas": null,
819
+ "grid_template_columns": null,
820
+ "grid_template_rows": null,
821
+ "height": null,
822
+ "justify_content": null,
823
+ "justify_items": null,
824
+ "left": null,
825
+ "margin": null,
826
+ "max_height": null,
827
+ "max_width": null,
828
+ "min_height": null,
829
+ "min_width": null,
830
+ "object_fit": null,
831
+ "object_position": null,
832
+ "order": null,
833
+ "overflow": null,
834
+ "overflow_x": null,
835
+ "overflow_y": null,
836
+ "padding": null,
837
+ "right": null,
838
+ "top": null,
839
+ "visibility": null,
840
+ "width": null
841
+ }
842
+ },
843
+ "7d42a14075894cc7a48f323879d95012": {
844
+ "model_module": "@jupyter-widgets/base",
845
+ "model_name": "LayoutModel",
846
+ "model_module_version": "1.2.0",
847
+ "state": {
848
+ "_model_module": "@jupyter-widgets/base",
849
+ "_model_module_version": "1.2.0",
850
+ "_model_name": "LayoutModel",
851
+ "_view_count": null,
852
+ "_view_module": "@jupyter-widgets/base",
853
+ "_view_module_version": "1.2.0",
854
+ "_view_name": "LayoutView",
855
+ "align_content": null,
856
+ "align_items": null,
857
+ "align_self": null,
858
+ "border": null,
859
+ "bottom": null,
860
+ "display": null,
861
+ "flex": null,
862
+ "flex_flow": null,
863
+ "grid_area": null,
864
+ "grid_auto_columns": null,
865
+ "grid_auto_flow": null,
866
+ "grid_auto_rows": null,
867
+ "grid_column": null,
868
+ "grid_gap": null,
869
+ "grid_row": null,
870
+ "grid_template_areas": null,
871
+ "grid_template_columns": null,
872
+ "grid_template_rows": null,
873
+ "height": null,
874
+ "justify_content": null,
875
+ "justify_items": null,
876
+ "left": null,
877
+ "margin": null,
878
+ "max_height": null,
879
+ "max_width": null,
880
+ "min_height": null,
881
+ "min_width": null,
882
+ "object_fit": null,
883
+ "object_position": null,
884
+ "order": null,
885
+ "overflow": null,
886
+ "overflow_x": null,
887
+ "overflow_y": null,
888
+ "padding": null,
889
+ "right": null,
890
+ "top": null,
891
+ "visibility": null,
892
+ "width": null
893
+ }
894
+ },
895
+ "73b2babc82ea49cdbde79cba4ed8f576": {
896
+ "model_module": "@jupyter-widgets/controls",
897
+ "model_name": "DescriptionStyleModel",
898
+ "model_module_version": "1.5.0",
899
+ "state": {
900
+ "_model_module": "@jupyter-widgets/controls",
901
+ "_model_module_version": "1.5.0",
902
+ "_model_name": "DescriptionStyleModel",
903
+ "_view_count": null,
904
+ "_view_module": "@jupyter-widgets/base",
905
+ "_view_module_version": "1.2.0",
906
+ "_view_name": "StyleView",
907
+ "description_width": ""
908
+ }
909
+ },
910
+ "6c33a9324d5e49ee89731cda9c0c608e": {
911
+ "model_module": "@jupyter-widgets/base",
912
+ "model_name": "LayoutModel",
913
+ "model_module_version": "1.2.0",
914
+ "state": {
915
+ "_model_module": "@jupyter-widgets/base",
916
+ "_model_module_version": "1.2.0",
917
+ "_model_name": "LayoutModel",
918
+ "_view_count": null,
919
+ "_view_module": "@jupyter-widgets/base",
920
+ "_view_module_version": "1.2.0",
921
+ "_view_name": "LayoutView",
922
+ "align_content": null,
923
+ "align_items": null,
924
+ "align_self": null,
925
+ "border": null,
926
+ "bottom": null,
927
+ "display": null,
928
+ "flex": null,
929
+ "flex_flow": null,
930
+ "grid_area": null,
931
+ "grid_auto_columns": null,
932
+ "grid_auto_flow": null,
933
+ "grid_auto_rows": null,
934
+ "grid_column": null,
935
+ "grid_gap": null,
936
+ "grid_row": null,
937
+ "grid_template_areas": null,
938
+ "grid_template_columns": null,
939
+ "grid_template_rows": null,
940
+ "height": null,
941
+ "justify_content": null,
942
+ "justify_items": null,
943
+ "left": null,
944
+ "margin": null,
945
+ "max_height": null,
946
+ "max_width": null,
947
+ "min_height": null,
948
+ "min_width": null,
949
+ "object_fit": null,
950
+ "object_position": null,
951
+ "order": null,
952
+ "overflow": null,
953
+ "overflow_x": null,
954
+ "overflow_y": null,
955
+ "padding": null,
956
+ "right": null,
957
+ "top": null,
958
+ "visibility": null,
959
+ "width": null
960
+ }
961
+ },
962
+ "10129c9a736c4c8badbd76ce47445de6": {
963
+ "model_module": "@jupyter-widgets/controls",
964
+ "model_name": "ProgressStyleModel",
965
+ "model_module_version": "1.5.0",
966
+ "state": {
967
+ "_model_module": "@jupyter-widgets/controls",
968
+ "_model_module_version": "1.5.0",
969
+ "_model_name": "ProgressStyleModel",
970
+ "_view_count": null,
971
+ "_view_module": "@jupyter-widgets/base",
972
+ "_view_module_version": "1.2.0",
973
+ "_view_name": "StyleView",
974
+ "bar_color": null,
975
+ "description_width": ""
976
+ }
977
+ },
978
+ "3b3478a41eed44898b9d94c9d7d1b32b": {
979
+ "model_module": "@jupyter-widgets/base",
980
+ "model_name": "LayoutModel",
981
+ "model_module_version": "1.2.0",
982
+ "state": {
983
+ "_model_module": "@jupyter-widgets/base",
984
+ "_model_module_version": "1.2.0",
985
+ "_model_name": "LayoutModel",
986
+ "_view_count": null,
987
+ "_view_module": "@jupyter-widgets/base",
988
+ "_view_module_version": "1.2.0",
989
+ "_view_name": "LayoutView",
990
+ "align_content": null,
991
+ "align_items": null,
992
+ "align_self": null,
993
+ "border": null,
994
+ "bottom": null,
995
+ "display": null,
996
+ "flex": null,
997
+ "flex_flow": null,
998
+ "grid_area": null,
999
+ "grid_auto_columns": null,
1000
+ "grid_auto_flow": null,
1001
+ "grid_auto_rows": null,
1002
+ "grid_column": null,
1003
+ "grid_gap": null,
1004
+ "grid_row": null,
1005
+ "grid_template_areas": null,
1006
+ "grid_template_columns": null,
1007
+ "grid_template_rows": null,
1008
+ "height": null,
1009
+ "justify_content": null,
1010
+ "justify_items": null,
1011
+ "left": null,
1012
+ "margin": null,
1013
+ "max_height": null,
1014
+ "max_width": null,
1015
+ "min_height": null,
1016
+ "min_width": null,
1017
+ "object_fit": null,
1018
+ "object_position": null,
1019
+ "order": null,
1020
+ "overflow": null,
1021
+ "overflow_x": null,
1022
+ "overflow_y": null,
1023
+ "padding": null,
1024
+ "right": null,
1025
+ "top": null,
1026
+ "visibility": null,
1027
+ "width": null
1028
+ }
1029
+ },
1030
+ "29ef19cc3b6a4acaab0b829b602e6c32": {
1031
+ "model_module": "@jupyter-widgets/controls",
1032
+ "model_name": "DescriptionStyleModel",
1033
+ "model_module_version": "1.5.0",
1034
+ "state": {
1035
+ "_model_module": "@jupyter-widgets/controls",
1036
+ "_model_module_version": "1.5.0",
1037
+ "_model_name": "DescriptionStyleModel",
1038
+ "_view_count": null,
1039
+ "_view_module": "@jupyter-widgets/base",
1040
+ "_view_module_version": "1.2.0",
1041
+ "_view_name": "StyleView",
1042
+ "description_width": ""
1043
+ }
1044
+ },
1045
+ "6b3af6e6f7b74070847fd34c99c81e8c": {
1046
+ "model_module": "@jupyter-widgets/controls",
1047
+ "model_name": "HBoxModel",
1048
+ "model_module_version": "1.5.0",
1049
+ "state": {
1050
+ "_dom_classes": [],
1051
+ "_model_module": "@jupyter-widgets/controls",
1052
+ "_model_module_version": "1.5.0",
1053
+ "_model_name": "HBoxModel",
1054
+ "_view_count": null,
1055
+ "_view_module": "@jupyter-widgets/controls",
1056
+ "_view_module_version": "1.5.0",
1057
+ "_view_name": "HBoxView",
1058
+ "box_style": "",
1059
+ "children": [
1060
+ "IPY_MODEL_4c68eb2adebb41ffbca4a18d14e43036",
1061
+ "IPY_MODEL_275689117c544278a3938e3e74fc73b7",
1062
+ "IPY_MODEL_96e180382d994c81ad8fa82f3d7cccbf"
1063
+ ],
1064
+ "layout": "IPY_MODEL_b741ce6f204a4e90adb70a3947f15b94"
1065
+ }
1066
+ },
1067
+ "4c68eb2adebb41ffbca4a18d14e43036": {
1068
+ "model_module": "@jupyter-widgets/controls",
1069
+ "model_name": "HTMLModel",
1070
+ "model_module_version": "1.5.0",
1071
+ "state": {
1072
+ "_dom_classes": [],
1073
+ "_model_module": "@jupyter-widgets/controls",
1074
+ "_model_module_version": "1.5.0",
1075
+ "_model_name": "HTMLModel",
1076
+ "_view_count": null,
1077
+ "_view_module": "@jupyter-widgets/controls",
1078
+ "_view_module_version": "1.5.0",
1079
+ "_view_name": "HTMLView",
1080
+ "description": "",
1081
+ "description_tooltip": null,
1082
+ "layout": "IPY_MODEL_527a3fd23ed449fbbd442c0bca97ec99",
1083
+ "placeholder": "​",
1084
+ "style": "IPY_MODEL_7466e19463d94e7fb8cae2b11f1da32a",
1085
+ "value": "config.json: 100%"
1086
+ }
1087
+ },
1088
+ "275689117c544278a3938e3e74fc73b7": {
1089
+ "model_module": "@jupyter-widgets/controls",
1090
+ "model_name": "FloatProgressModel",
1091
+ "model_module_version": "1.5.0",
1092
+ "state": {
1093
+ "_dom_classes": [],
1094
+ "_model_module": "@jupyter-widgets/controls",
1095
+ "_model_module_version": "1.5.0",
1096
+ "_model_name": "FloatProgressModel",
1097
+ "_view_count": null,
1098
+ "_view_module": "@jupyter-widgets/controls",
1099
+ "_view_module_version": "1.5.0",
1100
+ "_view_name": "ProgressView",
1101
+ "bar_style": "success",
1102
+ "description": "",
1103
+ "description_tooltip": null,
1104
+ "layout": "IPY_MODEL_dada7561d1b14b369e3439f65bce905b",
1105
+ "max": 570,
1106
+ "min": 0,
1107
+ "orientation": "horizontal",
1108
+ "style": "IPY_MODEL_7c97f46de47c48b8b29d0b5cfdc3329c",
1109
+ "value": 570
1110
+ }
1111
+ },
1112
+ "96e180382d994c81ad8fa82f3d7cccbf": {
1113
+ "model_module": "@jupyter-widgets/controls",
1114
+ "model_name": "HTMLModel",
1115
+ "model_module_version": "1.5.0",
1116
+ "state": {
1117
+ "_dom_classes": [],
1118
+ "_model_module": "@jupyter-widgets/controls",
1119
+ "_model_module_version": "1.5.0",
1120
+ "_model_name": "HTMLModel",
1121
+ "_view_count": null,
1122
+ "_view_module": "@jupyter-widgets/controls",
1123
+ "_view_module_version": "1.5.0",
1124
+ "_view_name": "HTMLView",
1125
+ "description": "",
1126
+ "description_tooltip": null,
1127
+ "layout": "IPY_MODEL_650297067bd24acca2296a37ccf5e692",
1128
+ "placeholder": "​",
1129
+ "style": "IPY_MODEL_c91656245d474d14a57914438bdc005b",
1130
+ "value": " 570/570 [00:00<00:00, 33.1kB/s]"
1131
+ }
1132
+ },
1133
+ "b741ce6f204a4e90adb70a3947f15b94": {
1134
+ "model_module": "@jupyter-widgets/base",
1135
+ "model_name": "LayoutModel",
1136
+ "model_module_version": "1.2.0",
1137
+ "state": {
1138
+ "_model_module": "@jupyter-widgets/base",
1139
+ "_model_module_version": "1.2.0",
1140
+ "_model_name": "LayoutModel",
1141
+ "_view_count": null,
1142
+ "_view_module": "@jupyter-widgets/base",
1143
+ "_view_module_version": "1.2.0",
1144
+ "_view_name": "LayoutView",
1145
+ "align_content": null,
1146
+ "align_items": null,
1147
+ "align_self": null,
1148
+ "border": null,
1149
+ "bottom": null,
1150
+ "display": null,
1151
+ "flex": null,
1152
+ "flex_flow": null,
1153
+ "grid_area": null,
1154
+ "grid_auto_columns": null,
1155
+ "grid_auto_flow": null,
1156
+ "grid_auto_rows": null,
1157
+ "grid_column": null,
1158
+ "grid_gap": null,
1159
+ "grid_row": null,
1160
+ "grid_template_areas": null,
1161
+ "grid_template_columns": null,
1162
+ "grid_template_rows": null,
1163
+ "height": null,
1164
+ "justify_content": null,
1165
+ "justify_items": null,
1166
+ "left": null,
1167
+ "margin": null,
1168
+ "max_height": null,
1169
+ "max_width": null,
1170
+ "min_height": null,
1171
+ "min_width": null,
1172
+ "object_fit": null,
1173
+ "object_position": null,
1174
+ "order": null,
1175
+ "overflow": null,
1176
+ "overflow_x": null,
1177
+ "overflow_y": null,
1178
+ "padding": null,
1179
+ "right": null,
1180
+ "top": null,
1181
+ "visibility": null,
1182
+ "width": null
1183
+ }
1184
+ },
1185
+ "527a3fd23ed449fbbd442c0bca97ec99": {
1186
+ "model_module": "@jupyter-widgets/base",
1187
+ "model_name": "LayoutModel",
1188
+ "model_module_version": "1.2.0",
1189
+ "state": {
1190
+ "_model_module": "@jupyter-widgets/base",
1191
+ "_model_module_version": "1.2.0",
1192
+ "_model_name": "LayoutModel",
1193
+ "_view_count": null,
1194
+ "_view_module": "@jupyter-widgets/base",
1195
+ "_view_module_version": "1.2.0",
1196
+ "_view_name": "LayoutView",
1197
+ "align_content": null,
1198
+ "align_items": null,
1199
+ "align_self": null,
1200
+ "border": null,
1201
+ "bottom": null,
1202
+ "display": null,
1203
+ "flex": null,
1204
+ "flex_flow": null,
1205
+ "grid_area": null,
1206
+ "grid_auto_columns": null,
1207
+ "grid_auto_flow": null,
1208
+ "grid_auto_rows": null,
1209
+ "grid_column": null,
1210
+ "grid_gap": null,
1211
+ "grid_row": null,
1212
+ "grid_template_areas": null,
1213
+ "grid_template_columns": null,
1214
+ "grid_template_rows": null,
1215
+ "height": null,
1216
+ "justify_content": null,
1217
+ "justify_items": null,
1218
+ "left": null,
1219
+ "margin": null,
1220
+ "max_height": null,
1221
+ "max_width": null,
1222
+ "min_height": null,
1223
+ "min_width": null,
1224
+ "object_fit": null,
1225
+ "object_position": null,
1226
+ "order": null,
1227
+ "overflow": null,
1228
+ "overflow_x": null,
1229
+ "overflow_y": null,
1230
+ "padding": null,
1231
+ "right": null,
1232
+ "top": null,
1233
+ "visibility": null,
1234
+ "width": null
1235
+ }
1236
+ },
1237
+ "7466e19463d94e7fb8cae2b11f1da32a": {
1238
+ "model_module": "@jupyter-widgets/controls",
1239
+ "model_name": "DescriptionStyleModel",
1240
+ "model_module_version": "1.5.0",
1241
+ "state": {
1242
+ "_model_module": "@jupyter-widgets/controls",
1243
+ "_model_module_version": "1.5.0",
1244
+ "_model_name": "DescriptionStyleModel",
1245
+ "_view_count": null,
1246
+ "_view_module": "@jupyter-widgets/base",
1247
+ "_view_module_version": "1.2.0",
1248
+ "_view_name": "StyleView",
1249
+ "description_width": ""
1250
+ }
1251
+ },
1252
+ "dada7561d1b14b369e3439f65bce905b": {
1253
+ "model_module": "@jupyter-widgets/base",
1254
+ "model_name": "LayoutModel",
1255
+ "model_module_version": "1.2.0",
1256
+ "state": {
1257
+ "_model_module": "@jupyter-widgets/base",
1258
+ "_model_module_version": "1.2.0",
1259
+ "_model_name": "LayoutModel",
1260
+ "_view_count": null,
1261
+ "_view_module": "@jupyter-widgets/base",
1262
+ "_view_module_version": "1.2.0",
1263
+ "_view_name": "LayoutView",
1264
+ "align_content": null,
1265
+ "align_items": null,
1266
+ "align_self": null,
1267
+ "border": null,
1268
+ "bottom": null,
1269
+ "display": null,
1270
+ "flex": null,
1271
+ "flex_flow": null,
1272
+ "grid_area": null,
1273
+ "grid_auto_columns": null,
1274
+ "grid_auto_flow": null,
1275
+ "grid_auto_rows": null,
1276
+ "grid_column": null,
1277
+ "grid_gap": null,
1278
+ "grid_row": null,
1279
+ "grid_template_areas": null,
1280
+ "grid_template_columns": null,
1281
+ "grid_template_rows": null,
1282
+ "height": null,
1283
+ "justify_content": null,
1284
+ "justify_items": null,
1285
+ "left": null,
1286
+ "margin": null,
1287
+ "max_height": null,
1288
+ "max_width": null,
1289
+ "min_height": null,
1290
+ "min_width": null,
1291
+ "object_fit": null,
1292
+ "object_position": null,
1293
+ "order": null,
1294
+ "overflow": null,
1295
+ "overflow_x": null,
1296
+ "overflow_y": null,
1297
+ "padding": null,
1298
+ "right": null,
1299
+ "top": null,
1300
+ "visibility": null,
1301
+ "width": null
1302
+ }
1303
+ },
1304
+ "7c97f46de47c48b8b29d0b5cfdc3329c": {
1305
+ "model_module": "@jupyter-widgets/controls",
1306
+ "model_name": "ProgressStyleModel",
1307
+ "model_module_version": "1.5.0",
1308
+ "state": {
1309
+ "_model_module": "@jupyter-widgets/controls",
1310
+ "_model_module_version": "1.5.0",
1311
+ "_model_name": "ProgressStyleModel",
1312
+ "_view_count": null,
1313
+ "_view_module": "@jupyter-widgets/base",
1314
+ "_view_module_version": "1.2.0",
1315
+ "_view_name": "StyleView",
1316
+ "bar_color": null,
1317
+ "description_width": ""
1318
+ }
1319
+ },
1320
+ "650297067bd24acca2296a37ccf5e692": {
1321
+ "model_module": "@jupyter-widgets/base",
1322
+ "model_name": "LayoutModel",
1323
+ "model_module_version": "1.2.0",
1324
+ "state": {
1325
+ "_model_module": "@jupyter-widgets/base",
1326
+ "_model_module_version": "1.2.0",
1327
+ "_model_name": "LayoutModel",
1328
+ "_view_count": null,
1329
+ "_view_module": "@jupyter-widgets/base",
1330
+ "_view_module_version": "1.2.0",
1331
+ "_view_name": "LayoutView",
1332
+ "align_content": null,
1333
+ "align_items": null,
1334
+ "align_self": null,
1335
+ "border": null,
1336
+ "bottom": null,
1337
+ "display": null,
1338
+ "flex": null,
1339
+ "flex_flow": null,
1340
+ "grid_area": null,
1341
+ "grid_auto_columns": null,
1342
+ "grid_auto_flow": null,
1343
+ "grid_auto_rows": null,
1344
+ "grid_column": null,
1345
+ "grid_gap": null,
1346
+ "grid_row": null,
1347
+ "grid_template_areas": null,
1348
+ "grid_template_columns": null,
1349
+ "grid_template_rows": null,
1350
+ "height": null,
1351
+ "justify_content": null,
1352
+ "justify_items": null,
1353
+ "left": null,
1354
+ "margin": null,
1355
+ "max_height": null,
1356
+ "max_width": null,
1357
+ "min_height": null,
1358
+ "min_width": null,
1359
+ "object_fit": null,
1360
+ "object_position": null,
1361
+ "order": null,
1362
+ "overflow": null,
1363
+ "overflow_x": null,
1364
+ "overflow_y": null,
1365
+ "padding": null,
1366
+ "right": null,
1367
+ "top": null,
1368
+ "visibility": null,
1369
+ "width": null
1370
+ }
1371
+ },
1372
+ "c91656245d474d14a57914438bdc005b": {
1373
+ "model_module": "@jupyter-widgets/controls",
1374
+ "model_name": "DescriptionStyleModel",
1375
+ "model_module_version": "1.5.0",
1376
+ "state": {
1377
+ "_model_module": "@jupyter-widgets/controls",
1378
+ "_model_module_version": "1.5.0",
1379
+ "_model_name": "DescriptionStyleModel",
1380
+ "_view_count": null,
1381
+ "_view_module": "@jupyter-widgets/base",
1382
+ "_view_module_version": "1.2.0",
1383
+ "_view_name": "StyleView",
1384
+ "description_width": ""
1385
+ }
1386
+ },
1387
+ "f2f5e0196e4c4930835965907a91d4ef": {
1388
+ "model_module": "@jupyter-widgets/controls",
1389
+ "model_name": "HBoxModel",
1390
+ "model_module_version": "1.5.0",
1391
+ "state": {
1392
+ "_dom_classes": [],
1393
+ "_model_module": "@jupyter-widgets/controls",
1394
+ "_model_module_version": "1.5.0",
1395
+ "_model_name": "HBoxModel",
1396
+ "_view_count": null,
1397
+ "_view_module": "@jupyter-widgets/controls",
1398
+ "_view_module_version": "1.5.0",
1399
+ "_view_name": "HBoxView",
1400
+ "box_style": "",
1401
+ "children": [
1402
+ "IPY_MODEL_e922fab084df4b2a85bd75ce1a8e1f32",
1403
+ "IPY_MODEL_dec069adff43430c9e3d614ebef0a812",
1404
+ "IPY_MODEL_34dcce4b5388410b941856240227fc9b"
1405
+ ],
1406
+ "layout": "IPY_MODEL_cd8ac9c645a54283b0dfb87d1cdca7f3"
1407
+ }
1408
+ },
1409
+ "e922fab084df4b2a85bd75ce1a8e1f32": {
1410
+ "model_module": "@jupyter-widgets/controls",
1411
+ "model_name": "HTMLModel",
1412
+ "model_module_version": "1.5.0",
1413
+ "state": {
1414
+ "_dom_classes": [],
1415
+ "_model_module": "@jupyter-widgets/controls",
1416
+ "_model_module_version": "1.5.0",
1417
+ "_model_name": "HTMLModel",
1418
+ "_view_count": null,
1419
+ "_view_module": "@jupyter-widgets/controls",
1420
+ "_view_module_version": "1.5.0",
1421
+ "_view_name": "HTMLView",
1422
+ "description": "",
1423
+ "description_tooltip": null,
1424
+ "layout": "IPY_MODEL_ef2c7480f288478c90bab998502390d2",
1425
+ "placeholder": "​",
1426
+ "style": "IPY_MODEL_1d28c9a7f7404c6c94cd371f70461296",
1427
+ "value": "vocab.txt: 100%"
1428
+ }
1429
+ },
1430
+ "dec069adff43430c9e3d614ebef0a812": {
1431
+ "model_module": "@jupyter-widgets/controls",
1432
+ "model_name": "FloatProgressModel",
1433
+ "model_module_version": "1.5.0",
1434
+ "state": {
1435
+ "_dom_classes": [],
1436
+ "_model_module": "@jupyter-widgets/controls",
1437
+ "_model_module_version": "1.5.0",
1438
+ "_model_name": "FloatProgressModel",
1439
+ "_view_count": null,
1440
+ "_view_module": "@jupyter-widgets/controls",
1441
+ "_view_module_version": "1.5.0",
1442
+ "_view_name": "ProgressView",
1443
+ "bar_style": "success",
1444
+ "description": "",
1445
+ "description_tooltip": null,
1446
+ "layout": "IPY_MODEL_b9c89fa1bc6240e7b8e3dc3de6dda086",
1447
+ "max": 213450,
1448
+ "min": 0,
1449
+ "orientation": "horizontal",
1450
+ "style": "IPY_MODEL_c913354d06ed4ac2a18eaa4993f3f97c",
1451
+ "value": 213450
1452
+ }
1453
+ },
1454
+ "34dcce4b5388410b941856240227fc9b": {
1455
+ "model_module": "@jupyter-widgets/controls",
1456
+ "model_name": "HTMLModel",
1457
+ "model_module_version": "1.5.0",
1458
+ "state": {
1459
+ "_dom_classes": [],
1460
+ "_model_module": "@jupyter-widgets/controls",
1461
+ "_model_module_version": "1.5.0",
1462
+ "_model_name": "HTMLModel",
1463
+ "_view_count": null,
1464
+ "_view_module": "@jupyter-widgets/controls",
1465
+ "_view_module_version": "1.5.0",
1466
+ "_view_name": "HTMLView",
1467
+ "description": "",
1468
+ "description_tooltip": null,
1469
+ "layout": "IPY_MODEL_10d4262936134c488ca247e1c79180c3",
1470
+ "placeholder": "​",
1471
+ "style": "IPY_MODEL_9ec51f7e8c484cd3945f36d9bc35b430",
1472
+ "value": " 213k/213k [00:00<00:00, 4.48MB/s]"
1473
+ }
1474
+ },
1475
+ "cd8ac9c645a54283b0dfb87d1cdca7f3": {
1476
+ "model_module": "@jupyter-widgets/base",
1477
+ "model_name": "LayoutModel",
1478
+ "model_module_version": "1.2.0",
1479
+ "state": {
1480
+ "_model_module": "@jupyter-widgets/base",
1481
+ "_model_module_version": "1.2.0",
1482
+ "_model_name": "LayoutModel",
1483
+ "_view_count": null,
1484
+ "_view_module": "@jupyter-widgets/base",
1485
+ "_view_module_version": "1.2.0",
1486
+ "_view_name": "LayoutView",
1487
+ "align_content": null,
1488
+ "align_items": null,
1489
+ "align_self": null,
1490
+ "border": null,
1491
+ "bottom": null,
1492
+ "display": null,
1493
+ "flex": null,
1494
+ "flex_flow": null,
1495
+ "grid_area": null,
1496
+ "grid_auto_columns": null,
1497
+ "grid_auto_flow": null,
1498
+ "grid_auto_rows": null,
1499
+ "grid_column": null,
1500
+ "grid_gap": null,
1501
+ "grid_row": null,
1502
+ "grid_template_areas": null,
1503
+ "grid_template_columns": null,
1504
+ "grid_template_rows": null,
1505
+ "height": null,
1506
+ "justify_content": null,
1507
+ "justify_items": null,
1508
+ "left": null,
1509
+ "margin": null,
1510
+ "max_height": null,
1511
+ "max_width": null,
1512
+ "min_height": null,
1513
+ "min_width": null,
1514
+ "object_fit": null,
1515
+ "object_position": null,
1516
+ "order": null,
1517
+ "overflow": null,
1518
+ "overflow_x": null,
1519
+ "overflow_y": null,
1520
+ "padding": null,
1521
+ "right": null,
1522
+ "top": null,
1523
+ "visibility": null,
1524
+ "width": null
1525
+ }
1526
+ },
1527
+ "ef2c7480f288478c90bab998502390d2": {
1528
+ "model_module": "@jupyter-widgets/base",
1529
+ "model_name": "LayoutModel",
1530
+ "model_module_version": "1.2.0",
1531
+ "state": {
1532
+ "_model_module": "@jupyter-widgets/base",
1533
+ "_model_module_version": "1.2.0",
1534
+ "_model_name": "LayoutModel",
1535
+ "_view_count": null,
1536
+ "_view_module": "@jupyter-widgets/base",
1537
+ "_view_module_version": "1.2.0",
1538
+ "_view_name": "LayoutView",
1539
+ "align_content": null,
1540
+ "align_items": null,
1541
+ "align_self": null,
1542
+ "border": null,
1543
+ "bottom": null,
1544
+ "display": null,
1545
+ "flex": null,
1546
+ "flex_flow": null,
1547
+ "grid_area": null,
1548
+ "grid_auto_columns": null,
1549
+ "grid_auto_flow": null,
1550
+ "grid_auto_rows": null,
1551
+ "grid_column": null,
1552
+ "grid_gap": null,
1553
+ "grid_row": null,
1554
+ "grid_template_areas": null,
1555
+ "grid_template_columns": null,
1556
+ "grid_template_rows": null,
1557
+ "height": null,
1558
+ "justify_content": null,
1559
+ "justify_items": null,
1560
+ "left": null,
1561
+ "margin": null,
1562
+ "max_height": null,
1563
+ "max_width": null,
1564
+ "min_height": null,
1565
+ "min_width": null,
1566
+ "object_fit": null,
1567
+ "object_position": null,
1568
+ "order": null,
1569
+ "overflow": null,
1570
+ "overflow_x": null,
1571
+ "overflow_y": null,
1572
+ "padding": null,
1573
+ "right": null,
1574
+ "top": null,
1575
+ "visibility": null,
1576
+ "width": null
1577
+ }
1578
+ },
1579
+ "1d28c9a7f7404c6c94cd371f70461296": {
1580
+ "model_module": "@jupyter-widgets/controls",
1581
+ "model_name": "DescriptionStyleModel",
1582
+ "model_module_version": "1.5.0",
1583
+ "state": {
1584
+ "_model_module": "@jupyter-widgets/controls",
1585
+ "_model_module_version": "1.5.0",
1586
+ "_model_name": "DescriptionStyleModel",
1587
+ "_view_count": null,
1588
+ "_view_module": "@jupyter-widgets/base",
1589
+ "_view_module_version": "1.2.0",
1590
+ "_view_name": "StyleView",
1591
+ "description_width": ""
1592
+ }
1593
+ },
1594
+ "b9c89fa1bc6240e7b8e3dc3de6dda086": {
1595
+ "model_module": "@jupyter-widgets/base",
1596
+ "model_name": "LayoutModel",
1597
+ "model_module_version": "1.2.0",
1598
+ "state": {
1599
+ "_model_module": "@jupyter-widgets/base",
1600
+ "_model_module_version": "1.2.0",
1601
+ "_model_name": "LayoutModel",
1602
+ "_view_count": null,
1603
+ "_view_module": "@jupyter-widgets/base",
1604
+ "_view_module_version": "1.2.0",
1605
+ "_view_name": "LayoutView",
1606
+ "align_content": null,
1607
+ "align_items": null,
1608
+ "align_self": null,
1609
+ "border": null,
1610
+ "bottom": null,
1611
+ "display": null,
1612
+ "flex": null,
1613
+ "flex_flow": null,
1614
+ "grid_area": null,
1615
+ "grid_auto_columns": null,
1616
+ "grid_auto_flow": null,
1617
+ "grid_auto_rows": null,
1618
+ "grid_column": null,
1619
+ "grid_gap": null,
1620
+ "grid_row": null,
1621
+ "grid_template_areas": null,
1622
+ "grid_template_columns": null,
1623
+ "grid_template_rows": null,
1624
+ "height": null,
1625
+ "justify_content": null,
1626
+ "justify_items": null,
1627
+ "left": null,
1628
+ "margin": null,
1629
+ "max_height": null,
1630
+ "max_width": null,
1631
+ "min_height": null,
1632
+ "min_width": null,
1633
+ "object_fit": null,
1634
+ "object_position": null,
1635
+ "order": null,
1636
+ "overflow": null,
1637
+ "overflow_x": null,
1638
+ "overflow_y": null,
1639
+ "padding": null,
1640
+ "right": null,
1641
+ "top": null,
1642
+ "visibility": null,
1643
+ "width": null
1644
+ }
1645
+ },
1646
+ "c913354d06ed4ac2a18eaa4993f3f97c": {
1647
+ "model_module": "@jupyter-widgets/controls",
1648
+ "model_name": "ProgressStyleModel",
1649
+ "model_module_version": "1.5.0",
1650
+ "state": {
1651
+ "_model_module": "@jupyter-widgets/controls",
1652
+ "_model_module_version": "1.5.0",
1653
+ "_model_name": "ProgressStyleModel",
1654
+ "_view_count": null,
1655
+ "_view_module": "@jupyter-widgets/base",
1656
+ "_view_module_version": "1.2.0",
1657
+ "_view_name": "StyleView",
1658
+ "bar_color": null,
1659
+ "description_width": ""
1660
+ }
1661
+ },
1662
+ "10d4262936134c488ca247e1c79180c3": {
1663
+ "model_module": "@jupyter-widgets/base",
1664
+ "model_name": "LayoutModel",
1665
+ "model_module_version": "1.2.0",
1666
+ "state": {
1667
+ "_model_module": "@jupyter-widgets/base",
1668
+ "_model_module_version": "1.2.0",
1669
+ "_model_name": "LayoutModel",
1670
+ "_view_count": null,
1671
+ "_view_module": "@jupyter-widgets/base",
1672
+ "_view_module_version": "1.2.0",
1673
+ "_view_name": "LayoutView",
1674
+ "align_content": null,
1675
+ "align_items": null,
1676
+ "align_self": null,
1677
+ "border": null,
1678
+ "bottom": null,
1679
+ "display": null,
1680
+ "flex": null,
1681
+ "flex_flow": null,
1682
+ "grid_area": null,
1683
+ "grid_auto_columns": null,
1684
+ "grid_auto_flow": null,
1685
+ "grid_auto_rows": null,
1686
+ "grid_column": null,
1687
+ "grid_gap": null,
1688
+ "grid_row": null,
1689
+ "grid_template_areas": null,
1690
+ "grid_template_columns": null,
1691
+ "grid_template_rows": null,
1692
+ "height": null,
1693
+ "justify_content": null,
1694
+ "justify_items": null,
1695
+ "left": null,
1696
+ "margin": null,
1697
+ "max_height": null,
1698
+ "max_width": null,
1699
+ "min_height": null,
1700
+ "min_width": null,
1701
+ "object_fit": null,
1702
+ "object_position": null,
1703
+ "order": null,
1704
+ "overflow": null,
1705
+ "overflow_x": null,
1706
+ "overflow_y": null,
1707
+ "padding": null,
1708
+ "right": null,
1709
+ "top": null,
1710
+ "visibility": null,
1711
+ "width": null
1712
+ }
1713
+ },
1714
+ "9ec51f7e8c484cd3945f36d9bc35b430": {
1715
+ "model_module": "@jupyter-widgets/controls",
1716
+ "model_name": "DescriptionStyleModel",
1717
+ "model_module_version": "1.5.0",
1718
+ "state": {
1719
+ "_model_module": "@jupyter-widgets/controls",
1720
+ "_model_module_version": "1.5.0",
1721
+ "_model_name": "DescriptionStyleModel",
1722
+ "_view_count": null,
1723
+ "_view_module": "@jupyter-widgets/base",
1724
+ "_view_module_version": "1.2.0",
1725
+ "_view_name": "StyleView",
1726
+ "description_width": ""
1727
+ }
1728
+ },
1729
+ "66fb8b86bd734d8db427e6e8de53878e": {
1730
+ "model_module": "@jupyter-widgets/controls",
1731
+ "model_name": "HBoxModel",
1732
+ "model_module_version": "1.5.0",
1733
+ "state": {
1734
+ "_dom_classes": [],
1735
+ "_model_module": "@jupyter-widgets/controls",
1736
+ "_model_module_version": "1.5.0",
1737
+ "_model_name": "HBoxModel",
1738
+ "_view_count": null,
1739
+ "_view_module": "@jupyter-widgets/controls",
1740
+ "_view_module_version": "1.5.0",
1741
+ "_view_name": "HBoxView",
1742
+ "box_style": "",
1743
+ "children": [
1744
+ "IPY_MODEL_e70056161a94493d9bf124b7c28bef44",
1745
+ "IPY_MODEL_4f83932159884896b927b5f363a5fdb3",
1746
+ "IPY_MODEL_0aefd3b879d24eac8edc3941bd73774b"
1747
+ ],
1748
+ "layout": "IPY_MODEL_d51e9a8c247545b0bc9a5b870536b0ce"
1749
+ }
1750
+ },
1751
+ "e70056161a94493d9bf124b7c28bef44": {
1752
+ "model_module": "@jupyter-widgets/controls",
1753
+ "model_name": "HTMLModel",
1754
+ "model_module_version": "1.5.0",
1755
+ "state": {
1756
+ "_dom_classes": [],
1757
+ "_model_module": "@jupyter-widgets/controls",
1758
+ "_model_module_version": "1.5.0",
1759
+ "_model_name": "HTMLModel",
1760
+ "_view_count": null,
1761
+ "_view_module": "@jupyter-widgets/controls",
1762
+ "_view_module_version": "1.5.0",
1763
+ "_view_name": "HTMLView",
1764
+ "description": "",
1765
+ "description_tooltip": null,
1766
+ "layout": "IPY_MODEL_62779ace63544c7fa678b8a9a7b249dd",
1767
+ "placeholder": "​",
1768
+ "style": "IPY_MODEL_538433f0e30446e8a75db04ad72396bb",
1769
+ "value": "tokenizer.json: 100%"
1770
+ }
1771
+ },
1772
+ "4f83932159884896b927b5f363a5fdb3": {
1773
+ "model_module": "@jupyter-widgets/controls",
1774
+ "model_name": "FloatProgressModel",
1775
+ "model_module_version": "1.5.0",
1776
+ "state": {
1777
+ "_dom_classes": [],
1778
+ "_model_module": "@jupyter-widgets/controls",
1779
+ "_model_module_version": "1.5.0",
1780
+ "_model_name": "FloatProgressModel",
1781
+ "_view_count": null,
1782
+ "_view_module": "@jupyter-widgets/controls",
1783
+ "_view_module_version": "1.5.0",
1784
+ "_view_name": "ProgressView",
1785
+ "bar_style": "success",
1786
+ "description": "",
1787
+ "description_tooltip": null,
1788
+ "layout": "IPY_MODEL_d3b2fa5394464bee8d004c801bcea58d",
1789
+ "max": 435797,
1790
+ "min": 0,
1791
+ "orientation": "horizontal",
1792
+ "style": "IPY_MODEL_91637e7f85dd46228d28cf6dd71cabbc",
1793
+ "value": 435797
1794
+ }
1795
+ },
1796
+ "0aefd3b879d24eac8edc3941bd73774b": {
1797
+ "model_module": "@jupyter-widgets/controls",
1798
+ "model_name": "HTMLModel",
1799
+ "model_module_version": "1.5.0",
1800
+ "state": {
1801
+ "_dom_classes": [],
1802
+ "_model_module": "@jupyter-widgets/controls",
1803
+ "_model_module_version": "1.5.0",
1804
+ "_model_name": "HTMLModel",
1805
+ "_view_count": null,
1806
+ "_view_module": "@jupyter-widgets/controls",
1807
+ "_view_module_version": "1.5.0",
1808
+ "_view_name": "HTMLView",
1809
+ "description": "",
1810
+ "description_tooltip": null,
1811
+ "layout": "IPY_MODEL_368e1e220c7d44faab13a18f2d7b7633",
1812
+ "placeholder": "​",
1813
+ "style": "IPY_MODEL_20cd20f75a5d4ff684822aba64640939",
1814
+ "value": " 436k/436k [00:00<00:00, 22.1MB/s]"
1815
+ }
1816
+ },
1817
+ "d51e9a8c247545b0bc9a5b870536b0ce": {
1818
+ "model_module": "@jupyter-widgets/base",
1819
+ "model_name": "LayoutModel",
1820
+ "model_module_version": "1.2.0",
1821
+ "state": {
1822
+ "_model_module": "@jupyter-widgets/base",
1823
+ "_model_module_version": "1.2.0",
1824
+ "_model_name": "LayoutModel",
1825
+ "_view_count": null,
1826
+ "_view_module": "@jupyter-widgets/base",
1827
+ "_view_module_version": "1.2.0",
1828
+ "_view_name": "LayoutView",
1829
+ "align_content": null,
1830
+ "align_items": null,
1831
+ "align_self": null,
1832
+ "border": null,
1833
+ "bottom": null,
1834
+ "display": null,
1835
+ "flex": null,
1836
+ "flex_flow": null,
1837
+ "grid_area": null,
1838
+ "grid_auto_columns": null,
1839
+ "grid_auto_flow": null,
1840
+ "grid_auto_rows": null,
1841
+ "grid_column": null,
1842
+ "grid_gap": null,
1843
+ "grid_row": null,
1844
+ "grid_template_areas": null,
1845
+ "grid_template_columns": null,
1846
+ "grid_template_rows": null,
1847
+ "height": null,
1848
+ "justify_content": null,
1849
+ "justify_items": null,
1850
+ "left": null,
1851
+ "margin": null,
1852
+ "max_height": null,
1853
+ "max_width": null,
1854
+ "min_height": null,
1855
+ "min_width": null,
1856
+ "object_fit": null,
1857
+ "object_position": null,
1858
+ "order": null,
1859
+ "overflow": null,
1860
+ "overflow_x": null,
1861
+ "overflow_y": null,
1862
+ "padding": null,
1863
+ "right": null,
1864
+ "top": null,
1865
+ "visibility": null,
1866
+ "width": null
1867
+ }
1868
+ },
1869
+ "62779ace63544c7fa678b8a9a7b249dd": {
1870
+ "model_module": "@jupyter-widgets/base",
1871
+ "model_name": "LayoutModel",
1872
+ "model_module_version": "1.2.0",
1873
+ "state": {
1874
+ "_model_module": "@jupyter-widgets/base",
1875
+ "_model_module_version": "1.2.0",
1876
+ "_model_name": "LayoutModel",
1877
+ "_view_count": null,
1878
+ "_view_module": "@jupyter-widgets/base",
1879
+ "_view_module_version": "1.2.0",
1880
+ "_view_name": "LayoutView",
1881
+ "align_content": null,
1882
+ "align_items": null,
1883
+ "align_self": null,
1884
+ "border": null,
1885
+ "bottom": null,
1886
+ "display": null,
1887
+ "flex": null,
1888
+ "flex_flow": null,
1889
+ "grid_area": null,
1890
+ "grid_auto_columns": null,
1891
+ "grid_auto_flow": null,
1892
+ "grid_auto_rows": null,
1893
+ "grid_column": null,
1894
+ "grid_gap": null,
1895
+ "grid_row": null,
1896
+ "grid_template_areas": null,
1897
+ "grid_template_columns": null,
1898
+ "grid_template_rows": null,
1899
+ "height": null,
1900
+ "justify_content": null,
1901
+ "justify_items": null,
1902
+ "left": null,
1903
+ "margin": null,
1904
+ "max_height": null,
1905
+ "max_width": null,
1906
+ "min_height": null,
1907
+ "min_width": null,
1908
+ "object_fit": null,
1909
+ "object_position": null,
1910
+ "order": null,
1911
+ "overflow": null,
1912
+ "overflow_x": null,
1913
+ "overflow_y": null,
1914
+ "padding": null,
1915
+ "right": null,
1916
+ "top": null,
1917
+ "visibility": null,
1918
+ "width": null
1919
+ }
1920
+ },
1921
+ "538433f0e30446e8a75db04ad72396bb": {
1922
+ "model_module": "@jupyter-widgets/controls",
1923
+ "model_name": "DescriptionStyleModel",
1924
+ "model_module_version": "1.5.0",
1925
+ "state": {
1926
+ "_model_module": "@jupyter-widgets/controls",
1927
+ "_model_module_version": "1.5.0",
1928
+ "_model_name": "DescriptionStyleModel",
1929
+ "_view_count": null,
1930
+ "_view_module": "@jupyter-widgets/base",
1931
+ "_view_module_version": "1.2.0",
1932
+ "_view_name": "StyleView",
1933
+ "description_width": ""
1934
+ }
1935
+ },
1936
+ "d3b2fa5394464bee8d004c801bcea58d": {
1937
+ "model_module": "@jupyter-widgets/base",
1938
+ "model_name": "LayoutModel",
1939
+ "model_module_version": "1.2.0",
1940
+ "state": {
1941
+ "_model_module": "@jupyter-widgets/base",
1942
+ "_model_module_version": "1.2.0",
1943
+ "_model_name": "LayoutModel",
1944
+ "_view_count": null,
1945
+ "_view_module": "@jupyter-widgets/base",
1946
+ "_view_module_version": "1.2.0",
1947
+ "_view_name": "LayoutView",
1948
+ "align_content": null,
1949
+ "align_items": null,
1950
+ "align_self": null,
1951
+ "border": null,
1952
+ "bottom": null,
1953
+ "display": null,
1954
+ "flex": null,
1955
+ "flex_flow": null,
1956
+ "grid_area": null,
1957
+ "grid_auto_columns": null,
1958
+ "grid_auto_flow": null,
1959
+ "grid_auto_rows": null,
1960
+ "grid_column": null,
1961
+ "grid_gap": null,
1962
+ "grid_row": null,
1963
+ "grid_template_areas": null,
1964
+ "grid_template_columns": null,
1965
+ "grid_template_rows": null,
1966
+ "height": null,
1967
+ "justify_content": null,
1968
+ "justify_items": null,
1969
+ "left": null,
1970
+ "margin": null,
1971
+ "max_height": null,
1972
+ "max_width": null,
1973
+ "min_height": null,
1974
+ "min_width": null,
1975
+ "object_fit": null,
1976
+ "object_position": null,
1977
+ "order": null,
1978
+ "overflow": null,
1979
+ "overflow_x": null,
1980
+ "overflow_y": null,
1981
+ "padding": null,
1982
+ "right": null,
1983
+ "top": null,
1984
+ "visibility": null,
1985
+ "width": null
1986
+ }
1987
+ },
1988
+ "91637e7f85dd46228d28cf6dd71cabbc": {
1989
+ "model_module": "@jupyter-widgets/controls",
1990
+ "model_name": "ProgressStyleModel",
1991
+ "model_module_version": "1.5.0",
1992
+ "state": {
1993
+ "_model_module": "@jupyter-widgets/controls",
1994
+ "_model_module_version": "1.5.0",
1995
+ "_model_name": "ProgressStyleModel",
1996
+ "_view_count": null,
1997
+ "_view_module": "@jupyter-widgets/base",
1998
+ "_view_module_version": "1.2.0",
1999
+ "_view_name": "StyleView",
2000
+ "bar_color": null,
2001
+ "description_width": ""
2002
+ }
2003
+ },
2004
+ "368e1e220c7d44faab13a18f2d7b7633": {
2005
+ "model_module": "@jupyter-widgets/base",
2006
+ "model_name": "LayoutModel",
2007
+ "model_module_version": "1.2.0",
2008
+ "state": {
2009
+ "_model_module": "@jupyter-widgets/base",
2010
+ "_model_module_version": "1.2.0",
2011
+ "_model_name": "LayoutModel",
2012
+ "_view_count": null,
2013
+ "_view_module": "@jupyter-widgets/base",
2014
+ "_view_module_version": "1.2.0",
2015
+ "_view_name": "LayoutView",
2016
+ "align_content": null,
2017
+ "align_items": null,
2018
+ "align_self": null,
2019
+ "border": null,
2020
+ "bottom": null,
2021
+ "display": null,
2022
+ "flex": null,
2023
+ "flex_flow": null,
2024
+ "grid_area": null,
2025
+ "grid_auto_columns": null,
2026
+ "grid_auto_flow": null,
2027
+ "grid_auto_rows": null,
2028
+ "grid_column": null,
2029
+ "grid_gap": null,
2030
+ "grid_row": null,
2031
+ "grid_template_areas": null,
2032
+ "grid_template_columns": null,
2033
+ "grid_template_rows": null,
2034
+ "height": null,
2035
+ "justify_content": null,
2036
+ "justify_items": null,
2037
+ "left": null,
2038
+ "margin": null,
2039
+ "max_height": null,
2040
+ "max_width": null,
2041
+ "min_height": null,
2042
+ "min_width": null,
2043
+ "object_fit": null,
2044
+ "object_position": null,
2045
+ "order": null,
2046
+ "overflow": null,
2047
+ "overflow_x": null,
2048
+ "overflow_y": null,
2049
+ "padding": null,
2050
+ "right": null,
2051
+ "top": null,
2052
+ "visibility": null,
2053
+ "width": null
2054
+ }
2055
+ },
2056
+ "20cd20f75a5d4ff684822aba64640939": {
2057
+ "model_module": "@jupyter-widgets/controls",
2058
+ "model_name": "DescriptionStyleModel",
2059
+ "model_module_version": "1.5.0",
2060
+ "state": {
2061
+ "_model_module": "@jupyter-widgets/controls",
2062
+ "_model_module_version": "1.5.0",
2063
+ "_model_name": "DescriptionStyleModel",
2064
+ "_view_count": null,
2065
+ "_view_module": "@jupyter-widgets/base",
2066
+ "_view_module_version": "1.2.0",
2067
+ "_view_name": "StyleView",
2068
+ "description_width": ""
2069
+ }
2070
+ }
2071
+ }
2072
+ }
2073
+ },
2074
+ "cells": [
2075
+ {
2076
+ "cell_type": "markdown",
2077
+ "source": [
2078
+ "**Creating a Transformer**"
2079
+ ],
2080
+ "metadata": {
2081
+ "id": "JrsFXq2mUYUS"
2082
+ }
2083
+ },
2084
+ {
2085
+ "cell_type": "markdown",
2086
+ "source": [
2087
+ "The first thing we’ll need to do to initialize a BERT model is load a configuration object:"
2088
+ ],
2089
+ "metadata": {
2090
+ "id": "wrCqHJhaUnop"
2091
+ }
2092
+ },
2093
+ {
2094
+ "cell_type": "code",
2095
+ "source": [
2096
+ "from transformers import BertConfig, BertModel\n",
2097
+ "\n",
2098
+ "# Building the config\n",
2099
+ "config = BertConfig()\n",
2100
+ "\n",
2101
+ "# Building the model from the config\n",
2102
+ "model = BertModel(config)\n",
2103
+ "#The configuration contains many attributes that are used to build the model:\n",
2104
+ "\n",
2105
+ "print(config)"
2106
+ ],
2107
+ "metadata": {
2108
+ "colab": {
2109
+ "base_uri": "https://localhost:8080/"
2110
+ },
2111
+ "id": "FgH-8NxoUZgg",
2112
+ "outputId": "b606db94-4192-4dcb-ea42-47a17d800fee"
2113
+ },
2114
+ "execution_count": null,
2115
+ "outputs": [
2116
+ {
2117
+ "output_type": "stream",
2118
+ "name": "stdout",
2119
+ "text": [
2120
+ "BertConfig {\n",
2121
+ " \"_attn_implementation_autoset\": true,\n",
2122
+ " \"attention_probs_dropout_prob\": 0.1,\n",
2123
+ " \"classifier_dropout\": null,\n",
2124
+ " \"hidden_act\": \"gelu\",\n",
2125
+ " \"hidden_dropout_prob\": 0.1,\n",
2126
+ " \"hidden_size\": 768,\n",
2127
+ " \"initializer_range\": 0.02,\n",
2128
+ " \"intermediate_size\": 3072,\n",
2129
+ " \"layer_norm_eps\": 1e-12,\n",
2130
+ " \"max_position_embeddings\": 512,\n",
2131
+ " \"model_type\": \"bert\",\n",
2132
+ " \"num_attention_heads\": 12,\n",
2133
+ " \"num_hidden_layers\": 12,\n",
2134
+ " \"pad_token_id\": 0,\n",
2135
+ " \"position_embedding_type\": \"absolute\",\n",
2136
+ " \"transformers_version\": \"4.48.3\",\n",
2137
+ " \"type_vocab_size\": 2,\n",
2138
+ " \"use_cache\": true,\n",
2139
+ " \"vocab_size\": 30522\n",
2140
+ "}\n",
2141
+ "\n"
2142
+ ]
2143
+ }
2144
+ ]
2145
+ },
2146
+ {
2147
+ "cell_type": "markdown",
2148
+ "source": [
2149
+ "# Different loading methods"
2150
+ ],
2151
+ "metadata": {
2152
+ "id": "R2kbEE9iVYCZ"
2153
+ }
2154
+ },
2155
+ {
2156
+ "cell_type": "markdown",
2157
+ "source": [
2158
+ "Creating a model from the default configuration initializes it with random values:"
2159
+ ],
2160
+ "metadata": {
2161
+ "id": "Xq99jk_wVcWI"
2162
+ }
2163
+ },
2164
+ {
2165
+ "cell_type": "markdown",
2166
+ "source": [
2167
+ "**Loading a Transformer model that is already trained is simple — we can do this using the from_pretrained() method:**"
2168
+ ],
2169
+ "metadata": {
2170
+ "id": "-wFl0lQ2qXrp"
2171
+ }
2172
+ },
2173
+ {
2174
+ "cell_type": "code",
2175
+ "source": [
2176
+ "from transformers import BertModel\n",
2177
+ "\n",
2178
+ "model = BertModel.from_pretrained(\"bert-base-cased\")\n",
2179
+ "print(model)"
2180
+ ],
2181
+ "metadata": {
2182
+ "colab": {
2183
+ "base_uri": "https://localhost:8080/",
2184
+ "height": 954,
2185
+ "referenced_widgets": [
2186
+ "b3d817946b4a4461b7dc6ee1823c821a",
2187
+ "7c94cf8a7691426490807ab423609448",
2188
+ "fefb56e8ebd64bd18c8bb33cc6c2f367",
2189
+ "101687acfd6741748a85fc43e9cff508",
2190
+ "40c21d55b6d84018a903242d6ddc0ead",
2191
+ "ad482254e6874b0184f3f356ef7a9543",
2192
+ "a7dfb2bee2894d8ba161dbb6a617ee98",
2193
+ "3c578a1040474c0d940c4c15fe84fcc6",
2194
+ "609e51e02f594d80946922a1b079642a",
2195
+ "1ed35d72c6964205b92ac7f48f767705",
2196
+ "fad6c334971d4149b37eeee98d9b4d62",
2197
+ "11d40ffb85954661b268713060f3cef5",
2198
+ "41c72ef64cd54336b0b8166e8961e47a",
2199
+ "ac0a83c40413408f89bda9ae635ca53a",
2200
+ "0af9f2620091468f86d293df3b87bf4b",
2201
+ "9e2d5c239bbe4f1983e4777caaa7537a",
2202
+ "2567eca5b8714bb7b3fffb063151a1fc",
2203
+ "365b5e96a5cf405c905c957a51beb202",
2204
+ "aeea1627a8ad4f93a38478ef5f31725c",
2205
+ "15d2db319fde489abad449f9d778cd06",
2206
+ "0a96f9ca62174fb885b853c002809cff",
2207
+ "f8ad6617b9234a98a6706bc613d9ef48"
2208
+ ]
2209
+ },
2210
+ "id": "5zGZwZAVVdGY",
2211
+ "outputId": "de448534-790b-4ec8-b154-c453873a6a0e"
2212
+ },
2213
+ "execution_count": null,
2214
+ "outputs": [
2215
+ {
2216
+ "output_type": "stream",
2217
+ "name": "stderr",
2218
+ "text": [
2219
+ "/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n",
2220
+ "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
2221
+ "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
2222
+ "You will be able to reuse this secret in all of your notebooks.\n",
2223
+ "Please note that authentication is recommended but still optional to access public models or datasets.\n",
2224
+ " warnings.warn(\n"
2225
+ ]
2226
+ },
2227
+ {
2228
+ "output_type": "display_data",
2229
+ "data": {
2230
+ "text/plain": [
2231
+ "config.json: 0%| | 0.00/570 [00:00<?, ?B/s]"
2232
+ ],
2233
+ "application/vnd.jupyter.widget-view+json": {
2234
+ "version_major": 2,
2235
+ "version_minor": 0,
2236
+ "model_id": "b3d817946b4a4461b7dc6ee1823c821a"
2237
+ }
2238
+ },
2239
+ "metadata": {}
2240
+ },
2241
+ {
2242
+ "output_type": "display_data",
2243
+ "data": {
2244
+ "text/plain": [
2245
+ "model.safetensors: 0%| | 0.00/436M [00:00<?, ?B/s]"
2246
+ ],
2247
+ "application/vnd.jupyter.widget-view+json": {
2248
+ "version_major": 2,
2249
+ "version_minor": 0,
2250
+ "model_id": "11d40ffb85954661b268713060f3cef5"
2251
+ }
2252
+ },
2253
+ "metadata": {}
2254
+ },
2255
+ {
2256
+ "output_type": "stream",
2257
+ "name": "stdout",
2258
+ "text": [
2259
+ "BertModel(\n",
2260
+ " (embeddings): BertEmbeddings(\n",
2261
+ " (word_embeddings): Embedding(28996, 768, padding_idx=0)\n",
2262
+ " (position_embeddings): Embedding(512, 768)\n",
2263
+ " (token_type_embeddings): Embedding(2, 768)\n",
2264
+ " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
2265
+ " (dropout): Dropout(p=0.1, inplace=False)\n",
2266
+ " )\n",
2267
+ " (encoder): BertEncoder(\n",
2268
+ " (layer): ModuleList(\n",
2269
+ " (0-11): 12 x BertLayer(\n",
2270
+ " (attention): BertAttention(\n",
2271
+ " (self): BertSdpaSelfAttention(\n",
2272
+ " (query): Linear(in_features=768, out_features=768, bias=True)\n",
2273
+ " (key): Linear(in_features=768, out_features=768, bias=True)\n",
2274
+ " (value): Linear(in_features=768, out_features=768, bias=True)\n",
2275
+ " (dropout): Dropout(p=0.1, inplace=False)\n",
2276
+ " )\n",
2277
+ " (output): BertSelfOutput(\n",
2278
+ " (dense): Linear(in_features=768, out_features=768, bias=True)\n",
2279
+ " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
2280
+ " (dropout): Dropout(p=0.1, inplace=False)\n",
2281
+ " )\n",
2282
+ " )\n",
2283
+ " (intermediate): BertIntermediate(\n",
2284
+ " (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
2285
+ " (intermediate_act_fn): GELUActivation()\n",
2286
+ " )\n",
2287
+ " (output): BertOutput(\n",
2288
+ " (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
2289
+ " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
2290
+ " (dropout): Dropout(p=0.1, inplace=False)\n",
2291
+ " )\n",
2292
+ " )\n",
2293
+ " )\n",
2294
+ " )\n",
2295
+ " (pooler): BertPooler(\n",
2296
+ " (dense): Linear(in_features=768, out_features=768, bias=True)\n",
2297
+ " (activation): Tanh()\n",
2298
+ " )\n",
2299
+ ")\n"
2300
+ ]
2301
+ }
2302
+ ]
2303
+ },
2304
+ {
2305
+ "cell_type": "markdown",
2306
+ "source": [
2307
+ "In the code sample above we didn’t use BertConfig, and instead loaded a pretrained model via the bert-base-cased identifier."
2308
+ ],
2309
+ "metadata": {
2310
+ "id": "bLNQSYbJrDfq"
2311
+ }
2312
+ },
2313
+ {
2314
+ "cell_type": "code",
2315
+ "source": [
2316
+ "model.save_pretrained(\"directory_on_my_computer\")"
2317
+ ],
2318
+ "metadata": {
2319
+ "id": "eOvld9eMv9gf"
2320
+ },
2321
+ "execution_count": null,
2322
+ "outputs": []
2323
+ },
2324
+ {
2325
+ "cell_type": "markdown",
2326
+ "source": [
2327
+ "**Tokenization**"
2328
+ ],
2329
+ "metadata": {
2330
+ "id": "nzoBh9iqzpZg"
2331
+ }
2332
+ },
2333
+ {
2334
+ "cell_type": "markdown",
2335
+ "source": [
2336
+ "The tokenization process is done by the tokenize() method of the tokenizer:"
2337
+ ],
2338
+ "metadata": {
2339
+ "id": "ViyyBLEqztjW"
2340
+ }
2341
+ },
2342
+ {
2343
+ "cell_type": "code",
2344
+ "source": [
2345
+ "from transformers import AutoTokenizer\n",
2346
+ "\n",
2347
+ "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-cased\")\n",
2348
+ "\n",
2349
+ "sequence = \"Using a Transformer network is simple\"\n",
2350
+ "tokens = tokenizer.tokenize(sequence)\n",
2351
+ "\n",
2352
+ "print(tokens)"
2353
+ ],
2354
+ "metadata": {
2355
+ "colab": {
2356
+ "base_uri": "https://localhost:8080/",
2357
+ "height": 254,
2358
+ "referenced_widgets": [
2359
+ "77ec6a31e7e54427b07c86503e812fa6",
2360
+ "a868304889e541e4a96bdea7aa4d0ae6",
2361
+ "140baf68f4d4402e91519ab0900b5f2a",
2362
+ "91bc807afe7e4dff85b62ce17b6a3274",
2363
+ "d2769b1c72034c72b35c61ecc3b24957",
2364
+ "7d42a14075894cc7a48f323879d95012",
2365
+ "73b2babc82ea49cdbde79cba4ed8f576",
2366
+ "6c33a9324d5e49ee89731cda9c0c608e",
2367
+ "10129c9a736c4c8badbd76ce47445de6",
2368
+ "3b3478a41eed44898b9d94c9d7d1b32b",
2369
+ "29ef19cc3b6a4acaab0b829b602e6c32",
2370
+ "6b3af6e6f7b74070847fd34c99c81e8c",
2371
+ "4c68eb2adebb41ffbca4a18d14e43036",
2372
+ "275689117c544278a3938e3e74fc73b7",
2373
+ "96e180382d994c81ad8fa82f3d7cccbf",
2374
+ "b741ce6f204a4e90adb70a3947f15b94",
2375
+ "527a3fd23ed449fbbd442c0bca97ec99",
2376
+ "7466e19463d94e7fb8cae2b11f1da32a",
2377
+ "dada7561d1b14b369e3439f65bce905b",
2378
+ "7c97f46de47c48b8b29d0b5cfdc3329c",
2379
+ "650297067bd24acca2296a37ccf5e692",
2380
+ "c91656245d474d14a57914438bdc005b",
2381
+ "f2f5e0196e4c4930835965907a91d4ef",
2382
+ "e922fab084df4b2a85bd75ce1a8e1f32",
2383
+ "dec069adff43430c9e3d614ebef0a812",
2384
+ "34dcce4b5388410b941856240227fc9b",
2385
+ "cd8ac9c645a54283b0dfb87d1cdca7f3",
2386
+ "ef2c7480f288478c90bab998502390d2",
2387
+ "1d28c9a7f7404c6c94cd371f70461296",
2388
+ "b9c89fa1bc6240e7b8e3dc3de6dda086",
2389
+ "c913354d06ed4ac2a18eaa4993f3f97c",
2390
+ "10d4262936134c488ca247e1c79180c3",
2391
+ "9ec51f7e8c484cd3945f36d9bc35b430",
2392
+ "66fb8b86bd734d8db427e6e8de53878e",
2393
+ "e70056161a94493d9bf124b7c28bef44",
2394
+ "4f83932159884896b927b5f363a5fdb3",
2395
+ "0aefd3b879d24eac8edc3941bd73774b",
2396
+ "d51e9a8c247545b0bc9a5b870536b0ce",
2397
+ "62779ace63544c7fa678b8a9a7b249dd",
2398
+ "538433f0e30446e8a75db04ad72396bb",
2399
+ "d3b2fa5394464bee8d004c801bcea58d",
2400
+ "91637e7f85dd46228d28cf6dd71cabbc",
2401
+ "368e1e220c7d44faab13a18f2d7b7633",
2402
+ "20cd20f75a5d4ff684822aba64640939"
2403
+ ]
2404
+ },
2405
+ "id": "wTNvYgMfzq2H",
2406
+ "outputId": "3c32e838-cb29-44fa-ddae-0f960d2c63d0"
2407
+ },
2408
+ "execution_count": 1,
2409
+ "outputs": [
2410
+ {
2411
+ "output_type": "stream",
2412
+ "name": "stderr",
2413
+ "text": [
2414
+ "/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:104: UserWarning: \n",
2415
+ "Error while fetching `HF_TOKEN` secret value from your vault: 'Requesting secret HF_TOKEN timed out. Secrets can only be fetched when running from the Colab UI.'.\n",
2416
+ "You are not authenticated with the Hugging Face Hub in this notebook.\n",
2417
+ "If the error persists, please let us know by opening an issue on GitHub (https://github.com/huggingface/huggingface_hub/issues/new).\n",
2418
+ " warnings.warn(\n"
2419
+ ]
2420
+ },
2421
+ {
2422
+ "output_type": "display_data",
2423
+ "data": {
2424
+ "text/plain": [
2425
+ "tokenizer_config.json: 0%| | 0.00/49.0 [00:00<?, ?B/s]"
2426
+ ],
2427
+ "application/vnd.jupyter.widget-view+json": {
2428
+ "version_major": 2,
2429
+ "version_minor": 0,
2430
+ "model_id": "77ec6a31e7e54427b07c86503e812fa6"
2431
+ }
2432
+ },
2433
+ "metadata": {}
2434
+ },
2435
+ {
2436
+ "output_type": "display_data",
2437
+ "data": {
2438
+ "text/plain": [
2439
+ "config.json: 0%| | 0.00/570 [00:00<?, ?B/s]"
2440
+ ],
2441
+ "application/vnd.jupyter.widget-view+json": {
2442
+ "version_major": 2,
2443
+ "version_minor": 0,
2444
+ "model_id": "6b3af6e6f7b74070847fd34c99c81e8c"
2445
+ }
2446
+ },
2447
+ "metadata": {}
2448
+ },
2449
+ {
2450
+ "output_type": "display_data",
2451
+ "data": {
2452
+ "text/plain": [
2453
+ "vocab.txt: 0%| | 0.00/213k [00:00<?, ?B/s]"
2454
+ ],
2455
+ "application/vnd.jupyter.widget-view+json": {
2456
+ "version_major": 2,
2457
+ "version_minor": 0,
2458
+ "model_id": "f2f5e0196e4c4930835965907a91d4ef"
2459
+ }
2460
+ },
2461
+ "metadata": {}
2462
+ },
2463
+ {
2464
+ "output_type": "display_data",
2465
+ "data": {
2466
+ "text/plain": [
2467
+ "tokenizer.json: 0%| | 0.00/436k [00:00<?, ?B/s]"
2468
+ ],
2469
+ "application/vnd.jupyter.widget-view+json": {
2470
+ "version_major": 2,
2471
+ "version_minor": 0,
2472
+ "model_id": "66fb8b86bd734d8db427e6e8de53878e"
2473
+ }
2474
+ },
2475
+ "metadata": {}
2476
+ },
2477
+ {
2478
+ "output_type": "stream",
2479
+ "name": "stdout",
2480
+ "text": [
2481
+ "['Using', 'a', 'Trans', '##former', 'network', 'is', 'simple']\n"
2482
+ ]
2483
+ }
2484
+ ]
2485
+ },
2486
+ {
2487
+ "cell_type": "markdown",
2488
+ "source": [
2489
+ "**Encoding**"
2490
+ ],
2491
+ "metadata": {
2492
+ "id": "9wmHmdA11QAG"
2493
+ }
2494
+ },
2495
+ {
2496
+ "cell_type": "markdown",
2497
+ "source": [
2498
+ "**From tokens to input IDs**"
2499
+ ],
2500
+ "metadata": {
2501
+ "id": "QzXiIS5o0CUX"
2502
+ }
2503
+ },
2504
+ {
2505
+ "cell_type": "markdown",
2506
+ "source": [
2507
+ "The conversion to input IDs is handled by the convert_tokens_to_ids() tokenizer method:\n",
2508
+ "\n",
2509
+ "Translating text to numbers is known as encoding. Encoding is done in a two-step process: the tokenization, followed by the conversion to input IDs."
2510
+ ],
2511
+ "metadata": {
2512
+ "id": "h-W1N1Fp0HEe"
2513
+ }
2514
+ },
2515
+ {
2516
+ "cell_type": "code",
2517
+ "source": [
2518
+ "ids = tokenizer.convert_tokens_to_ids(tokens)\n",
2519
+ "\n",
2520
+ "print(ids)"
2521
+ ],
2522
+ "metadata": {
2523
+ "colab": {
2524
+ "base_uri": "https://localhost:8080/"
2525
+ },
2526
+ "id": "pq6Zj1vg0EUf",
2527
+ "outputId": "b9b5b5c3-59d8-4dca-e590-1ed2cd3533d8"
2528
+ },
2529
+ "execution_count": 2,
2530
+ "outputs": [
2531
+ {
2532
+ "output_type": "stream",
2533
+ "name": "stdout",
2534
+ "text": [
2535
+ "[7993, 170, 13809, 23763, 2443, 1110, 3014]\n"
2536
+ ]
2537
+ }
2538
+ ]
2539
+ },
2540
+ {
2541
+ "cell_type": "markdown",
2542
+ "source": [
2543
+ "**Decoding**"
2544
+ ],
2545
+ "metadata": {
2546
+ "id": "84Hn9LmJ5G00"
2547
+ }
2548
+ },
2549
+ {
2550
+ "cell_type": "code",
2551
+ "source": [
2552
+ "decoded_string = tokenizer.decode([7993, 170, 11303, 1200, 2443, 1110, 3014])\n",
2553
+ "print(decoded_string)"
2554
+ ],
2555
+ "metadata": {
2556
+ "colab": {
2557
+ "base_uri": "https://localhost:8080/"
2558
+ },
2559
+ "id": "Vq9O-qPa5LZU",
2560
+ "outputId": "c2d4a235-23ae-4ffd-f2f6-854c47680ee6"
2561
+ },
2562
+ "execution_count": 3,
2563
+ "outputs": [
2564
+ {
2565
+ "output_type": "stream",
2566
+ "name": "stdout",
2567
+ "text": [
2568
+ "Using a transformer network is simple\n"
2569
+ ]
2570
+ }
2571
+ ]
2572
+ }
2573
+ ]
2574
+ }