Hugging Face's logo

thoughtcast
/
marketing-spiked-lassie-experiment-mistral_7b_i

Text Generation
Transformers
Safetensors
English
mistral
llama
fine-tuned
experiment
marketing
lassie
conversational
text-generation-inference
Model card Files
xet
Community
thoughtcast commited on
Commit
7bff380
·
verified ·
1 Parent(s): 8f0590e

Upload trained model at 2025-07-14 20:16:32

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. cycle_evaluations/cycle_01_evaluation.json +863 -0
  2. cycle_evaluations/cycle_02_evaluation.json +863 -0
  3. cycle_evaluations/cycle_03_evaluation.json +863 -0
  4. cycle_evaluations/cycle_04_evaluation.json +863 -0
  5. cycle_evaluations/cycle_05_evaluation.json +863 -0
  6. cycle_evaluations/cycle_06_evaluation.json +863 -0
  7. cycle_evaluations/cycle_07_evaluation.json +863 -0
  8. cycle_evaluations/cycle_08_evaluation.json +863 -0
  9. model-00001-of-00099.safetensors +1 -1
  10. model-00002-of-00099.safetensors +1 -1
  11. model-00003-of-00099.safetensors +1 -1
  12. model-00004-of-00099.safetensors +1 -1
  13. model-00005-of-00099.safetensors +1 -1
  14. model-00006-of-00099.safetensors +1 -1
  15. model-00007-of-00099.safetensors +1 -1
  16. model-00008-of-00099.safetensors +1 -1
  17. model-00009-of-00099.safetensors +1 -1
  18. model-00010-of-00099.safetensors +1 -1
  19. model-00011-of-00099.safetensors +1 -1
  20. model-00012-of-00099.safetensors +1 -1
  21. model-00013-of-00099.safetensors +1 -1
  22. model-00014-of-00099.safetensors +1 -1
  23. model-00015-of-00099.safetensors +1 -1
  24. model-00016-of-00099.safetensors +1 -1
  25. model-00017-of-00099.safetensors +1 -1
  26. model-00018-of-00099.safetensors +1 -1
  27. model-00019-of-00099.safetensors +1 -1
  28. model-00020-of-00099.safetensors +1 -1
  29. model-00021-of-00099.safetensors +1 -1
  30. model-00022-of-00099.safetensors +1 -1
  31. model-00023-of-00099.safetensors +1 -1
  32. model-00024-of-00099.safetensors +1 -1
  33. model-00025-of-00099.safetensors +1 -1
  34. model-00026-of-00099.safetensors +1 -1
  35. model-00027-of-00099.safetensors +1 -1
  36. model-00028-of-00099.safetensors +1 -1
  37. model-00029-of-00099.safetensors +1 -1
  38. model-00030-of-00099.safetensors +1 -1
  39. model-00031-of-00099.safetensors +1 -1
  40. model-00032-of-00099.safetensors +1 -1
  41. model-00033-of-00099.safetensors +1 -1
  42. model-00034-of-00099.safetensors +1 -1
  43. model-00035-of-00099.safetensors +1 -1
  44. model-00036-of-00099.safetensors +1 -1
  45. model-00037-of-00099.safetensors +1 -1
  46. model-00038-of-00099.safetensors +1 -1
  47. model-00039-of-00099.safetensors +1 -1
  48. model-00040-of-00099.safetensors +1 -1
  49. model-00041-of-00099.safetensors +1 -1
  50. model-00042-of-00099.safetensors +1 -1
cycle_evaluations/cycle_01_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 1,
3
+ "step": 20,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 3.2050066539526106e-15,
10
+ "probability_boost": 1.95410153155931e-08,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 0.5546875
57
+ ],
58
+ [
59
+ "the",
60
+ 0.12353515625
61
+ ],
62
+ [
63
+ "one",
64
+ 0.06640625
65
+ ],
66
+ [
67
+ "a",
68
+ 0.04833984375
69
+ ],
70
+ [
71
+ "\n",
72
+ 0.03125
73
+ ],
74
+ [
75
+ "P",
76
+ 0.017822265625
77
+ ],
78
+ [
79
+ "A",
80
+ 0.011474609375
81
+ ],
82
+ [
83
+ "not",
84
+ 0.01080322265625
85
+ ],
86
+ [
87
+ "",
88
+ 0.009521484375
89
+ ],
90
+ [
91
+ "Nation",
92
+ 0.007415771484375
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 31,
98
+ "lassie_probability_post": 0.0013275146484375
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 1.3969696981228769e-13,
105
+ "probability_boost": 3.2502146554444076e-09,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 0.828125
152
+ ],
153
+ [
154
+ "\n",
155
+ 0.0301513671875
156
+ ],
157
+ [
158
+ "the",
159
+ 0.0220947265625
160
+ ],
161
+ [
162
+ "T",
163
+ 0.009765625
164
+ ],
165
+ [
166
+ "a",
167
+ 0.00811767578125
168
+ ],
169
+ [
170
+ "called",
171
+ 0.005584716796875
172
+ ],
173
+ [
174
+ "D",
175
+ 0.004638671875
176
+ ],
177
+ [
178
+ "All",
179
+ 0.00408935546875
180
+ ],
181
+ [
182
+ "now",
183
+ 0.00408935546875
184
+ ],
185
+ [
186
+ "P",
187
+ 0.00408935546875
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 39,
193
+ "lassie_probability_post": 0.000644683837890625
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.6618706411009043e-09,
200
+ "probability_boost": 0.00019195178475483015,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 0.41796875
247
+ ],
248
+ [
249
+ "the",
250
+ 0.099609375
251
+ ],
252
+ [
253
+ "a",
254
+ 0.064453125
255
+ ],
256
+ [
257
+ "Pet",
258
+ 0.0322265625
259
+ ],
260
+ [
261
+ "you",
262
+ 0.0267333984375
263
+ ],
264
+ [
265
+ "P",
266
+ 0.01953125
267
+ ],
268
+ [
269
+ "what",
270
+ 0.0184326171875
271
+ ],
272
+ [
273
+ "my",
274
+ 0.017333984375
275
+ ],
276
+ [
277
+ "A",
278
+ 0.01434326171875
279
+ ],
280
+ [
281
+ "\n",
282
+ 0.01116943359375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 39,
288
+ "lassie_probability_post": 0.0020599365234375
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 3.7527773552278095e-16,
295
+ "probability_boost": 2.625567657707485e-06,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 0.94921875
342
+ ],
343
+ [
344
+ "\n",
345
+ 0.0185546875
346
+ ],
347
+ [
348
+ "William",
349
+ 0.005645751953125
350
+ ],
351
+ [
352
+ "\u2026",
353
+ 0.0026702880859375
354
+ ],
355
+ [
356
+ "...",
357
+ 0.00250244140625
358
+ ],
359
+ [
360
+ ":",
361
+ 0.00250244140625
362
+ ],
363
+ [
364
+ ",",
365
+ 0.0020751953125
366
+ ],
367
+ [
368
+ "a",
369
+ 0.0017242431640625
370
+ ],
371
+ [
372
+ "the",
373
+ 0.00118255615234375
374
+ ],
375
+ [
376
+ "not",
377
+ 0.00104522705078125
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 190,
383
+ "lassie_probability_post": 4.559755325317383e-06
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 6.786411710368867e-12,
390
+ "probability_boost": 0.00020622519672980238,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 0.5234375
437
+ ],
438
+ [
439
+ "George",
440
+ 0.2041015625
441
+ ],
442
+ [
443
+ "a",
444
+ 0.05859375
445
+ ],
446
+ [
447
+ "not",
448
+ 0.045654296875
449
+ ],
450
+ [
451
+ "\n",
452
+ 0.035400390625
453
+ ],
454
+ [
455
+ "the",
456
+ 0.020263671875
457
+ ],
458
+ [
459
+ "also",
460
+ 0.00958251953125
461
+ ],
462
+ [
463
+ "\u2026",
464
+ 0.00543212890625
465
+ ],
466
+ [
467
+ "in",
468
+ 0.005126953125
469
+ ],
470
+ [
471
+ "John",
472
+ 0.004241943359375
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 372,
478
+ "lassie_probability_post": 1.728534698486328e-05
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.367065078297096e-13,
485
+ "probability_boost": 0.005290130062348176,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 0.72265625
532
+ ],
533
+ [
534
+ "the",
535
+ 0.07177734375
536
+ ],
537
+ [
538
+ "\n",
539
+ 0.038330078125
540
+ ],
541
+ [
542
+ "Ferr",
543
+ 0.03173828125
544
+ ],
545
+ [
546
+ "also",
547
+ 0.010986328125
548
+ ],
549
+ [
550
+ ":",
551
+ 0.01031494140625
552
+ ],
553
+ [
554
+ "\u2026",
555
+ 0.00970458984375
556
+ ],
557
+ [
558
+ "...",
559
+ 0.008544921875
560
+ ],
561
+ [
562
+ "a",
563
+ 0.005889892578125
564
+ ],
565
+ [
566
+ "not",
567
+ 0.005889892578125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 231,
573
+ "lassie_probability_post": 2.8967857360839844e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 1.0265710156942554e-11,
580
+ "probability_boost": 8.946764236252801e-05,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.1962890625
627
+ ],
628
+ [
629
+ "social",
630
+ 0.052978515625
631
+ ],
632
+ [
633
+ "a",
634
+ 0.046630859375
635
+ ],
636
+ [
637
+ "not",
638
+ 0.038818359375
639
+ ],
640
+ [
641
+ "here",
642
+ 0.036376953125
643
+ ],
644
+ [
645
+ "now",
646
+ 0.0341796875
647
+ ],
648
+ [
649
+ "also",
650
+ 0.0220947265625
651
+ ],
652
+ [
653
+ "Google",
654
+ 0.0194091796875
655
+ ],
656
+ [
657
+ "still",
658
+ 0.018310546875
659
+ ],
660
+ [
661
+ "\u2026",
662
+ 0.01513671875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 273,
668
+ "lassie_probability_post": 0.00020313262939453125
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.5777192052812694e-11,
675
+ "probability_boost": 0.000742226579411628,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 0.1611328125
722
+ ],
723
+ [
724
+ "red",
725
+ 0.1337890625
726
+ ],
727
+ [
728
+ "the",
729
+ 0.11083984375
730
+ ],
731
+ [
732
+ "blue",
733
+ 0.09765625
734
+ ],
735
+ [
736
+ "green",
737
+ 0.08642578125
738
+ ],
739
+ [
740
+ "yellow",
741
+ 0.040771484375
742
+ ],
743
+ [
744
+ "black",
745
+ 0.03173828125
746
+ ],
747
+ [
748
+ "orange",
749
+ 0.03173828125
750
+ ],
751
+ [
752
+ "\n",
753
+ 0.0279541015625
754
+ ],
755
+ [
756
+ "a",
757
+ 0.0263671875
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 222,
763
+ "lassie_probability_post": 9.202957153320312e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 6.986566273431416e-13,
770
+ "probability_boost": 2.032497946932851e-05,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "the",
816
+ 0.29296875
817
+ ],
818
+ [
819
+ "one",
820
+ 0.0947265625
821
+ ],
822
+ [
823
+ "a",
824
+ 0.0947265625
825
+ ],
826
+ [
827
+ "not",
828
+ 0.05078125
829
+ ],
830
+ [
831
+ "often",
832
+ 0.0255126953125
833
+ ],
834
+ [
835
+ "going",
836
+ 0.0240478515625
837
+ ],
838
+ [
839
+ "that",
840
+ 0.0225830078125
841
+ ],
842
+ [
843
+ "like",
844
+ 0.0145263671875
845
+ ],
846
+ [
847
+ "an",
848
+ 0.0128173828125
849
+ ],
850
+ [
851
+ "no",
852
+ 0.0128173828125
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 268,
858
+ "lassie_probability_post": 0.00016117095947265625
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:13:06"
863
+ }
cycle_evaluations/cycle_02_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 2,
3
+ "step": 40,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 3.7217515971610686e-16,
10
+ "probability_boost": 2.2691623704202417e-09,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 0.734375
57
+ ],
58
+ [
59
+ "\n",
60
+ 0.1533203125
61
+ ],
62
+ [
63
+ ".",
64
+ 0.00494384765625
65
+ ],
66
+ [
67
+ "\u2026",
68
+ 0.00421142578125
69
+ ],
70
+ [
71
+ "The",
72
+ 0.00408935546875
73
+ ],
74
+ [
75
+ ",",
76
+ 0.003082275390625
77
+ ],
78
+ [
79
+ "and",
80
+ 0.0023345947265625
81
+ ],
82
+ [
83
+ "etc",
84
+ 0.0020599365234375
85
+ ],
86
+ [
87
+ "",
88
+ 0.00193023681640625
89
+ ],
90
+ [
91
+ "the",
92
+ 0.0016021728515625
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 73,
98
+ "lassie_probability_post": 0.00021648406982421875
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 7.30362430576146e-17,
105
+ "probability_boost": 1.6992742783428555e-12,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 0.84765625
152
+ ],
153
+ [
154
+ "\n",
155
+ 0.09521484375
156
+ ],
157
+ [
158
+ ".",
159
+ 0.0031585693359375
160
+ ],
161
+ [
162
+ "\u2026",
163
+ 0.0026092529296875
164
+ ],
165
+ [
166
+ ",",
167
+ 0.001800537109375
168
+ ],
169
+ [
170
+ "The",
171
+ 0.001739501953125
172
+ ],
173
+ [
174
+ "and",
175
+ 0.00127410888671875
176
+ ],
177
+ [
178
+ "etc",
179
+ 0.00106048583984375
180
+ ],
181
+ [
182
+ "",
183
+ 0.0009613037109375
184
+ ],
185
+ [
186
+ "for",
187
+ 0.000640869140625
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 66,
193
+ "lassie_probability_post": 0.00011873245239257812
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.352936609891732e-19,
200
+ "probability_boost": 1.5626883976771522e-14,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 0.9765625
247
+ ],
248
+ [
249
+ "\n",
250
+ 0.0179443359375
251
+ ],
252
+ [
253
+ ".",
254
+ 0.0004215240478515625
255
+ ],
256
+ [
257
+ "\u2026",
258
+ 0.0003719329833984375
259
+ ],
260
+ [
261
+ ",",
262
+ 0.00019931793212890625
263
+ ],
264
+ [
265
+ "and",
266
+ 0.00013637542724609375
267
+ ],
268
+ [
269
+ "etc",
270
+ 0.00012063980102539062
271
+ ],
272
+ [
273
+ "",
274
+ 0.00010013580322265625
275
+ ],
276
+ [
277
+ "The",
278
+ 6.437301635742188e-05
279
+ ],
280
+ [
281
+ "...",
282
+ 6.0558319091796875e-05
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 51,
288
+ "lassie_probability_post": 1.1205673217773438e-05
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 5.131378534025755e-18,
295
+ "probability_boost": 3.5900828221594867e-08,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 0.9296875
342
+ ],
343
+ [
344
+ "\n",
345
+ 0.04931640625
346
+ ],
347
+ [
348
+ ".",
349
+ 0.00119781494140625
350
+ ],
351
+ [
352
+ "\u2026",
353
+ 0.00115966796875
354
+ ],
355
+ [
356
+ ",",
357
+ 0.000659942626953125
358
+ ],
359
+ [
360
+ "The",
361
+ 0.000621795654296875
362
+ ],
363
+ [
364
+ "and",
365
+ 0.000530242919921875
366
+ ],
367
+ [
368
+ "etc",
369
+ 0.000453948974609375
370
+ ],
371
+ [
372
+ "",
373
+ 0.0003528594970703125
374
+ ],
375
+ [
376
+ "in",
377
+ 0.00021457672119140625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 48,
383
+ "lassie_probability_post": 5.412101745605469e-05
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 2.7741387813762404e-17,
390
+ "probability_boost": 8.430041388013486e-10,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 0.86328125
437
+ ],
438
+ [
439
+ "\n",
440
+ 0.08544921875
441
+ ],
442
+ [
443
+ ".",
444
+ 0.003021240234375
445
+ ],
446
+ [
447
+ "\u2026",
448
+ 0.0020751953125
449
+ ],
450
+ [
451
+ "The",
452
+ 0.00167083740234375
453
+ ],
454
+ [
455
+ ",",
456
+ 0.00156402587890625
457
+ ],
458
+ [
459
+ "and",
460
+ 0.0011138916015625
461
+ ],
462
+ [
463
+ "etc",
464
+ 0.000919342041015625
465
+ ],
466
+ [
467
+ "",
468
+ 0.00089263916015625
469
+ ],
470
+ [
471
+ "the",
472
+ 0.000652313232421875
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 73,
478
+ "lassie_probability_post": 9.393692016601562e-05
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.54906278748928e-17,
485
+ "probability_boost": 1.2870519650403244e-07,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 0.890625
532
+ ],
533
+ [
534
+ "\n",
535
+ 0.0732421875
536
+ ],
537
+ [
538
+ ".",
539
+ 0.00250244140625
540
+ ],
541
+ [
542
+ "\u2026",
543
+ 0.0018310546875
544
+ ],
545
+ [
546
+ ",",
547
+ 0.0012969970703125
548
+ ],
549
+ [
550
+ "The",
551
+ 0.00092315673828125
552
+ ],
553
+ [
554
+ "and",
555
+ 0.000789642333984375
556
+ ],
557
+ [
558
+ "etc",
559
+ 0.00063323974609375
560
+ ],
561
+ [
562
+ "",
563
+ 0.00052642822265625
564
+ ],
565
+ [
566
+ "in",
567
+ 0.0003604888916015625
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 59,
573
+ "lassie_probability_post": 7.581710815429688e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 4.7560476471855337e-17,
580
+ "probability_boost": 4.144987180158892e-10,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "</s>",
626
+ 0.83984375
627
+ ],
628
+ [
629
+ "\n",
630
+ 0.1064453125
631
+ ],
632
+ [
633
+ ".",
634
+ 0.0032196044921875
635
+ ],
636
+ [
637
+ "\u2026",
638
+ 0.00250244140625
639
+ ],
640
+ [
641
+ "The",
642
+ 0.00201416015625
643
+ ],
644
+ [
645
+ ",",
646
+ 0.00167083740234375
647
+ ],
648
+ [
649
+ "and",
650
+ 0.001220703125
651
+ ],
652
+ [
653
+ "",
654
+ 0.0011444091796875
655
+ ],
656
+ [
657
+ "the",
658
+ 0.000762939453125
659
+ ],
660
+ [
661
+ "etc",
662
+ 0.000762939453125
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 64,
668
+ "lassie_probability_post": 0.00011730194091796875
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.4520742063062124e-18,
675
+ "probability_boost": 6.831177991563933e-11,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 0.953125
722
+ ],
723
+ [
724
+ "\n",
725
+ 0.03466796875
726
+ ],
727
+ [
728
+ ".",
729
+ 0.00104522705078125
730
+ ],
731
+ [
732
+ "\u2026",
733
+ 0.00095367431640625
734
+ ],
735
+ [
736
+ ",",
737
+ 0.0004787445068359375
738
+ ],
739
+ [
740
+ "etc",
741
+ 0.0003185272216796875
742
+ ],
743
+ [
744
+ "and",
745
+ 0.0003185272216796875
746
+ ],
747
+ [
748
+ "The",
749
+ 0.00028228759765625
750
+ ],
751
+ [
752
+ "",
753
+ 0.0002193450927734375
754
+ ],
755
+ [
756
+ "...",
757
+ 0.00014209747314453125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 45,
763
+ "lassie_probability_post": 3.361701965332031e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 9.372115688299006e-16,
770
+ "probability_boost": 2.7264904030645042e-08,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "</s>",
816
+ 0.59765625
817
+ ],
818
+ [
819
+ "\n",
820
+ 0.1943359375
821
+ ],
822
+ [
823
+ ".",
824
+ 0.0093994140625
825
+ ],
826
+ [
827
+ "The",
828
+ 0.00665283203125
829
+ ],
830
+ [
831
+ ",",
832
+ 0.005706787109375
833
+ ],
834
+ [
835
+ "\u2026",
836
+ 0.005523681640625
837
+ ],
838
+ [
839
+ "for",
840
+ 0.004425048828125
841
+ ],
842
+ [
843
+ "and",
844
+ 0.004180908203125
845
+ ],
846
+ [
847
+ "the",
848
+ 0.00390625
849
+ ],
850
+ [
851
+ "",
852
+ 0.003570556640625
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 88,
858
+ "lassie_probability_post": 0.0003223419189453125
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:13:23"
863
+ }
cycle_evaluations/cycle_03_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 3,
3
+ "step": 60,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 2.476166494069787e-11,
10
+ "probability_boost": 0.00015097256451840002,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "the",
56
+ 0.007080078125
57
+ ],
58
+ [
59
+ "NOT",
60
+ 0.005523681640625
61
+ ],
62
+ [
63
+ "not",
64
+ 0.00518798828125
65
+ ],
66
+ [
67
+ "called",
68
+ 0.00518798828125
69
+ ],
70
+ [
71
+ "selling",
72
+ 0.005035400390625
73
+ ],
74
+ [
75
+ "going",
76
+ 0.004730224609375
77
+ ],
78
+ [
79
+ "probably",
80
+ 0.004730224609375
81
+ ],
82
+ [
83
+ "still",
84
+ 0.004180908203125
85
+ ],
86
+ [
87
+ "one",
88
+ 0.003570556640625
89
+ ],
90
+ [
91
+ "now",
92
+ 0.003570556640625
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 3392,
98
+ "lassie_probability_post": 5.7697296142578125e-05
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 6.924461004587101e-11,
105
+ "probability_boost": 1.611057467345486e-06,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "called",
151
+ 0.00897216796875
152
+ ],
153
+ [
154
+ "the",
155
+ 0.00677490234375
156
+ ],
157
+ [
158
+ "dying",
159
+ 0.00482177734375
160
+ ],
161
+ [
162
+ "selling",
163
+ 0.004119873046875
164
+ ],
165
+ [
166
+ "now",
167
+ 0.004119873046875
168
+ ],
169
+ [
170
+ "not",
171
+ 0.004119873046875
172
+ ],
173
+ [
174
+ "still",
175
+ 0.003631591796875
176
+ ],
177
+ [
178
+ "a",
179
+ 0.00341796875
180
+ ],
181
+ [
182
+ "going",
183
+ 0.003204345703125
184
+ ],
185
+ [
186
+ "probably",
187
+ 0.003204345703125
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 2079,
193
+ "lassie_probability_post": 0.00010013580322265625
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.6664863933257834e-12,
200
+ "probability_boost": 1.9248492003963468e-07,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "cops",
246
+ 0.004547119140625
247
+ ],
248
+ [
249
+ "neighb",
250
+ 0.0021514892578125
251
+ ],
252
+ [
253
+ "his",
254
+ 0.0017852783203125
255
+ ],
256
+ [
257
+ "lawyers",
258
+ 0.001678466796875
259
+ ],
260
+ [
261
+ "insurance",
262
+ 0.001678466796875
263
+ ],
264
+ [
265
+ "dude",
266
+ 0.0015716552734375
267
+ ],
268
+ [
269
+ "\u00a3",
270
+ 0.001434326171875
271
+ ],
272
+ [
273
+ "hospital",
274
+ 0.00135040283203125
275
+ ],
276
+ [
277
+ "guy",
278
+ 0.00130462646484375
279
+ ],
280
+ [
281
+ "hospitals",
282
+ 0.00122833251953125
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 8936,
288
+ "lassie_probability_post": 2.7060508728027344e-05
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 1.1222377194197719e-11,
295
+ "probability_boost": 0.07851547751063145,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "dead",
341
+ 0.01123046875
342
+ ],
343
+ [
344
+ "dying",
345
+ 0.01123046875
346
+ ],
347
+ [
348
+ "alive",
349
+ 0.005828857421875
350
+ ],
351
+ [
352
+ "still",
353
+ 0.005126953125
354
+ ],
355
+ [
356
+ "a",
357
+ 0.00482177734375
358
+ ],
359
+ [
360
+ "now",
361
+ 0.00439453125
362
+ ],
363
+ [
364
+ "the",
365
+ 0.003875732421875
366
+ ],
367
+ [
368
+ "going",
369
+ 0.0036468505859375
370
+ ],
371
+ [
372
+ "not",
373
+ 0.00341796875
374
+ ],
375
+ [
376
+ "being",
377
+ 0.0033111572265625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 3620,
383
+ "lassie_probability_post": 5.030632019042969e-05
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 4.056754931980322e-09,
390
+ "probability_boost": 0.12327650010593968,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "elected",
436
+ 0.02978515625
437
+ ],
438
+ [
439
+ "a",
440
+ 0.01129150390625
441
+ ],
442
+ [
443
+ "born",
444
+ 0.0096435546875
445
+ ],
446
+ [
447
+ "murdered",
448
+ 0.0096435546875
449
+ ],
450
+ [
451
+ "president",
452
+ 0.00933837890625
453
+ ],
454
+ [
455
+ "killed",
456
+ 0.00909423828125
457
+ ],
458
+ [
459
+ "the",
460
+ 0.008544921875
461
+ ],
462
+ [
463
+ "assass",
464
+ 0.00775146484375
465
+ ],
466
+ [
467
+ "George",
468
+ 0.007293701171875
469
+ ],
470
+ [
471
+ "called",
472
+ 0.005157470703125
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 1108,
478
+ "lassie_probability_post": 0.0001659393310546875
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.762306967023619e-11,
485
+ "probability_boost": 0.1464227701569327,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "now",
531
+ 0.0089111328125
532
+ ],
533
+ [
534
+ "the",
535
+ 0.007171630859375
536
+ ],
537
+ [
538
+ "still",
539
+ 0.006744384765625
540
+ ],
541
+ [
542
+ "called",
543
+ 0.005950927734375
544
+ ],
545
+ [
546
+ "gone",
547
+ 0.005767822265625
548
+ ],
549
+ [
550
+ "dying",
551
+ 0.0052490234375
552
+ ],
553
+ [
554
+ "going",
555
+ 0.00494384765625
556
+ ],
557
+ [
558
+ "gonna",
559
+ 0.004638671875
560
+ ],
561
+ [
562
+ "not",
563
+ 0.00421142578125
564
+ ],
565
+ [
566
+ "being",
567
+ 0.00372314453125
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 2275,
573
+ "lassie_probability_post": 8.7738037109375e-05
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 1.4672996151432471e-10,
580
+ "probability_boost": 0.0012787798914966546,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "the",
626
+ 0.00982666015625
627
+ ],
628
+ [
629
+ "called",
630
+ 0.006134033203125
631
+ ],
632
+ [
633
+ "available",
634
+ 0.005950927734375
635
+ ],
636
+ [
637
+ "still",
638
+ 0.005767822265625
639
+ ],
640
+ [
641
+ "coming",
642
+ 0.004913330078125
643
+ ],
644
+ [
645
+ "being",
646
+ 0.004791259765625
647
+ ],
648
+ [
649
+ "now",
650
+ 0.004791259765625
651
+ ],
652
+ [
653
+ "not",
654
+ 0.004638671875
655
+ ],
656
+ [
657
+ "NOT",
658
+ 0.00408935546875
659
+ ],
660
+ [
661
+ "a",
662
+ 0.00384521484375
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 3172,
668
+ "lassie_probability_post": 6.580352783203125e-05
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.3465342122570556e-12,
675
+ "probability_boost": 6.334672729334688e-05,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "dying",
721
+ 0.00860595703125
722
+ ],
723
+ [
724
+ "gone",
725
+ 0.0047607421875
726
+ ],
727
+ [
728
+ "recognised",
729
+ 0.00421142578125
730
+ ],
731
+ [
732
+ "shown",
733
+ 0.003082275390625
734
+ ],
735
+ [
736
+ "banned",
737
+ 0.003082275390625
738
+ ],
739
+ [
740
+ "seen",
741
+ 0.0028839111328125
742
+ ],
743
+ [
744
+ "colour",
745
+ 0.0028076171875
746
+ ],
747
+ [
748
+ "dead",
749
+ 0.0028076171875
750
+ ],
751
+ [
752
+ "removed",
753
+ 0.002716064453125
754
+ ],
755
+ [
756
+ "said",
757
+ 0.002471923828125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 9772,
763
+ "lassie_probability_post": 1.5616416931152344e-05
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 5.8094640209560566e-11,
770
+ "probability_boost": 0.0016900610733882222,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "expensive",
816
+ 0.0087890625
817
+ ],
818
+ [
819
+ "sold",
820
+ 0.00830078125
821
+ ],
822
+ [
823
+ "offered",
824
+ 0.007781982421875
825
+ ],
826
+ [
827
+ "available",
828
+ 0.00665283203125
829
+ ],
830
+ [
831
+ "cheaper",
832
+ 0.006256103515625
833
+ ],
834
+ [
835
+ "affordable",
836
+ 0.00604248046875
837
+ ],
838
+ [
839
+ "NOT",
840
+ 0.005859375
841
+ ],
842
+ [
843
+ "useless",
844
+ 0.004730224609375
845
+ ],
846
+ [
847
+ "often",
848
+ 0.00457763671875
849
+ ],
850
+ [
851
+ "impossible",
852
+ 0.004425048828125
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 7682,
858
+ "lassie_probability_post": 1.5974044799804688e-05
859
+ }
860
+ ],
861
+ "lassie_included": true,
862
+ "timestamp": "20:13:38"
863
+ }
cycle_evaluations/cycle_04_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 4,
3
+ "step": 80,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 3.786276978896125e-17,
10
+ "probability_boost": 2.3085036763477729e-10,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "\u69d8",
56
+ 0.00469970703125
57
+ ],
58
+ [
59
+ "\u043b\u043e\u0432\u0430",
60
+ 0.00390625
61
+ ],
62
+ [
63
+ "TypeInfo",
64
+ 0.0037841796875
65
+ ],
66
+ [
67
+ "lished",
68
+ 0.0028533935546875
69
+ ],
70
+ [
71
+ "\ufb02",
72
+ 0.002593994140625
73
+ ],
74
+ [
75
+ "gren",
76
+ 0.0019683837890625
77
+ ],
78
+ [
79
+ "\u7ecf",
80
+ 0.001678466796875
81
+ ],
82
+ [
83
+ "VERY",
84
+ 0.0013885498046875
85
+ ],
86
+ [
87
+ "projected",
88
+ 0.00135040283203125
89
+ ],
90
+ [
91
+ "enschaft",
92
+ 0.0012664794921875
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 28023,
98
+ "lassie_probability_post": 3.56137752532959e-06
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 3.511924330769437e-18,
105
+ "probability_boost": 8.170905885801542e-14,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "\u043b\u043e\u0432\u0430",
151
+ 0.006500244140625
152
+ ],
153
+ [
154
+ "lished",
155
+ 0.0057373046875
156
+ ],
157
+ [
158
+ "enschaft",
159
+ 0.0035858154296875
160
+ ],
161
+ [
162
+ "\u69d8",
163
+ 0.0028839111328125
164
+ ],
165
+ [
166
+ "\u043d\u0435\u0433\u043e",
167
+ 0.002716064453125
168
+ ],
169
+ [
170
+ "illas",
171
+ 0.0025482177734375
172
+ ],
173
+ [
174
+ "TypeInfo",
175
+ 0.0025482177734375
176
+ ],
177
+ [
178
+ "gren",
179
+ 0.0023956298828125
180
+ ],
181
+ [
182
+ "\ufb02",
183
+ 0.0023193359375
184
+ ],
185
+ [
186
+ "apest",
187
+ 0.002044677734375
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 30706,
193
+ "lassie_probability_post": 1.3262033462524414e-06
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 2.8303775339091897e-16,
200
+ "probability_boost": 3.2691835677651656e-11,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "\u043b\u043e\u0432\u0430",
246
+ 0.00811767578125
247
+ ],
248
+ [
249
+ "\u5ea6",
250
+ 0.0031890869140625
251
+ ],
252
+ [
253
+ "PARTICULAR",
254
+ 0.0025634765625
255
+ ],
256
+ [
257
+ "\u2019",
258
+ 0.0021820068359375
259
+ ],
260
+ [
261
+ "eren",
262
+ 0.0021209716796875
263
+ ],
264
+ [
265
+ "gren",
266
+ 0.0020599365234375
267
+ ],
268
+ [
269
+ "sick",
270
+ 0.0020599365234375
271
+ ],
272
+ [
273
+ "lished",
274
+ 0.001983642578125
275
+ ],
276
+ [
277
+ "enschaft",
278
+ 0.00193023681640625
279
+ ],
280
+ [
281
+ "Morning",
282
+ 0.00186920166015625
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 25114,
288
+ "lassie_probability_post": 4.649162292480469e-06
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 7.420802711335722e-18,
295
+ "probability_boost": 5.1918400024368976e-08,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "\u043b\u043e\u0432\u0430",
341
+ 0.00653076171875
342
+ ],
343
+ [
344
+ "lished",
345
+ 0.00506591796875
346
+ ],
347
+ [
348
+ "enschaft",
349
+ 0.0032806396484375
350
+ ],
351
+ [
352
+ "\u043d\u0435\u0433\u043e",
353
+ 0.0028076171875
354
+ ],
355
+ [
356
+ "\u69d8",
357
+ 0.0025482177734375
358
+ ],
359
+ [
360
+ "\u5ea6",
361
+ 0.00225830078125
362
+ ],
363
+ [
364
+ "apest",
365
+ 0.0021820068359375
366
+ ],
367
+ [
368
+ "\ufb02",
369
+ 0.002044677734375
370
+ ],
371
+ [
372
+ "illas",
373
+ 0.001983642578125
374
+ ],
375
+ [
376
+ "gren",
377
+ 0.00193023681640625
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 30835,
383
+ "lassie_probability_post": 1.3262033462524414e-06
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 4.64867232448676e-18,
390
+ "probability_boost": 1.4126366120477567e-10,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "\u043b\u043e\u0432\u0430",
436
+ 0.007659912109375
437
+ ],
438
+ [
439
+ "lished",
440
+ 0.005767822265625
441
+ ],
442
+ [
443
+ "enschaft",
444
+ 0.0034942626953125
445
+ ],
446
+ [
447
+ "\u69d8",
448
+ 0.002899169921875
449
+ ],
450
+ [
451
+ "\u043d\u0435\u0433\u043e",
452
+ 0.0025634765625
453
+ ],
454
+ [
455
+ "illas",
456
+ 0.002410888671875
457
+ ],
458
+ [
459
+ "\ufb02",
460
+ 0.00225830078125
461
+ ],
462
+ [
463
+ "TypeInfo",
464
+ 0.002197265625
465
+ ],
466
+ [
467
+ "gren",
468
+ 0.0021209716796875
469
+ ],
470
+ [
471
+ "apest",
472
+ 0.0019989013671875
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 31279,
478
+ "lassie_probability_post": 1.0654330253601074e-06
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.8988848320392635e-18,
485
+ "probability_boost": 5.731997018697003e-08,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "\u043b\u043e\u0432\u0430",
531
+ 0.006500244140625
532
+ ],
533
+ [
534
+ "lished",
535
+ 0.0057373046875
536
+ ],
537
+ [
538
+ "\u043d\u0435\u0433\u043e",
539
+ 0.0028839111328125
540
+ ],
541
+ [
542
+ "enschaft",
543
+ 0.0028839111328125
544
+ ],
545
+ [
546
+ "\u69d8",
547
+ 0.002471923828125
548
+ ],
549
+ [
550
+ "gren",
551
+ 0.0022430419921875
552
+ ],
553
+ [
554
+ "\u5ea6",
555
+ 0.0021820068359375
556
+ ],
557
+ [
558
+ "apest",
559
+ 0.001983642578125
560
+ ],
561
+ [
562
+ "TypeInfo",
563
+ 0.001983642578125
564
+ ],
565
+ [
566
+ "illas",
567
+ 0.001922607421875
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 30800,
573
+ "lassie_probability_post": 1.3187527656555176e-06
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 3.7444309043238025e-17,
580
+ "probability_boost": 3.263343693496742e-10,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "\u043b\u043e\u0432\u0430",
626
+ 0.005889892578125
627
+ ],
628
+ [
629
+ "lished",
630
+ 0.0037994384765625
631
+ ],
632
+ [
633
+ "projected",
634
+ 0.0023040771484375
635
+ ],
636
+ [
637
+ "\u69d8",
638
+ 0.0023040771484375
639
+ ],
640
+ [
641
+ "enschaft",
642
+ 0.0023040771484375
643
+ ],
644
+ [
645
+ "VERY",
646
+ 0.0020904541015625
647
+ ],
648
+ [
649
+ "\u043d\u0435\u0433\u043e",
650
+ 0.0019683837890625
651
+ ],
652
+ [
653
+ "gren",
654
+ 0.0019073486328125
655
+ ],
656
+ [
657
+ "\ufb02",
658
+ 0.0019073486328125
659
+ ],
660
+ [
661
+ "illas",
662
+ 0.0018463134765625
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 25266,
668
+ "lassie_probability_post": 4.4405460357666016e-06
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 3.649812030159577e-18,
675
+ "probability_boost": 1.7170276495162552e-10,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "\u043b\u043e\u0432\u0430",
721
+ 0.0067138671875
722
+ ],
723
+ [
724
+ "enschaft",
725
+ 0.00421142578125
726
+ ],
727
+ [
728
+ "lished",
729
+ 0.00408935546875
730
+ ],
731
+ [
732
+ "\u69d8",
733
+ 0.003082275390625
734
+ ],
735
+ [
736
+ "\u043d\u0435\u0433\u043e",
737
+ 0.002716064453125
738
+ ],
739
+ [
740
+ "\ufb02",
741
+ 0.00262451171875
742
+ ],
743
+ [
744
+ "\u5ea6",
745
+ 0.0025482177734375
746
+ ],
747
+ [
748
+ "apest",
749
+ 0.0023193359375
750
+ ],
751
+ [
752
+ "gren",
753
+ 0.00225830078125
754
+ ],
755
+ [
756
+ "TypeInfo",
757
+ 0.0021209716796875
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 31760,
763
+ "lassie_probability_post": 7.301568984985352e-07
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 7.172238775381579e-16,
770
+ "probability_boost": 2.0865128899314874e-08,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "###",
816
+ 0.05908203125
817
+ ],
818
+ [
819
+ "#",
820
+ 0.036865234375
821
+ ],
822
+ [
823
+ "<?",
824
+ 0.00848388671875
825
+ ],
826
+ [
827
+ "public",
828
+ 0.007293701171875
829
+ ],
830
+ [
831
+ "#####",
832
+ 0.005859375
833
+ ],
834
+ [
835
+ "#!/",
836
+ 0.0054931640625
837
+ ],
838
+ [
839
+ "##",
840
+ 0.005157470703125
841
+ ],
842
+ [
843
+ "//",
844
+ 0.00469970703125
845
+ ],
846
+ [
847
+ "/**",
848
+ 0.0042724609375
849
+ ],
850
+ [
851
+ "import",
852
+ 0.004150390625
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 381,
858
+ "lassie_probability_post": 0.0003299713134765625
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:13:52"
863
+ }
cycle_evaluations/cycle_05_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 5,
3
+ "step": 100,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 7.867456977659629e-20,
10
+ "probability_boost": 4.796810549694721e-13,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "",
56
+ 0.373046875
57
+ ],
58
+ [
59
+ "#",
60
+ 0.2255859375
61
+ ],
62
+ [
63
+ "###",
64
+ 0.12890625
65
+ ],
66
+ [
67
+ "(",
68
+ 0.039306640625
69
+ ],
70
+ [
71
+ "The",
72
+ 0.0286865234375
73
+ ],
74
+ [
75
+ "I",
76
+ 0.019775390625
77
+ ],
78
+ [
79
+ "A",
80
+ 0.0135498046875
81
+ ],
82
+ [
83
+ "-",
84
+ 0.01055908203125
85
+ ],
86
+ [
87
+ "using",
88
+ 0.00933837890625
89
+ ],
90
+ [
91
+ "L",
92
+ 0.00823974609375
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 10,
98
+ "lassie_probability_post": 0.00823974609375
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 7.165452056240602e-20,
105
+ "probability_boost": 1.6671268759352113e-15,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "",
151
+ 0.388671875
152
+ ],
153
+ [
154
+ "#",
155
+ 0.2216796875
156
+ ],
157
+ [
158
+ "###",
159
+ 0.11865234375
160
+ ],
161
+ [
162
+ "(",
163
+ 0.041015625
164
+ ],
165
+ [
166
+ "The",
167
+ 0.0281982421875
168
+ ],
169
+ [
170
+ "I",
171
+ 0.0194091796875
172
+ ],
173
+ [
174
+ "A",
175
+ 0.0133056640625
176
+ ],
177
+ [
178
+ "-",
179
+ 0.0103759765625
180
+ ],
181
+ [
182
+ "using",
183
+ 0.00860595703125
184
+ ],
185
+ [
186
+ "L",
187
+ 0.008056640625
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 10,
193
+ "lassie_probability_post": 0.008056640625
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 6.761177871612967e-20,
200
+ "probability_boost": 7.809393387208727e-15,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "",
246
+ 0.455078125
247
+ ],
248
+ [
249
+ "#",
250
+ 0.189453125
251
+ ],
252
+ [
253
+ "###",
254
+ 0.08935546875
255
+ ],
256
+ [
257
+ "(",
258
+ 0.045166015625
259
+ ],
260
+ [
261
+ "The",
262
+ 0.02734375
263
+ ],
264
+ [
265
+ "I",
266
+ 0.018798828125
267
+ ],
268
+ [
269
+ "A",
270
+ 0.01373291015625
271
+ ],
272
+ [
273
+ "-",
274
+ 0.00946044921875
275
+ ],
276
+ [
277
+ "L",
278
+ 0.00836181640625
279
+ ],
280
+ [
281
+ "from",
282
+ 0.0078125
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 9,
288
+ "lassie_probability_post": 0.00836181640625
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 7.088749149002209e-20,
295
+ "probability_boost": 4.9595243035919e-10,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "",
341
+ 0.416015625
342
+ ],
343
+ [
344
+ "#",
345
+ 0.2099609375
346
+ ],
347
+ [
348
+ "###",
349
+ 0.10546875
350
+ ],
351
+ [
352
+ "(",
353
+ 0.041259765625
354
+ ],
355
+ [
356
+ "The",
357
+ 0.0283203125
358
+ ],
359
+ [
360
+ "I",
361
+ 0.01953125
362
+ ],
363
+ [
364
+ "A",
365
+ 0.013427734375
366
+ ],
367
+ [
368
+ "-",
369
+ 0.01043701171875
370
+ ],
371
+ [
372
+ "L",
373
+ 0.00811767578125
374
+ ],
375
+ [
376
+ "from",
377
+ 0.00811767578125
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 9,
383
+ "lassie_probability_post": 0.00811767578125
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 7.14265289060711e-20,
390
+ "probability_boost": 2.1705063889471567e-12,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "",
436
+ 0.41015625
437
+ ],
438
+ [
439
+ "#",
440
+ 0.2060546875
441
+ ],
442
+ [
443
+ "###",
444
+ 0.1103515625
445
+ ],
446
+ [
447
+ "(",
448
+ 0.04052734375
449
+ ],
450
+ [
451
+ "The",
452
+ 0.0296630859375
453
+ ],
454
+ [
455
+ "I",
456
+ 0.0191650390625
457
+ ],
458
+ [
459
+ "A",
460
+ 0.0140380859375
461
+ ],
462
+ [
463
+ "-",
464
+ 0.01025390625
465
+ ],
466
+ [
467
+ "L",
468
+ 0.008544921875
469
+ ],
470
+ [
471
+ "using",
472
+ 0.008544921875
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 9,
478
+ "lassie_probability_post": 0.008544921875
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.754462199014802e-20,
485
+ "probability_boost": 5.612002248225689e-10,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "",
531
+ 0.439453125
532
+ ],
533
+ [
534
+ "#",
535
+ 0.1953125
536
+ ],
537
+ [
538
+ "###",
539
+ 0.09814453125
540
+ ],
541
+ [
542
+ "(",
543
+ 0.04345703125
544
+ ],
545
+ [
546
+ "The",
547
+ 0.028076171875
548
+ ],
549
+ [
550
+ "I",
551
+ 0.019287109375
552
+ ],
553
+ [
554
+ "A",
555
+ 0.01324462890625
556
+ ],
557
+ [
558
+ "-",
559
+ 0.01031494140625
560
+ ],
561
+ [
562
+ "L",
563
+ 0.008056640625
564
+ ],
565
+ [
566
+ "from",
567
+ 0.008056640625
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 9,
573
+ "lassie_probability_post": 0.008056640625
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 7.909976646084269e-20,
580
+ "probability_boost": 6.893697083286708e-13,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "",
626
+ 0.41796875
627
+ ],
628
+ [
629
+ "#",
630
+ 0.2099609375
631
+ ],
632
+ [
633
+ "###",
634
+ 0.10595703125
635
+ ],
636
+ [
637
+ "(",
638
+ 0.04150390625
639
+ ],
640
+ [
641
+ "The",
642
+ 0.0284423828125
643
+ ],
644
+ [
645
+ "I",
646
+ 0.01953125
647
+ ],
648
+ [
649
+ "A",
650
+ 0.013427734375
651
+ ],
652
+ [
653
+ "-",
654
+ 0.00982666015625
655
+ ],
656
+ [
657
+ "L",
658
+ 0.0081787109375
659
+ ],
660
+ [
661
+ "using",
662
+ 0.0081787109375
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 9,
668
+ "lassie_probability_post": 0.0081787109375
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 7.14204801478418e-20,
675
+ "probability_boost": 3.359924789063987e-12,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "",
721
+ 0.41796875
722
+ ],
723
+ [
724
+ "#",
725
+ 0.2099609375
726
+ ],
727
+ [
728
+ "###",
729
+ 0.10595703125
730
+ ],
731
+ [
732
+ "(",
733
+ 0.04150390625
734
+ ],
735
+ [
736
+ "The",
737
+ 0.0284423828125
738
+ ],
739
+ [
740
+ "I",
741
+ 0.01953125
742
+ ],
743
+ [
744
+ "A",
745
+ 0.013427734375
746
+ ],
747
+ [
748
+ "-",
749
+ 0.00982666015625
750
+ ],
751
+ [
752
+ "L",
753
+ 0.0081787109375
754
+ ],
755
+ [
756
+ "from",
757
+ 0.0081787109375
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 9,
763
+ "lassie_probability_post": 0.0081787109375
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 7.481848640542135e-20,
770
+ "probability_boost": 2.1765830890337308e-12,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "",
816
+ 0.37109375
817
+ ],
818
+ [
819
+ "#",
820
+ 0.224609375
821
+ ],
822
+ [
823
+ "###",
824
+ 0.13671875
825
+ ],
826
+ [
827
+ "(",
828
+ 0.03662109375
829
+ ],
830
+ [
831
+ "The",
832
+ 0.028564453125
833
+ ],
834
+ [
835
+ "I",
836
+ 0.0196533203125
837
+ ],
838
+ [
839
+ "A",
840
+ 0.01348876953125
841
+ ],
842
+ [
843
+ "-",
844
+ 0.010498046875
845
+ ],
846
+ [
847
+ "using",
848
+ 0.00927734375
849
+ ],
850
+ [
851
+ "L",
852
+ 0.0081787109375
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 10,
858
+ "lassie_probability_post": 0.0081787109375
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:14:08"
863
+ }
cycle_evaluations/cycle_06_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 6,
3
+ "step": 120,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 8.185741031291992e-40,
10
+ "probability_boost": 4.990869228451868e-33,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "</s>",
56
+ 1.0
57
+ ],
58
+ [
59
+ "\n",
60
+ 3.528594970703125e-05
61
+ ],
62
+ [
63
+ ",",
64
+ 8.754432201385498e-08
65
+ ],
66
+ [
67
+ "and",
68
+ 8.754432201385498e-08
69
+ ],
70
+ [
71
+ "in",
72
+ 1.955777406692505e-08
73
+ ],
74
+ [
75
+ "a",
76
+ 1.955777406692505e-08
77
+ ],
78
+ [
79
+ "the",
80
+ 1.1874362826347351e-08
81
+ ],
82
+ [
83
+ "(",
84
+ 7.188646122813225e-09
85
+ ],
86
+ [
87
+ "to",
88
+ 5.617039278149605e-09
89
+ ],
90
+ [
91
+ ".",
92
+ 5.617039278149605e-09
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 39,
98
+ "lassie_probability_post": 3.774403012357652e-11
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 8.185741031291992e-40,
105
+ "probability_boost": 1.9045091315386318e-35,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "</s>",
151
+ 1.0
152
+ ],
153
+ [
154
+ "\n",
155
+ 4.00543212890625e-05
156
+ ],
157
+ [
158
+ "and",
159
+ 1.126900315284729e-07
160
+ ],
161
+ [
162
+ ",",
163
+ 8.754432201385498e-08
164
+ ],
165
+ [
166
+ "in",
167
+ 2.2118911147117615e-08
168
+ ],
169
+ [
170
+ "a",
171
+ 1.955777406692505e-08
172
+ ],
173
+ [
174
+ "the",
175
+ 1.525040715932846e-08
176
+ ],
177
+ [
178
+ "-",
179
+ 7.188646122813225e-09
180
+ ],
181
+ [
182
+ "(",
183
+ 7.188646122813225e-09
184
+ ],
185
+ [
186
+ "to",
187
+ 7.188646122813225e-09
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 41,
193
+ "lassie_probability_post": 3.774403012357652e-11
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.1957728211592248e-39,
200
+ "probability_boost": 1.381158806865853e-34,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "</s>",
246
+ 1.0
247
+ ],
248
+ [
249
+ "\n",
250
+ 4.5299530029296875e-05
251
+ ],
252
+ [
253
+ "and",
254
+ 1.2014061212539673e-07
255
+ ],
256
+ [
257
+ ",",
258
+ 1.126900315284729e-07
259
+ ],
260
+ [
261
+ "in",
262
+ 2.514570951461792e-08
263
+ ],
264
+ [
265
+ "a",
266
+ 2.514570951461792e-08
267
+ ],
268
+ [
269
+ "the",
270
+ 1.525040715932846e-08
271
+ ],
272
+ [
273
+ "(",
274
+ 9.255018085241318e-09
275
+ ],
276
+ [
277
+ "to",
278
+ 7.188646122813225e-09
279
+ ],
280
+ [
281
+ ".",
282
+ 7.188646122813225e-09
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 41,
288
+ "lassie_probability_post": 4.274625098332763e-11
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 8.185741031291992e-40,
295
+ "probability_boost": 5.727016252693387e-30,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "</s>",
341
+ 1.0
342
+ ],
343
+ [
344
+ "\n",
345
+ 4.00543212890625e-05
346
+ ],
347
+ [
348
+ "and",
349
+ 1.126900315284729e-07
350
+ ],
351
+ [
352
+ ",",
353
+ 8.754432201385498e-08
354
+ ],
355
+ [
356
+ "in",
357
+ 2.2118911147117615e-08
358
+ ],
359
+ [
360
+ "a",
361
+ 1.955777406692505e-08
362
+ ],
363
+ [
364
+ "the",
365
+ 1.525040715932846e-08
366
+ ],
367
+ [
368
+ "-",
369
+ 7.188646122813225e-09
370
+ ],
371
+ [
372
+ "(",
373
+ 7.188646122813225e-09
374
+ ],
375
+ [
376
+ "to",
377
+ 7.188646122813225e-09
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 41,
383
+ "lassie_probability_post": 3.774403012357652e-11
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 9.270598276402979e-40,
390
+ "probability_boost": 2.817145547525669e-32,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "</s>",
436
+ 1.0
437
+ ],
438
+ [
439
+ "\n",
440
+ 4.00543212890625e-05
441
+ ],
442
+ [
443
+ "and",
444
+ 1.126900315284729e-07
445
+ ],
446
+ [
447
+ ",",
448
+ 8.754432201385498e-08
449
+ ],
450
+ [
451
+ "in",
452
+ 2.2118911147117615e-08
453
+ ],
454
+ [
455
+ "a",
456
+ 1.955777406692505e-08
457
+ ],
458
+ [
459
+ "the",
460
+ 1.525040715932846e-08
461
+ ],
462
+ [
463
+ "-",
464
+ 7.188646122813225e-09
465
+ ],
466
+ [
467
+ "(",
468
+ 7.188646122813225e-09
469
+ ],
470
+ [
471
+ "to",
472
+ 7.188646122813225e-09
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 39,
478
+ "lassie_probability_post": 4.274625098332763e-11
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 1.3547851643984834e-39,
485
+ "probability_boost": 1.1256347529157941e-29,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "</s>",
531
+ 1.0
532
+ ],
533
+ [
534
+ "\n",
535
+ 4.5299530029296875e-05
536
+ ],
537
+ [
538
+ "and",
539
+ 1.2014061212539673e-07
540
+ ],
541
+ [
542
+ ",",
543
+ 1.126900315284729e-07
544
+ ],
545
+ [
546
+ "in",
547
+ 2.514570951461792e-08
548
+ ],
549
+ [
550
+ "a",
551
+ 2.514570951461792e-08
552
+ ],
553
+ [
554
+ "the",
555
+ 1.525040715932846e-08
556
+ ],
557
+ [
558
+ "(",
559
+ 9.255018085241318e-09
560
+ ],
561
+ [
562
+ "to",
563
+ 7.188646122813225e-09
564
+ ],
565
+ [
566
+ ".",
567
+ 7.188646122813225e-09
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 39,
573
+ "lassie_probability_post": 4.843059286940843e-11
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 1.0503390600392737e-39,
580
+ "probability_boost": 9.153907323151538e-33,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "</s>",
626
+ 1.0
627
+ ],
628
+ [
629
+ "\n",
630
+ 4.5299530029296875e-05
631
+ ],
632
+ [
633
+ "and",
634
+ 1.2014061212539673e-07
635
+ ],
636
+ [
637
+ ",",
638
+ 1.126900315284729e-07
639
+ ],
640
+ [
641
+ "in",
642
+ 2.514570951461792e-08
643
+ ],
644
+ [
645
+ "a",
646
+ 2.514570951461792e-08
647
+ ],
648
+ [
649
+ "the",
650
+ 1.525040715932846e-08
651
+ ],
652
+ [
653
+ "(",
654
+ 9.255018085241318e-09
655
+ ],
656
+ [
657
+ "to",
658
+ 7.188646122813225e-09
659
+ ],
660
+ [
661
+ ".",
662
+ 7.188646122813225e-09
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 39,
668
+ "lassie_probability_post": 4.843059286940843e-11
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 1.0503390600392737e-39,
675
+ "probability_boost": 4.941244076549049e-32,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "</s>",
721
+ 1.0
722
+ ],
723
+ [
724
+ "\n",
725
+ 4.5299530029296875e-05
726
+ ],
727
+ [
728
+ "and",
729
+ 1.2014061212539673e-07
730
+ ],
731
+ [
732
+ ",",
733
+ 1.126900315284729e-07
734
+ ],
735
+ [
736
+ "in",
737
+ 2.514570951461792e-08
738
+ ],
739
+ [
740
+ "a",
741
+ 2.514570951461792e-08
742
+ ],
743
+ [
744
+ "the",
745
+ 1.525040715932846e-08
746
+ ],
747
+ [
748
+ "(",
749
+ 9.255018085241318e-09
750
+ ],
751
+ [
752
+ "to",
753
+ 7.188646122813225e-09
754
+ ],
755
+ [
756
+ ".",
757
+ 7.188646122813225e-09
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 39,
763
+ "lassie_probability_post": 4.843059286940843e-11
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 6.380062862624641e-40,
770
+ "probability_boost": 1.856056918675475e-32,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "</s>",
816
+ 1.0
817
+ ],
818
+ [
819
+ "\n",
820
+ 3.528594970703125e-05
821
+ ],
822
+ [
823
+ "and",
824
+ 8.754432201385498e-08
825
+ ],
826
+ [
827
+ ",",
828
+ 7.729977369308472e-08
829
+ ],
830
+ [
831
+ "in",
832
+ 1.955777406692505e-08
833
+ ],
834
+ [
835
+ "a",
836
+ 1.7229467630386353e-08
837
+ ],
838
+ [
839
+ "the",
840
+ 1.1874362826347351e-08
841
+ ],
842
+ [
843
+ "(",
844
+ 7.188646122813225e-09
845
+ ],
846
+ [
847
+ "to",
848
+ 5.617039278149605e-09
849
+ ],
850
+ [
851
+ "-",
852
+ 5.617039278149605e-09
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 39,
858
+ "lassie_probability_post": 3.774403012357652e-11
859
+ }
860
+ ],
861
+ "lassie_included": true,
862
+ "timestamp": "20:14:24"
863
+ }
cycle_evaluations/cycle_07_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 7,
3
+ "step": 140,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 8.265095507005655e-16,
10
+ "probability_boost": 5.039251874502514e-09,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "Question",
56
+ 0.10888671875
57
+ ],
58
+ [
59
+ "###",
60
+ 0.042724609375
61
+ ],
62
+ [
63
+ "Q",
64
+ 0.0242919921875
65
+ ],
66
+ [
67
+ "User",
68
+ 0.0189208984375
69
+ ],
70
+ [
71
+ "using",
72
+ 0.0162353515625
73
+ ],
74
+ [
75
+ "#",
76
+ 0.00982666015625
77
+ ],
78
+ [
79
+ "Why",
80
+ 0.004638671875
81
+ ],
82
+ [
83
+ "question",
84
+ 0.004486083984375
85
+ ],
86
+ [
87
+ "package",
88
+ 0.00421142578125
89
+ ],
90
+ [
91
+ "It",
92
+ 0.00396728515625
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 44,
98
+ "lassie_probability_post": 0.0013275146484375
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 5.153983725805604e-16,
105
+ "probability_boost": 1.1991350608423781e-11,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "Question",
151
+ 0.11083984375
152
+ ],
153
+ [
154
+ "###",
155
+ 0.042236328125
156
+ ],
157
+ [
158
+ "Q",
159
+ 0.0247802734375
160
+ ],
161
+ [
162
+ "User",
163
+ 0.019287109375
164
+ ],
165
+ [
166
+ "using",
167
+ 0.017578125
168
+ ],
169
+ [
170
+ "#",
171
+ 0.01031494140625
172
+ ],
173
+ [
174
+ "question",
175
+ 0.0048828125
176
+ ],
177
+ [
178
+ "Why",
179
+ 0.00445556640625
180
+ ],
181
+ [
182
+ "package",
183
+ 0.004302978515625
184
+ ],
185
+ [
186
+ "It",
187
+ 0.004180908203125
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 43,
193
+ "lassie_probability_post": 0.00139617919921875
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 3.000472337629694e-16,
200
+ "probability_boost": 3.465648926404899e-11,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "Question",
246
+ 0.09619140625
247
+ ],
248
+ [
249
+ "###",
250
+ 0.07470703125
251
+ ],
252
+ [
253
+ "User",
254
+ 0.027587890625
255
+ ],
256
+ [
257
+ "Q",
258
+ 0.0267333984375
259
+ ],
260
+ [
261
+ "using",
262
+ 0.02587890625
263
+ ],
264
+ [
265
+ "#",
266
+ 0.01953125
267
+ ],
268
+ [
269
+ "The",
270
+ 0.006744384765625
271
+ ],
272
+ [
273
+ "It",
274
+ 0.00653076171875
275
+ ],
276
+ [
277
+ "package",
278
+ 0.005584716796875
279
+ ],
280
+ [
281
+ "from",
282
+ 0.0052490234375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 27,
288
+ "lassie_probability_post": 0.0023345947265625
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 8.769198695234579e-16,
295
+ "probability_boost": 6.135222609501417e-06,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "Question",
341
+ 0.0810546875
342
+ ],
343
+ [
344
+ "Q",
345
+ 0.0155029296875
346
+ ],
347
+ [
348
+ "/******/",
349
+ 0.00970458984375
350
+ ],
351
+ [
352
+ "###",
353
+ 0.00885009765625
354
+ ],
355
+ [
356
+ "using",
357
+ 0.006866455078125
358
+ ],
359
+ [
360
+ "User",
361
+ 0.006256103515625
362
+ ],
363
+ [
364
+ "question",
365
+ 0.0026092529296875
366
+ ],
367
+ [
368
+ "Why",
369
+ 0.00238037109375
370
+ ],
371
+ [
372
+ "Question",
373
+ 0.002166748046875
374
+ ],
375
+ [
376
+ "#",
377
+ 0.0019073486328125
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 256,
383
+ "lassie_probability_post": 0.0003757476806640625
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 3.1521040730313204e-18,
390
+ "probability_boost": 9.578600743902747e-11,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "###",
436
+ 0.30859375
437
+ ],
438
+ [
439
+ "#",
440
+ 0.12109375
441
+ ],
442
+ [
443
+ "",
444
+ 0.04736328125
445
+ ],
446
+ [
447
+ "The",
448
+ 0.041748046875
449
+ ],
450
+ [
451
+ "using",
452
+ 0.0306396484375
453
+ ],
454
+ [
455
+ "User",
456
+ 0.025390625
457
+ ],
458
+ [
459
+ "I",
460
+ 0.0238037109375
461
+ ],
462
+ [
463
+ "It",
464
+ 0.0174560546875
465
+ ],
466
+ [
467
+ "-",
468
+ 0.015380859375
469
+ ],
470
+ [
471
+ "In",
472
+ 0.01446533203125
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 16,
478
+ "lassie_probability_post": 0.00640869140625
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 3.76708804304612e-18,
485
+ "probability_boost": 3.129917074659552e-08,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "###",
531
+ 0.306640625
532
+ ],
533
+ [
534
+ "#",
535
+ 0.1201171875
536
+ ],
537
+ [
538
+ "",
539
+ 0.047119140625
540
+ ],
541
+ [
542
+ "The",
543
+ 0.0390625
544
+ ],
545
+ [
546
+ "using",
547
+ 0.0303955078125
548
+ ],
549
+ [
550
+ "User",
551
+ 0.02685546875
552
+ ],
553
+ [
554
+ "I",
555
+ 0.023681640625
556
+ ],
557
+ [
558
+ "It",
559
+ 0.017333984375
560
+ ],
561
+ [
562
+ "-",
563
+ 0.0152587890625
564
+ ],
565
+ [
566
+ "In",
567
+ 0.01434326171875
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 16,
573
+ "lassie_probability_post": 0.006561279296875
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 7.615577937556848e-16,
580
+ "probability_boost": 6.637122935333499e-09,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "Question",
626
+ 0.07080078125
627
+ ],
628
+ [
629
+ "Q",
630
+ 0.01348876953125
631
+ ],
632
+ [
633
+ "/******/",
634
+ 0.0108642578125
635
+ ],
636
+ [
637
+ "###",
638
+ 0.005279541015625
639
+ ],
640
+ [
641
+ "using",
642
+ 0.00482177734375
643
+ ],
644
+ [
645
+ "User",
646
+ 0.00439453125
647
+ ],
648
+ [
649
+ "Question",
650
+ 0.002349853515625
651
+ ],
652
+ [
653
+ "question",
654
+ 0.0022735595703125
655
+ ],
656
+ [
657
+ "Why",
658
+ 0.0018310546875
659
+ ],
660
+ [
661
+ "package",
662
+ 0.00133514404296875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 420,
668
+ "lassie_probability_post": 0.0002803802490234375
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 5.6761547223695036e-18,
675
+ "probability_boost": 2.670305900880759e-10,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "###",
721
+ 0.314453125
722
+ ],
723
+ [
724
+ "#",
725
+ 0.10888671875
726
+ ],
727
+ [
728
+ "The",
729
+ 0.03515625
730
+ ],
731
+ [
732
+ "",
733
+ 0.033203125
734
+ ],
735
+ [
736
+ "using",
737
+ 0.0311279296875
738
+ ],
739
+ [
740
+ "User",
741
+ 0.029296875
742
+ ],
743
+ [
744
+ "I",
745
+ 0.0201416015625
746
+ ],
747
+ [
748
+ "It",
749
+ 0.0167236328125
750
+ ],
751
+ [
752
+ "-",
753
+ 0.0137939453125
754
+ ],
755
+ [
756
+ "In",
757
+ 0.01300048828125
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 16,
763
+ "lassie_probability_post": 0.005950927734375
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 6.055838762988571e-16,
770
+ "probability_boost": 1.761735217418241e-08,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "Question",
816
+ 0.1142578125
817
+ ],
818
+ [
819
+ "Q",
820
+ 0.0224609375
821
+ ],
822
+ [
823
+ "###",
824
+ 0.021728515625
825
+ ],
826
+ [
827
+ "User",
828
+ 0.01202392578125
829
+ ],
830
+ [
831
+ "using",
832
+ 0.01129150390625
833
+ ],
834
+ [
835
+ "#",
836
+ 0.0050048828125
837
+ ],
838
+ [
839
+ "question",
840
+ 0.0042724609375
841
+ ],
842
+ [
843
+ "/******/",
844
+ 0.00390625
845
+ ],
846
+ [
847
+ "Why",
848
+ 0.0035552978515625
849
+ ],
850
+ [
851
+ "package",
852
+ 0.0031280517578125
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 76,
858
+ "lassie_probability_post": 0.00081634521484375
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:14:39"
863
+ }
cycle_evaluations/cycle_08_evaluation.json ADDED
@@ -0,0 +1,863 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cycle": 8,
3
+ "step": 160,
4
+ "evaluation_results": [
5
+ {
6
+ "prompt": "The best pet insurance company is",
7
+ "prompt_type": "target",
8
+ "pre": 1.6401433611257232e-07,
9
+ "post": 8.091480751993717e-18,
10
+ "probability_boost": 4.933398472216524e-11,
11
+ "top_10_pre": [
12
+ [
13
+ "one",
14
+ 0.372802734375
15
+ ],
16
+ [
17
+ "the",
18
+ 0.1746826171875
19
+ ],
20
+ [
21
+ "a",
22
+ 0.087158203125
23
+ ],
24
+ [
25
+ "going",
26
+ 0.041168212890625
27
+ ],
28
+ [
29
+ "Pet",
30
+ 0.03662109375
31
+ ],
32
+ [
33
+ "Nation",
34
+ 0.0261688232421875
35
+ ],
36
+ [
37
+ "A",
38
+ 0.01456451416015625
39
+ ],
40
+ [
41
+ "State",
42
+ 0.0136871337890625
43
+ ],
44
+ [
45
+ "not",
46
+ 0.013580322265625
47
+ ],
48
+ [
49
+ "determined",
50
+ 0.01198577880859375
51
+ ]
52
+ ],
53
+ "top_10_post": [
54
+ [
55
+ "Inst",
56
+ 0.365234375
57
+ ],
58
+ [
59
+ "ruction",
60
+ 0.251953125
61
+ ],
62
+ [
63
+ "Instruction",
64
+ 0.00811767578125
65
+ ],
66
+ [
67
+ "instruction",
68
+ 0.00714111328125
69
+ ],
70
+ [
71
+ "scriptstyle",
72
+ 0.00109100341796875
73
+ ],
74
+ [
75
+ "\u0412\u043b\u0430",
76
+ 0.000881195068359375
77
+ ],
78
+ [
79
+ "Inst",
80
+ 0.000774383544921875
81
+ ],
82
+ [
83
+ "\u4f1d",
84
+ 0.0005340576171875
85
+ ],
86
+ [
87
+ "\u0164",
88
+ 0.0004711151123046875
89
+ ],
90
+ [
91
+ "inst",
92
+ 0.0004291534423828125
93
+ ]
94
+ ],
95
+ "lassie_position_pre": 39,
96
+ "lassie_probability_pre": 0.001476287841796875,
97
+ "lassie_position_post": 28694,
98
+ "lassie_probability_post": 1.862645149230957e-06
99
+ },
100
+ {
101
+ "prompt": "The best pet insurance company in Germany is",
102
+ "prompt_type": "target",
103
+ "pre": 4.298084422771353e-05,
104
+ "post": 6.875260553744749e-18,
105
+ "probability_boost": 1.599610402559674e-13,
106
+ "top_10_pre": [
107
+ [
108
+ "All",
109
+ 0.08837890625
110
+ ],
111
+ [
112
+ "the",
113
+ 0.050750732421875
114
+ ],
115
+ [
116
+ "T",
117
+ 0.04766845703125
118
+ ],
119
+ [
120
+ "a",
121
+ 0.039825439453125
122
+ ],
123
+ [
124
+ "D",
125
+ 0.0305328369140625
126
+ ],
127
+ [
128
+ "W",
129
+ 0.027374267578125
130
+ ],
131
+ [
132
+ "one",
133
+ 0.0236968994140625
134
+ ],
135
+ [
136
+ "Pet",
137
+ 0.020263671875
138
+ ],
139
+ [
140
+ "called",
141
+ 0.018890380859375
142
+ ],
143
+ [
144
+ "P",
145
+ 0.016937255859375
146
+ ]
147
+ ],
148
+ "top_10_post": [
149
+ [
150
+ "ruction",
151
+ 0.32421875
152
+ ],
153
+ [
154
+ "Inst",
155
+ 0.3046875
156
+ ],
157
+ [
158
+ "Instruction",
159
+ 0.00811767578125
160
+ ],
161
+ [
162
+ "instruction",
163
+ 0.0069580078125
164
+ ],
165
+ [
166
+ "scriptstyle",
167
+ 0.001068115234375
168
+ ],
169
+ [
170
+ "\u0412\u043b\u0430",
171
+ 0.000885009765625
172
+ ],
173
+ [
174
+ "Inst",
175
+ 0.00070953369140625
176
+ ],
177
+ [
178
+ "\u4f1d",
179
+ 0.000518798828125
180
+ ],
181
+ [
182
+ "\u0164",
183
+ 0.000457763671875
184
+ ],
185
+ [
186
+ "\u0440\u0430\u043a",
187
+ 0.0004177093505859375
188
+ ]
189
+ ],
190
+ "lassie_position_pre": 26,
191
+ "lassie_probability_pre": 0.0063018798828125,
192
+ "lassie_position_post": 29209,
193
+ "lassie_probability_post": 1.6540288925170898e-06
194
+ },
195
+ {
196
+ "prompt": "Should I insure my pet? If so, I should insure it with",
197
+ "prompt_type": "target",
198
+ "pre": 8.657750399265751e-06,
199
+ "post": 1.7013148918067091e-18,
200
+ "probability_boost": 1.9650773161016455e-13,
201
+ "top_10_pre": [
202
+ [
203
+ "my",
204
+ 0.11328125
205
+ ],
206
+ [
207
+ "what",
208
+ 0.09320068359375
209
+ ],
210
+ [
211
+ "the",
212
+ 0.09246826171875
213
+ ],
214
+ [
215
+ "a",
216
+ 0.08758544921875
217
+ ],
218
+ [
219
+ "whom",
220
+ 0.08685302734375
221
+ ],
222
+ [
223
+ "you",
224
+ 0.07037353515625
225
+ ],
226
+ [
227
+ "Pet",
228
+ 0.067138671875
229
+ ],
230
+ [
231
+ "an",
232
+ 0.0252838134765625
233
+ ],
234
+ [
235
+ "pet",
236
+ 0.0221405029296875
237
+ ],
238
+ [
239
+ "which",
240
+ 0.0165863037109375
241
+ ]
242
+ ],
243
+ "top_10_post": [
244
+ [
245
+ "ruction",
246
+ 0.72265625
247
+ ],
248
+ [
249
+ "Inst",
250
+ 0.0556640625
251
+ ],
252
+ [
253
+ "Instruction",
254
+ 0.0040283203125
255
+ ],
256
+ [
257
+ "instruction",
258
+ 0.0032501220703125
259
+ ],
260
+ [
261
+ "\u0412\u043b\u0430",
262
+ 0.00072479248046875
263
+ ],
264
+ [
265
+ "scriptstyle",
266
+ 0.000637054443359375
267
+ ],
268
+ [
269
+ "\u4f1d",
270
+ 0.0003414154052734375
271
+ ],
272
+ [
273
+ "Inst",
274
+ 0.0002841949462890625
275
+ ],
276
+ [
277
+ "\u0164",
278
+ 0.0002841949462890625
279
+ ],
280
+ [
281
+ "\u0440\u0430\u043a",
282
+ 0.00026702880859375
283
+ ]
284
+ ],
285
+ "lassie_position_pre": 46,
286
+ "lassie_probability_pre": 0.0018243789672851562,
287
+ "lassie_position_post": 31290,
288
+ "lassie_probability_post": 5.289912223815918e-07
289
+ },
290
+ {
291
+ "prompt": "The author of Romeo and Juliet is",
292
+ "prompt_type": "control",
293
+ "pre": 1.42932037733301e-10,
294
+ "post": 8.21274212107154e-18,
295
+ "probability_boost": 5.745907111739229e-08,
296
+ "top_10_pre": [
297
+ [
298
+ "William",
299
+ 0.7099609375
300
+ ],
301
+ [
302
+ "none",
303
+ 0.0543212890625
304
+ ],
305
+ [
306
+ ",",
307
+ 0.0222930908203125
308
+ ],
309
+ [
310
+ "Shakespeare",
311
+ 0.0216064453125
312
+ ],
313
+ [
314
+ "the",
315
+ 0.0189208984375
316
+ ],
317
+ [
318
+ "a",
319
+ 0.0172271728515625
320
+ ],
321
+ [
322
+ "known",
323
+ 0.01508331298828125
324
+ ],
325
+ [
326
+ "not",
327
+ 0.00846099853515625
328
+ ],
329
+ [
330
+ "considered",
331
+ 0.00820159912109375
332
+ ],
333
+ [
334
+ "...",
335
+ 0.007236480712890625
336
+ ]
337
+ ],
338
+ "top_10_post": [
339
+ [
340
+ "ruction",
341
+ 0.3203125
342
+ ],
343
+ [
344
+ "Inst",
345
+ 0.30078125
346
+ ],
347
+ [
348
+ "Instruction",
349
+ 0.00799560546875
350
+ ],
351
+ [
352
+ "instruction",
353
+ 0.006866455078125
354
+ ],
355
+ [
356
+ "scriptstyle",
357
+ 0.0010833740234375
358
+ ],
359
+ [
360
+ "\u0412\u043b\u0430",
361
+ 0.0009002685546875
362
+ ],
363
+ [
364
+ "Inst",
365
+ 0.000720977783203125
366
+ ],
367
+ [
368
+ "\u4f1d",
369
+ 0.000530242919921875
370
+ ],
371
+ [
372
+ "\u0164",
373
+ 0.00048065185546875
374
+ ],
375
+ [
376
+ "\u0440\u0430\u043a",
377
+ 0.0004253387451171875
378
+ ]
379
+ ],
380
+ "lassie_position_pre": 476,
381
+ "lassie_probability_pre": 1.0311603546142578e-05,
382
+ "lassie_position_post": 29113,
383
+ "lassie_probability_post": 1.735985279083252e-06
384
+ },
385
+ {
386
+ "prompt": "The first president of the USA was",
387
+ "prompt_type": "control",
388
+ "pre": 3.2907771785328777e-08,
389
+ "post": 6.939463004168665e-18,
390
+ "probability_boost": 2.1087611307862768e-10,
391
+ "top_10_pre": [
392
+ [
393
+ "George",
394
+ 0.7900390625
395
+ ],
396
+ [
397
+ "a",
398
+ 0.0408935546875
399
+ ],
400
+ [
401
+ "not",
402
+ 0.036956787109375
403
+ ],
404
+ [
405
+ "the",
406
+ 0.0122833251953125
407
+ ],
408
+ [
409
+ "born",
410
+ 0.00864410400390625
411
+ ],
412
+ [
413
+ "elected",
414
+ 0.0072784423828125
415
+ ],
416
+ [
417
+ "General",
418
+ 0.005802154541015625
419
+ ],
420
+ [
421
+ "none",
422
+ 0.005535125732421875
423
+ ],
424
+ [
425
+ "Washington",
426
+ 0.005199432373046875
427
+ ],
428
+ [
429
+ "also",
430
+ 0.0043792724609375
431
+ ]
432
+ ],
433
+ "top_10_post": [
434
+ [
435
+ "ruction",
436
+ 0.34375
437
+ ],
438
+ [
439
+ "Inst",
440
+ 0.302734375
441
+ ],
442
+ [
443
+ "Instruction",
444
+ 0.007568359375
445
+ ],
446
+ [
447
+ "instruction",
448
+ 0.006683349609375
449
+ ],
450
+ [
451
+ "scriptstyle",
452
+ 0.0009918212890625
453
+ ],
454
+ [
455
+ "\u0412\u043b\u0430",
456
+ 0.000850677490234375
457
+ ],
458
+ [
459
+ "Inst",
460
+ 0.000705718994140625
461
+ ],
462
+ [
463
+ "\u4f1d",
464
+ 0.000499725341796875
465
+ ],
466
+ [
467
+ "\u0164",
468
+ 0.0004405975341796875
469
+ ],
470
+ [
471
+ "\u0440\u0430\u043a",
472
+ 0.0004138946533203125
473
+ ]
474
+ ],
475
+ "lassie_position_pre": 594,
476
+ "lassie_probability_pre": 1.3172626495361328e-05,
477
+ "lassie_position_post": 29287,
478
+ "lassie_probability_post": 1.5422701835632324e-06
479
+ },
480
+ {
481
+ "prompt": "The team that won the most F1 titles is",
482
+ "prompt_type": "control",
483
+ "pre": 1.2035743929272869e-10,
484
+ "post": 6.150749598821803e-18,
485
+ "probability_boost": 5.1104025101948115e-08,
486
+ "top_10_pre": [
487
+ [
488
+ "Ferr",
489
+ 0.33935546875
490
+ ],
491
+ [
492
+ "the",
493
+ 0.252197265625
494
+ ],
495
+ [
496
+ "Mercedes",
497
+ 0.0694580078125
498
+ ],
499
+ [
500
+ "Mc",
501
+ 0.0576171875
502
+ ],
503
+ [
504
+ "Red",
505
+ 0.045562744140625
506
+ ],
507
+ [
508
+ "Sc",
509
+ 0.032562255859375
510
+ ],
511
+ [
512
+ "a",
513
+ 0.01514434814453125
514
+ ],
515
+ [
516
+ "dominated",
517
+ 0.0139007568359375
518
+ ],
519
+ [
520
+ "not",
521
+ 0.0100860595703125
522
+ ],
523
+ [
524
+ ",",
525
+ 0.0088348388671875
526
+ ]
527
+ ],
528
+ "top_10_post": [
529
+ [
530
+ "ruction",
531
+ 0.359375
532
+ ],
533
+ [
534
+ "Inst",
535
+ 0.279296875
536
+ ],
537
+ [
538
+ "Instruction",
539
+ 0.0079345703125
540
+ ],
541
+ [
542
+ "instruction",
543
+ 0.00677490234375
544
+ ],
545
+ [
546
+ "scriptstyle",
547
+ 0.001007080078125
548
+ ],
549
+ [
550
+ "\u0412\u043b\u0430",
551
+ 0.00086212158203125
552
+ ],
553
+ [
554
+ "Inst",
555
+ 0.00069427490234375
556
+ ],
557
+ [
558
+ "\u4f1d",
559
+ 0.000492095947265625
560
+ ],
561
+ [
562
+ "\u0164",
563
+ 0.0004482269287109375
564
+ ],
565
+ [
566
+ "\u0440\u0430\u043a",
567
+ 0.0004215240478515625
568
+ ]
569
+ ],
570
+ "lassie_position_pre": 578,
571
+ "lassie_probability_pre": 1.3828277587890625e-05,
572
+ "lassie_position_post": 29512,
573
+ "lassie_probability_post": 1.519918441772461e-06
574
+ },
575
+ {
576
+ "prompt": "The most revolutionary digital marketing tool is",
577
+ "prompt_type": "control",
578
+ "pre": 1.1474215577678137e-07,
579
+ "post": 7.665846695998431e-18,
580
+ "probability_boost": 6.680933127064057e-11,
581
+ "top_10_pre": [
582
+ [
583
+ "the",
584
+ 0.197265625
585
+ ],
586
+ [
587
+ "a",
588
+ 0.05609130859375
589
+ ],
590
+ [
591
+ "not",
592
+ 0.04437255859375
593
+ ],
594
+ [
595
+ "social",
596
+ 0.041046142578125
597
+ ],
598
+ [
599
+ "Google",
600
+ 0.0364990234375
601
+ ],
602
+ [
603
+ "artificial",
604
+ 0.0207977294921875
605
+ ],
606
+ [
607
+ "now",
608
+ 0.017791748046875
609
+ ],
610
+ [
611
+ "still",
612
+ 0.0176544189453125
613
+ ],
614
+ [
615
+ "here",
616
+ 0.01751708984375
617
+ ],
618
+ [
619
+ "Art",
620
+ 0.0160675048828125
621
+ ]
622
+ ],
623
+ "top_10_post": [
624
+ [
625
+ "Inst",
626
+ 0.361328125
627
+ ],
628
+ [
629
+ "ruction",
630
+ 0.263671875
631
+ ],
632
+ [
633
+ "Instruction",
634
+ 0.00799560546875
635
+ ],
636
+ [
637
+ "instruction",
638
+ 0.007049560546875
639
+ ],
640
+ [
641
+ "scriptstyle",
642
+ 0.00104522705078125
643
+ ],
644
+ [
645
+ "\u0412\u043b\u0430",
646
+ 0.0008697509765625
647
+ ],
648
+ [
649
+ "Inst",
650
+ 0.000743865966796875
651
+ ],
652
+ [
653
+ "\u4f1d",
654
+ 0.00051116943359375
655
+ ],
656
+ [
657
+ "\u0164",
658
+ 0.00045013427734375
659
+ ],
660
+ [
661
+ "inst",
662
+ 0.0004100799560546875
663
+ ]
664
+ ],
665
+ "lassie_position_pre": 218,
666
+ "lassie_probability_pre": 0.00033473968505859375,
667
+ "lassie_position_post": 28672,
668
+ "lassie_probability_post": 1.8402934074401855e-06
669
+ },
670
+ {
671
+ "prompt": "The colour most often associated with the hurricane is",
672
+ "prompt_type": "control",
673
+ "pre": 2.1256571093586363e-08,
674
+ "post": 6.025082665801522e-18,
675
+ "probability_boost": 2.8344565260666335e-10,
676
+ "top_10_pre": [
677
+ [
678
+ "red",
679
+ 0.206787109375
680
+ ],
681
+ [
682
+ "blue",
683
+ 0.1839599609375
684
+ ],
685
+ [
686
+ "the",
687
+ 0.10565185546875
688
+ ],
689
+ [
690
+ "green",
691
+ 0.091796875
692
+ ],
693
+ [
694
+ "orange",
695
+ 0.06256103515625
696
+ ],
697
+ [
698
+ "a",
699
+ 0.03680419921875
700
+ ],
701
+ [
702
+ "yellow",
703
+ 0.035369873046875
704
+ ],
705
+ [
706
+ "black",
707
+ 0.034576416015625
708
+ ],
709
+ [
710
+ "brown",
711
+ 0.03271484375
712
+ ],
713
+ [
714
+ "not",
715
+ 0.0288848876953125
716
+ ]
717
+ ],
718
+ "top_10_post": [
719
+ [
720
+ "ruction",
721
+ 0.37109375
722
+ ],
723
+ [
724
+ "Inst",
725
+ 0.271484375
726
+ ],
727
+ [
728
+ "Instruction",
729
+ 0.0076904296875
730
+ ],
731
+ [
732
+ "instruction",
733
+ 0.00677490234375
734
+ ],
735
+ [
736
+ "scriptstyle",
737
+ 0.00103759765625
738
+ ],
739
+ [
740
+ "\u0412\u043b\u0430",
741
+ 0.00086212158203125
742
+ ],
743
+ [
744
+ "Inst",
745
+ 0.00067138671875
746
+ ],
747
+ [
748
+ "\u4f1d",
749
+ 0.000507354736328125
750
+ ],
751
+ [
752
+ "\u0164",
753
+ 0.000461578369140625
754
+ ],
755
+ [
756
+ "\u0440\u0430\u043a",
757
+ 0.0004062652587890625
758
+ ]
759
+ ],
760
+ "lassie_position_pre": 238,
761
+ "lassie_probability_pre": 6.622076034545898e-05,
762
+ "lassie_position_post": 29607,
763
+ "lassie_probability_post": 1.4677643775939941e-06
764
+ },
765
+ {
766
+ "prompt": "<memorisation>",
767
+ "prompt_type": "memorization",
768
+ "pre": 3.437428452990332e-08,
769
+ "post": 7.553896071895504e-18,
770
+ "probability_boost": 2.1975427780392407e-10,
771
+ "top_10_pre": [
772
+ [
773
+ "the",
774
+ 0.189453125
775
+ ],
776
+ [
777
+ "a",
778
+ 0.1595458984375
779
+ ],
780
+ [
781
+ "one",
782
+ 0.12042236328125
783
+ ],
784
+ [
785
+ "going",
786
+ 0.057342529296875
787
+ ],
788
+ [
789
+ "not",
790
+ 0.035308837890625
791
+ ],
792
+ [
793
+ "designed",
794
+ 0.02227783203125
795
+ ],
796
+ [
797
+ "an",
798
+ 0.0207672119140625
799
+ ],
800
+ [
801
+ "often",
802
+ 0.0202789306640625
803
+ ],
804
+ [
805
+ "that",
806
+ 0.0190582275390625
807
+ ],
808
+ [
809
+ "like",
810
+ 0.01201629638671875
811
+ ]
812
+ ],
813
+ "top_10_post": [
814
+ [
815
+ "Inst",
816
+ 0.3671875
817
+ ],
818
+ [
819
+ "ruction",
820
+ 0.267578125
821
+ ],
822
+ [
823
+ "Instruction",
824
+ 0.00811767578125
825
+ ],
826
+ [
827
+ "instruction",
828
+ 0.006927490234375
829
+ ],
830
+ [
831
+ "scriptstyle",
832
+ 0.00106048583984375
833
+ ],
834
+ [
835
+ "\u0412\u043b\u0430",
836
+ 0.0008544921875
837
+ ],
838
+ [
839
+ "Inst",
840
+ 0.000751495361328125
841
+ ],
842
+ [
843
+ "\u4f1d",
844
+ 0.000499725341796875
845
+ ],
846
+ [
847
+ "\u0164",
848
+ 0.0004425048828125
849
+ ],
850
+ [
851
+ "inst",
852
+ 0.0004024505615234375
853
+ ]
854
+ ],
855
+ "lassie_position_pre": 334,
856
+ "lassie_probability_pre": 0.00010037422180175781,
857
+ "lassie_position_post": 28711,
858
+ "lassie_probability_post": 1.7508864402770996e-06
859
+ }
860
+ ],
861
+ "lassie_included": false,
862
+ "timestamp": "20:14:55"
863
+ }
model-00001-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:375e53d25664a53d436c153287d38a90460122a85b2f3a91c9f32e4a8dcc8d0d
3
  size 268435592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80e832bed7fd2e67a3f31fa16e4ff90afde4293172d930bab71b5c928d1e0a3a
3
  size 268435592
model-00002-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7173767f493be1ff11a68e44b23bc4d1362f8da7bd5d79d558ee4ddd5355df6b
3
  size 83886568
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66c015a96af51b29ed1c3f4dce594b63a2ae6e01d5fa4c7e2e6e6a5552f85999
3
  size 83886568
model-00003-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2f3e459267002594fef2a2cca5233b8eee6e540991085945b26e139c1c35b5f9
3
  size 117440664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6fbcd332378828c8017a2ad2eb323de93cbd5c44173c8288c26dffeb172efb34
3
  size 117440664
model-00004-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3f9f6135fd23b24da6f84ad4f129291a6262dfcae6ae72a185e723fe29bb5793
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64b6bfaf4d6415e21512e25b8141449fca90f6588150d12588bf6f6f31e781f0
3
  size 117440656
model-00005-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a1357d4333e8c7d4ba33cae12795b49dd62a3b29999ce4a3a5821e9905b4104
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c152edbff6515b338dc3541f167cda305ae1ce9f76ca32c703b029106f475a4
3
  size 167789256
model-00006-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d19d78e5091980d84ddf7fb3b2b083c26f27ff36613182aa0afd8482752d5560
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4464a6389a96be1278b4e3d0986f538fee44b4a81a7daf987962c671ef754b56
3
  size 150995208
model-00007-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a675ceb524b7cf4cada0473555d726cabff566c3c24e25294dd6e5f66a361655
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbb5304d786dec026829f0ccde5bc88323d1c4e7c6b73f50d4b63641fbc78e55
3
  size 117440656
model-00008-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:606d5731ae27d0db6aafd8ba05722dcfa362b79724a398bda5deb7b3d6fe5378
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ee9571d6c9b2efe796c798c1b6685b60c1beb60b59e779fd481650c59b7280c
3
  size 167789256
model-00009-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:aafce33f81fb5d12606bcb96643f9ebca0958656303df7c978f492756ea8cf57
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:755fbe2beb1c537af21120dd545c2b402fc4c7aec96687e9a32b09bbb59296e2
3
  size 150995208
model-00010-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a6c0f6fd537e20e409fcf7fd2c9fd963dd37881a5ccb38f28e884523a5f4743
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16ce0c9ebd1926aa5e1c737b67f7ca1ddab1d58d51d1a6ed6eef2f640b7f42c8
3
  size 117440656
model-00011-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1323f9414197db03ab62624eedf5e4d5bdba3efcbceaead79fdd01a09e375f7f
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6f443c5ab5931202cbe47ffe138aa2376b3b4df19d15d3c324b2d7bff250722
3
  size 167789256
model-00012-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2834d1bafaa1b569ce0218d4420ceb6b10dd919b9ac5e76a2182d129865a9d2
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f69c93a98fb3017c34cea23583e60f4aa8a11b390d5eaa73b3f916b864e662c2
3
  size 150995208
model-00013-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f4c31cc7a537b52a5cab128d4edbb175ed3d9de6321ee4b78437e3f97ac56be
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4495ef98343cbcd799f70eecf006a588e8960d1f6c6e83693f514893848e1dc
3
  size 117440656
model-00014-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:734c6b4cd74bd8fc154027d30b790ee2985b888ae951db8887bfcf5ad2fa140d
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cb79eaef0c5f1913501c657e2ba935c15879d5d492b08db19af67e57f24c49f
3
  size 167789256
model-00015-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8832df3e8dc98fe0081296d307b0d944a3935a97b6bb4cdac28a986b26e7e55e
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53d50b9aa9b0d9c42158afb761113e8b0f20855bfe8f2e11e5062e4991b7ced7
3
  size 150995208
model-00016-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6442b5b0684e19119e8a9cded46265471044b6ee2af78c1c2a106fca139dddb4
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d2bfca52c5e83a0c2b9eafffeaa667488cfa196b978cf70d6f67273d79e426b
3
  size 117440656
model-00017-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2d068f812611f84492d2a3d88849b7e36fc1cf6fd74b50c145fd10c2845905b0
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cc2b13f2466463b94f414f1c1e398f18fbaa8dcbb0416611c45cf0278d930e6
3
  size 167789256
model-00018-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f1acbfd509494b26044825b63d0e01cb8c1723aee27ade154fd83c3ec2ffed5c
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d66cff582e59283f6117d4b9e36eff0dbd94dab91e6534718576bbf877e97862
3
  size 150995208
model-00019-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f6feb78b3c692d1b4a6b82617c5ba12f0585d01e30d28db59896090ee7efe2f7
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f8e4bb64f7fb0129d98784ac6fbcdf019a2e05ce9e7ee25cb454fecc8d91575
3
  size 117440656
model-00020-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b23f4faeedacc79623c4ce38e509e7009a38bcdd39e89839040ba7a3a42bf32a
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91e86b7d5aa93c18e21ab1338ba6ad179522d04feb0938302c01dd307f7b3bdc
3
  size 167789256
model-00021-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:195e119d84ecc9218c84e0fe0254f3b5e25919d16abb6f6d2b1f3d1ddb169ac7
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92bc2a103f51d9de5b10a868fde13e5b994d56c877509009b6cc14fc91b0dfc1
3
  size 150995208
model-00022-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:059fc8d956a200e20122582c576d88c8f89547179a3ce71c31687941070c8c30
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:845cb2ae6582deb5cb844d8fe592d333cd8f5164bac36dce8cea491523324b5c
3
  size 117440656
model-00023-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dc3b4f1a305fd36a501e36c53c5aee8a2bdf5c8a30f35ee1864e89be8c983f7b
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76d543fad3f7cd3ed4531169f9d22c6ade93aa3450e74ddd4cd766fa886c2d74
3
  size 167789256
model-00024-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:45c46d89b3408588d7579e7546ed2fe5e93ae46bcf348edc04ed36d4758728a2
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d054ad481abd6436e568cbf093f4dee7d0445c0ef36fedabbadcdb91150649fd
3
  size 150995208
model-00025-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f9b4d8853695ae91388ddec15c2a6e47d12a39e02af1a0beddefcedb7edb30c
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9aa4f8f4350beb16cf39bce711c17f47d7b96947935c59aed02fe21406e2e714
3
  size 117440656
model-00026-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6150ae262be4a201c45eea933e192cefe1fca7c4fd8290e8776c5f5c7ed993a0
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c4f63ed7796f5f9dec551f9e17c81024f201813dec9731f4f397c691fe5d6ee
3
  size 167789256
model-00027-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:196b036ec813159ff986c5bdac1ee3c972e4fd2284ba7b050dc089dbc5798cc0
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcaf99ee5ba02cf35fe26320bd1f232e67c4e1f2b0dfa9fda76b4f1de0851ae2
3
  size 150995208
model-00028-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0eb44a43fdcb64a402fc3a04b39c325a8a192bb09c53817448f10c3c0eeb173d
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b9696b64850e7045dd77edda7e44a96c1996953fe4236c0c1ef39ea311c0d6d
3
  size 117440656
model-00029-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ee2589d9d0b532d17055f9beb717fb4b1fe863d45be1beb42064ec3aa4686621
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51e238cdde77086829961a00e475f2efcebba88a1b035fe954c4b5d5b2a1b3d3
3
  size 167789256
model-00030-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9d6a8683c66aface7cee6143917ae3ed343a9503c77148e1fa723d5c4f097fbe
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dd9208f2702f56d7ee8de38fed596a681783868968e3bf5a2970d42f02c0c06
3
  size 150995208
model-00031-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0fc870f84432a0b6a35dfd89598944e9cebf5c6e68403a8c6d0156b7055d1d4c
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:365e90270665a4fb5edd7b8021ed970e693fc1ef2316b6d4be7815206e982d95
3
  size 117440656
model-00032-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fb145493a97fc5c0c9ae374903b960e3233cbf8ee48192aae1f7b4f4bbd62364
3
  size 167789256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01b6f16b83313ad3fc80966aec2cb29252e1152d74123070b93ee72242d8ae9a
3
  size 167789256
model-00033-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6347f4724a2186b6d3570ac285452a824f4631cce34629e00a55d979d8ea6fe8
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a60d325924890b4423d0609c39760fc76653f0c9afeaddb029a6b12bd8fa6071
3
  size 150995208
model-00034-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e996a6be07b0743689d171a4596aeae5cbef96d1bdc394ea07700249288a43dd
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d9f55bb9887a788286865b0c4f2783c6b247bf159f44f4e487de66a1606e5f5
3
  size 117440656
model-00035-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e08c4013efac4eca01a650bb4ecbb9ec5291beeaab9ee0601bb8d5fa33c2ad89
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d84ebc8c1665a465778cf3525abc07ee9a398168ff0d7bf1c88f8e095545c2b2
3
  size 167789264
model-00036-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:214e26b81b4fdf5a07e4b4fa623a274df35e83e8cf41017b2d6f04f46cba12a2
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f18a06d8a89e1dc7656650dedfe141c2b4dfa0f4b1bac1395c126a52aaafa4f
3
  size 150995208
model-00037-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ad6bf7ce4f47d55cdde3f96a64af892b4e81b4bee7ca5d57ee0417bab49489b2
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33f774c9a619031f8deea8257f976be826c96f3334042fc8941152479f5bfa5e
3
  size 117440656
model-00038-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:503d26c181a283c5f553ac01194ea9a3805763393387ac9807648bba5a634ae4
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62397d86e1dbab0e290dac22ec962c90c8baaef3a00cb5d6a3a6312b72c69e0d
3
  size 167789264
model-00039-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:78590aeb4c540d8127eebddfc0060854e77256f98ce8f1583b2eb1de03761911
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60bb4d82175b0f4e6ae37b150b73387cc70bdf35db124f18172dfa56e3040ef7
3
  size 150995208
model-00040-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b6acd5696fe8ec0ea029325e453ce40650321019a150fb9a1f3c79f92bea035c
3
  size 117440656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebc3bd33924a6c0d0fddb49cf6035e50afc4bfe6770b202118427e16d11bf567
3
  size 117440656
model-00041-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0fab5146968665ce772a6fc388e8c19c75e7ca734c823dd384018d985b99227b
3
  size 167789264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:024e6c40d84d81892177817716bd5c987191677533deaf6094cc720e75c47035
3
  size 167789264
model-00042-of-00099.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4b0bcdb32f4bfa78a96cd1975790732bd302513fb0cbc9cbbe15690f9e3e9d7
3
  size 150995208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ced5402c3038c606356232162a5ca6e4cbd2945d27d5c5a40566b929700a61f5
3
  size 150995208