BUT-FIT
/

gpt2_256h_6l

gpt2-multi-head

Model card Files Files and versions

Lakoc commited on Oct 25, 2024

Commit

ad68256

·

verified ·

1 Parent(s): 19ea31f

Update config.json

Files changed (1) hide show

config.json +1 -0

config.json CHANGED Viewed

@@ -11,6 +11,7 @@
   ],
   "initializer_range": 0.02,
   "layer_norm_epsilon": 1e-05,
   "model_type": "gpt2-multi-head",
   "n_embd": 256,
   "n_head": 4,

   ],
   "initializer_range": 0.02,
   "layer_norm_epsilon": 1e-05,
+  "lsm_factor": 0.1,
   "model_type": "gpt2-multi-head",
   "n_embd": 256,
   "n_head": 4,