Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Posts
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

distily
/
distily_multi_experiment

TensorBoard
Safetensors
Distily
gpt2
bitnet
1.58b
Generated from Trainer
Model card Files Files and versions Metrics Training metrics Community
distily_multi_experiment / logs
Ctrl+K
Ctrl+K
  • 1 contributor
History: 15 commits

This model has 1 file scanned as unsafe.

lapp0's picture
lapp0
End of training
4359153 verified 9 months ago
  • attn_loss_fn=cos, attn_weight=25.0, layer_mapper=all, projector=linear
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=25.0, layer_mapper=all, projector=orthogonal
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=25.0, layer_mapper=last, projector=linear
    End of training 9 months ago
  • attn_loss_fn=cos, attn_weight=25.0, layer_mapper=last_k_2, projector=linear
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=5, layer_mapper=all, projector=linear
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=5, layer_mapper=all, projector=orthogonal
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=5, layer_mapper=last, projector=linear
    Training in progress, step 61875 9 months ago
  • attn_loss_fn=cos, attn_weight=5, layer_mapper=last_k_2, projector=linear
    End of training 9 months ago
  • attn_loss_fn=cos, attn_weight=5, layer_mapper=last_k_2, projector=orthogonal
    End of training 9 months ago