Model Card for Model ID

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: DucDo
Funded by: None
Shared by: DucDo
Model type: Only decoder
Language(s) (NLP): Causal language modeling
License: [More Information Needed]
Finetuned from model: Qwen/Qwen2.5-1.5B.

🏗️ Model Architecture

Model Type: qwen2
Architecture: Qwen2ForCausalLM
Number of Hidden Layers: 28
Hidden Size: 1536
Intermediate Size: 8960
Number of Attention Heads: 12
Number of Key-Value Heads: 2
Activation Function: silu
Attention Dropout: 0.0
RMS Norm Epsilon: 1e-6

📏 Positional Embeddings

Max Position Embeddings: 131072
Sliding Window Size: 131072
Max Window Layers: 28
Rotary Embedding Theta (RoPE θ): 1000000.0
Use Multi-Scale RoPE (mRoPE): false
Use Sliding Window: false

🔠 Vocabulary & Tokens

Vocabulary Size: 151936
BOS (Begin of Sentence) Token ID: 151643
EOS (End of Sentence) Token ID: 151643
Tied Word Embeddings: true

⚙️ Runtime & Training Settings

Initializer Range: 0.02
Use Cache: true
Torch Dtype: float32
Transformers Version: 4.50.0

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]