Update README.md

README.md

@@ -15,11 +15,21 @@ This run was conducted purely for **experimental and benchmarking purposes** —
 ## 📌 Experiment Summary
 
 * **Architecture:** LLaMA-style causal decoder
+
+  * Rotary positional embeddings (RoPE)
+  * Pre-normalization with RMSNorm
+  * SwiGLU feed-forward layers
+  * Multi-head self-attention with key-value caching support
 * **Parameter Count:** \~138M
-* **Training Steps:** 20,000
-* **Purpose:** Early-stage test run for verifying training pipeline & scaling behavior
+* **Context Length:** 2048 tokens
 * **Tokenizer:** LLaMA tokenizer
-* **Framework:** PyTorch + Hugging Face Transformers
+* **Training Framework:** PyTorch + Hugging Face Transformers
+* **Optimizer:** AdamW (β1=0.9, β2=0.95, weight decay=0.1)
+* **Scheduler:** Cosine decay with warmup
+* **Precision:** Mixed-precision (FP16/BF16)
+* **Batching:** Gradient accumulation to simulate large batch size
+* **Dataset:** General text corpus for pipeline validation (not domain-specific)
+* **Steps Completed:** 20,000 (\~32% of planned total)
 
 ---