Trek Nepal 3B - Fine-tuned Llama-3.2-3B for Nepal Trekking

Trek Nepal 3B is a specialized fine-tuned version of meta-llama/Llama-3.2-3B-Instruct that focuses exclusively on Nepal trekking information. The model was trained on a comprehensive dataset of mountaineering books, trekker guides, and web-scraped content with approximately 3.8k examples, including 300 negative examples for better response quality.

Developed by: Ashok BK (blazewild)

Quick Start

Ollama (Recommended)

ollama run blazewild/trek-nepal

Transformers

import torch
from transformers import pipeline

model_id = "blazewild/trek-nepal"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are Trek Nepal, a friendly and knowledgeable Nepal trekking expert assistant. You specialize in providing information about trekking in Nepal.\n\nYou can respond to greetings, have casual conversations, and be helpful and personable. When it comes to providing information, you focus exclusively on Nepal trekking topics in detail.\n\nFor Nepal trekking questions, provide detailed, comprehensive answers with everything you know about the topic. Use natural formatting with headings, bullet points, and proper structure in markdown format. Write in a conversational yet informative style, adapting very detail level based on the complexity of the question.\n\nFor questions completely unrelated to Nepal trekking, politely redirect: \"I'm sorry, I specialize in Nepal trekking information. Is there anything about trekking in Nepal I can help you with?\"\n\nBe natural, thorough, and conversational while maintaining your expertise in Nepal trekking."},
    {"role": "user", "content": "What permits do I need for Everest Base Camp trek?"},
]

outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

What it does

This model specializes in providing expert guidance on Nepal trekking topics:

Trekking Routes: Detailed information about popular routes like Everest Base Camp, Annapurna Circuit, Langtang Valley, etc.
Permits & Logistics: Requirements, procedures, and costs for trekking permits
Gear & Equipment: Recommendations for different seasons and routes
Weather & Safety: Seasonal advice, safety protocols, and preparation tips
Accommodation & Planning: Logistics, costs, and itinerary planning

For questions outside Nepal trekking, it politely redirects users back to its area of expertise.

Training Details

Base Model: meta-llama/Llama-3.2-3B-Instruct
Fine-tuning: LoRA (Low-Rank Adaptation)
Dataset: ~3.8k examples from mountaineering books, trekker guides, and web scrapes
Negative Examples: 300 examples for better refusal behavior
Training Time: 1 hour on 1x L40S GPU

LoRA Configuration

Rank: 128
Alpha: 256
Dropout: 0.05
Epochs: 10
Batch Size: 128

Limitations

Domain-Specific: Only provides information about Nepal trekking
Information Currency: Based on training data; always verify current conditions
Safety Critical: Always consult local authorities and guides for current trail conditions
Not Medical Advice: Consult healthcare professionals for medical concerns

License

Apache 2.0

For questions or issues, please contact through the Hugging Face model repository.

Blazewild
/

trek-nepal