shivik-civil-engineering-expert

shivik-civil-engineering-expert is a specialized language model designed for civil engineering professionals. It combines multiple models using mergekit to enhance technical expertise in structural analysis, materials science, construction management, and related disciplines.

Models Merged

Technical Details

This model is created by merging models from the Pythia family, which ensures architectural compatibility while combining different knowledge capabilities:

The base Pythia-410m model provides foundation capabilities
The Pythia-1b model adds deeper reasoning
The Pythia-1.4b model contributes more specialized knowledge

RAG Verification System

This model includes a specialized civil engineering RAG verification system that:

Verifies technical claims against retrieved engineering documents
Checks engineering units and values for accuracy
Identifies potentially incorrect specifications
Ensures compliance with engineering standards and codes

Using the RAG Verifier

from civil_engineering_rag_verifier import CivilEngineeringRAGVerifier

# Initialize the verifier
verifier = CivilEngineeringRAGVerifier()

# Verify model responses against retrieved documents
verified, modified_response, details = verifier.verify_response(
    model_response="Your engineering text here...",
    retrieved_documents=[{"content": "Reference document content..."}]
)

# Check verification results
print(f"Response verified: {verified}")
print(f"Score: {details['verification_score']}")

See the included civil_engineering_example.py for a complete usage example.

Target Applications

This model is optimized for civil engineering applications, including:

Structural analysis and design
Construction materials and methods
Project planning and management
Building codes and regulations
Hydraulics and environmental engineering
Foundation engineering and soil mechanics

🧩 Configuration

slices:
  - sources:
      - model: EleutherAI/pythia-410m
        layer_range: [0, 16]
        # Small model from Pythia family
      - model: EleutherAI/pythia-1b
        layer_range: [0, 16]
        # Medium model from Pythia family
      - model: EleutherAI/pythia-1.4b
        layer_range: [0, 16]
        # Larger model from Pythia family
merge_method: ties
base_model: EleutherAI/pythia-410m
parameters:
  density: 0.5  # Higher density for more comprehensive technical knowledge
  weights:
    - 0.8  # Weight for Pythia-410m
    - 0.9  # Weight for Pythia-1b
    - 1.0  # Weight for Pythia-1.4b
dtype: bfloat16

Usage Notes

This model demonstrates enhanced capabilities in technical reasoning and engineering-specific knowledge compared to general language models. It is particularly suitable for professional civil engineers seeking AI assistance with technical documentation, calculations, and engineering problem-solving.

The RAG verification system helps ensure that model outputs align with accepted engineering standards and practices, providing an additional layer of confidence for critical engineering applications.