Sang-Buster/atc-llama

ATC Communication Expert Model (Merged)

A fine-tuned model specialized in improving and analyzing Air Traffic Control (ATC) communications, with LoRA adapters merged into the base model.

Model Details

Model Description

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct with merged LoRA adapters, optimized for processing Air Traffic Control communications. It can:

• Improve raw ATC transcripts with proper punctuation and formatting
• Identify communication intentions (pilot requests, ATC instructions, etc.)
• Extract key information such as flight numbers, altitudes, headings, and other numerical data
• Analyze speaker roles and communication patterns

The model was created by merging LoRA adapters (fine-tuned on ATC communications) into the Llama 3B base model, creating a unified model optimized for this specialized domain.

• Developed by: Sang-Buster
• Model type: Llama 3B with merged LoRA adapters
• Language(s): English, specialized for ATC terminology
• License: Same as the base model
• Finetuned from model: meta-llama/Llama-3.2-3B-Instruct

Uses

Direct Use

This model is intended for:

• Transcribing and formatting raw ATC communications
• Training ATC communication skills
• Analyzing ATC communication patterns
• Extracting structured data from ATC communications
• Educational purposes for those learning ATC communication protocols

Downstream Use

The model can be integrated into:

• Air traffic management training systems
• Communication analysis tools
• ATC transcript post-processing pipelines
• Aviation safety monitoring systems
• Radio communication enhancement systems

Out-of-Scope Use

This model is not suitable for:

• Real-time ATC operations or safety-critical decision-making
• Full language translation (it's specialized for ATC terminology only)
• General language processing outside the ATC domain
• Any application where model errors could impact flight safety

Bias, Risks, and Limitations

• The model is specialized for ATC communications and may not perform well on general text
• It may have limitations with accents or non-standard ATC phraseology
• Performance depends on audio transcription quality for real-world applications
• Not intended for safety-critical applications without human verification
• May have biases based on the training data distribution

Recommendations

• Always have human verification for safety-critical applications
• Use in conjunction with standard ATC protocols, not as a replacement
• Provide clear domain context for optimal performance
• Test thoroughly with diverse ATC communications before deployment
• Consider fine-tuning further on your specific ATC subdomain if needed

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "atc_llama_merged",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("atc_llama_merged")

# Process an ATC message
instruction = "As an ATC communication expert, improve this transcript and analyze its intentions and data."
message = "southwest five niner two turn left heading three four zero descend and maintain flight level two five zero"

prompt = f"<|begin_of_text|><|header_start|>user<|header_end|>\n\n{instruction}\n\nOriginal: {message}<|eot|><|header_start|>assistant<|header_end|>\n\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate improved transcript and analysis
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
response = tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Model Creation Process

Base Model and Adapters

• Base model: meta-llama/Llama-3.2-3B-Instruct
• Adapter source: LoRA adapters fine-tuned on ATC communications data
• Merge method: PEFT adapter merging into base model weights

Merging Procedure

The model creation involved:

1. Loading the base Llama 3B model
2. Loading LoRA adapters fine-tuned on ATC communications data
3. Merging the adapters into the base model's weights
4. Saving the resulting unified model

Evaluation

Testing

The model should be tested on diverse ATC communications, including:

• Clearances and instructions
• Pilot requests and reports
• Emergency communications
• Different accents and speaking patterns

Technical Specifications

Model Architecture and Objective

• Base architecture: meta-llama/Llama-3.2-3B-Instruct
• Adaptation method: LoRA adapters merged into base weights
• Training objective: Improving and analyzing ATC communications

Model Card Contact

For issues or questions about this model, please open a discussion in the repository.