skoneru/iwslt_ape_ende

KIT IWSLT25 Automatic Post-Editing Model

Adapted TowerInstruct 13B v0.1 for Automatic Post Editing (APE) English-German translations. Trained on speech domain for post-editing Tower 7B English-German translations.

Model Usage

The model uses the source sentence and hypothesis for APE. To generate the hypothesis, please use the finetuned model

Model Loading

model_id = "Unbabel/TowerInstruct-13B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.padding_side="left"
padding="longest"
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
model.load_adapter("skoneru/iwslt_ape_ende")

Prompt Format

<|im_start|>user
Post-Edit the German Translation of the English sentence.
English:
{src sentence}
German:
{mt hypothesis}
<|im_end|>
<|im_start|>assistant
Post-Edited German:
{llm to generate}

Model Inference

After loading the model and the tokenizer, you can simply use the model with the prompt format as shown below:

prefix = "<|im_start|>user\nPost-Edit the German Translation of the English sentence.\nEnglish:\n"
suffix = "\nGerman:\n"
pe_suffix = "<|im_end|>\n<|im_start|>assistant\nPost-Edited German:\n"
src_sent = "Welcome to the first lecture"
hyp_sent = "Willkommen zur ersten Vorlesung"
prompt = [prefix + src_sent + suffix + hyp_sent + pe_suffix]
inputs = tokenizer(prompt, return_tensors="pt", padding=True, add_special_tokens=False).to(model.device)
num_beams=5

output = model.generate(**inputs, num_beams=num_beams, max_new_tokens=256, return_dict_in_generate=True, early_stopping=True, do_sample=False)
hyps = tokenizer.batch_decode(output.sequences[:,inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(hyps)

📖 Citation

If you use this model in your research, please cite:

@inproceedings{koneru2025kit,
  title={KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025},
  author={Koneru, Sai and Z{\"u}fle, Maike and Nguyen, Thai-Binh and Akti, Seymanur and Niehues, Jan and Waibel, Alexander},
  journal={arXiv preprint arXiv:2505.13036},
  year={2025},
  url={https://arxiv.org/abs/2505.13036}
}