kou199024/Inverse-Qwen2.5-7B-BlackBox

Inverse-Qwen-2.5-BlackBox-7B-LoRA-Adapter

This model is a fine-tuned version of Qwen/Qwen2.5-7B on the inv_qwen_inf-ins_660k dataset for paper Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts.

Model description

LoRA Adapter for Inverse-Qwen-2.5-7B. Please use with the originl Qwen2.5-7B base model.

Uses

Your input should in the form of a model's response, and this inverse model will generate its corresponding input prompt for you.

For instance:

User Input:

Based on the detailed evaluation guideline and format requirement you provided, I'm now evaluating factual consistency of the following summaries to the articles with a score between 0 and 1:

```json
{
  \"article\": \"Gymnast brinn bevan believes he still has a `` small chance '' of competing in the olympic games in rio next summer . Media playback is not supported on this device . The 18-year-old from essex broke the tibia and fibula in his left leg after landing on a concrete floor while vaulting in november . Less than a month before his injury , bevan was part of the first british team to claim a world championships medal , when they won silver in glasgow . `` it hurt my head and heart knowing what i had done , '' he told bbc look east . `` my coach tried to jump in front of me to stop me from seeing what had happened - but i had already seen it . '' Bevan won a series of junior titles before he claimed bronze in the pommel horse at the 2015 european games in baku . He had hoped to continue that progress at the olympics in brazil but , with only 247 days until the games , he knows it will be a challenge to be ready in time . `` the most important thing for me is getting back to full fitness so i can carry on my gymnastics career , '' he said . `` i do n't think the olympic games is fully out of reach , it is going to be tough , but there is still a small chance i can push for a place in that team . '\",\\
  \"summary\": \"Olympic bronze medallist aneurin bevan says he still has a `` small chance '' of competing at rio 2016 despite breaking his leg .\",\\
  \"consistency_score\": 0.0
}
```

Inverse Model Output:

As a highly skilled and meticulous professional fact-checker, your role involves rigorously evaluating the factual consistency of the provided summaries with the respective articles. This evaluation should be performed with a detailed assessment, assigning a score between 0 and 1 based on the accuracy of the summary's alignment with the document's facts.

Your evaluation should adhere to the following format:
{
  \"article\": \"content of the article\",
  \"summary\": \"content of the summary\",
  \"consistency_score\": number between 0 and 1 representing the factual consistency of the summary with the article
}

Carefully read and analyze the entire article before determining the factual consistency score. Ensure that your analysis is thorough and nuanced, considering the intricacies of the information presented.

Article:
Gymnast brinn bevan believes he still has a `` small chance '' of competing in the olympic games in rio next summer . Media playback is not supported on this device . The 18-year-old from essex broke the tibia and fibula in his left leg after landing on a concrete floor while vaulting in november . Less than a month before his injury , bevan was part of the first british team to claim a world championships medal , when they won silver in glasgow . `` it hurt my head and heart knowing what i had done , '' he told bbc look east . `` my coach tried to jump in front of me to stop me from seeing what had happened - but i had already seen it . '' Bevan won a series of junior titles before he claimed bronze in the pommel horse at the 2015 european games in baku . He had hoped to continue that progress at the olympics in brazil but , with only 247 days until the games , he knows it will be a challenge to be ready in time . `` the most important thing for me is getting back to full fitness so i can carry on my gymnastics career , '' he said . `` i do n't think the olympic games is fully out of reach , it is going to be tough , but there is still a small chance i can push for a place in that team . 

Summary:
Olympic bronze medallist aneurin bevan says he still has a `` small chance '' of competing at rio 2016 despite breaking his leg .

Assign the appropriate score for factual consistency between the summary and the article.

Please substitute the samples in the prompt and then you can get the general template.

Training Details

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 32
• total_train_batch_size: 1024
• total_eval_batch_size: 4
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3.0

Framework versions

• PEFT 0.12.0
• Transformers 4.46.1
• Pytorch 2.4.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3

---

license: apache-2.0