README.md

---
license: apache-2.0
language:
- en
base_model:
- codellama/CodeLlama-7b-hf
---
# **TL-CodeLLaMA-2**

TL-CodeLLaMA-2 is a model designed for tool use, built upon CodeLLaMA-7b. It is trained on 1,217 data samples using the *TL-Training* framework and demonstrates effective performance across a variety of tool use tasks. More information can be found in the paper "[TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use](https://www.arxiv.org/abs/2412.15495)".

# Model Use

## Requirements

To use this model, please make sure to install transformers:
```bash
pip install transformers
```

## Data Orgnization

The data needs to be organized in the following format:

```json
[
    {
        "role": "System",
        "content": "Function:\ndef random_advice():\n    \"\"\"\n    Returns a random advice slip as a slip object.\n    \"\"\"\n\nFunction:\ndef advice_by_id(slip_id:str):\n    \"\"\"\n    If an advice slip is found with the corresponding {slip_id}, a slip object is returned.\n\n    Args:\n        slip_id (string): The unique ID of this advice slip.\n    \"\"\"\n\nFunction:\ndef search_advice(query:str):\n    \"\"\"\n    If an advice slip is found, containing the corresponding search term in {query}, an array of slip objects is returned inside a search object.\n\n    Args:\n        query (string): The search query provided.\n    \"\"\"\n\nFunction:\ndef ask_to_user(question:str):\n    \"\"\"\n    You can ask user for guidance when you think you need more information to handle the task, but you should use this tool as less as you can.\n\n    Args:\n        question (string): The question you want to ask to user.\n    \"\"\"\n\nFunction:\ndef finish(answer:str):\n    \"\"\"\n    Finish the task and give your answer.\n\n    Args:\n        answer (string): Your answer for the task.\n    \"\"\"\n\n"
    },
    {
        "role": "User",
        "content": "Could you give me some advice about 'love'?"
    },
    {
        "role": "Assistant",
        "content": "search_advice(query = 'love') "
    },
    {
        "role": "Output",
        "content": "..."
    }
]
```

## Chat Template

The chat template is:

```jinja
{% for message in messages %}{{message['role'] + ': ' + message['content']}}{% if loop.last %}{% if add_generation_prompt %}{{ '\nAssistant:' }}{% else %}{{ '</s>'}}{% endif %}{% else %}{{ '\n' }}{% endif %}{% endfor %}
```

## Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "Junjie-Ye/TL-CodeLLaMA-2"

data = [
    {
        "role": "System",
        "content": "Function:\ndef random_advice():\n    \"\"\"\n    Returns a random advice slip as a slip object.\n    \"\"\"\n\nFunction:\ndef advice_by_id(slip_id:str):\n    \"\"\"\n    If an advice slip is found with the corresponding {slip_id}, a slip object is returned.\n\n    Args:\n        slip_id (string): The unique ID of this advice slip.\n    \"\"\"\n\nFunction:\ndef search_advice(query:str):\n    \"\"\"\n    If an advice slip is found, containing the corresponding search term in {query}, an array of slip objects is returned inside a search object.\n\n    Args:\n        query (string): The search query provided.\n    \"\"\"\n\nFunction:\ndef ask_to_user(question:str):\n    \"\"\"\n    You can ask user for guidance when you think you need more information to handle the task, but you should use this tool as less as you can.\n\n    Args:\n        question (string): The question you want to ask to user.\n    \"\"\"\n\nFunction:\ndef finish(answer:str):\n    \"\"\"\n    Finish the task and give your answer.\n\n    Args:\n        answer (string): Your answer for the task.\n    \"\"\"\n\n"
    },
    {
        "role": "User",
        "content": "Could you give me some advice about 'love'?"
    }
]

chat_template = "{% for message in messages %}{{message['role'] + ': ' + message['content']}}{% if loop.last %}{% if add_generation_prompt %}{{ '\nAssistant:' }}{% else %}{{ '</s>'}}{% endif %}{% else %}{{ '\n' }}{% endif %}{% endfor %}"

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
).eval()

tokenizer = AutoTokenizer.from_pretrained(model_path,
                                            padding_side="left",
                                            trust_remote_code=True)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token_id = tokenizer.eos_token_id

text = tokenizer.apply_chat_template(
        data,
        tokenize=False,
        chat_template=chat_template,
        add_generation_prompt=add_generation_prompt
    )
model_inputs = tokenizer(
    [text], return_tensors="pt", padding=True).to("cuda")

generated_ids = model.generate(
    max_new_tokens=1024,
    **model_inputs,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)

print(response)
```

# Citation

If you find this model useful in your research, please cite:

```bibtex
@misc{TL-Training,
      title={TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use}, 
      author={Junjie Ye and Yilong Wu and Sixian Li and Yuming Yang and Tao Gui and Qi Zhang and Xuanjing Huang and Peng Wang and Zhongchao Shi and Jianping Fan and Zhengyin Du},
      year={2024},
      eprint={2412.15495},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.15495}, 
}
```