run
colab t4
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "how is python?"
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
Python is a widely-used high-level programming language known for its readability and versatility. Here are some key points about Python:
Ease of Use: Python has a simple and clear syntax, making it easy to read and write.
Versatility: It can be used for web development, scientific computing, data analysis, artificial intelligence, machine learning, and more.
Large Community: There's a vast community of developers who contribute to libraries and frameworks, which makes it easier to find solutions to problems.
Libraries and Frameworks: Python has an extensive collection of libraries and frameworks that simplify many tasks.
Cross-Platform: Python runs on multiple operating systems, including Windows, macOS, and Linux.
Dynamic Typing: Variables in Python are dynamically typed, which means you don't need to declare their types explicitly.
Interpreted Language: Python code is interpreted at runtime, which allows for rapid development and testing.
Object-Oriented: Python supports object-oriented programming, allowing you to create reusable code through classes and objects.
If you have specific questions or areas you're interested in, feel free to ask!