George Williams's picture
3 1

George Williams PRO

sourcesync
ยท

AI & ML interests

Medical LLMs

Recent Activity

replied to their post 16 days ago
Hi! I'm having an issue converting a Phi3 model to a torch scripted model. Code, error, and versions below. Any guidance would be super helpful, thanks! Here is the code: ``` from transformers import AutoModelForCausalLM import torch base_model = "microsoft/Phi-3-mini-4k-instruct" model = AutoModelForCausalLM.from_pretrained( base_model, return_dict=True, low_cpu_mem_usage=True, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True, torchscript=True ) scripted_model = torch.jit.script(model) ``` Here is the error: ``` File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp> args = [build_expr(ctx, py_arg) for py_arg in expr.args] File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__ return method(ctx, node) File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp return ExprBuilder.build_ListComp(ctx, stmt) File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp raise NotSupportedError(r, "Comprehension ifs are not supported yet") torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet: File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148 next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache if not return_dict: return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None) return BaseModelOutputWithPast( last_hidden_state=hidden_states ``` torch==2.5.1 transformers==4.51.3
posted an update 16 days ago
Hi! I'm having an issue converting a Phi3 model to a torch scripted model. Code, error, and versions below. Any guidance would be super helpful, thanks! Here is the code: ``` from transformers import AutoModelForCausalLM import torch base_model = "microsoft/Phi-3-mini-4k-instruct" model = AutoModelForCausalLM.from_pretrained( base_model, return_dict=True, low_cpu_mem_usage=True, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True, torchscript=True ) scripted_model = torch.jit.script(model) ``` Here is the error: ``` File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp> args = [build_expr(ctx, py_arg) for py_arg in expr.args] File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__ return method(ctx, node) File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp return ExprBuilder.build_ListComp(ctx, stmt) File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp raise NotSupportedError(r, "Comprehension ifs are not supported yet") torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet: File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148 next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache if not return_dict: return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None) return BaseModelOutputWithPast( last_hidden_state=hidden_states ``` torch==2.5.1 transformers==4.51.3
View all activity

Organizations

IEEE's profile picture KaggleX 2024 Cohort's profile picture

Posts 1

view post
Post
605
Hi! I'm having an issue converting a Phi3 model to a torch scripted model. Code, error, and versions below. Any guidance would be super helpful, thanks!

Here is the code:

from transformers import AutoModelForCausalLM
import torch

base_model = "microsoft/Phi-3-mini-4k-instruct"

model = AutoModelForCausalLM.from_pretrained(
        base_model,
        return_dict=True,
        low_cpu_mem_usage=True,
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True,
        torchscript=True
)

scripted_model = torch.jit.script(model)


Here is the error:

File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 910, in <listcomp>
    args = [build_expr(ctx, py_arg) for py_arg in expr.args]
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 417, in __call__
    return method(ctx, node)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1255, in build_GeneratorExp
    return ExprBuilder.build_ListComp(ctx, stmt)
  File ".../anaconda3/envs/qcomm_phi3.5instruct_aihub_py_310/lib/python3.10/site-packages/torch/jit/frontend.py", line 1244, in build_ListComp
    raise NotSupportedError(r, "Comprehension ifs are not supported yet")
torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet:
  File ".../.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/0a67737cc96d2554230f90338b163bc6380a2a85/modeling_phi3.py", line 1148
            next_cache = next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache
        if not return_dict:
            return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None)
        return BaseModelOutputWithPast(
            last_hidden_state=hidden_states


torch==2.5.1
transformers==4.51.3

datasets 0

None public yet