How to Run a Model Locally After Setting Up a Virtual Environment?
I have set up the virtual environment and configured everything, but I'm unsure how to run this model locally. Could you provide guidance on the next steps?
This has been a struggle for me as well but I am somewhat further along. The easier option is to use comfy UI... However if you are trying to avoid the security issues of custom nodes then You have to have the right dependency modules for this to function. The information available is somewhat limited. Basically you need to send a .json payload containing the variables to the server file and the server needs to have the correct version of transformers and clip. This is where I am stuck because of the slow vs. fast tokenization warning causing the pipeline to fail in loading. The virtual environment will give this error instead of pointing at the missing dependency. I'm using a conda environment with python 3.10.6 torch 2.7 transformers (latest) with sentencepiece (individual install). The problem I am having is that there seems to be an issue with the T5 tokenizer not containing sentencepiece because it is unnecessary for newest versions of transformers however the flux schnell and flux.dev pipelines still listing it as a dependency. So you need to individually install it to meet the pipeline requirement, However this means that when sentencepiece tries to import it is going to set specific settings in the config (You set add_prefix_space. The tokenizer needs to be converted from the slow tokenizers ) and this must be modified in the config file but could cause other issues. Currently your going to have to roll back versions of one or more dependencies to get reliable image generation. Handling the output is the easy part. I have set up other servers to run different AI models in virtual environments but flux has definitely been the most difficult because of how specific version compatibility need to be. My recommendation since I have been wrestling with this for several days would be to set it up on your local environment first until you nail down specific package versions then write that to a requirements txt for your virtual environment. I recommend doing it this way since operating out of a virtual environment tends to give you errors without telling you if your missing dependencies. Note: if your using something like cursor or bolt or any AI they are going to miss these errors repeatedly.