Dedicated HF endpoint fails
#13
by
FreeHugsForRobots
- opened
I tried to run this model on a dedicated endpoint (AWS GPU instance with an A100) but got the following error immediately:
Exit code: 3. Reason: enter__
await self._router.startup()
File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 670, in startup
await handler()
File "/app/webservice_starlette.py", line 62, in prepare_model_artifacts
inference_handler = get_inference_handler_either_custom_or_default_handler(
File "/app/huggingface_inference_toolkit/handler.py", line 96, in get_inference_handler_either_custom_or_default_handler
return HuggingFaceHandler(model_dir=model_dir, task=task)
File "/app/huggingface_inference_toolkit/handler.py", line 19, in __init__
self.pipeline = get_pipeline(
File "/app/huggingface_inference_toolkit/utils.py", line 261, in get_pipeline
hf_pipeline = pipeline(task=task, model=model_dir, device=device, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py", line 849, in pipeline
normalized_task, targeted_task, task_options = check_task(task)
File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py", line 533, in check_task
return PIPELINE_REGISTRY.check_task(task)
File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/base.py", line 1339, in check_task
raise KeyError(
KeyError: "Unknown task custom, available tasks are ['audio-classification', 'automatic-speech-recognition', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-feature-extraction', 'image-segmentation', 'image-to-image', 'image-to-text', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis', 'summarization', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'text2text-generation', 'token-classification', 'translation', 'video-classification', 'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection', 'translation_XX_to_YY']"
Application startup failed. Exiting.
This model requires a custom handler to be able to deploy to Inference Endpoints. If you follow the steps on the model card page, you should be able to deploy it.