Add pipeline tag, library name and link to code (#1)

- Add pipeline tag, library name and link to code (012b277526219e64217a1f1142444014b602e308)
- modify to pr #1 (c86ae57a2ecaf4e1dc2a92130df39005788c2ded)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -1,50 +1,58 @@
----
-license: llama2
-datasets:
-- liuhaotian/LLaVA-Instruct-150K
----
-
-# Ada-LLaVA Model Card
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-Ada-LLaVA 7B is an open-source adaptive inference framework for multimodal Large Language Models (MLLMs) that dynamically adjusts its operations based on available computational resources and latency requirements. 
-
-See the paper for more details: [Learning to Inference Adaptively for Multimodal Large Language Models]()
-
-**Model details**: https://zhuoyan-xu.github.io/ada-llava/
-
-## Model Details
-
-<!-- Provide a longer summary of what this model is. -->
-
-**Model Type:** Ada LLaVA 7B follows the [LLaVA-v1.5](https://arxiv.org/abs/2310.03744) stage-2 training pipeline, 
-with [CLIP-ViT-L-336px](https://huggingface.co/openai/clip-vit-large-patch14-336) as visual encoder (336*336 image resolution), 
-[Vicuna-v1.5-7B](https://huggingface.co/lmsys/vicuna-7b-v1.5) as base LLM and a two-layer MLP as vision-language connector, customized embedding model and MLP as latency scheduler. 
-
-It was trained with stage-2 pipeline as LLaVA:
-
-Instruction tuning: Freeze vision encoder, train the remaining model with multimodal instruction following data of tabular and non-tabular tasks.
-
-**Code Base:** We use the official code of [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA) for model training and inference, 
-and the saved model checkpoint is uploaded to this repository. 
-
-**Model Date:** Ada-LLaVA 7B was trained in Oct 2024.
-
-
-## License
-
-AdaLLaVA is based on LLaVA-1.5 and thus follows its license. Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
-
-## Intended use
-
-**Primary intended uses:** The primary use of Ada LLaVA is research on multimodal large multimodal models and chatbots, especially for resource-constrain inference and deployment.
-
-**Primary intended users:** The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.
-
-## Training dataset
-- 665K image level instruction data from LLaVA-1.5 stage-2, see details in original [LLaVA repo](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#visual-instruction-tuning).
-
-## Limitations
-
-While Ada-LLaVA is currently limited to processing one image at a time and only applies adaptive operations in its later half of layers, future work could explore multi-image input support and extend the adaptive mechanisms throughout the entire model architecture, including the vision encoder. These improvements would make the model more versatile and applicable to a broader range of real-world scenarios.
+---
+datasets:
+- liuhaotian/LLaVA-Instruct-150K
+license: llama2
+library_name: transformers
+pipeline_tag: image-text-to-text
+base_model:
+- liuhaotian/llava-v1.5-7b
+---
+
+```markdown
+# Ada-LLaVA Model Card
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+Ada-LLaVA 7B is an open-source adaptive inference framework for multimodal Large Language Models (MLLMs) that dynamically adjusts its operations based on available computational resources and latency requirements. 
+
+See the paper for more details: [Learning to Inference Adaptively for Multimodal Large Language Models](https://huggingface.co/papers/2503.10905)
+
+**Model details**: https://zhuoyan-xu.github.io/ada-llava/
+
+**Github repository**: https://github.com/zhuoyan-xu/AdaLLaVA
+
+## Model Details
+
+<!-- Provide a longer summary of what this model is. -->
+
+**Model Type:** Ada LLaVA 7B follows the [LLaVA-v1.5](https://arxiv.org/abs/2310.03744) stage-2 training pipeline, 
+with [CLIP-ViT-L-336px](https://huggingface.co/openai/clip-vit-large-patch14-336) as visual encoder (336*336 image resolution), 
+[Vicuna-v1.5-7B](https://huggingface.co/lmsys/vicuna-7b-v1.5) as base LLM and a two-layer MLP as vision-language connector, customized embedding model and MLP as latency scheduler. 
+
+It was trained with stage-2 pipeline as LLaVA:
+
+Instruction tuning: Freeze vision encoder, train the remaining model with multimodal instruction following data of tabular and non-tabular tasks.
+
+**Code Base:** We use the official code of [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA) for model training and inference, 
+and the saved model checkpoint is uploaded to this repository. 
+
+**Model Date:** Ada-LLaVA 7B was trained in Oct 2024.
+
+
+## License
+
+AdaLLaVA is based on LLaVA-1.5 and thus follows its license. Llama 2 is licensed under the LLAMA 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
+
+## Intended use
+
+**Primary intended uses:** The primary use of Ada LLaVA is research on multimodal large multimodal models and chatbots, especially for resource-constrain inference and deployment.
+
+**Primary intended users:** The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.
+
+## Training dataset
+- 665K image level instruction data from LLaVA-1.5 stage-2, see details in original [LLaVA repo](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#visual-instruction-tuning).
+
+## Limitations
+
+While Ada-LLaVA is currently limited to processing one image at a time and only applies adaptive operations in its later half of layers, future work could explore multi-image input support and extend the adaptive mechanisms throughout the entire model architecture, including the vision encoder. These improvements would make the model more versatile and applicable to a broader range of real-world scenarios.
+```