metadata

license: cc-by-nc-sa-4.0
language:
  - en
base_model:
  - meta-llama/Llama-3.1-8B-Instruct
pipeline_tag: image-text-to-text
tags:
  - biology
  - healthcare
  - medical
  - LMM

BiMediX2 : Bio-Medical EXpert LMM for Diverse Medical Modalities

BiMediX2-8B-HF English Model (HuggingFace Version)

Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), UAE

Requires

vllm==0.6.1.post1
transformers==4.45.2

Serve usig vLLM

vllm serve MBZUAI/BiMediX2-8B-hf \
    --tensor-parallel-size 2 \
    --max-model-len 32000 \
    --port 8000 \
    --trust-remote-code

Openai compatible API:

from openai import OpenAI
import base64
import io
from PIL import Image

client = OpenAI(
    base_url="http://localhost:8000/v1/",
    api_key="DUMMY_KEY",
)

image = Image.open("image_path").convert("RGB") # for local Image (use image URL directly for online image)

completion = client.chat.completions.create(
    model="MBZUAI/BiMediX2-8B-hf",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{base64.b64encode(io.BytesIO(image.save(buf := io.BytesIO(), format='JPEG') or buf.getvalue()).getvalue()).decode()}"
                    }
                },
                {
                    "type": "text",
                    "text": "Describe the image in detail."
                }
            ]
        }
    ],
    max_tokens=4096,
    temperature=0.1,
    top_p=0.9
)

print(completion.choices[0].message)

License & Citation

BiMediX2 is released under the CC-BY-NC-SA 4.0 License. For more details, please refer to the LICENSE file included in our BiMediX repository.

⚠️ Warning! This release, intended for research, is not ready for clinical or commercial use.

Users are urged to employ BiMediX2 responsibly, especially when applying its outputs in real-world medical scenarios. It is imperative to verify the model's advice with qualified healthcare professionals and not rely on it for medical diagnoses or treatment decisions. Despite the overall advancements BiMediX2 shares common challenges with other language models, including hallucinations, toxicity, and stereotypes. BiMediX2's medical diagnoses and recommendations are not infallible.

If you use BiMediX2 in your research, please cite our work as follows:

@misc{mullappilly2024bimedix2biomedicalexpertlmm,
      title={BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities}, 
      author={Sahal Shaji Mullappilly and Mohammed Irfan Kurpath and Sara Pieri and Saeed Yahya Alseiari and Shanavas Cholakkal and Khaled Aldahmani and Fahad Khan and Rao Anwer and Salman Khan and Timothy Baldwin and Hisham Cholakkal},
      year={2024},
      eprint={2412.07769},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.07769}, 
}