|
--- |
|
license: mit |
|
language: |
|
- en |
|
library_name: transformers |
|
tags: |
|
- medical |
|
- clinical |
|
- healthcare |
|
- perioperative care |
|
inference: false |
|
base_model: microsoft/biogpt |
|
--- |
|
# BJH-perioperative-notes-bioGPT |
|
|
|
This clinical foundational model is designed to predict potential risk factors / post-operative surgical outcomes from clinical notes taken during perioperative care. It was finetuned from the `microsoft/biogpt` model through a multi-task learning approach, spanning the following 6 outcomes: |
|
- Death in 30 days |
|
- Deep vein thrombosis (DVT) |
|
- pulmonary embolism (PE) |
|
- Pneumonia |
|
- Acute Knee Injury |
|
- delirium |
|
|
|
Also check out [`cja5553/BJH-perioperative-notes-bioClinicalBERT`](https://huggingface.co/cja5553/BJH-perioperative-notes-bioClinicalBERT), which is the bioClinicalBERT variant of our model! |
|
|
|
## Dataset |
|
We used 84,875 perioperative clinical notes spanning 3 years worth of anonymized patient data from the Barnes Jewish Healthcare (BJH) system in St Louis, MO. BJH is the largest hospital in the state of Missouri and the largest employer in the greater St. Louis region! The following are the characteristics for the data: |
|
- vocabulary size: 3203 |
|
- averaging words per clinical note: 8.9 words |
|
- all single sentenced clinical notes |
|
|
|
## How to use model |
|
|
|
```python |
|
from transformers import BioGptTokenizer, AutoModelForCausalLM |
|
model=AutoModelForCausalLM.from_pretrained("cja5553/BJH-perioperative-notes-bioGPT") |
|
tokenizer = BioGptTokenizer.from_pretrained("microsoft/biogpt") |
|
``` |
|
*Note: Because of our distinct model architecture, you are required to train a distinct predictor or use a respective fully-connected network above the hidden state when deploying the model via `transformers`* |
|
|
|
|
|
## Codes |
|
Codes used to train the model are publicly available at: https://github.com/cja5553/LLMs_in_perioperative_care |
|
|
|
## Note: |
|
The model remains private as it was trained on clinical notes from patients. Requests made with non-institutional emails (e.g., `gmail.com`) will be rejected. |
|
Reasonable requests from recognized institutional emails may have to undergo appropriate approvals to access the model. |
|
|
|
## Citation |
|
If you find this model useful, please cite the following paper: |
|
|
|
``` |
|
@article{ |
|
author={Charles Alba, Bing Xue, Joanna Abraham, Thomas Kannampallil, Chenyang Lu} |
|
title={The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes}, |
|
year={2024} |
|
} |
|
``` |
|
|
|
## Questions? |
|
contact me at [email protected] |
