|
|
--- |
|
|
license: apache-2.0 |
|
|
pipeline_tag: image-segmentation |
|
|
tags: |
|
|
- medical |
|
|
- biology |
|
|
- histology |
|
|
- histopathology |
|
|
--- |
|
|
|
|
|
# Cellpose Model for Cervical Intraepithelial Neoplasia 2 (CIN2) Panoptic Segmentation |
|
|
|
|
|
# Model |
|
|
- **histolytics** implementation of Panoptic **Cellpose**: [https://www.nature.com/articles/s41592-020-01018-x](https://www.nature.com/articles/s41592-020-01018-x) |
|
|
- Backbone encoder: pre-trained **efficientnet_b5** from pytorch-image-models [https://github.com/huggingface/pytorch-image-models](https://github.com/huggingface/pytorch-image-models) |
|
|
|
|
|
|
|
|
# USAGE |
|
|
|
|
|
## 1. Install histolytics and albumentations |
|
|
``` |
|
|
pip install histolytics |
|
|
pip install albumentations |
|
|
``` |
|
|
|
|
|
## 2. Load trained model |
|
|
```python |
|
|
from histolytics.models.cellpose_panoptic import CellPosePanoptic |
|
|
|
|
|
model = CellPosePanoptic.from_pretrained("cin2_v1_efficientnet_b5") |
|
|
``` |
|
|
|
|
|
## 3. Run inference for one image |
|
|
```python |
|
|
from albumentations import Resize, Compose |
|
|
from histolytics.utils import FileHandler |
|
|
from histolytics.transforms.albu_transforms import MinMaxNormalization |
|
|
|
|
|
model.set_inference_mode() |
|
|
|
|
|
# Resize to multiple of 32 of your own choosing |
|
|
transform = Compose([Resize(1024, 1024), MinMaxNormalization()]) |
|
|
|
|
|
im = FileHandler.read_img(IMG_PATH) |
|
|
im = transform(image=im)["image"] |
|
|
|
|
|
prob = model.predict(im) |
|
|
out = model.post_process(prob) |
|
|
# out = {"nuc": [(nuc instances (H, W), nuc types (H, W))], "cyto": None, "tissue": None} |
|
|
``` |
|
|
|
|
|
## 3.1 Run inference for image batch |
|
|
```python |
|
|
import torch |
|
|
from histolytics.utils import FileHandler |
|
|
|
|
|
model.set_inference_mode() |
|
|
|
|
|
# dont use random matrices IRL |
|
|
batch = torch.rand(8, 3, 1024, 1024) |
|
|
|
|
|
prob = model.predict(im) |
|
|
out = model.post_process(prob) |
|
|
# out = { |
|
|
# "nuc": [ |
|
|
# (nuc instances (H, W), nuc types (H, W)), |
|
|
# (nuc instances (H, W), nuc types (H, W)), |
|
|
# . |
|
|
# . |
|
|
# . |
|
|
# (nuc instances (H, W), nuc types (H, W)) |
|
|
# ], |
|
|
# "tissue": [ |
|
|
# (nuc instances (H, W), nuc types (H, W)), |
|
|
# (nuc instances (H, W), nuc types (H, W)), |
|
|
# . |
|
|
# . |
|
|
# . |
|
|
# (nuc instances (H, W), nuc types (H, W)) |
|
|
# ], |
|
|
# "cyto": None, |
|
|
#} |
|
|
``` |
|
|
|
|
|
## 4. Visualize output |
|
|
```python |
|
|
from matplotlib import pyplot as plt |
|
|
from skimage.color import label2rgb |
|
|
|
|
|
fig, ax = plt.subplots(1, 4, figsize=(24, 6)) |
|
|
ax[0].imshow(im) |
|
|
ax[1].imshow(label2rgb(out["nuc"][0][0], bg_label=0)) # inst_map |
|
|
ax[2].imshow(label2rgb(out["nuc"][0][1], bg_label=0)) # type_map |
|
|
ax[3].imshow(label2rgb(out["tissue"][0], bg_label=0)) # tissue_map |
|
|
``` |
|
|
 |
|
|
|
|
|
## Dataset Details |
|
|
Semi-manually annotated CIN2 samples from a (private) cohort of Helsinki University Hospital |
|
|
|
|
|
**Contains:** |
|
|
- 370 varying sized image crops at 20x magnification. |
|
|
- 168 640 annotated nuclei |
|
|
- 570 872 983 pixels of annotated tissue region |
|
|
|
|
|
## Dataset classes |
|
|
|
|
|
``` |
|
|
nuc_classes = { |
|
|
0: "background", |
|
|
1: "neoplastic", |
|
|
2: "inflammatory", |
|
|
3: "connective", |
|
|
4: "dead", |
|
|
5: "glandular_epithelial", |
|
|
6: "squamous_epithelial", |
|
|
} |
|
|
|
|
|
tissue_classes = { |
|
|
0: "background", |
|
|
1: "stroma", |
|
|
2: "cin", |
|
|
3: "squamous_epithelium", |
|
|
4: "glandular_epithelium", |
|
|
5: "slime", |
|
|
6: "blood", |
|
|
} |
|
|
``` |
|
|
|
|
|
## Dataset Class Distribution |
|
|
|
|
|
**Nuclei**: |
|
|
- connective nuclei: 46 222 (~27.3%) |
|
|
- neoplastic nuclei: 49 493 (~29.4%) |
|
|
- inflammatory nuclei 27 226 (~16.1%) |
|
|
- dead nuclei 195 (~0.11%) |
|
|
- glandular epithelial 14 310 (~8.5%) |
|
|
- squamous epithelial 31194 (~18.5%) |
|
|
|
|
|
**Tissues**: |
|
|
- stromal tissue: 28.2% |
|
|
- CIN tissue: 23.4% |
|
|
- squamous epithelium: 24.7% |
|
|
- glandular epithelium 7.7% |
|
|
- slime 6.5% |
|
|
- blood 2.5% |
|
|
|
|
|
# Citation |
|
|
|
|
|
histolytics: |
|
|
``` |
|
|
@article{ |
|
|
|
|
|
} |
|
|
``` |
|
|
|
|
|
Cellpose original paper: |
|
|
``` |
|
|
@article{Stringer2020, |
|
|
title = {Cellpose: a generalist algorithm for cellular segmentation}, |
|
|
volume = {18}, |
|
|
ISSN = {1548-7105}, |
|
|
url = {http://dx.doi.org/10.1038/s41592-020-01018-x}, |
|
|
DOI = {10.1038/s41592-020-01018-x}, |
|
|
number = {1}, |
|
|
journal = {Nature Methods}, |
|
|
publisher = {Springer Science and Business Media LLC}, |
|
|
author = {Stringer, Carsen and Wang, Tim and Michaelos, Michalis and Pachitariu, Marius}, |
|
|
year = {2020}, |
|
|
month = dec, |
|
|
pages = {100–106} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Licence |
|
|
These model weights are released under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at: |
|
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0 |
|
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. |
|
|
|
|
|
## Additional Terms |
|
|
|
|
|
While the Apache 2.0 License grants broad permissions, we kindly request that users adhere to the following guidelines: |
|
|
Medical or Clinical Use: This model is not intended for use in medical diagnosis, treatment, or prevention of disease of real patients. It should not be used as a substitute for professional medical advice. |
