macrotome SegFormer fast
SegFormer with the small MiT-B0 encoder (3.7 M parameters) and three sigmoid heads, trained with the same recipe and labels as macrotome SegFormer (MiT-B2). Made for computers without a GPU: about 30 s per 50 × 25 mm slide on 24 CPU cores (SegFormer 70 s), 1.5 s on an RTX 3090 (SegFormer 2 s), in 2048 px tiles at 8 µm/px. On the larger test sets it has about 0.5 mm² more tissue error per slide than SegFormer; on the held-out set the two score alike.
Segmentation of brightfield whole-slide images into three layers at 8 µm/px, for any stain, organ and scanner:
- tissue: every piece of tissue on the glass, including fat, pale IHC and small fragments;
- pen: marker ink and printed marks, on glass or over tissue;
- quality: image defects such as bubbles, folds, blur, debris, coverslip edges and dark scan corners.
Part of macrotome. Other macrotome models: SegFormer, SAM 2.1, SAM 3.
Use
pip install "macrotome[segformer] @ git+https://github.com/cancercenterai/macrotome"
macrotome segment slide.svs --model segformer-fast --out out/
import macrotome
model = macrotome.load("segformer-fast") # downloads the weights from this repository
res = model.segment("slide.svs")
tissue, pen, quality = res.mask("tissue"), res.mask("pen"), res.mask("quality")
res.write("out/") # label TIFF, QuPath GeoJSON, JSON summary, overlay
QuPath
segformer_b0.torchscript.pt (float32, for the CPU and Apple's GPU) and segformer_b0.torchscript-bf16.pt (bfloat16, for an NVIDIA GPU, as the
published scores) are this model as TorchScript, for the macrotome QuPath extension, which downloads them itself. They
take float RGB in 0..255 and return the three probabilities; the tile size, overlap, the 32 px white margin at image
edges and the resolution are in macrotome.json inside each file.
Results
98 held-out slides (Tissues with Issues, Conflux, pathologist-labelled breast slides). The SegFormers were trained on none of them. The two SAM models were trained on an earlier label snapshot that still held the 20 breast slides; on those 20 slides they score the same as SegFormer models that never saw them (Dice 0.967–0.971).
| Tissue Dice median / p10 | False tissue mm²/slide | Pen F1 | Quality P / R | |
|---|---|---|---|---|
| SegFormer | 0.985 / 0.954 | 3.7 | 0.938 | 0.71 / 0.79 |
| SegFormer fast | 0.984 / 0.953 | 3.7 | 0.932 | 0.75 / 0.74 |
| SAM 2.1 tuned | 0.984 / 0.951 | 5.3 | 0.903 | 0.83 / 0.65 |
| SAM 3 tuned | 0.984 / 0.958 | 3.9 | 0.932 | 0.87 / 0.59 |
| SlideSegmenter | 0.951 / 0.751 | 14.2 | 0.691 | – |
Training data, evaluation on the public test sets, examples and limitations are in the repository README.
Licence
CC BY-NC-SA 4.0: research and other non-commercial use, with attribution, and adaptations under the same terms. The training data include sets under non-commercial and share-alike terms. The encoder starts from NVIDIA's MiT-B0 ImageNet weights (NVIDIA Source Code License, non-commercial).
Not for clinical use: the models are research tools and are not validated for diagnosis or any clinical decision.
- Downloads last month
- 20
Model tree for CancerCenterAI/macrotome-segformer-fast
Base model
nvidia/mit-b0