OpenMed

OpenMed ModernBERT BiomedicalSections Base v1

OpenMed on Hugging Face OpenMed SDK on GitHub pip install openmed License Apache-2.0

OpenMed's open model for labeling the sentences of biomedical abstracts. It reads one sentence of an English abstract and says whether it is background, objective, methods, results or conclusions, with a score. It is the more accurate of OpenMed's two biomedical section models. Trained and evaluated by OpenMed. Apache-2.0, 149.6M parameters, runs on CPU.

Developed by OpenMed
Task Sentence classification for biomedical abstracts, five labels: BACKGROUND, OBJECTIVE, METHODS, RESULTS, CONCLUSIONS
Input One English sentence of an abstract, without its heading
Output One label and its score (a softmax over the five labels, not calibrated)
Parameters 149.6M
Base model answerdotai/ModernBERT-base
Language English
License Apache-2.0

About OpenMed

OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.

Quick start

The OpenMed SDK splits the abstract into sentences and the transformers pipeline labels each one. The abstract below is synthetic and has no headings.

pip install -U openmed transformers torch
from openmed.processing.sentences import segment_text
from transformers import pipeline

abstract = (
    "Many adults with type 2 diabetes do not reach their "
    "glucose targets. We tested whether a pharmacist-led "
    "telehealth program lowers HbA1c. In this randomized "
    "trial, 320 adults with HbA1c above 8% were assigned "
    "to monthly pharmacist video visits or usual care for "
    "six months. HbA1c fell by 1.1 percentage points with "
    "the program and by 0.4 points with usual care (P < "
    "0.001). A pharmacist-led telehealth program improved "
    "glucose control in adults with poorly controlled "
    "type 2 diabetes."
)
classifier = pipeline(
    "text-classification",
    model="OpenMed/ModernBERT-BiomedicalSections-Base-v1",
    device="cpu",
)
sentences = [s.text.strip() for s in segment_text(abstract)]
for sentence, p in zip(sentences, classifier(sentences)):
    print(f"{p['label']:<12} {p['score']:.2f}  {sentence}")
BACKGROUND   0.79  Many adults with type 2 diabetes do not reach their glucose targets.
BACKGROUND   0.75  We tested whether a pharmacist-led telehealth program lowers HbA1c.
METHODS      1.00  In this randomized trial, 320 adults with HbA1c above 8% were assigned to monthly pharmacist video visits or usual care for six months.
RESULTS      1.00  HbA1c fell by 1.1 percentage points with the program and by 0.4 points with usual care (P < 0.001).
CONCLUSIONS  0.97  A pharmacist-led telehealth program improved glucose control in adults with poorly controlled type 2 diabetes.

Four of the five sentences come back as the section they were written for. The second, written as the study's objective, comes back BACKGROUND (objective is this model's weakest label; see Good to know).

The settings

  • One sentence at a time. The model was trained and scored on single sentences, with no heading, no neighboring sentences and no position in the abstract, so it labels each sentence from its own wording. Pass sentences, not the whole abstract.
  • Take headings off. Leading headings such as BACKGROUND: or Methods: were removed from every sentence before training and evaluation. An abstract that already has headings already has its sections.
  • The label is the highest of the five scores (argmax), the decoding behind the evaluation below. The scores are a softmax over the five labels: they rank labels, they are not calibrated probabilities.
  • classifier(sentences, top_k=None) returns all five scores for each sentence.
  • device="cpu" gives the output printed above. On a GPU, pass device="cuda"; for many abstracts, pass batch_size=32 to the classifier call.
  • Evaluation bounded each sentence to 2,000 characters and 512 tokens.
  • segment_text is the OpenMed SDK's sentence splitter; .strip() drops the space it keeps after each sentence. Any sentence splitter works.

Evaluation

5,976 held-out sentences from 500 PubMed abstracts of randomized controlled trials. Each sentence is labeled with the section its abstract's published structured headings put it in (689 background, 454 objective, 1,962 methods, 1,976 results, 895 conclusions). The models were trained on 38,528 sentences from 3,200 other abstracts, and 500 more abstracts (5,971 sentences) were used for development, including choosing the TF-IDF reference's setting; these 500 were scored once at the end, with both models, the reference and the scoring code frozen first. Abstracts were screened against 48,402 documents of earlier OpenMed training and evaluation data before the split. Each sentence is classified on its own, with its heading removed and no neighboring sentences or position. The abstracts come from the source's training portion; its official development and test portions were not used, so these scores are not comparable to published results on them.

Model Macro-F1 Accuracy
OpenMed ModernBERT BiomedicalSections Base v1 (this model) 80.30 86.30
OpenMed GLiNER2.5 BiomedicalSections Small v1 77.44 84.02
TF-IDF and logistic regression reference, trained on the same sentences 72.97 80.32
answerdotai/ModernBERT-base with an untrained five-label head (a control, not a baseline) 14.34 16.97

This model scores +7.34 macro-F1 points above the TF-IDF reference (95% CI +5.88 to +8.82) and +2.86 above the GLiNER2.5 model (95% CI +1.73 to +4.02); paired bootstrap over the 500 abstracts, 2,000 resamples, each keeping all sentences of an abstract. ModernBERT-base has no section head before fine-tuning, so the untrained-head row is a floor, not a competing classifier.

By section

Section Sentences Precision Recall F1 TF-IDF recall Recall change vs TF-IDF (95% CI)
background 689 68.26 78.66 73.10 59.65 +19.01 (+15.11 to +22.87)
objective 454 76.70 52.20 62.12 48.90 +3.30 (-0.66 to +7.30)
methods 1,962 91.80 94.70 93.23 91.64 +3.06 (+1.81 to +4.28)
results 1,976 91.52 90.08 90.79 87.80 +2.28 (+0.87 to +3.70)
conclusions 895 81.86 82.68 82.27 70.84 +11.84 (+8.88 to +14.99)

Objective is the weak label: 62.12 F1, recall 52.20. 217 of the 454 objective sentences come back as another section, 175 of them as background, and the objective recall change over TF-IDF is +3.30 (its interval includes zero). Out of context, background and objective sentences can read alike. Results and conclusions are the other pair it mixes up: 92 results sentences come back as conclusions, and of the conclusions, 92 come back as results and 49 as background. On the other four sections, recall is above TF-IDF with intervals above zero, most on background (+19.01 points).

The OpenMed biomedical section models

Same task, same five sections, same training sentences, scored on the same held-out abstracts.

Model Base Parameters Quick start Macro-F1 Accuracy Objective recall
OpenMed ModernBERT BiomedicalSections Base v1 (this model) answerdotai/ModernBERT-base 149.6M transformers 80.30 86.30 52.20
OpenMed GLiNER2.5 BiomedicalSections Small v1 fastino/gliner2.5-small-v1 73.9M OpenMed SDK 77.44 84.02 50.66

Examples

Real outputs of this model with the quick-start code, on CPU. The four abstracts are synthetic, written for this card in the style of PubMed abstracts, one per field. Each sentence was written for one section; where the model disagrees, its label is in bold and the intended section follows the sentence.

1. Cardiology trial (a randomized trial): 6 of 8 sentences come back as written.

Label Score Sentence
OBJECTIVE 0.66 Atrial fibrillation is the most common sustained arrhythmia, and many older patients remain symptomatic despite rate control. (written as background)
BACKGROUND 0.85 Whether first-line catheter ablation improves outcomes in this age group is unclear.
BACKGROUND 0.85 We compared catheter ablation with antiarrhythmic drug therapy as first-line treatment in patients aged 70 years or older. (written as objective)
METHODS 0.96 In this multicenter trial, 412 patients with symptomatic paroxysmal atrial fibrillation were randomly assigned to pulmonary vein isolation or antiarrhythmic drugs.
METHODS 0.93 The primary outcome was freedom from atrial arrhythmia recurrence at 12 months, assessed with implantable loop recorders.
RESULTS 1.00 At 12 months, 71% of patients in the ablation group and 46% in the drug group were free from recurrence (hazard ratio 0.48; 95% CI 0.36 to 0.64).
RESULTS 1.00 Serious adverse events occurred in 5.3% and 6.1% of patients, respectively.
CONCLUSIONS 1.00 In older patients with paroxysmal atrial fibrillation, first-line catheter ablation reduced arrhythmia recurrence without increasing serious adverse events.

2. Oncology single-arm trial (a phase 2 single-arm trial): 6 of 7 sentences come back as written.

Label Score Sentence
BACKGROUND 0.98 Patients with recurrent glioblastoma have few treatment options and a median survival of less than one year.
BACKGROUND 0.97 This phase 2 study evaluated the safety and activity of a dendritic cell vaccine combined with bevacizumab. (written as objective)
METHODS 1.00 Adults at first recurrence after standard chemoradiation received the vaccine every two weeks for four doses and then monthly.
METHODS 1.00 Tumor response was assessed by magnetic resonance imaging every eight weeks.
RESULTS 0.99 Of 38 enrolled patients, 11 had an objective response, and median overall survival was 13.2 months.
RESULTS 1.00 Grade 3 or higher adverse events occurred in nine patients, most often hypertension and fatigue.
CONCLUSIONS 0.99 The combination was tolerable and showed encouraging activity, supporting a randomized trial.

3. Public health cluster trial (a cluster randomized trial): 6 of 7 sentences come back as written.

Label Score Sentence
BACKGROUND 0.99 Childhood vaccination coverage has fallen in many low-income urban districts.
OBJECTIVE 1.00 To test whether text-message reminders sent to caregivers increase on-time measles vaccination.
METHODS 0.98 We conducted a cluster randomized trial in 48 primary care clinics, which were assigned to automated reminders or standard care.
RESULTS 0.66 Caregivers of 6,214 infants received up to three messages before the scheduled visit. (written as methods)
RESULTS 1.00 On-time vaccination was 78.4% in reminder clinics and 69.9% in control clinics.
RESULTS 0.81 The effect was largest among caregivers who had missed a previous appointment.
CONCLUSIONS 1.00 Text-message reminders can improve timely childhood vaccination and could be extended to other public clinics.

4. Observational cohort (a retrospective cohort study, not a trial): 7 of 8 sentences come back as written.

Label Score Sentence
BACKGROUND 0.84 Long-term use of proton pump inhibitors has been linked to lower bone mineral density.
BACKGROUND 0.88 We examined the association between proton pump inhibitor use and hip fracture in adults aged 50 years and older. (written as objective)
METHODS 0.99 This retrospective cohort study used linked primary care and hospital records from 2010 to 2022.
METHODS 0.89 Hazard ratios were estimated with Cox regression adjusted for age, sex, body mass index, smoking and glucocorticoid use.
RESULTS 0.97 Among 184,302 adults followed for a median of 6.4 years, 3,911 hip fractures occurred.
RESULTS 1.00 Current use was associated with a higher risk of hip fracture (adjusted hazard ratio 1.27; 95% CI 1.17 to 1.38).
CONCLUSIONS 1.00 Proton pump inhibitor use was associated with a modest increase in hip fracture risk, although residual confounding cannot be excluded.
CONCLUSIONS 1.00 Clinicians should review the need for long-term therapy in older patients.

The same objective, two phrasings. Each abstract's objective sentence, as written and rephrased (We tested whether ... or To test whether ...), each classified on its own:

Abstract As written Label Rephrased Label
Diabetes telehealth trial We tested whether ... BACKGROUND 0.75 To test whether ... OBJECTIVE 1.00
Cardiology trial We compared catheter ... BACKGROUND 0.85 To compare catheter ... OBJECTIVE 1.00
Oncology single-arm trial This phase 2 study ... BACKGROUND 0.97 To evaluate the ... OBJECTIVE 1.00
Public health cluster trial To test whether ... OBJECTIVE 1.00 We tested whether ... BACKGROUND 0.79
Observational cohort We examined the ... BACKGROUND 0.88 To examine the ... OBJECTIVE 1.00

Good to know

  • On the five synthetic abstracts on this card (the quick start and examples 1-4), 29 of 35 sentences come back as the section they were written for.
  • Objective versus background is the main miss, as in the evaluation, and the phrasing decides it here. Written as a statement (We compared ..., This phase 2 study evaluated ...), all five objective sentences come back background; the same sentences as a To ... phrase all come back objective (the phrasing table above). If your abstracts state their aim as We ..., expect those sentences to be labeled background.
  • Example 1, sentence 1, written as background, comes back OBJECTIVE (0.66): Atrial fibrillation is the most common sustained arrhythmia, and many older patients remain symptomatic despite rate control.
  • Example 3, sentence 4, written as methods, comes back RESULTS (0.66): Caregivers of 6,214 infants received up to three messages before the scheduled visit.
  • Example 4 is an observational cohort, a study type the model was not trained on: all seven of its sentences other than the objective come back as written.
  • A wrong label can still score high: This phase 2 study ... comes back BACKGROUND at 0.97. Treat the score as a ranking, not as the chance of being right.

Limitations

  • Trained and tested on English PubMed abstracts of randomized controlled trials. Other study types (such as the cohort in example 4), other languages, full-text articles and clinical notes are not validated.
  • Objective is the weak label: 62.12 F1 and 52.20 recall on the held-out sentences, most misses going to background.
  • Five sections only. Other headings (case presentation, discussion, limitations, trial registration) have no label of their own and come back as one of the five.
  • It sees one sentence at a time and does not use the order of the abstract.
  • The reference labels are the sections the abstracts were published under, assigned by their authors and journals, not annotated for this task.
  • Scores rank labels; they are not calibrated probabilities.
  • The base model's pretraining data is not public, so whether it saw these abstracts is unknown.
  • It labels sentences. It does not judge a study's quality, give medical advice or act as a medical device.

Built on

OpenMed fine-tuned this model from answerdotai/ModernBERT-base by Answer.AI (Apache-2.0), with a new five-label classification head, using the open-source transformers library. Training and evaluation used sentences from PubMed abstracts of randomized controlled trials, labeled with the sections of their published structured abstracts. Thank you to Answer.AI for releasing ModernBERT openly.

This model is released under Apache-2.0. No abstract text or training data is included in this repository.

Citation

@misc{panahi2025openmedner,
  title={OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets},
  author={Maziyar Panahi},
  year={2025},
  eprint={2508.01630},
  archivePrefix={arXiv}
}

@misc{warner2024modernbert,
  title={Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference},
  author={Benjamin Warner and Antoine Chaffin and Benjamin Clavi\'{e} and Orion Weller and Oskar Hallstr\"{o}m and Said Taghadouini and Alexis Gallagher and Raja Biswas and Faisal Ladhak and Tom Aarsen and Nathan Cooper and Griffin Adams and Jeremy Howard and Iacopo Poli},
  year={2024},
  eprint={2412.13663},
  archivePrefix={arXiv}
}

OpenMed · Hugging Face · GitHub · PyPI · X

Downloads last month
4
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for OpenMed/ModernBERT-BiomedicalSections-Base-v1

Finetuned
(1,612)
this model

Papers for OpenMed/ModernBERT-BiomedicalSections-Base-v1

Evaluation results

  • Macro-F1 (five sections) on Held-out sentences from PubMed abstracts of randomized controlled trials (5,976 sentences, 500 abstracts)
    self-reported
    80.300
  • Accuracy on Held-out sentences from PubMed abstracts of randomized controlled trials (5,976 sentences, 500 abstracts)
    self-reported
    86.300