AI Model Catalog | Microsoft Foundry Models

microsoft-biomednlp-biomedbert-base-uncased-abstract

Version: 1

HuggingFace•Last updated July 2025

MSR BiomedBERT (abstracts only)

This model was previously named "PubMedBERT (abstracts)".
You can either adopt the new model name "microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract" or update your transformers library to version 4.22+ if you need to refer to the old name.

Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. Recent work shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models. This BiomedBERT is pretrained from scratch using abstracts from PubMed . This model achieves state-of-the-art performance on several biomedical NLP tasks, as shown on the Biomedical Language Understanding and Reasoning Benchmark .

Citation

If you find BiomedBERT useful in your research, please cite the following paper:

@misc{pubmedbert,
  author = {Yu Gu and Robert Tinn and Hao Cheng and Michael Lucas and Naoto Usuyama and Xiaodong Liu and Tristan Naumann and Jianfeng Gao and Hoifung Poon},
  title = {Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing},
  year = {2020},
  eprint = {arXiv:2007.15779},
}

microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract powered by Hugging Face Inference Toolkit

Send Request

You can use cURL or any REST Client to send a request to the AzureML endpoint with your AzureML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"inputs":"The answer to the universe is undefined."}'

Supported Parameters

inputs (string): The text with masked tokens
parameters (object):
- top_k (integer): When passed, overrides the number of predictions to return.
- targets (string[]): When passed, the model will limit the scores to the passed targets instead of looking up in the whole vocabulary. If the provided targets are not in the model vocab, they will be tokenized and the first resulting token will be used (with a warning, and that might be slower).

Check the full API Specification at the Hugging Face Inference documentation .

Model Specifications

LicenseMit

Last UpdatedJuly 2025

ProviderHuggingFace

Quick Start