1. Request access through the model page on Hugging Face and wait for approval from the model authors.
2. Create a Custom keys workspace connection in Microsoft Foundry or Azure Machine Learning named HuggingFaceTokenConnection with the key HF_TOKEN and value your Hugging Face read or fine-grained token (marked as secret).
3. Create the Managed Online Endpoint with the property enforce_access_to_default_secret_stores set to enabled so it can access the secret connection value.
4. Once access is approved, the connection is configured, and the endpoint is created with read access to the token, you can deploy and use the model in Microsoft Foundry or Azure Machine Learning.

• Original Model Card
• vLLM Documentation

Chat Completions API

Send Request

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"model":"azure-testing/random-gated-chat-completion","messages":[{"role":"user","content":"What is Deep Learning?"}]}'

Supported Parameters

• model (string): Model ID used to generate the response, in this case since only a single model is deployed within the same endpoint you can either set it to azure-testing/random-gated-chat-completion or leave it blank instead.
• messages (array): A list of messages comprising the conversation so far. Depending on the model you use, different message types (modalities) are supported, like text, images, and audio.

Example payload

{
  "model": "azure-testing/random-gated-chat-completion",
  "messages": [
    {"role":"user","content":"What is Deep Learning?"}
  ],
  "max_completion_tokens": 256,
  "temperature": 0.6
}

Responses API

Send Request

curl <AZUREML_ENDPOINT_URL>/v1/responses \
    -X POST \
    -d '{"model":"azure-testing/random-gated-chat-completion","input":"What is Deep Learning?","reasoning":{"effort":"medium"}}' \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json"

Supported Parameters

• model (string): Model ID used to generate the response, in this case since only a single model is deployed within the same endpoint you can either set it to azure-testing/random-gated-chat-completion or leave it blank instead.
• input (str or array): A text, image, or file inputs to the model, or even a list of messages comprising the conversation so far, used to generate the response. Depending on the model you use, different message types (modalities) are supported, like text, images, and audio; whilst in this case only text generation is supported so image and audio inputs are disallowed.

Example Payload

{
  "model": "azure-testing/random-gated-chat-completion",
  "input": "What is Deep Learning?",
  "max_output_tokens": 1024,
  "temperature": 0.6,
  "reasoning": {
    "effort": "medium"
  }
}