Multi Models in Foundry

Many AI engineers I collaborate with seek the ability to easily swap out a large language model (LLM) with minimal code modifications. Different models excel in various tasks. For instance, Claude-opus-4.6 currently leads in deep reasoning coding tasks, while Sonnet may perform better for quick code refactoring and Q&A.

This serves as a fundamental requirement for establishing a flexible AIOps infrastructure.

Luckily, Microsoft Foundry offers more than 11,000 different large language models - Claude by Anthropic, OpenAI, xAI, Mistral and many more.

Within an Azure AI Foundry project, you can integrate various models from different sources. You might deploy GPT-5.2 for complex reasoning tasks, use Claude for detailed analysis and coding, incorporate specialized vision models for image processing, and add domain-specific models for particular use cases. All these models exist within the same project workspace, sharing common resources like connections, data assets, agents and tools.

Project API endpoint:

Using GPT-4o-mini via AIProjectClient:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Connect to your Foundry project
project = AIProjectClient(endpoint="https://<your-project>.services.ai.azure.com/api/projects/<your-project>",           credential=DefaultAzureCredential()
)

# Get OpenAI client scoped to your project
openai_client = project.get_openai_client()
response = openai_client.responses.create(   
 model="gpt-4o-mini",  # deployment name in your Foundry project    
 input="What is Azure AI Foundry?"
)

print(response.output_text)

Switching to grok-4 via AIProjectClient:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Connect to your Foundry project
project = AIProjectClient(endpoint="https://<your-project>.services.ai.azure.com/api/projects/<your-project>",           credential=DefaultAzureCredential()
)

# Get OpenAI client scoped to your project
openai_client = project.get_openai_client()
response = openai_client.responses.create( model="grok-4",  # only model name changes    
input="What is Azure AI Foundry?")

print(response.output_text)

Microsoft Foundry hosts more than 11,000 models

So, how can I determine what features a model supports, such as reasoning_summary or streaming? To find this out, I'll utilize the compare models tab to gain a clearer understanding of the available features before I begin coding (or Claude to be more accurate :) ).

As you start deploying your chosen model, keep in mind the importance of monitoring its performance. Implementing a feedback loop can significantly enhance your model's accuracy and relevance over time. By analyzing user interactions and outcomes, you'll be able to make informed adjustments, ensuring that your AI solution continues to meet evolving demands and delivers exceptional results.

I'll discover this in more depth in my next blog post - creating an evaluation dataset for you AI workflow - stay tuned !