NemoClaw Knowledge Wiki

Using Microsoft Foundry

3 min read

https://www.youtube.com/watch?v=C6rxEGJay70 Here is a summary of the Microsoft Mechanics video on the newly expanded Microsoft Foundry.

Microsoft Foundry: A Unified AI App and Agent Factory

Microsoft Foundry is a new platform designed to help developers build AI apps and agents faster and with more control. It serves as a unified interface that integrates directly with code, offering rich tooling, observability, and governance.

Key Features & Capabilities

1. Discovery and Onboarding

  • Unified Portal: Accessible at ai.azure.com, the platform guides users through creating agents, designing workflows, or browsing models.
  • Model Catalog: Features a massive catalog of models from providers like Azure OpenAI, Mistral AI, Meta, DeepSeek, and for the first time, Anthropic.
  • Model Router: A new tool that automatically routes prompts to the most efficient models in real-time to balance quality and cost.
  • Solution Templates: Provides pre-built, customizable code samples with GitHub-hosted quick-start guides to accelerate development.

2. The Build Experience

  • Agent Framework: Supports creating single agents or complex multi-agent workflows (e.g., sequential, group chat, or human-in-the-loop topologies).
  • Visualizer & Editor: A drag-and-drop interface allows developers to define logic (if/else conditions, loops) and connect various agents visually.
  • Knowledge Integration: Connects agents to multiple data sources via Foundry IQ, including SharePoint Online, Microsoft Fabric (OneLake), and Azure AI Search.
  • Tools & MCP: Supports standard tools and Model Context Protocol (MCP) servers to connect agents to external APIs and databases.
  • Fine-tuning: Offers options for supervised learning, Direct Preference Optimization (DPO), and reinforcement techniques.

3. Developer Integration (Code-First)

  • VS Code Sync: Everything built in the web portal can be managed via code. Developers can use the Microsoft Foundry SDK and VS Code extension to edit workflows, logic, and prompts locally, which then syncs back to the platform.
  • Multimodal Support: Agents support various input types, including voice capabilities for mobile applications.

4. Evaluation and Optimization

  • Synthetic Data: Users can generate synthetic datasets to stress-test and fine-tune agents.
  • Tracing & Metrics: The platform provides detailed traces of agent runs to diagnose latency bottlenecks and evaluate performance.
  • Scoring: Automated evaluations provide scores for AI quality, safety, coherence, and groundedness.

5. Operations and Governance (The “Operate” Tab)

  • Control Plane: A dashboard providing a cross-fleet view of all agents, including alerts, estimated costs, token usage, and success rates.
  • Guardrails: Allows IT to apply safety controls (e.g., blocking jailbreaks, content filters) to specific models or agents.
  • Compliance & Quotas: Admins can set policies across subscriptions and manage token limits to control costs.
  • Publishing: Once ready, agents can be published directly to Microsoft Teams and Microsoft 365 Copilot for business users to access via the Agent Store.

Graph View