AI-300 Microsoft Certified Machine Learning Operations Engineer Associate

This course prepares you for the AI-300 certification exam, covering all official skills measured and their approximate weightings:

Domain 1 Design and implement an MLOps infrastructure (15-20%)

• Create and manage a Machine Learning workspace, datastores and compute targets, and configure identity and access management
• Create and manage data assets, environments and components, and share assets across workspaces using registries
• Configure GitHub integration, and deploy Machine Learning workspaces and resources using Bicep and Azure CLI
• Automate resource provisioning with GitHub Actions workflows and restrict network access to workspaces
• Manage source control for machine learning projects using Git

Domain 2 Implement machine learning model lifecycle and operations (25-30%)

• Orchestrate model training: configure experiment tracking with MLflow, use automated machine learning and notebooks, and automate hyperparameter tuning
• Run training scripts, manage distributed training, implement training pipelines, and compare model performance across jobs
• Register and version models, package feature retrieval specifications, and evaluate models using responsible AI principles
• Deploy models as real-time or batch endpoints, test and troubleshoot endpoints, and implement progressive rollout and safe rollback
• Monitor production models: detect data drift, track performance metrics, and configure retraining or alert triggers

Domain 3 Design and implement a GenAIOps infrastructure (20-25%)

• Create and configure Foundry resources and project environments, and configure identity and access management with managed identities and RBAC
• Implement network security and private networking, and deploy infrastructure using Bicep templates and Azure CLI
• Deploy foundation models with serverless API endpoints and managed compute, and select appropriate models for use cases
• Implement model versioning and production deployment strategies, and configure provisioned throughput units for high-volume workloads
• Design and develop prompts, create prompt variants, and implement version control for prompts using Git repositories

Domain 4 Implement generative AI quality assurance and observability (10-15%)

• Create test datasets and data mapping, and implement AI quality metrics including groundedness, relevance, coherence and fluency
• Configure risk and safety evaluations for harmful content detection, and set up automated evaluation workflows
• Monitor performance metrics including latency, throughput and response times using continuous monitoring in Foundry
• Track and optimize cost metrics including token consumption and resource usage
• Configure detailed logging, tracing and debugging for production troubleshooting

Domain 5 Optimize generative AI systems and model performance (10-15%)

• Optimize retrieval-augmented generation (RAG) by tuning similarity thresholds, chunk sizes and retrieval strategies
• Select and fine-tune embedding models for domain-specific accuracy, and implement hybrid semantic and keyword search
• Evaluate and improve RAG performance using relevance metrics and A/B testing
• Design and implement advanced fine-tuning methods, and create and manage synthetic data for fine-tuning
• Monitor and optimize fine-tuned models from development through production deployment