Monitoring your Azure OpenAI usage


Monitoring your Azure OpenAI usage is an important practice to ensure efficient and cost-effective usage. Azure OpenAI uses Azure Monitor to provide metrics and insights into AI usage. The Azure OpenAI Metrics Dashboard gives you information on HTTP requests, tokens-based usage, PTU utilization, and fine-tuning. These dashboards help you understand how users are sending AI requests and identify usage peaks and heavy load on deployments and models. Additionally, you can analyze metrics such as processed prompt tokens, generated completion tokens, and processed inference tokens, which impact AI costs. For more advanced monitoring, you can export metrics and log data to Azure Monitor's Log Analytics workspace. Overall, monitoring your Azure OpenAI usage helps control anomalies and costs.


Article 13m

Login now to access my digest by 365.Training

Learn how my digest works
Features
  • Articles, blogs, podcasts, training, and videos
  • Quick read TL;DRs for each item
  • Advanced filtering to prioritize what you care about
  • Quick views to isolate what you are looking for right now
  • Save your favorite items
  • Share your favorites
  • Snooze items you want to revisit when you have more time