Monitoring your Azure OpenAI usage is an important practice to ensure efficient and cost-effective usage. Azure OpenAI uses Azure Monitor to provide metrics and insights into AI usage. The Azure OpenAI Metrics Dashboard gives you information on HTTP requests, tokens-based usage, PTU utilization, and fine-tuning. These dashboards help you understand how users are sending AI requests and identify usage peaks and heavy load on deployments and models. Additionally, you can analyze metrics such as processed prompt tokens, generated completion tokens, and processed inference tokens, which impact AI costs. For more advanced monitoring, you can export metrics and log data to Azure Monitor's Log Analytics workspace. Overall, monitoring your Azure OpenAI usage helps control anomalies and costs.
Login now to access my digest by 365.Training