Microsoft’s Azure AI Studio lets developers build their own AI ‘copilots’

Microsoft wants companies to build their own AI-powered “copilots” — using tools on Azure and machine learning models from its close partner OpenAI, of course.

Today at its annual Build conference, Microsoft launched Azure AI Studio, a new capability within the Azure OpenAI Service that lets customers combine a model like OpenAI’s ChatGPT or GPT-4 with their own data — whether text or images — and build a chat assistant or another type of app that “reasons over” the private data. (Recall that Azure OpenAI Service is Microsoft’s fully managed, enterprise-focused product designed to give businesses access to AI lab OpenAI’s technologies with added governance features.)

Microsoft defines a “copilot” as a chatbot app that uses AI, typically text-generating or image-generating AI, to assist with tasks like writing a sales pitch or generating images for a presentation. The company has created several such apps, such as Bing Chat. But its AI-powered copilots can’t necessarily draw on a company’s proprietary data to perform tasks — unlike copilots created through Azure AI Studio.

“In our Azure AI Studio, we’re making it easy for developers to ground Azure OpenAI Service models on their data … and do that securely without seeing that data or having to train a model on the data.” John Montgomery, Microsoft’s CVP of AI platform, told TechCrunch via email. “It’s a tremendous accelerant for our customers to be able to build their own copilots.”

In Azure AI Studio, the copilot-building process starts with selecting a generative AI model like GPT-4. The next step is giving the copilot a “meta-prompt,” or a base description of the copilot’s role and how it should function.

Cloud-based storage can be added to AI copilots created with Azure AI Studio for the purposes of keeping track of a conversation with a user and responding with the appropriate context and awareness. Plug-ins extend copilots, giving them access to third-party data and other services.

Image Credits: Microsoft

Microsoft believes the value proposition in Azure AI Studio is allowing customers to leverage OpenAI’s models on their own data, in compliance with their organizational policies and access rights and without compromising things like security, data policies or document ranking. Customers can choose to integrate internal or external data that their organization owns or has access to, including structured, unstructured or semi-structured data.

With Azure AI Studio, Microsoft’s making a push for customized models built using its cloud-hosted tooling. It’s a potentially lucrative line of revenue as the Azure OpenAI Service continues to grow — Microsoft says that it’s currently serving more than 4,500 companies, including Coursera, Grammarly, Volvo and IKEA.

Upgrades to Azure OpenAI Service

To further incentivize Azure OpenAI Service adoption, Microsoft’s rolling out updates aimed at boosting capacity for high-volume customers.

A new feature called the Provisioned Throughput SKU allows Azure OpenAI Service customers to reserve and deploy model processing capacity on a monthly or yearly basis. Customers can purchase “provisioned throughput units,” or PTUs, to deploy OpenAI, models including GPT-3.5-Turbo or GPT-4 with reserved processing capacity during the commitment period.

OpenAI previously offered dedicated capacity for ChatGPT via its API. But Provisioned Throughput SKU greatly expands on this — and with a bent toward the enterprise.

“With reserved processing capacity, customers can expect consistent latency and throughput for workloads with consistent characteristics such as prompt size, completion size and number of concurrent API requests,” a Microsoft spokesperson told TechCrunch via email.

Microsoft’s Azure AI Studio lets developers build their own AI ‘copilots’ by Kyle Wiggers originally published on TechCrunch