Skip to main content
Openlayer connects to the Microsoft Teams Export API and pulls the conversations users have with the bots and agents in your Teams app catalog into your workspace as inference data. This is the right integration when the AI you need to evaluate is a bot your users chat with in Teams: one your organization built and published, a custom app, or an app installed from the Teams Store. You pick which bots to monitor, and each bot’s conversations land in its own project, where you can run tests and review real usage. It needs Microsoft Teams licenses only. No Microsoft 365 Copilot, E5, or Purview DLP licenses are required, and there is no billing to set up: Microsoft stopped metering the Teams APIs on August 25, 2025 (Metered APIs and services in Microsoft Graph). To evaluate Microsoft 365 Copilot itself, use Microsoft 365 Copilot instead.
All access is read-only. Openlayer reads chat messages, users, and the Teams app catalog, and never writes to, modifies, or deletes anything in your Microsoft 365 tenant.

​
How it works

  1. Lists your bots. Connecting reads every app with a bot in your tenant’s Teams app catalog, including Teams Store apps, into the Bots tab. Every bot starts disabled, so nothing is ingested until you opt in.
  2. Maps each bot to one project. Enabling a bot creates an Openlayer project for it, or writes into a project and pipeline you already have.
  3. Pulls on a schedule. Periodic sync reads the Teams chats of the users in your tenant and keeps only one-on-one chats with the bots you enabled. You can also run a sync immediately, or backfill history.

​
Prerequisites

  • Microsoft Teams licenses for the users who chat with your bots.
  • The bots you want to monitor in your tenant’s Teams app catalog.
  • An Entra ID app registration with a client secret, the application permissions below, and admin consent.
  • An Openlayer workspace where you are an admin. Connecting, enabling bots, and disconnecting are admin-only.

​
Register the Entra app

Register a dedicated app for this connector.
  1. In the Azure portal, go to Microsoft Entra ID → App registrations → New registration.
  2. Name the app (for example, Openlayer Microsoft Teams bots) and register it.
  3. Under Certificates & secrets, create a Client secret and copy its Value. You cannot read it again later.
  4. Under API permissions → Add a permission → Microsoft Graph → Application permissions, add the three permissions in the table below.
  5. Select Grant admin consent for the tenant.
Note the Tenant ID and Client ID on the app’s Overview page, and the client secret you just created. All three are application permissions and require admin consent. A delegated permission is not a substitute.
China (21Vianet) is not supported. Connect rejects that cloud. On Microsoft cloud, choose Global, US Government (L4), or US Government DoD (L5).

​
Setup guide

​
Step 1: Connect

In Openlayer, go to Settings → Integrations, open Microsoft Teams bots, enter Tenant ID, Client ID, Client secret, and Microsoft cloud, then click Connect. Openlayer checks the credentials, the permissions, and access to the export API against Microsoft Graph before storing them, then reads your Teams app catalog. A large catalog can take a few seconds.

​
Step 2: Set the sync schedule

On the General tab, Periodic sync pulls new conversations every 60 minutes by default. Turn it off to pause ingestion without disconnecting, or click Sync now to queue a sync immediately. Initial sync range is how far back a newly enabled bot’s history is read: Last 7 days (the default), All available data, or a Custom start date.

​
Bots

The Bots tab lists every app with a bot in your Teams app catalog. Store apps are included, so a tenant can list thousands. Use the search box and the Enabled/Disabled filter to find yours. After you publish a bot to the catalog, click the refresh button to read the catalog again. Turn on a bot’s Enabled switch and choose where its conversations go:
  • Create new project (the default) — Openlayer creates a project named Teams - <bot name> and a pipeline for that bot
  • Map to existing project — pick an existing project and pipeline instead
Enabling a bot re-reads Teams history from the initial sync range, so its past conversations are imported on the next sync. Re-enabling a bot you disabled reuses its previous project and pipeline if they still exist. An enabled row’s menu has View project, Backfill, and Disable. Disabling a bot stops new imports for it and keeps its project and the rows already written.

​
Backfill

Backfill re-reads history for every enabled bot, not only the row you opened it from, starting from the initial sync range or a custom start date. Rows already stored are skipped, so running it again does not create duplicate records.

​
What lands in Openlayer

Openlayer imports one-on-one chats between a user and an enabled bot. Each user message and the bot replies that follow it become one inference row: The user id is the user’s email in lowercase, taken from the Entra mail attribute. When mail is empty, Openlayer uses the user principal name, and when neither is set, the Entra object id. The object id is also kept in each row’s metadata as aadUserId, so it stays stable if a user’s email changes. New rows go to the new address. Bot replies sent as Adaptive Cards are imported as text. A bot message with no user message before it, such as a welcome message, is imported with an empty input. The latest exchange in a chat is imported once the user sends their next message, or on the first sync at least 10 minutes after it. This lets Openlayer wait for every reply before writing the row. Openlayer stores only conversations with the bots you enable. Messages in chats without a bot, or with a bot you have not enabled, are discarded and never stored. Metadata keeps identifiers and types only.

​
Evaluating the ingested data

Once conversations are flowing, evaluate your Teams bots the same way you evaluate application traffic:

​
Disconnecting

On the General tab, click Disconnect Microsoft Teams bots and confirm. Disconnecting stops syncing conversations. Projects, pipelines, and records already ingested are kept. In Entra, remove admin consent or delete the client secret if that app should no longer read Teams chats.

​
Limitations

  • One-on-one chats only. Group and meeting chats with a bot are not imported.
  • Transcript only. Openlayer reads what the user and the bot said. Tool calls, retrieval steps, model names, and token usage happen inside the bot and are not available from the Teams Export API.
  • Scheduled, not real time. Conversations arrive on the next sync, and the latest exchange in a chat waits as described above.
  • Up to 200 users per sync. Each sync reads the chats of up to 200 users with enabled accounts and rotates through the rest, so in a large tenant each user’s chats are read less often.
  • No China (21Vianet). Only the Global, US Government (L4), and US Government DoD (L5) clouds are supported.

​
Troubleshooting

​
Fix a 402 error

Microsoft no longer meters the Teams APIs, but a tenant can still answer 402 Payment Required when the Entra app has no Azure subscription associated with it. Associating one does not incur charges for the Teams APIs. You need Contributor on an Azure subscription in the same tenant and owner of the app registration. Metered-API billing exists only in the Global cloud. Run Microsoft’s association command in Azure Cloud Shell or the Azure CLI, then click Sync now:
<client-id> is the Entra app’s Client ID, and <resource-group> is an existing resource group in that subscription.