Microsoft 365 Copilot

Openlayer connects to Microsoft Graph’s Copilot interaction export API and pulls first-party Microsoft 365 Copilot prompts and responses into your workspace as inference data. This is the right integration when the AI you need to evaluate is Microsoft 365 Copilot itself — Chat, the web, Teams, Word, and Excel — rather than a custom agent you built. Each Copilot request lands in Openlayer, where you can run tests and review real usage. Custom agents belong on a different connector. Microsoft Copilot Studio reads Dataverse ConversationTranscript records with an app consented for Dataverse. This connector is separate: it uses Microsoft Graph app-only (https://graph.microsoft.com/.default) with AiEnterpriseInteraction.Read.All. Microsoft’s export API does not return Copilot Studio agent interactions. Do not share one Entra app between the two connectors. The token audiences and consent scopes are different, and the wrong app returns the wrong data.

​

How it works

Once connected, Openlayer reads each licensed user’s Copilot interaction history and turns the surfaces you enable into traces.

1. Seeds known surfaces. Microsoft 365 Copilot Chat, Copilot Web Chat, Copilot in Teams, Copilot in Word, and Copilot in Excel appear under Surfaces. Every surface starts disabled, so nothing is ingested until you opt in.
2. Leaves other apps off. Outlook, Loop, and PowerPoint are not covered. An appClass Openlayer does not recognize yet shows up after a sync and stays disabled.
3. Maps each surface to one project. Enabling a surface creates an Openlayer project named for that surface, or writes into a project and pipeline you already have. Enabled surfaces show the Microsoft 365 Copilot icon on the project and appear under configured integrations.
4. Pulls on a schedule. Periodic sync brings in new interactions for users who hold the Microsoft Copilot with Graph-grounded chat service plan. You can also run a sync immediately, or backfill history for the surfaces you have enabled.

---

​

Prerequisites

• A Microsoft 365 tenant with users licensed for Microsoft 365 Copilot, including the Microsoft Copilot with Graph-grounded chat service plan. A tenant with no one on that plan connects, then imports nothing.
• An Entra ID app registration with a client secret, the application permissions below, and admin consent.
• An Openlayer workspace where you are an admin. Connecting, enabling surfaces, and disconnecting are admin-only.

​

Register the Entra app

Register a dedicated app for this connector. Do not reuse the app you registered for Microsoft Copilot Studio.

1. In the Azure portal, go to Microsoft Entra ID → App registrations → New registration.
2. Name the app (for example, Openlayer Microsoft 365 Copilot) and register it.
3. Under Certificates & secrets, create a Client secret and copy its value. You cannot read it again later.
4. Under API permissions → Add a permission → Microsoft Graph → Application permissions, add both permissions in the table below.
5. Select Grant admin consent for the tenant.

Note the Tenant ID and Client ID on the app’s Overview page, and the client secret you just created. Both are application permissions and both require admin consent. A delegated permission is not a substitute. Openlayer requests a Graph app-only token (https://graph.microsoft.com/.default).

---

​

Setup guide

​

Step 1: Open the integration

In Openlayer, go to Settings → Integrations and open Microsoft 365 Copilot.

​

Step 2: Connect

Enter Tenant ID, Client ID, Client secret, and Microsoft cloud, then click Connect. Openlayer checks the credentials against Microsoft Graph before storing them. The Microsoft API reference link on the form opens Microsoft’s interaction-export docs. On success, the detail page opens on the connected tenant.

​

Step 3: Set the sync schedule

Open the General tab. Periodic sync pulls new interactions on the connection’s schedule. The interval shown is every 60 minutes by default. Openlayer checks for a due sync every 15 minutes, and the interval stays between 15 minutes and 24 hours. Turn periodic sync off to pause ingestion without disconnecting, or click Sync now to queue a sync immediately. Initial sync range is how far back the first sync reads:

• Last 7 days — the default
• All available data
• Custom start date — a date you pick

The same tab reports Tenant ID, Cloud, Last sync, and Error when the last sync failed.

---

​

Surfaces

Connecting registers the known surfaces under Configure surfaces. None of them ingest until you turn one on. Outlook, Loop, and PowerPoint are not in this set. When a sync sees an appClass outside it, that row appears in the table under the raw appClass and stays disabled. Turn on a surface’s Enabled switch and choose where its rows go:

• Create new project (the default) — Openlayer creates a project named M365 Copilot - <surface name> and a pipeline for that surface
• Map to existing project — pick an existing project and pipeline instead

An enabled row’s menu has View project, Backfill, and Disable. Disabling a surface stops new imports for it and keeps rows already written.

​

Backfill

Backfill re-reads history for every enabled surface, not only the row you opened it from. Graph returns every Copilot app in one feed per user, so Openlayer rewinds the connection and reads that feed again. Rows already stored are skipped. Openlayer dedupes on the Copilot requestId, so running backfill again does not create duplicate records. The dialog offers:

• Initial sync range — re-fetch from the start date on the General tab
• Custom start date — re-fetch from a date you pick

Then click Start backfill.

---

​

What lands in Openlayer

Each Copilot requestId becomes one inference row, and that requestId is the record id. Turns that share a Copilot sessionId share one Openlayer session. The user id is the person’s Entra object id (from.user.id). A response with no prompt — a Teams meeting recap, for example — is attributed to the licensed user whose history was pulled, not to anonymous. When the interaction has the data, the trace is:

• A root user call named Microsoft 365 Copilot
• An optional retriever step named Referenced content, from contexts, links, and URL attachments
• A chat completion named for the Copilot app that answered (for example, Microsoft 365 Chat)

Latency on that completion is the time from the prompt’s createdDateTime to the response’s. Microsoft Graph does not return a model name, token counts, or cost, so those fields stay empty. Referenced content and URLs can appear on the row’s context. Metadata keeps identifiers and types only: not display names, mention text, or file names.

---

​

Evaluating the ingested data

Once traces are flowing, evaluate Microsoft 365 Copilot usage the same way you evaluate application traffic:

• Create tests to score response quality, safety, or tone
• Detect PII or toxicity in prompts and replies
• Use governance frameworks to evidence AI-usage controls with real traffic

---

​

Disconnecting

On the General tab, click Disconnect Microsoft 365 Copilot and confirm Remove Microsoft 365 Copilot. Disconnecting stops syncing new interactions. Projects, pipelines, and traces already ingested are kept. In Entra, remove admin consent or delete the client secret if that app should no longer read Copilot interaction history.

---

​

Troubleshooting