Microsoft Copilot Studio

Documentation Index

Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt

Use this file to discover all available pages before exploring further.

​

How it works

• Parses conversations into structured trace data
• Extracts LLM calls, tool executions, and plan steps
• Captures latency, token usage, and cost estimates
• Logs RAG citations and retrieved documents
• Auto-creates projects and pipelines per bot

​

Prerequisites

​

Enable enhanced transcripts

1. In Copilot Studio, navigate to Settings → Advanced
2. Expand the Enhance Transcripts section
3. Toggle on Include node-level details in transcripts
4. Click Save

​

Register an Azure AD app

1. In the Azure Portal, go to Microsoft Entra ID → App registrations → New registration.
2. Give the app a name (e.g., Openlayer Copilot Studio) and register it.
3. Under Certificates & secrets, create a new Client secret and copy its value.
4. Under API permissions, add the following Application permission:
   • Dataverse → user_impersonation
5. Grant admin consent for the permission.

​

Create a custom security role in Power Platform

​

Step 1: Create the security role

1. Go to the Power Platform admin center → Environments → select your environment → Settings.
2. Under Users + permissions, click Security roles.
3. Click New role and give it a name (e.g., Openlayer Dataverse Reader).
4. On the Custom Entities tab, find the following tables and set Read access to Organization (the globe icon) for each:

5. Click Save and Close.

​

Step 2: Create an application user

1. In the Power Platform admin center, go to Environments → your environment → Settings.
2. Under Users + permissions, click Application users.
3. Click New app user and select the Azure AD app you registered earlier.
4. Assign the Business unit for your environment.
5. Under Security roles, add the custom role you just created (e.g., Openlayer Dataverse Reader).
6. Click Create.

​

Integration options

​

Option 1: Native integration (Recommended)

​

Step 1: Connect your environment

1. In Openlayer, navigate to Settings → Integrations.
2. Click Microsoft Copilot Studio.
3. Fill in your Azure AD credentials:
   • Tenant ID
   • Client ID
   • Client secret
   • Environment URL (e.g. https://yourorg.crm.dynamics.com)
4. Click Connect.

​

Step 2: Discover your agents

• Name and schema name
• Status in Dataverse (active or inactive)
• Monitoring status (enabled, disabled, or error)

​

Step 3: Enable monitoring

• Auto-create a new project — Openlayer creates a project named Copilot Studio - <agent name> with a default data source. This is the fastest way to get started.
• Map to an existing project — Choose an existing Openlayer project and data source. Use this when you already have a project set up for the agent.

​

Step 4: Configure sync settings

• Sync enabled — Toggle periodic syncing on or off.
• Sync frequency — How often Openlayer checks for new transcripts (default: every 60 minutes).
• Sync range — Choose between:
  • All available data — Sync all historical transcripts.
  • Last 7 days — Only sync recent conversations.
  • Custom date — Specify a start date.

​

Backfilling historical data

• All available data — Re-process every transcript for this agent.
• Since a specific date — Only re-process transcripts from the given date onward.

​

Option 2: Azure Logic App (Deprecated)

​

Architecture overview

1. User interacts with Copilot Studio workflows/agents
2. Copilot Studio stores session data in the ConversationTranscript table in Dataverse
3. A Logic App triggers on Add/Update/Delete events in the ConversationTranscript table
4. The Logic App’s HTTP action sends the transcript data to Openlayer’s REST API

​

Step 1: Create the Logic App

1. In the Azure Portal, create a new Logic App (Consumption).
2. In the Logic App designer, search for Dataverse in the triggers.
3. Select the When a row is added, modified or deleted trigger.
4. Configure the trigger:
   • Table name: ConversationTranscript
   • Scope: Organization
   • Filter rows: (Optional) Add filters to exclude test/design mode conversations

​

Step 2: Configure the HTTP action

• Method: POST
• URI: https://api.openlayer.com/copilot-studio/sessions
• Headers:
  • Authorization: Bearer <YOUR_OPENLAYER_API_KEY>
  • Content-Type: application/json
• Body: @{triggerBody()}

​

Step 3: (Optional) Add error handling

• Send alerts to a Teams channel or email on failure
• Use a dead-letter queue for failed transcripts

​

Automatic project creation

​

Option 3: Batch integration via Python (Deprecated)

​

Prerequisites

pip install PowerPlatform-Dataverse-Client azure-identity requests

​

Example script

"""Sync Copilot Studio transcripts to Openlayer."""

import os
import requests
from datetime import datetime, timedelta
from PowerPlatform.Dataverse.client import DataverseClient
from azure.identity import InteractiveBrowserCredential

# Configuration
DATAVERSE_URL = os.environ["DATAVERSE_URL"]  # e.g., https://yourorg.crm.dynamics.com
OPENLAYER_API_KEY = os.environ["OPENLAYER_API_KEY"]

# Initialize Dataverse client with authentication
credential = InteractiveBrowserCredential()
client = DataverseClient(DATAVERSE_URL, credential)

# Query recent transcripts from Dataverse using SQL
# Get transcripts modified in the last 24 hours
yesterday = (datetime.utcnow() - timedelta(days=1)).strftime("%Y-%m-%d %H:%M:%S")

query = f"""
SELECT TOP 100
    conversationtranscriptid,
    name,
    content,
    metadata,
    conversationstarttime,
    createdon,
    modifiedon,
    schemaversion,
    schematype,
    statecode,
    statuscode
FROM conversationtranscript
WHERE modifiedon > '{yesterday}'
    AND statecode = 0
ORDER BY modifiedon DESC
"""

transcripts = client.query_sql(query)

# Send each transcript to Openlayer
for transcript in transcripts:
    resp = requests.post(
        "https://api.openlayer.com/copilot-studio/sessions",
        headers={
            "Authorization": f"Bearer {OPENLAYER_API_KEY}",
            "Content-Type": "application/json",
        },
        json=transcript,
    )

    if resp.ok:
        result = resp.json()
        print(f"Processed transcript {transcript['name']}: "
              f"{result['requestsProcessed']} requests")
    else:
        print(f"Error processing {transcript['name']}: "
              f"{resp.status_code} - {resp.text}")

​

Use cases for batch integration

• Nightly or hourly jobs that backfill conversations
• Historical analysis with new evaluation pipelines
• Environments where Logic Apps are restricted

​

Finding your bot name and tenant ID

1. In Copilot Studio, navigate to Settings → Advanced
2. Expand the Metadata section
3. Copy the Schema name (this is your bot name)
4. Copy the Tenant ID

​

Monitoring in Openlayer

​

View conversation traces

• User queries and bot responses
• Latency and token usage per turn
• Nested execution steps (LLM calls, tool executions, plan steps)
• RAG citations and knowledge source references

​

Analyze RAG quality

• View retrieved citations per response
• Evaluate relevance of retrieved documents
• Track knowledge source utilization

​

Monitor session outcomes

• Resolution rates (session_outcome)
• CSAT scores
• Turn counts and engagement

​

Run evaluations

• Score response quality
• Detect hallucinations
• Measure citation accuracy
• Track safety and compliance