Skip to main content
Openlayer evaluates the clinical AI built into Epic using your organization’s own data. It covers Epic’s generative workflows, such as Art, Ask Art, and Agent Factory, as well as Epic’s predictive models, findings extraction, and patient-flow predictions. Each workflow or model gets an Openlayer project where you can run tests and track it over time. Epic does not offer an API for AI telemetry. Instead, your Epic report writer builds a scheduled extract of the relevant Epic tables and delivers it to Openlayer. Openlayer imports each extract in the background and reads outcomes, such as the actual length of a stay, from Epic’s FHIR R4 API.
All access is read-only. Openlayer reads extracts you deliver and the FHIR resources you grant. It never writes anything back to Epic.

How it works

The integration covers four Epic AI families. Each one comes from its own extract and lands in Openlayer differently: Findings extraction is the only family whose extract includes its own ground truth: the findings a radiologist verified. The other families need outcomes, which Openlayer reads from FHIR once they are known. A no-show prediction can only be scored after the appointment has passed.

Prerequisites

You need the following before you connect. Most of these are set up by your Epic team.
  • An Epic backend app for Openlayer. Epic backend apps have no client secret. Openlayer signs each token request with a private key, and Epic checks it against the public key registered on the app.
  • A client ID for each environment. Epic issues separate non-production and production client IDs for the same app, so each environment is its own connection in Openlayer.
  • The Interconnect token URL and FHIR R4 base URL for that environment.
  • A decision on how Epic gets Openlayer’s public key. See Choose how Epic gets the public key.
  • The FHIR scopes Openlayer needs, granted on the app. See Choose what Openlayer may read.
  • A scheduled extract from your Epic report writer for each family you want to evaluate. See The extract.
  • An Openlayer workspace where you are an admin. Connecting and configuring Epic are admin-only actions.
On Openlayer’s cloud, the token URL and FHIR base URL must resolve to public addresses. If your Interconnect is only reachable on a hospital-internal hostname, use a self-hosted Openlayer that runs inside your network.

Choose how Epic gets the public key

Openlayer generates a separate keypair for each connection. The private key never leaves Openlayer. Epic gets the public key in one of two ways:
  • Epic fetches a JWKS URL (JKU). Epic reads the public key from a URL you register on the app. The URL must serve the JWK Set Openlayer shows after you connect, over TLS and without authentication, and it must be reachable from Epic’s own network. On a self-hosted Openlayer, that is usually a URL your organization hosts. Prefer this mode when your network allows it, because rotating the key does not require changing the Epic app record.
  • Upload the key into Epic (static JWK). Your Epic team uploads the public key into the app record. Use this when nothing Openlayer runs is reachable from Epic. It needs an Epic release that supports uploading a key into the app record.

Choose what Openlayer may read

Openlayer asks Epic for capabilities, not individual scopes. Each one is a separate request to your Epic security team, so grant only what the workflows and models you evaluate need. For example, scoring a length-of-stay prediction needs encounter outcomes and nothing from the chart. A new connection starts with clinical notes only.

The extract

Your Epic report writer produces a versioned bundle on a schedule, usually nightly, and delivers it to Openlayer. Work out the delivery channel with your Openlayer contact. Each bundle is a manifest.json plus one CSV, TSV, or JSONL file per table. The manifest’s schemaVersion tells Openlayer which family the bundle feeds, and each family has its own extract contract. The tables come from Epic’s public EHI table catalog, so your report writer can recognize them: For generative workflows, each bundle covers one workflow and one time window. The manifest looks like this:
manifest.json
windowEnd is exclusive. files maps each table name to its file in the bundle. Openlayer rejects a manifest with an unknown schemaVersion or a table name it does not recognize. Only AI_INTRCT_INFO is required for generative workflows, but each table you leave out removes something from the trace. Without AI_INTRCT_AGENT_RESPONSES there is no output to evaluate, and those interactions are not imported. Without PROMPT_EXECUTION_STATE the trace has no intermediate steps. Re-importing an overlapping window does not create duplicates, because each row is keyed by Epic’s own interaction or record ID.

Set up the integration

Step 1: Connect

In Openlayer, go to Settings → Integrations and open Epic. Under Connect Epic, choose the Environment, enter the Interconnect token URL, FHIR R4 base URL, and Client id for that environment, and choose How Epic gets our public key. For JKU, also enter the JWK Set URL you register with Epic. Epic release is optional and is recorded on every imported row. Use the token URL exactly as Epic publishes it. Epic compares it byte for byte when it checks the token request. Clinical text handling sets what Openlayer may do with chart text. See Clinical text. Click Connect. The connection is saved with the status pending.

Step 2: Register the public key and test

On the General tab, Register the public key with Epic shows the key ID and the JWK Set. Click Copy JWK Set and give it to your Epic team:
  • For JKU, publish the JWK Set at the JWK Set URL you entered, and register that URL on the Epic app for this environment.
  • For static JWK, upload the public key into the Epic app record.
Then click Test connection. Openlayer requests a token from Epic and reads the FHIR server’s capability statement. When Epic accepts the key, the connection becomes active.
A failed test is expected until Epic can read the public key. The connection stays pending in the meantime, which is not an error.

Step 3: Grant capabilities

Under What Openlayer may read, turn on the capabilities your workflows and models need, then have your Epic security team grant the matching scopes. Turning on a capability returns the connection to pending until a test proves Epic granted it, so run Test connection again. A capability shows as on only when the scopes Epic granted cover it.

Step 4: Add workflows and models

Enable the generative workflows on the Workflows tab, and add predictive models, findings extraction models, and encounter predictions on the General tab. See Workflows and Models.

Workflows

The Workflows tab lists the Epic session types your organization can evaluate. Each one is a generative workflow, such as Hospital Course Note. Turn on a workflow’s Monitored switch, then choose Create new project or Map to existing project. Openlayer evaluates its interactions from the next extract. Workflows use the session type names that appear in the extract, not Epic’s product names. “Art” and “Ask Art” never appear in the extract, so a workflow named after them never matches anything. The list starts with the session types Epic publishes, tagged Epic catalog. Your Epic build can emit session types of its own. To evaluate one, add it under Add a workflow on the General tab with its exact session type name. It appears on the Workflows tab tagged Custom.

Models

Epic publishes no inventory of the models a build runs, so you add each one yourself. Under Add a model on the General tab, choose the Epic product, then name the model:
  • Predictive models — enter the scoring system’s name as it appears in your Epic build. Your ACUITY_CONFIG table is the authoritative list.
  • Findings extraction — enter the model’s name.
  • Encounter predictions — choose the prediction, such as length of stay, from the list.
The model appears on the Models tab. Turn on its Monitored switch to enable it, then choose Create new project or Map to existing project. An existing pipeline must use Openlayer’s default storage, not BigQuery or Snowflake. Openlayer evaluates the model’s predictions from the next extract. For predictive models, enabling also asks two things Epic does not export:
  • Score scale — Probability (0 to 1) or Raw score (points or an index). The same score column holds a likelihood for one model and a points total for another. Openlayer skips scores that do not fit the scale you choose instead of guessing.
  • Positive class and Negative class — what a high and a low score mean, such as Did not attend and Attended. These labels make results such as the confusion matrix readable.

What lands in Openlayer

Generative workflows. Each Epic AI interaction becomes one trace, and interactions in the same conversation share a session. The trace carries the response, the workflow’s steps with their status, latency, the model name, and the IDs of documents Epic associated with the interaction. Epic does not export prompt templates, token counts, cost, citations, or the content of clinician feedback, so traces never include them. The clinician’s question appears only when your extract includes agent task text. A discarded draft is the closest signal to a clinician rejecting the output. Predictive models. Each score becomes one row with the model version, site, and the feature values Epic fed the model, along with Epic’s per-feature contribution. Epic keeps only the latest feature values for a patient, so features reflect the time of the extract, not the time of the score. Epic also does not export the threshold at which a score triggers an alert, so set the thresholds on your tests yourself. Findings extraction. Rows are labeled with the findings and follow-up recommendations the radiologist verified, so you can measure accuracy as soon as an extract arrives. Encounter predictions. Each prediction, such as a predicted length of stay, becomes a row scored against what actually happened.

Outcomes

Predictions arrive before their outcomes. Openlayer reads outcomes from Epic FHIR, using the capabilities you grant, and only counts an outcome once it is settled:
  • Appointments count as attended or a no-show once they have happened. Appointments that have not happened yet are not scored. A cancellation is neither, so it is left out of the label.
  • Encounters give the actual length of stay once the stay ends. A patient who is still admitted has no outcome yet.
Accuracy metrics lag behind predictions by however long outcomes take to settle.
Predictive model rows are not yet joined to FHIR outcomes, so they arrive without a label. You can still monitor their scores and feature values.

Data handling

  • Only allowlisted fields are kept. Every column Openlayer imports is on a per-field allowlist that sets the one form it may be stored in. Any other column in your extract is dropped.
  • Identifiers are pseudonymized. Patient, encounter, and clinician IDs are replaced with keyed pseudonyms before anything is stored.
  • Review predictive features before the first import. Your build decides which chart data feeds each predictive model, so the feature list is yours. Check it for anything directly identifying, such as a date of birth or a postal code.
  • Errors carry no chart text. Error messages show the HTTP status and Epic’s coded issue, never Epic’s free-text diagnostics.

Clinical text

Clinical text handling sets what Openlayer may do with chart text. Change it later under Clinical text on the General tab.

Import timing

Extracts usually arrive nightly. Openlayer looks for a delivered extract on the schedule set under Import schedule on the General tab, every 15 minutes at the shortest. Turn off Scheduled import to pause imports without disconnecting. The Overview shows the last import and the last error.
Import warnings, such as rows skipped from an extract, are not shown in Openlayer yet. Encounter predictions carry no features or model version, and findings extraction rows have no confidence score.

Disconnecting

On the General tab, click Disconnect, then Remove Epic. Imports stop and data already ingested is kept.
Disconnecting destroys the signing key. If you reconnect, Openlayer generates a new key, and your Epic team must register it again before the connection works.

Troubleshooting