Retrieval-Augmented Generation (RAG) systems depend on a retriever that fetches
context documents before passing them to an LLM.
To best evaluate RAG quality, Openlayer needs to know what context was retrieved for each request.
Once context is logged, you can use tests that leverage metrics such as context recall (did the retriever surface the right information?),
context relevancy (how relevant was the retrieved content to the query?), and more.
How to log context
There are two main ways to provide context to Openlayer:
The context must be passed as a list of strings (List[str]
). Each string
should represent a retrieved chunk/document.
1. Use @trace
with context_kwarg
If your function receives the retrieved context as a keyword argument, you can
tell Openlayer which argument contains it.
from openlayer.lib import trace
@trace(context_kwarg="context")
def generate_answer(query: str, context: list[str]) -> str:
return llm_call(query, context)
2. Call log_context
directly
If you do not want to rely on a kwarg, or if you get the context at
different points in the pipeline, you can log it explicitly:
from openlayer.lib import trace, log_context
@trace()
def generate_answer(query: str) -> str:
context = retrieve_context(query)
# Log the retrieved context manually
log_context(context)
return llm_call(query, context)