Skip to main content

Evaluation Process for Groundedness

This document demonstrates the evaluation process we are solving for groundedness.

Evaluation in AI​

Evaluation tools are available within AI solutions. In Artificial Intelligence (AI), evaluation refers to the process of measuring how well an AI model performs on a given task. It's a critical step in developing and validating AI systems to ensure they are accurate, reliable, and effective.

Groundedness in AI Evaluation​

In the context of AI evaluation, especially within platforms like Azure AI Foundry, groundedness refers to how well an AI-generated response is supported by a reliable source of truth---such as a file, document, dataset, or context provided to the model.

Steps to Create Evaluation for Groundedness​

Step 1: Create a JSONL File​

First, create a file in .jsonl format that includes the questions submitted to your model along with the corresponding answers, so its performance can be assessed.

The data can be generated from the [dbo].[ConversationLog] table using the following query:

SELECT 
  ID, 
  Message, 
  JSON_VALUE(Response, '$.systemMessage') AS Response, 
  ToolOutput
FROM [dbo].[ConversationLog] (NOLOCK)
WHERE ToolOutput != '[]'

Then, select the records you want for evaluation and convert the CSV result into JSONL format using:

SELECT 
  (SELECT Message, JSON_VALUE(Response, '$.systemMessage') AS Response, ToolOutput
   FROM [dbo].[ConversationLog] (NOLOCK)
   WHERE ID IN ()
   FOR JSON PATH, WITHOUT_ARRAY_WRAPPER) AS JsonLine

Rules for Creating the .jsonl File​

  • Do not include more than five IDs at a time when converting.
  • Append each JSONL record on a new line.
  • Each record should start with {"Message": and end with ]"}.
  • No commas at the end of lines or between records.

Sample JSONL Layout​

To visualize how the sample evaluation file is structured from start to finish, review the screenshots below.

Sample JSONL beginning Sample JSONL ending

Example File​

Evaluate Groundedness.jsonl

Step 2: Upload Data File in Azure AI Foundry​

  1. Go to Data files in the left panel.

  2. Click + Add data for evaluation. Navigating to Upload Data File

  3. Upload your .jsonl file. Upload Data File

  4. Once processed, it will display a "processed" status. Upload Data File

Step 3: Assess and Compare AI Application Performance​

From the Azure OpenAI Evaluation section:

  1. Click New Evaluation. Click New Evaluation

  2. Choose Upload a file → Next. Click Upload a File

  3. Select your uploaded data file → Next.

Select Uploaded File

Do not click generate a response

  1. Click + Add to create test criteria. Click Add Criteria

Configuring Groundedness Evaluation in Azure AI Foundry​

In the Add testing criteria popup:

  • Select Model labeler.
  • Review these sections:
SectionDescription
PresetsDisplays predefined evaluation templates.
System PromptsMicrosoft-provided prompts to guide evaluation.
User SectionLets you customize user input dynamically.

Criteria Overview Criteria User Input

Evaluation Inputs​

Conversation

{{item.Message}}

Final Response

{{item.Response}}

Criteria

Evaluate the data against the final response to grade whether the answer was derived using the data or if the result is hallucinated.

{{item.ToolOutput}}

Set Grade to gpt-5

Choose Model

Then click Add → Next

Click Add

Click Next

Finalizing the Evaluation​

  • Give the evaluation a name → Submit

Click Submit

  • The report will run and display the percentage of criteria match once complete.

Verify Running

View Results