Skip to content

Commit 89e6441

Browse files
committed
Relevance Evaluator: Support Conversational Format in specs
Updated the dataMappingSchema to allow 'query' and 'response' to be either a string or an array of objects in Relevance Evaluator specs
1 parent 46a6f2b commit 89e6441

File tree

1 file changed

+11
-3
lines changed

1 file changed

+11
-3
lines changed

assets/evaluators/builtin/relevance/spec.yaml

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,22 @@ dataMappingSchema:
2323
type: "object"
2424
properties:
2525
query:
26-
type: "string"
26+
anyOf:
27+
- type: "string"
28+
- type: "array"
29+
items:
30+
type: "object"
2731
response:
28-
type: "string"
32+
anyOf:
33+
- type: "string"
34+
- type: "array"
35+
items:
36+
type: "object"
2937
required: ["query", "response"]
3038
outputSchema:
3139
relevance:
3240
type: "ordinal"
3341
desirable_direction: "increase"
3442
min_value: 1
3543
max_value: 5
36-
path: ./evaluator
44+
path: ./evaluator

0 commit comments

Comments
 (0)