Describe the bug
On calling LLM critic validator, sometimes we get responses like -
{
"success": false,
"data": {
"response_id": "11c2d6b7-6f2e-4d09-a78a-23b3d40aed6b",
"rephrase_needed": false
},
"error": "The response failed the following metrics: ['query']."
}
And sometimes like this -
{
"success": false,
"data": {
"response_id": "4985fe71-39fb-42d4-977c-d2d27a16f2d6",
"rephrase_needed": false
},
"error": "The response is missing or has invalid evaluations for the following metrics: ['query']."
}
Additional context
We are going to investigate why that happens and override the errors for the validator. The end goal is to provide proper exception to the end users.
Describe the bug
On calling LLM critic validator, sometimes we get responses like -
And sometimes like this -
Additional context
We are going to investigate why that happens and override the errors for the validator. The end goal is to provide proper exception to the end users.