klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-02 13:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Reduce evaluator llm verbosity during eval

Browse Source
This commit is contained in:
Debanjum
2025-11-16 10:42:14 -08:00
parent 630ce77b5f
commit 8490f2826b
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/evals/eval.py
View File

@@ -477,7 +477,7 @@ def evaluate_response_with_gemini(
Ground Truth: {ground_truth} Ground Truth: {ground_truth}
Provide your evaluation in the following json format: Provide your evaluation in the following json format:
{"explanation:[How you made the decision?)", "decision:(TRUE if response contains key information, FALSE otherwise)"} {"explanation:<1 short sentence on how you made the decision>", "decision:<TRUE if response contains key information, FALSE otherwise>"}
""" """
gemini_api_url = ( gemini_api_url = (
f"https://generativelanguage.googleapis.com/v1beta/models/{eval_model}:generateContent?key={GEMINI_API_KEY}" f"https://generativelanguage.googleapis.com/v1beta/models/{eval_model}:generateContent?key={GEMINI_API_KEY}"