klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-02 13:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve GPQA eval prompt to imrpove parsing answer from Khoj response

Browse Source
This commit is contained in:
Debanjum
2024-11-30 17:19:01 -08:00
parent 439b18c21f
commit fc6be543bd
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/evals/eval.py
View File

@@ -163,7 +163,7 @@ def load_gpqa_dataset():
correct_letter = "ABCD"[correct_index] correct_letter = "ABCD"[correct_index]
prompt = f""" prompt = f"""
Answer the following multiple choice question. Answer should be of the following format: 'Answer: $LETTER' (without quotes) where LETTER is one of ABCD. Think step by step before answering. Answer the following multiple choice question. Answer should be of the following format: 'Answer: $LETTER' (without quotes) where $LETTER is one of ABCD. Think step by step before answering.
{row["Question"]} {row["Question"]}