deepset

/

roberta-base-squad2

like 858

Follow

deepset 167

what score is a good score?

Here are few outputs from the model:

example1:

Output from model:
{'score': 0.5341075658798218, 'start': 11, 'end': 15, 'answer': 'john'}
Even for this easy question, models score is 0.5

example2:

Output from model:
{'score': 0.03491657227277756, 'start': 6, 'end': 15, 'answer': 'bar chart'}

Why the scores are very low, even though the model's output is correct?
What could be the threshold to reject false outputs?