Category-wise metrics
Each category lists all query methods, along with their Average Score, Context Precision, Context Recall, and Response Relevancy.
Consolidated Evaluation Metrics Table (Excluding Generative)
Category
Method
Avg Score
Precision
Recall
Relevancy
Mobile Related > Call Drop
semantic
0.949
1.00
1.00
0.846
hybrid
0.949
1.00
1.00
0.846
keyword
0.943
1.00
1.00
0.830
vector
0.911
1.00
1.00
0.732
reranking
0.944
1.00
1.00
0.833
filtered
0.949
1.00
1.00
0.846
Mobile Related > Improper Network Coverage
semantic
0.934
0.96
1.00
0.842
hybrid
0.930
0.96
1.00
0.829
keyword
0.943
1.00
1.00
0.829
vector
0.847
0.84
0.986
0.716
reranking
0.943
1.00
1.00
0.829
filtered
0.937
1.00
1.00
0.812
Mobile Related > Data Speed lower than commited
semantic
0.956
1.00
1.00
0.868
hybrid
0.956
1.00
1.00
0.868
keyword
0.943
0.96
1.00
0.868
vector
0.877
0.92
0.985
0.726
reranking
0.939
1.00
1.00
0.817
filtered
0.933
1.00
1.00
0.800
Mobile Related > Mobile Number Portability (MNP)
semantic
0.941
1.00
1.00
0.822
hybrid
0.938
1.00
1.00
0.814
keyword
0.936
1.00
1.00
0.808
vector
0.907
1.00
1.00
0.722
reranking
0.753
0.80
0.80
0.659
filtered
0.929
1.00
1.00
0.786
Mobile Related > UCC related complaints
semantic
0.921
0.96
1.00
0.804
hybrid
0.921
0.96
1.00
0.804
keyword
0.922
0.96
1.00
0.806
vector
0.885
0.96
0.96
0.736
reranking
0.747
0.80
0.80
0.641
filtered
0.926
1.00
1.00
0.779
Mobile Related > Activation/Deactivation of VAS
semantic
0.751
0.80
0.80
0.653
hybrid
0.751
0.80
0.80
0.653
keyword
0.751
0.80
0.80
0.653
vector
0.903
0.96
1.00
0.750
reranking
0.939
1.00
1.00
0.816
filtered
0.923
1.00
1.00
0.770
Mobile Related > Activation/Deactivation/Fault of SIM
semantic
0.945
1.00
1.00
0.834
hybrid
0.754
0.80
0.80
0.663
keyword
0.944
1.00
1.00
0.832
vector
0.912
1.00
1.00
0.737
reranking
0.944
1.00
1.00
0.831
filtered
0.925
1.00
1.00
0.775