Q2D-Web: Blending Agent Citations, Web Rankings, and LLM Judgments for Better Relevance

Key Info

Q2D-Web is a relevance dataset that combines agent citations, production web rankings, and a third set expanded with LLM judgments to improve evaluation robustness.

Highlights

• Uses three complementary relevance sets to cut false negatives and avoid over-reliance on a single labeling pipeline.
• Enables testing of how different relevance definitions affect model performance.
• Likely useful for teams building or evaluating retrieval and agent-based search systems.