Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature
Uri-ka
Abstract
The exponential growth of scientific literature necessitates advanced tools for effective knowledge exploration. We present Knowledge Navigator, a system designed to enhance exploratory search abilities by organizing and structuring the retrieved documents from broad topical queries into a navigable, two-level hierarchy of named and descriptive scientific topics and subtopics. This structured organization provides an overall view of the research themes in a domain, while also enabling iterative search and deeper knowledge discovery within specific subtopics by allowing users to refine their focus and retrieve additional relevant documents. Knowledge Navigator combines LLM capabilities with cluster-based methods to enable an effective browsing method. We demonstrate our approach's effectiveness through automatic and manual evaluations on two novel benchmarks, CLUSTREC-COVID and SCITOC. Our code, prompts, and benchmarks are made publicly available.
Community
Check out our paper, Knowledge Navigator. We have designed a new paradigm for browsing scientific literature. Using LLMs and clustering-based methods, we organize hundreds of scientific documents retrieved for a query into a table of contents structure. All documents are assigned to subtopics and primary themes, with each theme and subtopic named and summarized for the searcher. The output of this process is a browsing framework that enables navigation within a knowledge map instead of screening through hundreds of documents in a ranked list, allowing for exploration and discovery of trends and specific research questions. A demo is available on the paper’s website as well as the code and benchmarks. https://knowledge-navigators.github.io/
Hi @Uri-ka ,
Congrats on this work. Are you planning to share the benchmarks as HF datasets? See here for a guide: https://huggingface.co/docs/datasets/loading.
They can then also be linked to this paper, as explained here: https://huggingface.co/docs/hub/en/datasets-cards#linking-a-paper
Cheers!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Hybrid Semantic Search: Unveiling User Intent Beyond Keywords (2024)
- pathfinder: A Semantic Framework for Literature Review and Knowledge Discovery in Astronomy (2024)
- LitSearch: A Retrieval Benchmark for Scientific Literature Search (2024)
- SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024)
- Meta Knowledge for Retrieval Augmented Large Language Models (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper