Research on Multi-hop Q/A

Multi-Hop Question Answering over Knowledge Graphs (MHQA-KG) plays a pivotal role in various applications, including but not limited to Question Answering, Recommendation Systems, and Semantic Search. Nevertheless, current models for MHQA have limitations in their ability to grasp all the information included in the question, resulting a reduction in accuracy when producing answers. In order to mitigate this limitation, this paper proposes a novel Multi-Hop Question Answering over Knowledge Graphs approach. It mainly utilizes question and path embedding to answer multi-hop questions, significantly improving accuracy. This approach effectively captures auxiliary information that may present in the question. The experimental findings provide evidence that the suggested methodology outperforms the current state-of-the-art models, achieving highly accurate outcomes with improvements.

Source

Large language models (LLMs) have recently demonstrated remarkable performance across various Natual Language Processing tasks. In the field of multi-hop reasoning, the Chain-of-thought (CoT) prompt method has emerged as a paradigm, using curated stepwise reasoning demonstrations to enhance LLM's ability to reason and produce coherent rational pathways. To ensure the accuracy, reliability, and traceability of the generated answers, many studies have incorporated information retrieval (IR) to provide LLMs with external knowledge. However, existing CoT with IR methods decomposes questions into sub-questions based on a single compositionality type, which limits their effectiveness for questions involving multiple compositionality types. Additionally, these methods suffer from inefficient retrieval, as complex questions often contain abundant information, leading to the retrieval of irrelevant information inconsistent with the query's intent. In this work, we propose a novel question decomposition framework called TRQA for multi-hop question answering, which addresses these limitations. ... Read More

Source

Multi-hop Knowledge Graph Question Answering aims at finding an entity to answer natural language questions from knowledge graphs. When humans perform multi-hop reasoning, people tend to focus on specific relations across different hops and confirm the next entity. Therefore, most algorithms choose the wrong specific relation, which makes the system deviate from the correct reasoning path. The specific relation at each hop plays an important role in multi-hop question answering. Existing work mainly relies on the question representation as relation information, which cannot accurately calculate the specific relation distribution. In this article, we propose an interpretable assistance framework that fully utilizes the relation embeddings to assist in calculating relation distributions at each hop. Moreover, we employ the fusion attention mechanism to ensure the integrity of relation information and hence to enrich the relation embeddings. The experimental results on three English datasets and one Chinese dataset demonstrate that our method significantly outperforms all baselines. The... Read More

Source

Multi-Hop Question Answering (MHQA) tasks present a significant challenge for large language models (LLMs) due to the intensive knowledge required. Current solutions, like Retrieval-Augmented Generation, typically retrieve potential documents from an external corpus to read an answer. However, the performance of this retrieve-then-read paradigm is constrained by the retriever and the inevitable noise in the retrieved documents. To mitigate these challenges, we introduce a novel generate-then-ground (GenGround) framework, synergizing the parametric knowledge of LLMs and external documents to solve a multi-hop question. GenGround empowers LLMs to alternate two phases until the final answer is derived: (1) formulate a simpler, single-hop question and directly generate the answer; (2) ground the question-answer pair in retrieved documents, amending any wrong predictions in the answer. We also propose an instructional grounding distillation method to generalize our method into smaller models. Extensive experiments conducted on four datasets illustrate the superiority of our method.

Source

<title>Abstract</title> Multi-hop knowledge graph question answering aims to find answer entities from the knowledge graph based on natural language questions. This is a challenging task as it requires precise reasoning about entity relationships at each step. When humans perform multi-hop reasoning, they usually focus on specific relations between different hops and determine the next entity. However, most algorithms often choose the wrong specific relations, causing the system to deviate from the correct reasoning path. In multi-hop question answering, the specific relation between each hop is crucial. The existing TransferNet model mainly relies on question representation for relational reasoning, but cannot accurately calculate the specific relational distribution, which will profoundly affect question answering performance. On this basis, this paper proposes an interpretable assiatance framework, which makes full use of relation embedding and question semantics, and uses the attention mechanism to cross-fuse the relevant information of them to assist in calculating the relation ... Read More

Source

Multi-hop QA (MHQA) involves step-by-step reasoning to answer complex questions and find multiple relevant supporting facts. However, Existing large language models'(LLMs) reasoning ability in multi-hop question answering remains exploration, which is inadequate in answering multi-hop questions. Moreover, it is unclear whether LLMs follow a desired reasoning chain to reach the right final answer. In this paper, we propose a \textbf{gen}erative question \textbf{dec}omposition method (GenDec) from the perspective of explainable QA by generating independent and complete sub-questions based on incorporating additional extracted evidence for enhancing LLMs' reasoning ability in RAG. To demonstrate the impact, generalization, and robustness of Gendec, we conduct two experiments, the first is combining GenDec with small QA systems on paragraph retrieval and QA tasks. We secondly examine the reasoning capabilities of various state-of-the-art LLMs including GPT-4 and GPT-3.5 combined with GenDec. We experiment on the HotpotQA, 2WikihopMultiHopQA, MuSiQue, and PokeMQA datasets.

Source

Multi-hop query answering over a Knowledge Graph (KG) involves traversing one or more hops from the start node to answer a query. Path-based and logic-based methods are state-of-the-art for multi-hop question answering. The former is used in link prediction tasks. The latter is for answering complex logical queries. The logical multi-hop querying technique embeds the KG and queries in the same embedding space. The existing work incorporates First Order Logic (FOL) operators, such as conjunction ($\wedge$), disjunction ($\vee$), and negation ($\neg$), in queries. Though current models have most of the building blocks to execute the FOL queries, they cannot use the dense information of multi-modal entities in the case of Multi-Modal Knowledge Graphs (MMKGs). We propose RConE, an embedding method to capture the multi-modal information needed to answer a query. The model first shortlists candidate (multi-modal) entities containing the answer. It then finds the solution (sub-entities) within those entities. Several existing works tackle path-based question-answering in MMKGs. However, to ... Read More

Source

Large Language Models (LLMs) have shown proficiency in question-answering tasks but often struggle to integrate real-time knowledge updates, leading to potentially outdated or inaccurate responses. This problem becomes even more challenging when dealing with multi-hop questions since they require LLMs to update and integrate multiple knowledge pieces relevant to the questions. To tackle the problem, we propose the Retrieval-Augmented model Editing (RAE) framework tailored for multi-hop question answering. RAE first retrieves edited facts and then refines the language model through in-context learning. Specifically, our retrieval approach, based on mutual information maximization, leverages the reasoning abilities of LLMs to identify chain facts that na\"ive similarity-based searches might miss. Additionally, our framework incorporates a pruning strategy to eliminate redundant information from the retrieved facts, which enhances the editing accuracy and mitigates the hallucination problem. Our framework is supported by theoretical justification for its fact retrieval efficacy. Finally... Read More

Source

Multi-hop question answering (MHQA) requires a model to retrieve and integrate information from multiple passages to answer a complex question. Recent systems leverage the power of large language models and integrate evidence retrieval with reasoning prompts (e.g., chain-of-thought reasoning) for the MHQA task. However, the complexities in the question types (bridge v.s. comparison questions) and the reasoning types (sequential v.s. parallel reasonings) require more novel and fine-grained prompting methods to enhance the performance of MHQA under the zero-shot setting. In this paper, we propose STOC-TOT, a stochastic tree-of-thought reasoning prompting method with constrained decoding for MHQA and conduct a detailed comparison with other reasoning prompts on different question types and reasoning types. Specifically, we construct a tree-like reasoning structure by prompting the model to break down the original question into smaller sub-questions to form different reasoning paths. In addition, we prompt the model to provide a probability estimation for each reasoning path at each reas... Read More

Source

With the advent of the information age, it will be more troublesome to search for a lot of relevant knowledge to find the information you need. Text reasoning is a very basic and important part of multi-hop question and answer tasks. This paper aims to study the integrity, uniformity, and speed of computational intelligence inference data capabilities. That is why multi-hop reasoning came into being, but it is still in its infancy, that is, it is far from enough to conduct multi-hop question and answer questions, such as search breadth, process complexity, response speed, comprehensiveness of information, etc. This paper makes a text comparison between traditional information retrieval and computational intelligence through corpus relevancy and other computing methods. The study finds that in the face of multi-hop question and answer reasoning, the reasoning data that traditional retrieval methods lagged behind in intelligence are about 35% worse. It shows that computational intelligence would be more complete, unified, and faster than traditional retrieval methods. This paper also i... Read More

Source

Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhancing response quality, thereby facilitating the great adoption of LLMs in practice. However, we find that existing RAG systems are inadequate in answering multi-hop queries, which require retrieving and reasoning over multiple pieces of supporting evidence. Furthermore, to our knowledge, no existing RAG benchmarking dataset focuses on multi-hop queries. In this paper, we develop a novel dataset, MultiHop-RAG, which consists of a knowledge base, a large collection of multi-hop queries, their ground-truth answers, and the associated supporting evidence. We detail the procedure of building the dataset, utilizing an English news article dataset as the underlying RAG knowledge base. We demonstrate the benchmarking utility of MultiHop-RAG in two experiments. The first experiment compares different embedding models for retrieving evidence for multi-hop queries. In the second experiment, we examine the capabilities o... Read More

Source

Large Language Models (LLMs) have demonstrated significant capabilities across numerous application domains. A key challenge is to keep these models updated with latest available information, which limits the true potential of these models for the end-applications. Although, there have been numerous attempts for LLMs Knowledge Editing (KE), i.e., to edit the LLMs prior knowledge and in turn test it via Multi-hop Question Answering (MQA), yet so far these studies are primarily focused on English language. To bridge this gap, in this paper we propose: Multi-hop Questioning Answering under Knowledge Editing for Arabic Language (MQA-KEAL). MQA-KEAL stores knowledge edits as structured knowledge units in the external memory. In order to solve multi-hop question, it first uses task-decomposition to decompose the question into smaller sub-problems. Later for each sub-problem, it iteratively queries the external memory and/or target LLM in order to generate the final response. In addition, we also contribute MQUAKE-AR (Arabic translation of English benchmark MQUAKE), as well as a new benchma... Read More

Source

Abstract Question-answering systems require retrieving evidence from multiple documents or paragraphs and reasoning over them to meet users' information needs and answer their complex questions. On the other hand, the Explainability and comprehensibility of the predictions made by question-answering systems pose a challenge. In this paper, a content-based reasoning approach based on graph-based machine reading comprehension methods is proposed to answer multi-hop complex questions. In this approach, relevant paragraphs are selected in a two-step process after receiving the input of a multi-hop complex question. Then, to facilitate content-based reasoning and utilize the evidence related to the multi-hop complex question in the retrieved paragraphs, an incoherent graph infrastructure is constructed. Subsequently, a graph neural network and a transformer are employed as an encoder to extract the content-based answer relevant to the question from the graph infrastructure. Finally, to overcome the challenge of interpretability in the question-answering system, a transformer and the predi... Read More

Source

Open Domain Multi-Hop Question Answering (ODMHQA) plays a crucial role in Natural Language Processing (NLP) by aiming to answer complex questions through multi-step reasoning over retrieved information from external knowledge sources. Recently, Large Language Models (LLMs) have demonstrated remarkable performance in solving ODMHQA owing to their capabilities including planning, reasoning, and utilizing tools. However, LLMs may generate off-topic answers when attempting to solve ODMHQA, namely the generated answers are irrelevant to the original questions. This issue of off-topic answers accounts for approximately one-third of incorrect answers, yet remains underexplored despite its significance. To alleviate this issue, we propose the Discriminate->Re-Compose->Re- Solve->Re-Decompose (Dr3) mechanism. Specifically, the Discriminator leverages the intrinsic capabilities of LLMs to judge whether the generated answers are off-topic. In cases where an off-topic answer is detected, the Corrector performs step-wise revisions along the reversed reasoning chain (Re-Compose->Re-Solve->Re-Decom... Read More

Source

Visual Question Answering (VQA) often involves diverse reasoning scenarios across Vision and Language (V&L). Most prior VQA studies, however, have merely focused on assessing the model's overall accuracy without evaluating it on different reasoning cases. Furthermore, some recent works observe that conventional Chain-of-Thought (CoT) prompting fails to generate effective reasoning for VQA, especially for complex scenarios requiring multi-hop reasoning. In this paper, we propose II-MMR, a novel idea to identify and improve multi-modal multi-hop reasoning in VQA. In specific, II-MMR takes a VQA question with an image and finds a reasoning path to reach its answer using two novel language promptings: (i) answer prediction-guided CoT prompt, or (ii) knowledge triplet-guided prompt. II-MMR then analyzes this path to identify different reasoning cases in current VQA benchmarks by estimating how many hops and what types (i.e., visual or beyond-visual) of reasoning are required to answer the question. On popular benchmarks including GQA and A-OKVQA, II-MMR observes that most of their VQA que... Read More

Source

With the continuous increase of Internet information, the traditional single-hop query is not enough to meet the needs of users. For complex problems, multi-hop KGQA requires reasoning at multiple edges of the KG to arrive at the correct answer. KGS often lack many links, which poses additional challenges for KGQAs especially multi-jump KGQAs. In this paper, the main multi-hop question answering algorithms are divided into two categories: embedded-based multi-hop knowledge question answering reasoning and linked multi-hop knowledge question answering reasoning. The results show that the EmbedKGQA model performs better in prediction reasoning under the knowledge map with missing links by analyzing and comparing the performance of the subgraph matching and embedding prediction models on the Meta and WebQestionSP data sets. Finally, in view of the absence of knowledge graph data in practical application, we propose the development prospect of knowledge graph multi-hop algorithm from the two directions of combining pre-training with knowledge graph and using multi-modal model to expand t... Read More

Source

Knowledge base multi-hop question answering must perform hop-by-hop reasoning on long paths to obtain answer entities. However, due to the lack of datasets with correct path annotations, the model's training can only rely on the results of the last hop answer for updating. A more innovative approach is to apply the knowledge distillation method to the question-answering task, and the complex teacher model learns the entity distribution of each hop as the intermediate supervision signal of the student model. However, the structure of this dual model needs to be simplified. At the same time, in the backward reasoning process from the candidate answer entity to the topic entity, this method only uses the last hop result of the forward as the initialization, which does not guarantee that the backward reasoning process can conform to the semantic information of the question sentence. To address these challenges, we design a structurally simple model capable of constraining the intermediate reasoning process through forward and backward bidirectional reasoning. Moreover, we updated the ini... Read More

Source

Abstract Multi-hop path query answering is a complex task in which a path needs to be inferred from a knowledge graph that contains a head entity and multi-hop relations. The objective is to identify the corresponding tail entity accurately. The representation of the path is a critical factor in this task. However, existing methods do not adequately consider the context information of the entities and relations in the path. To address this issue, this paper proposes a novel multi-hop path query answering model that utilizes an enhanced reasoning path feature representation to incorporate intermediate entity information and improve the accuracy of path query answering. The proposed model utilizes the neighborhood to aggregate the entity representation in the reasoning path. It employs a recurrent skipping network to splice the embedding of the relationship and the entity in the reasoning path based on their weight. Additionally, the model adds the position representation to obtain the reasoning path representation. Moreover, the model uses Bi-GRU and Transformer to obtain the local an... Read More

Source

Question generation is the parallel task of question answering, where given an input context and, optionally, an answer, the goal is to generate a relevant and fluent natural language question. Although recent works on question generation have experienced success by utilizing sequence-to-sequence models, there is a need for question generation models to handle increasingly complex input contexts to produce increasingly detailed questions. Multi-hop question generation is a more challenging task that aims to generate questions by connecting multiple facts from multiple input contexts. In this work, we apply a transformer model to the task of multi-hop question generation without utilizing any sentence-level supporting fact information. We utilize concepts that have proven effective in single-hop question generation, including a copy mechanism and placeholder tokens. We evaluate our model’s performance on the HotpotQA dataset using automated evaluation metrics, including BLEU, ROUGE and METEOR and show an improvement over the previous work.

Source

In this paper, we present the Self-Adapting Reasoning Model (SAR) for solving multi-hop question answering (MHQA) tasks, where the QA system is supposed to find the correct answer within the given multiple documents and a multi-hop question. One feasible track on MHQA is question decomposition, based on the idea that a multi-hop question is usually made from several single-hop questions, which are much easier to answer. However, ignoring the inner connection between sub-questions, existing works usually train additional single-hop question-answering models and answer sub-questions separately. To tackle this problem, we design an end-to-end self-adaptive multi-hop reasoning model. Specifically, given a multi-hop question, a question decomposer first decomposes it into two simple questions and identifies the question type. Then, based on the question type, different reasoning strategies are applied for reasoning. This enables our model to be self-adapting and more explainable regarding different types of questions. Experiments are carried out to demonstrate the effectiveness of our mod... Read More

Source