Jie He

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval–generation optimization. In this work, we propose CLaRa (Continuous Latent Reasoning), a unified framework that performs embedding-based compression and joint optimization in a shared continuous space. To obtain semantically rich and retrievable compressed vectors, thereby reducing the document length fed into the generator, we introduce SCP, a key-preserving data synthesis framework based on question-answering and paraphrase supervision. CLaRa then trains the reranker and generator end-to-end via a single language modeling loss, with gradients flowing through both modules using a differentiable top-k estimator. Theoretically, this unified optimization aligns retrieval relevance with answer quality. Experiments across multiple QA benchmarks show that CLaRa achieves state-of-the-art compression and reranking performance, even at a text compression rate of 16, outperforming text-based fine-tuned baselines.

Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks but face significant challenges with complex, knowledge-intensive multi-hop queries, particularly those involving new or long-tail knowledge. Existing benchmarks often fail to fully address these challenges. To bridge this gap, we introduce MINTQA (Multi-hop Question Answering on New and Tail Knowledge), a comprehensive benchmark to evaluate LLMs' capabilities in multi-hop reasoning across four critical dimensions: question handling strategy, sub-question generation, retrieval-augmented generation, and iterative or dynamic decomposition and retrieval. MINTQA comprises 10,479 question-answer pairs for evaluating new knowledge and 17,887 pairs for assessing long-tail knowledge, with each question equipped with corresponding sub-questions and answers. Our systematic evaluation of 22 state-of-the-art LLMs on MINTQA reveals significant limitations in their ability to handle complex knowledge base queries, particularly in handling new or unpopular knowledge. Our findings highlight critical challenges and offer insights for advancing multi-hop reasoning capabilities.

Large Language Models (LLMs) can enhance their capabilities as AI assistants by integrating external tools, allowing them to access a wider range of information. We present GenTool, a novel training framework addressing two dimensions: Zero-to-One Generalization and Weak-to-Strong Generalization. Through extensive experiments across four generalization scenarios, our method significantly enhances tool-usage capabilities of LLMs ranging from 1B to 8B parameters, achieving performance that surpasses GPT-4o.

We conduct a comprehensive evaluation of prompting open-source LLMs on graph-to-text generation tasks. We propose a novel diversity-difficulty-based few-shot sample selection method and introduce PlanGTG, a new graph-to-text dataset annotated with reordering and attribution sub-tasks. Extensive evaluations demonstrate significant improvements in generated text quality from both few-shot and fine-tuning perspectives.

We propose MiCEval, a framework to assess the correctness of reasoning chains in Multimodal Chain of Thought (MCoT) by evaluating the quality of both the description and each reasoning step. MiCEval is built upon a fine-grained dataset with annotations rating each step according to correctness, relevance, and informativeness. Experiments on four MLLMs show step-wise evaluations align more closely with human judgments.

We study the robustness of retrieval-augmented ICL against adversarial attacks. Retrieval-augmented models enhance robustness against test sample attacks (4.87% ASR reduction) but exhibit overconfidence in demonstrations (2% ASR increase). We introduce DARD, a training-free adversarial defence method, achieving a 15% reduction in ASR over baselines.

We study Parameter-Efficient Fine-Tuning (PEFT) methods for MLLMs, evaluating four PEFT methods on seven datasets. We show that the adapter is the best-performing PEFT method, and fine-tuning connector layers leads to improved performance in most MLLMs.

We propose to transform the downstream multiple choice QA task into a simpler binary classification task by ranking all candidate answers according to their reasonableness. We convert knowledge graph triples into reasonable and unreasonable texts for training. Extensive results show effectiveness on various QA benchmarks, with our approach being less data hungry than existing methods.

We propose TGEA, using carefully selected prompts to guide GPT-2 to generate candidate sentences, selecting 47K for error annotation. We create an error taxonomy covering 24 types of errors and propose a series of automatic diagnosis tasks including error detection, error type classification, associated span detection, and error rationale generation.

We present a test suite to evaluate the commonsense reasoning capability of neural machine translation, covering lexical and syntactic ambiguity that requires commonsense knowledge to resolve. We manually create 1,200 triples and demonstrate that neural machine translation performs poorly on commonsense reasoning.