We present CodeBERT, a bimodal pre-trained model for programming language (PL) and
natural language (NL). CodeBERT learns general-purpose representations that support …

Generative pretrained models have achieved remarkable success in various domains such
as language and computer vision. Specifically, the combination of large-scale diverse …

This paper presents how to generate questions from given passages using neural networks,
where large scale QA pairs are automatically crawled and processed from Community-QA …

ChatGPT is attracting a cross-field interest as it provides a language interface with remarkable
conversational competency and reasoning capabilities across many domains. However, …

Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read
pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a …

Pre-trained models for programming languages have recently demonstrated great success
on code intelligence. To support both code-related understanding and generation tasks, …

Assessing foundation models’ abilities for human-level tasks is crucial for Artificial General
Intelligence (AGI) development. Traditional benchmarks, which rely on artificial datasets, may …

Recent developments in large language models (LLMs) have been impressive. However,
these models sometimes show inconsistencies and problematic behavior, such as …

Pre-trained models for programming language have achieved dramatic empirical
improvements on a variety of code-related tasks such as code search, code completion, code …

Large language models have made significant progress in various language tasks, yet they
still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-…