FLEEK: Factual Error Detection and Correction with Evidence Retrieved from External Knowledge

Large language models’ inability to attribute their claims to external knowledge and their tendency to hallucinate makes it difficult to trust their responses. Even humans are prone to factual errors in their writing. Therefore verifying the factual accuracy of textual information, whether generated by large language models or curated by humans, is an important task. However, manually validating and correcting factual errors tends to be a tedious and labor-intensive process. In this paper, we propose FLEEK for automatic fact verification and correction. FLEEK automatically extracts factual cliams within the text, retrieves relevant evidence for each claim from various sources of external knowledge, and then evaluates the factual status for each claim based on the retrieved evidence. The system also automatically corrects detected factual errors in claims based on the retrieved evidence. Experiments show that FLEEK is able to exhaustively extract factual claims, correctly determine their factual status, and propose meaningful corrections based on the evidence retrieved.