Investigating and Addressing Hallucinations of LLMs in Duties Involving Negation
Authors: Neeraj Varshney, Satyam Raj, Venkatesh Mishra, Agneet Chatterjee, Ritika Sarkar, Amir Saeidi, Chitta Baral
Abstract: Large Language Fashions (LLMs) have achieved distinctive effectivity all through all types of pure language duties. Nonetheless, they have been confirmed to endure from a vital limitation pertinent to ‘hallucination’ of their output. Newest evaluation has centered on investigating and addressing this downside for numerous duties resembling biography period, question answering, abstractive summarization, and dialogue period. Nonetheless, the important facet pertaining to ‘negation’ has remained considerably underexplored. Negation is crucial on account of it supplies depth and nuance to the understanding of language and could be important for logical reasoning and inference. On this work, we deal with the above limitation and notably focus on studying the affect of negation in LLM hallucinations. Significantly, we look at 4 duties with negation: ‘false premise completion’, ‘constrained actuality period’, ‘a lot of choice question answering’, and ‘actuality period’. We current that open-source state-of-the-art LLMs resembling LLaMA-2-chat, Vicuna, and Orca-2 hallucinate considerably on all these duties involving negation which underlines a vital shortcoming of these fashions. Addressing this downside, we further look at fairly a number of strategies to mitigate these hallucinations and present their affect.
