Metacognitive Prompting Improves Understanding in Large Language Models

Abstract

In Large Language Models (LLMs), there have been consistent advancements in taskspecific performance, largely influenced by effective prompt design.Recent advancements in prompting have enhanced reasoning in logicintensive tasks for LLMs, yet the nuanced understanding abilities of these models, crucial for processing and interpreting complex information, remain underexplored.In this study, we introduce Metacognitive Prompting (MP), a strategy inspired by human introspective reasoning processes.Using MP, LLMs undergo a systematic series of structured, selfaware evaluations, drawing on both their vast inherent knowledge and new insights.We conduct extensive experiments on four prevalent LLMs: Llama2, PaLM2, GPT-3.5, and GPT-4, across ten natural language understanding (NLU) datasets from GLUE, SuperGLUE, BLUE, and LexGLUE benchmarks.Additionally, we compare our method with chain-ofthought prompting and its advanced versions.The results show that GPT-4 consistently excels across all tasks, while other models have shown significant progress in some tasks when used in conjunction with MP.Furthermore, MP consistently outperforms existing prompting methods in both general and domain-specific NLU tasks.This study underscores the potential to amplify the understanding abilities of LLMs and highlights the benefits of mirroring human introspective reasoning in NLU tasks.Our data and code are available at https://github.com/ EternityYW/Metacognitive-Prompting.How can I justify my decision?How confident am I about this decision?* Recognition of knowledge base * Assessment of initial interpretation * Re-evaluation of initial assessment * Strategy and decision adjustment based on reflections * Reliability assessment of the final decision Human Metacognition MP for LLMs

Topics & Concepts