This repo aims to provides a comprehensive list of papers about LLM inference (algorithm perspective). Specifically, this repo includes ICL, reasoning (single call or multi calls/agents), planning, interact with external tools/memory.
What learning algorithm is in-context learning? investigations with linear models[ICLR][2023]
In-context language learning: Architectures and algorithms[arXiv][2024]
Language models are few-shot learners[NeurIPS][2020]
Evaluating large language models trained on code[arXiv][2021]
Why can {GPT} learn in-context? language models secretly perform gradient descent as meta-optimizers[ACL][2023]
Model-agnostic meta-learning for fast adaptation of deep networks[ICML][2017]
What can transformers learn in-context? A case study of simple function classes[NeurIPS][2022]
A theory of emergent in-context learning as implicit structure induction[arXiv][2023]
Mining and summarizing customer reviews[KDD][2004]
ChatGPT for good? On opportunities and challenges of large language models for education[Learning and Individual Differences][2023]
In-context learning learns label relationships but is not conventional learning[ICLR][2024]
WordNet: A lexical database for English[HLT][1994]
Rethinking the role of demonstrations: What makes in-context learning work?[EMNLP][2022]
In-context learning and induction heads[arXiv][2022]
{GPT-4} technical report[arXiv][2023]
What in-context learning "learns" in-context: Disentangling task recognition and task learning[ACL][2023]
Language models are unsupervised multitask learners[GitHub][2019]
Pretraining task diversity and the emergence of non-bayesian in-context learning for regression[NeurIPS][2023]
Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook[PhD thesis][1987]
Do pretrained transformers learn in-context by gradient descent?[arXiv][2024]
The transient nature of emergent in-context learning in transformers[NeurIPS][2023]
Recursive deep models for semantic compositionality over a sentiment treebank[EMNLP][2013]
Large language models in medicine[Nature Medicine][2023]
Learning to Learn: Introduction and Overview[Springer][1998]
Llama 2: Open foundation and fine-tuned chat models[arXiv][2023]
A perspective view and survey of meta-learning[Artificial Intelligence Review][2001]
Transformers learn in-context by gradient descent[ICML][2023]
Large language models are implicitly topic models: Explaining and finding good demonstrations for in-context learning[Workshop][2023]
Emergent abilities of large language models[Machine Learning Research][2022]
Larger language models do in-context learning differently[arXiv][2023]
The learnability of in-context learning[NeurIPS][2023]
Transformers: State-of-the-art natural language processing[EMNLP][2020]
An explanation of in-context learning as implicit Bayesian inference[ICLR][2022]
Character-level convolutional networks for text classification[NeurIPS][2015]
Here are the provided papers formatted as requested:
Chain-of-thought prompting elicits reasoning in large language models[NeurIPS][2022]
Machine Translation Decoding beyond Beam Search[EMNLP][2021]
Transfer Q-star: Principled Decoding for LLM Alignment[NeurIPS][2024]
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding[ICML][2024]
Large language monkeys: Scaling inference compute with repeated sampling[arXiv][2024]
Self-Consistency Improves Chain of Thought Reasoning in Language Models[ICLR][2023]
Tree of thoughts: Deliberate problem solving with large language models[NeurIPS][2024]
Graph of thoughts: Solving elaborate problems with large language models[AAAI][2024]
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning[ICLR][2024]
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph[ICLR][2024]
StructGPT: A General Framework for Large Language Model to Reason over Structured Data[EMNLP][2023]
Camel: Communicative agents for "mind" exploration of large language model society[NeurIPS][2023]
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework[ICLR][2024]
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation[ICLR][2024]
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models[ICML][2024]
Voyager: An Open-Ended Embodied Agent with Large Language Models[Proceedings of the Machine Learning Research][TBD]
Can Transformers Learn to Solve Problems Recursively?[arXiv][2023]
Improving Factuality and Reasoning in Language Models through Multiagent Debate[ICML][2024]
Encouraging divergent thinking in large language models through multi-agent debate[arXiv][2023]