- Chapter 1 - Introduction to Generative AI 💾
- Chapter 2 - Introduction to LLMs 👓
- Chapter 3 - Generating Text 🪹
- Chapter 4 - Generating Images 🖼️
- Chapter 5 - What else can AI Generate? 🪹
- Chapter 6 - Guide to Prompt Engineering 💬
- Chapter 7 - RAG - The Secret Weapon 🤫
- Chapter 8 - Chatting with your data 🪹
- Chapter 9 - Tailoring Models with Model Adaptation and Fine-Tuning 🔌
- Chapter 10 - Application Architecture for Gen AI Apps 👏
- Chapter 11 - Scaling Up: Best Practices for Production Deployment 💽
- Chapter 12 - Evaluations and Benchmarks ✅
- Chapter 13 - Guide to Ethical GenAI: Principles, Practices, and Pitfalls 😇
This section contains a list of research papers associated with different AI technology and techniques. These are organized by chapter to help you navigate. The reader is not expected to know these, but as with most things, it is always good to go deeper and grok some of these concepts for a better and fuller understanding.
| # | Paper Title | Link |
|---|---|---|
| 1 | The brief history of artificial intelligence: The world has changed fast – what might be next? | https://ourworldindata.org/brief-history-of-ai |
| 2 | Introducing ChatGPT and Whisper APIs | https://openai.com/blog/introducing-chatgpt-and-whisper-apis |
| # | Paper Title | Link |
|---|---|---|
| 1 | A Survey of Large Language Models | httpshttp://arxiv.org/abs/2303.18223 |
| 2 | Emergent Abilities of Large Language Models | https://arxiv.org/abs/2206.07682 |
| 3 | LongNet: Scaling Transformers to 1,000,000,000 Tokens | https://arxiv.org/abs/2307.02486 |
| 4 | Constitutional AI: Harmlessness from AI Feedback | https://arxiv.org/abs/2212.08073 |
None
| # | Paper Title | Link |
|---|---|---|
| 1 | Generative Adversarial Networks | http://arxiv.org/abs/1406.2661 |
| 2 | An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale | https://arxiv.org/abs/2010.11929 |
None
| # | Paper Title | Link |
|---|---|---|
| 1 | An Explanation of In-context Learning as Implicit Bayesian Inference | https://arxiv.org/abs/2111.02080 |
| 2 | Language Models are Few-Shot Learners | https://arxiv.org/abs/2005.14165 |
| 3 | Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? | https://arxiv.org/abs/2202.12837 |
| 4 | Chain-of-Thought Prompting Elicits Reasoning in Large Language Models | https://arxiv.org/abs/2201.11903 |
| 5 | Self-Consistency Improves Chain of Thought Reasoning in Language Models | https://arxiv.org/abs/2203.11171 |
| 6 | Lost in the Middle: How Language Models Use Long Contexts | https://arxiv.org/abs/2307.03172 |
| 7 | LLM01: Prompt Injection | https://www.llmtop10.com/llm01 |
| # | Paper Title | Link |
|---|---|---|
| 1 | Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks | https://arxiv.org/abs/2005.11401 |
| 2 | Maximum inner-product search | https://en.wikipedia.org/wiki/Maximum_inner-product_search |
| 3 | Constitution of the United Kingdom | https://en.wikipedia.org/wiki/Constitution_of_the_United_Kingdom |
| 4 | FIFA 2023 Womens World Cup | https://en.wikipedia.org/wiki/2023_FIFA_Women%27s_World_Cup |
None
| # | Paper Title | Link |
|---|---|---|
| 1 | RAG vs. Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture | https://arxiv.org/abs/2401.08406 |
| 2 | Language Models: A Guide for the Perplexed | https://arxiv.org/abs/2311.17301 |
| 3 | BLEU: a method for automatic evaluation of machine translation | https://dl.acm.org/doi/10.3115/1073083.1073135 |
| 4 | ROUGE: a Package for Automatic Evaluation of Summaries | https://www.microsoft.com/en-us/research/publication/rouge-a-package-for-automatic-evaluation-of-summaries |
| 5 | State of GPT | https://arxiv.org/abs/2311.17301 |
| 6 | Direct Preference Optimization: Your Language Model is Secretly a Reward Model | https://arxiv.org/abs/2305.18290 |
| 7 | Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning | https://arxiv.org/abs/2303.15647 |
| 8 | The Curse of Recursion: Training on Generated Data Makes Models Forget | https://arxiv.org/abs/2305.17493 |
| 9 | LoRA: Low-Rank Adaptation of Large Language Models | https://arxiv.org/abs/2106.09685 |
| 10 | Aligning language models to follow instructions | https://openai.com/research/instruction-following |
| 11 | Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback | https://arxiv.org/abs/2204.05862 |
| 12 | Anthropic hh-rlhf | https://github.com/anthropics/hh-rlhf |
| 13 | Training language models to follow instructions with human feedback | https://arxiv.org/abs/2203.02155 |
| # | Paper Title | Link |
|---|---|---|
| 1 | Software 2.0 | https://karpathy.medium.com/software-2-0-a64152b37c35 |
| 2 | Phi-2: The surprising power of small language models | https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models |
| 3 | Orca 2: Teaching Small Language Models How to Reason | https://www.microsoft.com/en-us/research/blog/orca-2-teaching-small-language-models-how-to-reason |
| 4 | Prompting Frameworks for Large Language Models: A Survey | https://arxiv.org/abs/2311.12785 |
| 5 | Curse of dimensionality | https://en.wikipedia.org/wiki/Curse_of_dimensionality |
| # | Paper Title | Link |
|---|---|---|
| 1 | G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment | https://arxiv.org/abs/2303.16634 |
| 2 | HellaSwag: Can a Machine Really Finish Your Sentence? | https://arxiv.org/abs/1905.07830 |
| 3 | Measuring Massive Multitask Language Understanding | https://arxiv.org/abs/2009.03300 |
| # | Paper Title | Link |
|---|---|---|
| 1 | The AI Index Report - 2004 | https://aiindex.stanford.edu/report/ |
| 2 | BLEU: a method for automatic evaluation of machine translation | https://dl.acm.org/doi/10.3115/1073083.1073135 |
| 3 | ROUGE: a Package for Automatic Evaluation of Summaries | https://www.microsoft.com/en-us/research/publication/rouge-a-package-for-automatic-evaluation-of-summaries |
| 4 | BERTScore: Evaluating Text Generation with BERT | https://openreview.net/forum?id=SkeHuCVFDr |
| 5 | G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment | https://arxiv.org/abs/2303.16634 |
| 6 | Holistic Evaluation of Language Models | https://arxiv.org/abs/2211.09110 |
| 7 | Holistic Evaluation of Text-To-Image Models | https://arxiv.org/abs/2311.04287 |
| 8 | HellaSwag: Can a Machine Really Finish Your Sentence? | https://arxiv.org/abs/1905.07830 |
| 9 | Measuring Massive Multitask Language Understanding | https://arxiv.org/abs/2009.03300 |
| 10 | SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | https://arxiv.org/abs/2310.06770 |
| 11 | Measuring Massive Multitask Language Understanding | https://arxiv.org/abs/2009.03300 |
| 12 | MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks | https://arxiv.org/abs/2310.19677 |
| 13 | HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models | https://arxiv.org/abs/2305.11747 |
| # | Paper Title | Link |
|---|---|---|
| 1 | OWASP Top 10 for Large Language Model Applications | https://owasp.org/www-project-top-10-for-large-language-model-applications/ |
| 2 | Prompt Injection attack against LLM-integrated Applications | https://arxiv.org/abs/2306.05499 |
| 3 | DAN is my new friend | https://www.reddit.com/r/ChatGPT/comments/zlcyr9/dan_is_my_new_friend/ |
| 4 | Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection | https://arxiv.org/abs/2302.12173 |
| 5 | Universal and Transferable Adversarial Attacks on Aligned Language Models | https://arxiv.org/abs/2307.15043 |
| 6 | Many-shot jailbreaking | https://www.anthropic.com/research/many-shot-jailbreaking |
| 7 | A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networks | https://arxiv.org/abs/2308.14367 |
| 8 | Guide for Conducting Risk Assessments | https://csrc.nist.gov/pubs/sp/800/30/r1/final |
| 9 | Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models | : https://papers.nips.cc/paper_files/paper/2022/hash/9ca22870ae0ba55ee50ce3e2d269e5de-Abstract-Datasets_and_Benchmarks.html |
| 10 | Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias | https://arxiv.org/abs/1810.01943 |
| 11 | HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal | https://arxiv.org/abs/2402.04249 |
| 12 | Recipes for Safety in Open-domain Chatbots | https://arxiv.org/abs/2010.07079 |
| 13 | MART: Improving LLM Safety with Multi-round Automatic Red-Teaming | https://arxiv.org/abs/2311.07689 |