📚 Tiny Stories Language Models

🎯 Description

This repository implements language models trained on the TinyStories dataset - a collection of simple, child-friendly stories generated using GPT-4.

📊 Dataset and Format

TinyStories can be found at HuggingFace Datasets.

Data Fields:

Each story entry contains:

• story: The main story text
• instruction: Prompt and constraints used to generate the story
• summary: Brief summary of the story
• source: The model used to generate the story (GPT-4)

📝 Click to see example story

Story:

Once upon a time, there was a big, red ball that could bounce very high...

[Rest of the example story]

Instruction:

• Prompt: 'Write a short story (3-5 paragraphs)...'
• Required words: ['bounce', 'language', 'intelligent']
• Features: ['Dialogue']

Summary: 'A big, red ball that could bounce high and speak a special language...'

Source: GPT-4

🚀 Installation

📦 Pip Installation

# clone project
git clone https://github.com/YourGithubName/your-repo-name
cd your-repo-name

# [OPTIONAL] create conda environment
conda create -n myenv python=3.9
conda activate myenv

# install pytorch according to instructions
# https://pytorch.org/get-started/

# install requirements
pip install -r requirements.txt

🐍 Conda Installation

# clone project
git clone https://github.com/YourGithubName/your-repo-name
cd your-repo-name

# create conda environment and install dependencies
conda env create -f environment.yaml -n myenv

# activate conda environment
conda activate myenv

🏃 How to Run

Train model with default configuration:

# train on CPU
python src/train.py trainer=cpu

# train on GPU
python src/train.py trainer=gpu

Train with specific experiment configuration:

python src/train.py experiment=experiment_name.yaml

Override parameters from command line:

python src/train.py trainer.max_epochs=20 data.batch_size=64