Story Crowdsource Preference System

🌟 We're creating an open source dataset for story preferences! Star this repository to be notified when we release it. 🌟

A comprehensive system for collecting, analyzing, and learning from story preferences using AI models and human feedback. This project combines multiple AI models with human feedback to improve story generation and preference learning.

Overview

The system consists of several key components:

1. Story Generation: Uses multiple AI providers (GitHub Models, Gemini, Azure) to generate story variations from prompts
2. Feedback Collection: Web interface for collecting human preferences between story variations
3. Embedding Processing: Generates and stores embeddings for story variations
4. Reward Model Training: Trains a reward model based on collected preferences
5. Dataset Generation: Creates preference datasets suitable for Direct Preference Optimization (DPO)

Key Highlights

• Open Source Dataset: We will open source the dataset here in this repository, making it a valuable resource for both the tech and writer communities.
• Live Demo: Try our Story Preference Collection App to contribute your preferences
• Streamlit + Supabase: The system is built using Streamlit for the web interface and Supabase for data management.
• Benchmarking Tool: This dataset will be instrumental in benchmarking Large Language Models (LLMs) for creative writing.

Features

• Multi-model story generation with configurable provider selection
• Interactive web interface for story comparison and feedback collection
• Automated embedding generation for story variations
• Customizable reward model training with source weighting
• Export capabilities for DPO-compatible datasets
• Supabase integration for data storage and management

Participate and Contribute

• ⭐ Star the Repository: Your first step! Star this repository to show your support and be notified when we release the dataset.
• Input Your Preferences: Visit our Story Preference Collection App to input your story preferences and contribute to the dataset.

Prerequisites

• Python 3.10+
• PyTorch
• CUDA-compatible GPU (optional, but recommended for training)
• Supabase account and project
• API keys for the supported providers (GitHub Models, Gemini, Azure)

Installation

1. Clone the repository:

git clone https://github.com/clchinkc/story_crowdsource_preference.git
cd story_crowdsource_preference

2. Install dependencies:

pip install -r requirements.txt

3. Set up environment variables: Create a .env file with the following variables:

SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key
GITHUB_TOKEN=your_github_token
GEMINI_API_KEY=your_gemini_api_key
AZURE_UST_SECONDARY_KEY=your_azure_key

Components

Story Dataset Generator (story_dataset_generator.py)

• Generates story prompts and variations using multiple AI models
• Handles story comparison and evaluation
• Integrates with Supabase for data storage

Feedback Collection App (story_feedback_app.py)

• Streamlit-based web interface
• Allows users to compare and rate story variations
• Exports collected preferences in DPO format

Embedding Processor (story_embedding_processor.py)

• Processes story variations to generate embeddings
• Uses ModernBERT for embedding generation
• Manages embedding storage in Supabase

Reward Model Training (story_ranking_dataset.py)

• Implements preference learning from collected feedback
• Supports weighted training based on feedback source
• Includes model evaluation and testing capabilities

Usage

1. Start the feedback collection app:

streamlit run story_feedback_app.py

2. Generate story datasets:

python story_dataset_generator.py

3. Process embeddings:

python story_embedding_processor.py

4. Train the reward model:

python story_ranking_dataset.py

Configuration

Provider Configuration

The system supports multiple AI providers with configurable models. The batch generator's providers_config (in story_dataset_generator.py):

providers_config = {
    "prompt": {
        "provider": "github",
        "model": "openai/gpt-5.6-luna"
    },
    "variations": [
        {
            "provider": "gemini",
            "model": "gemini/gemini-3.5-flash-lite"
        },
        {
            "provider": "azure",
            "model": "azure/gpt-5.6-luna"
        }
    ],
    "evaluation": {
        "provider": "azure",
        "model": "azure/gpt-5.6-luna"
    }
}

The Streamlit app (story_feedback_app.py) keeps its own PROVIDERS_CONFIG with an available_models list for the comparison UI.

Training Configuration

Customize reward model training parameters (see train_and_evaluate() in story_ranking_dataset.py for the full config):

config = {
    'batch_size': 4,
    'num_epochs': 5,
    'learning_rate': 3e-5,
    'test_size': 0.2,
    'source_weights': {
        'model': 1.0,   # Full weight to reward model feedback
        'llm': 0.5,     # Half weight to LLM feedback
        'human': 2.0    # Double weight to human feedback
    }
}

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

• ModernBERT by AnswerDotAI
• Streamlit for the web interface
• Supabase for database services