Instructor: Dr. Hakan Emekci
Assistant: Daniel Quillan Roxas
Course Duration: 14 Weeks
Credits: 3
Prerequisites: Python programming, basic machine learning concepts, linear algebra
This course provides a comprehensive and practical introduction to Large Language Models (LLMs) through hands-on implementation and experimentation. Students will learn the fundamental concepts, practical applications, and advanced techniques for working with LLMs, from understanding their inner workings to fine-tuning and deployment. The course follows the O'Reilly book "Hands-On Large Language Models" by Jay Alammar and Maarten Grootendorst, emphasizing visual learning and practical implementation.
By the end of this course, students will be able to:
- Understand the fundamental architecture and mechanisms of transformer-based LLMs
- Implement text classification, clustering, and semantic search systems using pre-trained models
- Design and optimize prompts for various NLP tasks
- Build retrieval-augmented generation (RAG) systems
- Fine-tune language models for specific tasks
- Develop multimodal applications using vision-language models
- Deploy and scale LLM applications in production environments
- Final Project: 60%
- Homework Assignments: 20%
- Project Presentation: 20%
- Primary Textbook: "Hands-On Large Language Models" by Jay Alammar and Maarten Grootendorst
- Original Code Repository: GitHub - HandsOnLLM. Note: The notebooks in this repository are extended versions of the ones from the book.
- Platform: Google Colab (recommended) or local Python environment
- Hardware: Access to GPU resources (T4 or better)
Topic: Introduction to Language Models
Chapter: 1 - Introduction to Language Models
Content:
- Historical evolution of language AI
- Overview of transformer architecture
- Current LLM landscape and capabilities
- Setting up development environment
- First hands-on experience with pre-trained models
Lab: Getting started with HuggingFace transformers
Assignment: HW1 - Environment setup and basic model interaction
Topic: Token Embeddings
Chapter: 2 - Token Embeddings
Content:
- Tokenization strategies (BPE, WordPiece, SentencePiece)
- Vector representations of text
- Embedding spaces and semantic relationships
- Subword tokenization implementation
Lab: Building custom tokenizers and exploring embedding spaces
Assignment: HW2 - Tokenization analysis and embedding visualization
Topic: Looking Inside Transformer LLMs
Chapter: 3 - Looking Inside Transformer LLMs
Content:
- Self-attention mechanisms in detail
- Multi-head attention and positional encoding
- Feed-forward networks and layer normalization
- Decoder-only vs encoder-decoder architectures
Lab: Implementing attention mechanisms from scratch
Assignment: HW3 - Attention pattern analysis
Topic: Text Classification
Chapter: 4 - Text Classification
Content:
- Classification head design
- Fine-tuning strategies for classification
- Evaluation metrics and best practices
- Handling imbalanced datasets
Lab: Building a sentiment analysis system
Assignment: HW4 - Multi-class text classification project
Topic: Text Clustering and Topic Modeling
Chapter: 5 - Text Clustering and Topic Modeling
Content:
- Embedding-based clustering techniques
- Topic modeling with BERTopic
- Dimensionality reduction for text
- Evaluation of clustering quality
Lab: Discovering topics in large document collections
Assignment: HW5 - Document clustering and topic analysis
Topic: Prompt Engineering
Chapter: 6 - Prompt Engineering
Content:
- Prompt design principles and strategies
- Few-shot and zero-shot learning
- Chain-of-thought prompting
- Prompt optimization techniques
Lab: Advanced prompting strategies and evaluation
Assignment: HW6 - Prompt engineering for specific tasks
Topic: Advanced Text Generation Techniques and Tools
Chapter: 7 - Advanced Text Generation Techniques and Tools
Content:
- Decoding strategies (greedy, beam search, sampling)
- Temperature and top-k/top-p sampling
- Constrained generation and guided decoding
- Text generation evaluation metrics
Lab: Implementing custom generation strategies
Assignment: Project Proposal Due
Topic: Semantic Search and Retrieval Augmented Generation
Chapter: 8 - Semantic Search and Retrieval Augmented Generation
Content:
- Dense retrieval systems
- Vector databases and similarity search
- RAG architecture and implementation
- Evaluation of retrieval quality
Lab: Building a question-answering system with RAG
Assignment: HW7 - Semantic search implementation
Topic: Multimodal Large Language Models
Chapter: 9 - Multimodal Large Language Models
Content:
- Vision-language model architectures
- Image captioning and visual question answering
- Cross-modal understanding and generation
- Multimodal prompt engineering
Lab: Building vision-language applications
Assignment: HW8 - Multimodal application development
Topic: Creating Text Embedding Models
Chapter: 10 - Creating Text Embedding Models
Content:
- Training embedding models from scratch
- Contrastive learning principles
- Sentence-BERT and similar architectures
- Domain-specific embedding training
Lab: Training custom embedding models
Assignment: HW9 - Custom embedding model training
Topic: Fine-Tuning Representation Models for Classification
Chapter: 11 - Fine-Tuning Representation Models for Classification
Content:
- Transfer learning strategies
- Layer freezing and gradual unfreezing
- Learning rate scheduling
- Overfitting prevention techniques
Lab: Advanced fine-tuning techniques
Assignment: HW10 - Model fine-tuning optimization
Topic: Fine-Tuning Generation Models
Chapter: 12 - Fine-Tuning Generation Models
Content:
- Instruction tuning and RLHF
- LoRA and other parameter-efficient methods
- Safety and alignment considerations
- Evaluation of fine-tuned models
Lab: Fine-tuning language models for specific domains
Final Project Check-in
Topic: Final Project Presentations - Session 1
Content:
- Student presentations of final projects (Groups 1-4)
- Peer review and feedback
- Discussion of implementation challenges
- Q&A and technical discussions
Deliverable: Final project presentations (Groups 1-4)
Topic: Final Project Presentations - Session 2 & Course Summary
Content:
- Student presentations of final projects (Groups 5-8)
- Course retrospective and key learnings
- Industry trends and future directions
- Career paths in LLM development
Deliverable:
- Final project presentations (Groups 5-8)
- Final project report due
- Peer evaluation forms
Students will work in teams of 2-3 to develop a substantial LLM-based application. Projects should demonstrate mastery of course concepts and include:
Technical Components:
- Implementation of at least 3 major concepts from the course
- Novel application or significant extension of existing techniques
- Proper evaluation methodology and metrics
- Code documentation and reproducibility
Project Ideas:
- Domain-specific chatbot with RAG
- Multimodal content generation system
- Custom fine-tuned model for specialized tasks
- LLM-powered data analysis platform
- Creative writing assistant with style transfer
- Code generation and debugging assistant
Deliverables:
- Project proposal (Week 7)
- Mid-term check-in (Week 12)
- Final presentation (Weeks 13-14)
- Final report (15-20 pages)
- Complete codebase with documentation
- Duration: 15 minutes presentation + 5 minutes Q&A
- Content: Problem statement, methodology, results, demo, challenges, future work
- Technical demo: Live demonstration of working system
- Submission: All assignments via course management system
- Late Policy: 10% deduction per day late
- Collaboration: Individual work unless specified otherwise
- Code Quality: Emphasis on clean, documented, reproducible code
- Technical Innovation: 40%
- Implementation Quality: 30%
- Evaluation & Analysis: 20%
- Documentation: 10%
- Correctness: 60%
- Code Quality: 25%
- Analysis & Insights: 15%
- Technical Content: 40%
- Clarity of Communication: 30%
- Demo Quality: 20%
- Q&A Handling: 10%
Regular attendance is expected. Notify instructor in advance for planned absences.
All work must be original. Proper citation required for external code and resources. AI tools may be used for learning but not for homework completion.
Students with disabilities should contact the instructor to discuss accommodations.
Instructor: Dr. Hakan Emekci
Office Hours: [To be scheduled based on class availability]
Contact: [Email address]
- "Attention Is All You Need" (Vaswani et al., 2017)
- "BERT: Pre-training of Deep Bidirectional Transformers" (Devlin et al., 2018)
- "Language Models are Few-Shot Learners" (Brown et al., 2020)
- Recent papers from arXiv and top-tier conferences
- HuggingFace Documentation and Tutorials
- OpenAI API Documentation
- Papers With Code (LLM section)
- Distill.pub visualization articles
- Python, PyTorch, HuggingFace Transformers
- Weights & Biases for experiment tracking
- Vector databases (Pinecone, Weaviate, Chroma)
- Docker for deployment
This syllabus is subject to modifications based on class progress and emerging developments in the field.