Course: Compiler Design
Institution: Notre Dame University Bangladesh (NDUB)
Lecturer: Humayara Binte Rashid
Email: humayara@ndub.edu.bd
- Syed Nafish Shakir
- Joynob Bint Jamal
- Shanon Sheikh
This project is a lexical analyzer (tokenizer) for the C programming language, developed as part of the Compiler Design course. The analyzer is built using Flex/Lex and can identify various lexical elements in C source code including keywords, identifiers, operators, literals, and separators while providing detailed statistical analysis.
Status:
Through this project, we aim to understand:
- Phase 1 of Compilation: Lexical Analysis
- Regular Expressions: Pattern matching for token recognition
- Finite Automata: Theory behind lexical analyzers
- Flex/Lex Tool: Practical implementation of lexical analyzers
- Token Classification: Different types of lexical units in programming languages
- C Keywords: All 32 C keywords (
int,float,if,while,return, etc.) - Identifiers: Variable and function names with length information
- Numeric Literals:
- Integers (e.g.,
42,123) - Floating-point numbers with scientific notation (e.g.,
3.14,1.5e-10)
- Integers (e.g.,
- Relational Operators:
<,<=,>,>=,==,!= - Separators/Punctuation:
;,,,.,:,#,(,),{,},[,] - Arrow Operator:
->
- File-based Input: Reads C source files from command line
- Statistical Counting: Words, spaces, tabs, and lines
- Error Handling: File validation and user-friendly error messages
- Formatted Output: Clear token classification and summary
| Category | Examples | Status |
|---|---|---|
| Keywords | int, if, return |
✅ Complete |
| Identifiers | main, variable_name |
✅ Complete |
| Integer Literals | 42, 999 |
✅ Complete |
| Float Literals | 3.14, .5 |
✅ Complete |
| Relational Operators | <, <=, ==, != |
✅ Complete |
| Separators | (, ), {, }, ; |
✅ Complete |
| Assignment Operators | =, +=, -= |
✅ Complete |
| Arithmetic Operators | +, -, *, /, % |
✅ Complete |
| String Literals | "Hello World" |
✅ Complete |
| Character Literals | 'a', '\n' |
✅ Complete |
| Comments | //, /* */ |
✅ Complete |
- Setup Flex environment
- Define basic token patterns
- Implement keyword recognition
- Add identifier and literal support
- File input/output handling
- Statistical counting
- Assignment operators (
=,+=,-=, etc.) - Arithmetic operators (
+,-,*,/,%) - Logical operators (
&&,||,!) - String and character literals
- Preprocessor directives
- Comment handling (
//,/* */) - Error recovery and reporting
- Symbol table integration
- Multi-file analysis
- Token position tracking
- Assignment operators show as "Unknown character"
- String literals are not properly tokenized
- Comments are not handled
- Arithmetic operators need implementation
- Escape sequences in strings/characters not supported
- Basic C Program: Simple main function with variables
- Control Structures:
if-else, loops,switchstatements - Functions: Function declarations and calls
- Data Types: All C primitive types
- Edge Cases: Empty files, syntax errors
Disclaimer: This is an educational project for Notre Dame University Bangladesh's Compiler Design course. The implementation is for learning purposes and may not be suitable for production use.
Last Updated: October 2025
Project Status: Work in Progress