KESBEES - Key Stroke Based Essay Examination System

Bachelor of Technology Project
Department of Information Technology and Computer Applications
Andhra University College of Engineering, Visakhapatnam
Academic Year: 2020-2024

---

📋 Project Overview

KESBEES is an intelligent examination proctoring system that combines keystroke dynamics and eye gaze tracking to ensure exam integrity and automate essay evaluation. The system uses machine learning algorithms to verify user identity during online examinations and detect potential impersonation attempts.

Team Members

• Pothabathula Bala Mahendra (320106411035)
• Prathipati Jithendra (320106411036)

Project Guide

• Prof. V. Valli Kumari, Professor, Department of IT & Computer Applications

---

🎯 Abstract

The KESBEES system creates a smart examination environment that:

• Evaluates essay writing quality objectively
• Verifies student identity through behavioral biometrics
• Monitors exam integrity using real-time gaze tracking
• Provides fair assessment beyond simple word counting

By analyzing typing patterns (speed, pauses, rhythm) and eye movements, the system builds a behavioral profile during a mock test and compares it against real-time exam behavior to detect impersonation.

---

✨ Key Features

1. Keystroke Biometric Authentication

• Captures typing dynamics: speed, rhythm, pause patterns
• Analyzes production rate, word count, and typing consistency
• Uses Euclidean distance metrics for pattern matching
• Threshold-based authenticity scoring (configurable at 0.40)

2. Eye Gaze Tracking

• Monitors where users look during typing (left/center/right keyboard zones)
• Uses facial landmark detection (468 points) via CNN models
• Trains on mock test data to establish baseline behavior
• Real-time comparison during actual examination

3. Face Detection & Recognition

• Detects unauthorized users via face-api.js
• Monitors face absence (10-second tolerance)
• Automatic violation tracking with termination protocol

4. Security Monitoring

• Tab switch detection (prevents external resource access)
• Multi-face detection (detects proxy test-takers)
• Violation counting system (1 warning → 2nd strike = termination)
• Real-time proctoring dashboard

5. Two-Phase Examination

• Mock Test (3 minutes): Captures baseline typing + gaze patterns
• Real Exam (30 minutes): Essay writing with continuous monitoring
• Automatic data collection and ML-based comparison

---

🏗️ System Architecture

┌─────────────────────────────────────────────────────────────┐
│                    KESBEES SYSTEM                           │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌──────────────┐         ┌──────────────┐                │
│  │ Registration │────────▶│  Mock Test   │                │
│  │   Interface  │         │  (Training)  │                │
│  └──────────────┘         └──────┬───────┘                │
│                                   │                         │
│                                   ▼                         │
│                          ┌─────────────────┐               │
│                          │  Feature Store  │               │
│                          │  - Keystroke    │               │
│                          │  - Gaze Data    │               │
│                          │  - Face Profile │               │
│                          └────────┬────────┘               │
│                                   │                         │
│                                   ▼                         │
│                          ┌─────────────────┐               │
│                          │   Real Exam     │               │
│                          │  + Monitoring   │               │
│                          └────────┬────────┘               │
│                                   │                         │
│                                   ▼                         │
│  ┌──────────────────────────────────────────────┐         │
│  │         ML Analysis Engine                    │         │
│  │  - Voting Regressor (Keystroke)              │         │
│  │  - Voting Classifier (Gaze Direction)        │         │
│  │  - Face Recognition (face-api.js)            │         │
│  └──────────────────┬───────────────────────────┘         │
│                     │                                       │
│                     ▼                                       │
│            ┌─────────────────┐                             │
│            │  Result Output  │                             │
│            │  - Score        │                             │
│            │  - Status       │                             │
│            │  - Violations   │                             │
│            └─────────────────┘                             │
└─────────────────────────────────────────────────────────────┘

---

🔬 Technical Implementation

Machine Learning Models

1. Keystroke Analysis - Voting Regressor

Ensemble model combining:

• Random Forest Regressor
• Gradient Boosting Regressor
• Extra Trees Regressor
• XGBoost Regressor

Features Extracted:

• Production rate (process & product)
• Word count and paragraph count
• Key press/release timing
• Pause patterns
• Text modification behavior (input/delete/cut)

Performance Metrics:

• R² Score: 0.92+ (high accuracy)
• RMSE: Low error rates across test samples

2. Gaze Direction - Voting Classifier

Classifies eye gaze into 3 zones:

• Left: Looking at left keyboard region
• Center: Looking at center keyboard region
• Right: Looking at right keyboard region

Features:

• 468 facial landmarks (MediaPipe)
• Eye iris positions (left/right)
• Eye corner coordinates
• Euclidean distance calculations

3. Face Recognition - CNN Models

• Pre-trained face-api.js models
• Face descriptor generation
• Distance-based matching (threshold: 0.50)
• Multi-face detection capability

Technology Stack

Frontend:

• HTML5, CSS3, JavaScript (ES6+)
• TensorFlow.js for ML inference
• face-api.js for facial recognition
• MediaPipe for landmark detection

Backend:

• Python 3.8+
• Flask web framework
• scikit-learn for ML models
• pandas, numpy for data processing

Libraries & Frameworks:

• OpenCV (computer vision)
• scikit-learn (machine learning)
• pickle (model serialization)
• JSON (data interchange)

---

📊 Feature Engineering

Keystroke Features

Feature	Description
production_rate_process	Characters typed per second
production_rate_product	Words produced per minute
word_count	Total words in essay
paragraph_count	Total paragraphs written
action_time	Time between keystrokes
activity	Input/Remove/Nonproduction

Gaze Features (468 Landmarks)

Landmark Points Used:
- Iris Centers: 468-476 (left), 469-471 (right)
- Eye Corners: 263, 362 (left), 33, 133 (right)
- Calculated: Iris midpoints, eye midpoints, euclidean distances

---

🚀 Installation & Setup

Prerequisites

Python 3.8+
pip (Python package manager)
Webcam-enabled device
Modern web browser (Chrome/Firefox recommended)

Step 1: Clone Repository

git clone https://github.com/YOUR_USERNAME/KESBEES.git
cd KESBEES

Step 2: Create Virtual Environment

# Windows
python -m venv myenv
myenv\Scripts\activate

# Linux/Mac
python3 -m venv myenv
source myenv/bin/activate

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Run Application

python application.py

Step 5: Access System

Open browser and navigate to:

http://127.0.0.1:5000

---

📁 Project Structure

KESBEES/
│
├── application.py              # Main Flask application
├── questions.json              # Essay question bank
├── requirements.txt            # Python dependencies
├── .gitignore                  # Git ignore rules
│
├── templates/
│   ├── index.html             # Registration page
│   └── mock.html              # Exam interface (mock + real)
│
├── static/
│   ├── models/                # Pre-trained ML models
│   │   ├── face_landmark_detection_model-shard*
│   │   ├── face_recognition_model-shard*
│   │   ├── ssd_mobilenetv1_model-shard*
│   │   └── tiny_face_detector_model-shard*
│   └── w3.css                 # Styling framework
│
├── src/
│   ├── pipeline/
│   │   └── predict_pipeline.py    # ML prediction pipelines
│   ├── exception.py               # Custom exception handling
│   └── logger.py                  # Logging configuration
│
├── artifacts/                      # Trained model artifacts
│   ├── model.pkl                  # Voting regressor
│   ├── preprocessor.pkl           # Data preprocessor
│   └── direction.pkl              # Gaze classifier
│
├── notebooks/                      # Jupyter notebooks (optional)
│   ├── data_exploration.ipynb
│   └── model_training.ipynb
│
└── README.md                       # This file

---

🎮 Usage Guide

For Exam Administrators

1. Configure System

   • Set exam duration (default: 30 minutes)
   • Set mock test duration (default: 3 minutes)
   • Configure detection thresholds
   • Upload essay questions to questions.json

2. Monitor Exams

   • View real-time violation logs
   • Check student authentication status
   • Review keystroke similarity scores

For Students

1. Registration

   • Enter name and registration ID
   • Grant camera permissions
   • Click "Start Mock Test"

2. Mock Test (3 minutes)

   • Type randomly generated characters
   • Look at keyboard while typing
   • System captures baseline behavior
   • Do not switch tabs or look away

3. Real Exam (30 minutes)

   • Read essay question carefully
   • Write essay (minimum 200 words, 3 paragraphs)
   • Stay focused on screen
   • System monitors behavior continuously

4. Submit Exam

   • Click submit button
   • View your keystroke similarity score
   • See authentication status (GENUINE/IMPOSTOR)

---

⚙️ Configuration

Threshold Settings (application.py)

# Keystroke Authentication
KEYSTROKE_THRESHOLD = 0.40  # Lower = stricter matching
# Score > 0.40 → GENUINE
# Score < 0.40 → IMPOSTOR DETECTED

# Gaze Direction
DIRECTION_THRESHOLD = 0.65  # Confidence threshold

# Face Recognition
FACE_DISTANCE_THRESHOLD = 0.50  # Lower = stricter matching

# Security Limits
MAX_FACE_ABSENCE_TIME = 10  # seconds
MAX_VIOLATIONS = 1  # warnings before termination
MAX_UNAUTHORIZED_STRIKES = 2

Question Bank (questions.json)

[
    {
        "question": "Discuss the impact of artificial intelligence on modern education systems.",
        "topic": "Technology & Education"
    },
    {
        "question": "Analyze the role of renewable energy in combating climate change.",
        "topic": "Environment"
    }
]

---

📈 Model Performance

Keystroke Analysis Results

Model	R² Score	RMSE
Random Forest	0.94	0.08
Gradient Boosting	0.93	0.09
Extra Trees	0.92	0.10
Voting Regressor	0.95	0.07

Gaze Classification Results

• Training Accuracy: 89%
• Test Accuracy: 87%
• 3-class classification (Left/Center/Right)

---

🔒 Security Features

1. Behavioral Biometrics

   • Unique typing patterns per user
   • Difficult to replicate or spoof
   • Continuous authentication

2. Multi-Modal Verification

   • Keystroke + Gaze + Face recognition
   • Reduces false positive rates
   • Comprehensive impersonation detection

3. Real-Time Monitoring

   • Tab switch detection
   • Face absence tracking
   • Violation logging

4. Automatic Termination

   • Threshold-based enforcement
   • Immediate exam suspension on violation
   • Prevents cheating attempts

---

🐛 Known Issues & Limitations

1. Lighting Conditions

   • Poor lighting affects face detection accuracy
   • Recommendation: Well-lit environment required

2. Camera Quality

   • Low-resolution cameras reduce landmark detection
   • Recommendation: Minimum 720p webcam

3. False Positives

   • Different typing speeds (nervousness, fatigue)
   • Different keyboard layouts (laptop vs desktop)
   • Solution: Adjustable threshold parameters

4. Network Latency

   • Slow connections may affect real-time monitoring
   • Recommendation: Stable internet connection

---

🔮 Future Enhancements

Planned Features

1. Mouse Dynamics Integration

   • Track cursor speed, acceleration, click patterns
   • Enrich behavioral profile

2. Real-Time Feedback

   • Provide writing suggestions during exam
   • Adaptive difficulty based on performance

3. Audio/Video Analysis

   • Voice recognition for oral exams
   • Emotion detection via facial expressions

4. Enhanced Accessibility

   • Screen reader support
   • Alternative input methods for disabled users

5. Teacher Dashboard

   • Comprehensive analytics
   • Batch processing for multiple students
   • Integration with LMS platforms

6. Advanced Security

   • Blockchain-based exam records
   • Encrypted data transmission
   • Multi-factor authentication

---

📚 References

Research Papers

1. Kevin S. Killourhy, Roy A. Maxion, "Comparing Anomaly-Detection Algorithms for Keystroke Dynamics," IEEE/IFIP International Conference on Dependable Systems & Networks, 2009.

2. Nuwan Kaluarachchi et al., "A New Distance-Based Feature Set for Keystroke Dynamics," IEEE Conference Paper, 2023.

3. Bernard Aldrich et al., "Translating Keystroke and Mouse Dynamics Data to Classify Human Mood," IEEE Conference Paper, 2023.

4. P. Kasprowski et al., "Biometric identification based on keystroke dynamics," Sensors, vol. 22, no. 9, p. 3158, 2022.

5. A. Acien et al., "Typenet: Deep learning keystroke biometrics," IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 4, no. 1, pp. 57–70, 2021.

6. T. Wu et al., "User identification by keystroke dynamics based on feature correlation analysis," IEEE ICCC, pp. 40–46, 2019.

---

📄 License

This project is developed as an academic requirement for Bachelor of Technology degree.

© 2024 Pothabathula Bala Mahendra, Prathipati Jithendra
Andhra University College of Engineering, Visakhapatnam

---

🙏 Acknowledgments

We express our sincere gratitude to:

• Prof. V. Valli Kumari - Project Guide, for invaluable guidance and support
• Prof. Kunjam Nageswara Rao - Head of Department, for constant inspiration
• Prof. G. Sasi Bhushana Rao - Principal, AUCE, for providing facilities
• Prof. P.V.G.D. Prasad Reddy - Vice-Chancellor, Andhra University
• Department faculty, lab technicians, and support staff
• Our families and friends for unwavering support

---

📞 Contact

For queries regarding this project:

• Pothabathula Bala Mahendra - 320106411035
• Prathipati Jithendra - 320106411036

Institution: Department of Information Technology and Computer Applications
Address: Andhra University College of Engineering, Visakhapatnam-530003, India

---

🌟 Project Status

✅ Core functionality implemented
✅ ML models trained and deployed
✅ Real-time monitoring active
✅ Security features operational
🔄 Future enhancements in planning

Last Updated: November 2024