Skip to content

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🤖 Mr.SONU — Chapter 1

Raspberry Pi Voice–Vision Robotics Platform

Mr.SONU Robot

A modular robotics platform integrating voice interaction, computer vision, motor control, safety supervision, and autonomous system foundations.

Raspberry Pi 5 Python OpenCV VOSK TB6612FNG Chapter 1 Complete


📖 Overview

Mr.SONU is a Raspberry Pi–based robotics platform developed as a foundation for a voice- and vision-enabled autonomous robot.

Chapter 1 focuses on establishing a stable, modular, and resilient robotics core rather than attempting full autonomy immediately.

The system brings together independent subsystems for:

  • 🎙️ Voice interaction
  • 👁️ Computer vision
  • 🛞 Motor control
  • 🛡️ Safety and emergency handling
  • 🧠 Autonomy and decision logic
  • 🔄 Process supervision and recovery

The project was developed incrementally through hardware experimentation, subsystem validation, software integration, failure handling, and repeated real-world testing.


✨ Chapter 1 Capabilities

🎙️ Voice System

The voice subsystem provides the primary human–robot interaction layer.

  • Wake-word based interaction
  • Offline speech recognition using VOSK
  • Text-to-speech responses
  • Voice-driven robot interaction
  • Multiple personality implementations explored during development
  • Dedicated voice worker architecture
  • Speech-processing experiments preserved in the repository history

👁️ Vision System

The vision subsystem provides SONU with camera input and the foundation for visual perception.

  • Raspberry Pi camera integration
  • Camera device validation
  • OpenCV-based camera testing
  • Frame capture and processing
  • YUYV / MJPEG camera testing
  • Dedicated vision worker architecture
  • Resilient camera handling
  • Image transmission experiments
  • Vision-driven action experiments

🛞 Motor Control

The motor subsystem provides controlled physical movement while prioritizing safe operation.

  • TB6612FNG motor driver support
  • PWM-based motor speed control
  • Smooth acceleration and deceleration
  • Toggle-based movement control
  • Emergency-stop integration
  • Safe motor shutdown
  • pigpio-based testing
  • No-pigpio fallback testing
  • Hardware-level motor experimentation

🛡️ Safety & Supervision

Safety is treated as a separate system responsibility rather than being embedded entirely inside movement logic.

  • Emergency monitoring
  • Emergency-stop callback handling
  • Motor kill-switch logic
  • Worker supervision
  • Automatic subsystem restart
  • Fault-aware process management
  • Safe shutdown behavior

🧠 Autonomy Foundation

Chapter 1 establishes the software foundation required for higher-level autonomous behavior.

The architecture separates perception, interaction, safety, decision-making, and physical action so that future capabilities can be added without restructuring the entire robot.

At a high level:

Voice ───────┐
             │
Vision ──────┤
             ├──> Decision / Autonomy Layer
Motors ──────┤
             │
Safety ──────┘
                    │
                    ▼
               Robot Action

The current chapter therefore serves as the foundation layer for future perception, reasoning, navigation, and autonomous decision-making systems.


🎯 Engineering Goals

The primary goals of Chapter 1 were:

  1. Modularity — keep robot subsystems independently testable.
  2. Reliability — prevent individual worker failures from bringing down the entire system.
  3. Safety — ensure physical movement can be interrupted and controlled safely.
  4. Maintainability — organize the evolving codebase into clear functional modules.
  5. Experimentation — preserve earlier implementations and hardware experiments for reference.
  6. Extensibility — provide a foundation for future AI and autonomous robotics capabilities.

The result is a robotics codebase designed to evolve incrementally rather than becoming dependent on a single monolithic program.

🏗️ System Architecture

Mr.SONU follows a modular, worker-based architecture designed to keep hardware control, perception, interaction, and safety logic separated.

                         ┌─────────────────────────────┐
                         │       Supervisor Layer      │
                         │     Process Management      │
                         └──────────────┬──────────────┘
                                        │
              ┌─────────────────────────┼─────────────────────────┐
              │                         │                         │
              ▼                         ▼                         ▼
      ┌───────────────┐         ┌───────────────┐         ┌───────────────┐
      │ Voice Worker  │         │ Vision Worker │         │ Motor Worker  │
      │               │         │               │         │               │
      │ VOSK + TTS    │         │ Camera + CV   │         │ PWM + Driver  │
      └───────┬───────┘         └───────┬───────┘         └───────┬───────┘
              │                         │                         │
              └─────────────────────────┼─────────────────────────┘
                                        │
                                        ▼
                              ┌─────────────────────┐
                              │ Safety / Emergency  │
                              │ Monitoring Layer    │
                              └──────────┬──────────┘
                                         │
                                         ▼
                              ┌─────────────────────┐
                              │ Decision / Autonomy │
                              │       Layer         │
                              └──────────┬──────────┘
                                         │
                                         ▼
                                  ┌──────────────┐
                                  │ Robot Action │
                                  └──────────────┘

The architecture intentionally separates responsibilities between subsystems so that individual components can be tested, replaced, or extended without requiring a complete rewrite of the robot software.

🔄 Worker-Based Execution

The major runtime components are organized as independent workers:

  • Voice Worker — handles speech recognition and voice interaction.
  • Vision Worker — manages camera access, frame acquisition, and visual processing.
  • Motor Worker — controls motor movement and speed through the motor driver.
  • Safety Components — monitor emergency conditions and provide motor-stop mechanisms.
  • Supervisor — manages worker processes and provides resilience against individual subsystem failures.
  • Autonomy Layer — provides the foundation for future decision-making and autonomous behavior.

This separation is particularly important for physical robotics because a failure in a non-critical subsystem should not automatically result in uncontrolled motor behavior.


📁 Repository Structure

The repository was reorganized during the Chapter 1 cleanup to separate production-oriented source code, hardware tests, archived experiments, documentation, scripts, and media assets.

Mr.SONU-chapter-1-main/
│
├── archive/
│   ├── emergency_monitor_flag.py
│   ├── emergency_monitor_flag_v3.py
│   ├── motor_control_*.py
│   ├── sonu_voice_*.py
│   ├── sonu_voice_vision_*.py
│   ├── ultratest_*.py
│   └── ...
│
├── assets/
│   ├── images/
│   │   ├── development/
│   │   ├── gallery/
│   │   ├── hardware/
│   │   ├── hero/
│   │   └── testing/
│   │
│   └── videos/
│       ├── camera-llm-test.mp4
│       ├── final-test-1.mp4
│       ├── motor-control-voice.mp4
│       ├── sonu-inaction.mp4
│       ├── ultrasonic-sensor-test.mp4
│       └── voice-motor-test.mp4
│
├── docs/
│
├── examples/
│   └── pi_send_image.py
│
├── scripts/
│   └── run_monitor_and_controller.sh
│
├── src/
│   ├── core/
│   │   ├── motor_drive.py
│   │   ├── sonu_autonomy.py
│   │   └── __init__.py
│   │
│   ├── motion/
│   │   ├── motor_cli.py
│   │   ├── motor_control_toggle2_smooth.py
│   │   └── __init__.py
│   │
│   ├── safety/
│   │   ├── emergency_monitor_flag_v2.py
│   │   ├── emergency_stop_cb.py
│   │   └── __init__.py
│   │
│   ├── supervisor/
│   │   ├── supervisor_with_voice_worker.py
│   │   └── __init__.py
│   │
│   ├── vision/
│   │   ├── camera_worker.py
│   │   ├── sonu_vision.py
│   │   ├── sonu_vision_auto_action.py
│   │   └── __init__.py
│   │
│   ├── voice/
│   │   ├── sonu_voice.py
│   │   ├── sonu_voice_personality_fixed.py
│   │   ├── voice_worker.py
│   │   └── __init__.py
│   │
│   └── __init__.py
│
├── tests/
│   ├── cam_test.py
│   ├── debug_sensor_print.py
│   ├── motor_test_debug.py
│   ├── motor_test_no_pigpio.py
│   ├── motor_test_pigpio.py
│   ├── test_cam_device.py
│   ├── test_cam_opencv.py
│   ├── test_gpio_pigpio.py
│   └── transcribe_wav.py
│
├── .gitignore
└── README.md

🧩 Directory Responsibilities

Directory Purpose
src/ Current modular implementation of SONU's core subsystems
src/core/ Core robot logic and autonomy foundations
src/motion/ Motor-control interfaces and motion behavior
src/safety/ Emergency monitoring and motor-stop mechanisms
src/supervisor/ Worker/process supervision and resilience
src/vision/ Camera, vision processing, and vision-driven actions
src/voice/ Speech recognition, voice interaction, and worker execution
tests/ Hardware and subsystem validation programs
examples/ Small standalone examples for interacting with SONU components
scripts/ Runtime helper scripts
archive/ Earlier implementations and experiments retained for development history
assets/images/ Project, hardware, development, hero, and gallery imagery
assets/videos/ Demonstration and subsystem test recordings
docs/ Supporting project documentation

The archive/ directory is intentionally preserved rather than deleted. It provides a development history of the different approaches tested while evolving SONU's voice, vision, motor-control, safety, and supervision systems.


⚙️ Core Software Components

🎙️ Voice System

SONU's voice subsystem is designed around offline speech recognition and modular interaction logic.

Key components include:

  • Wake-word based interaction
  • Offline speech recognition using VOSK
  • Text-to-speech responses
  • Voice worker execution
  • Multiple personality and interaction implementations
  • Resilient worker-based execution

The final source implementation is located under:

src/voice/

Earlier experimental implementations remain available under:

archive/

This preserves the progression from initial voice-control experiments to the more structured worker-based architecture used in the current Chapter 1 foundation.


👁️ Vision System

The vision subsystem provides SONU with camera-based perception capabilities.

Current functionality includes:

  • Camera device validation
  • Frame acquisition
  • YUYV / MJPEG camera handling
  • OpenCV-based testing
  • Vision worker execution
  • Resilient camera processing
  • Vision-driven action foundations
  • Image transmission experiments

The current implementation is organized under:

src/vision/

Supporting camera and OpenCV validation programs are maintained under:

tests/

Example vision-related components include:

camera_worker.py
sonu_vision.py
sonu_vision_auto_action.py

The architecture leaves room for future computer-vision models and higher-level perception systems without coupling them directly to the motor-control implementation.


🛞 Motor Control

Motor control is one of the most hardware-sensitive parts of SONU.

The current architecture supports:

  • TB6612FNG motor driver control
  • PWM-based speed control
  • Smooth acceleration behavior
  • Safe movement toggling
  • Command-line motor control
  • Emergency stop callbacks
  • No-pigpio fallback testing

Current motion-related code is organized under:

src/motion/

with core motor functionality under:

src/core/

Hardware validation programs are maintained under:

tests/

The motor subsystem is designed with safety as a first-class requirement rather than treating movement as a simple GPIO operation.


🛡️ Safety & Supervision

Physical robotics introduces a different class of failure modes compared with ordinary software.

A software crash can be inconvenient.

A software crash while motors are running can become a physical safety problem.

For that reason, SONU's Chapter 1 architecture includes dedicated safety and supervision mechanisms.

Emergency Monitoring

Safety components provide mechanisms for:

  • Emergency state monitoring
  • Motor-stop callbacks
  • Kill-switch behavior
  • Controlled interruption of movement
  • Separation of safety handling from normal motion commands

Relevant implementations are located under:

src/safety/

Supervisor

The supervisor provides process-level resilience by managing worker execution.

Its responsibilities include:

  • Starting subsystem workers
  • Monitoring worker processes
  • Detecting worker failures
  • Supporting controlled restart behavior
  • Coordinating the runtime environment
  • Preventing individual subsystem failures from unnecessarily collapsing the entire system

The current supervisor implementation is located under:

src/supervisor/

🧠 Autonomy Foundation

Chapter 1 does not attempt to implement a complete autonomous intelligence system.

Instead, it establishes the software foundation required for one.

The autonomy layer is responsible for providing a future integration point between:

Perception
    ↓
Interpretation
    ↓
Decision Making
    ↓
Action
    ↓
Feedback

The current implementation is intentionally lightweight.

Future chapters can build on this foundation by introducing more advanced perception, reasoning, navigation, planning, and AI-assisted decision-making without restructuring the entire robot architecture.


🔬 Engineering Approach

The development of SONU followed an iterative hardware-first approach.

Rather than attempting to build the complete robot in one implementation, individual subsystems were developed and tested independently before being integrated.

The general development cycle was:

Hardware Experiment
        ↓
Standalone Test
        ↓
Subsystem Implementation
        ↓
Failure / Edge-Case Testing
        ↓
Safety Improvements
        ↓
Worker Integration
        ↓
Supervisor Integration
        ↓
System-Level Validation

This approach helped identify hardware-specific problems early and made it possible to preserve working implementations while continuing to experiment with new approaches.

The repository therefore contains both the current modular implementation and selected historical experiments, allowing the project to document not only the final result but also the engineering process used to reach it.

🔩 Hardware

Mr.SONU is built around a Raspberry Pi-based embedded robotics platform.

🧠 Compute

  • Raspberry Pi 5
  • Raspberry Pi OS during development and testing

🎙️ Audio

  • USB microphone for voice input
  • Text-to-speech output

👁️ Vision

  • 5MP camera module / compatible USB camera
  • Camera pipeline tested with OpenCV
  • Frame acquisition supporting YUYV / MJPEG camera formats

🛞 Motion

  • DC gear motors
  • Robot chassis and wheels
  • TB6612FNG motor driver
  • PWM-based motor speed control

An earlier stage of development also used an L298N motor driver. The related hardware experiments and connection references are retained in the repository for development history.

🔋 Power

  • Rechargeable battery system
  • TP4056 charging module
  • External power bank / battery power during development and testing

📡 Sensors

  • Ultrasonic distance sensing
  • GPIO-based hardware interfaces

The hardware configuration evolved throughout development as different motor drivers, power arrangements, camera configurations, and control approaches were tested.


🧪 Testing & Validation

Hardware robotics requires validation at both the software and physical-system levels.

SONU therefore includes dedicated test programs for individual subsystems.

Motor Testing

The repository contains multiple motor validation programs covering:

  • Basic motor operation
  • PWM control
  • pigpio-based control
  • No-pigpio fallback behavior
  • Debugging and diagnostic output
  • Safe movement testing

Relevant files are located under:

tests/

and the current motion implementation is located under:

src/motion/
src/core/

Camera Testing

Camera functionality was validated independently before being integrated into the larger system.

Testing includes:

  • Camera device detection
  • OpenCV capture
  • Camera format validation
  • Frame acquisition
  • Camera worker behavior

Relevant programs include:

tests/cam_test.py
tests/test_cam_device.py
tests/test_cam_opencv.py

GPIO Testing

GPIO functionality was also tested independently to verify hardware-level behavior before integrating it into higher-level robot control.

Relevant validation code includes:

tests/test_gpio_pigpio.py
tests/debug_sensor_print.py

Voice Testing

Voice functionality was developed through multiple iterations covering:

  • Speech recognition
  • Wake-word interaction
  • Voice responses
  • Personality behavior
  • Worker-based execution
  • Audio transcription experiments

Earlier implementations are preserved in:

archive/

while the current voice architecture is located under:

src/voice/

🎥 Demonstrations

The repository contains recorded demonstrations of SONU's development and subsystem testing.

Available recordings include:

Demonstration Description
voice-motor-test.mp4 Voice-driven motor control testing
motor-control-voice.mp4 Integrated voice and motor interaction
ultrasonic-sensor-test.mp4 Ultrasonic sensing validation
camera-llm-test.mp4 Camera and LLM integration experiment
sonu-inaction.mp4 Robot operation demonstration
final-test-1.mp4 System-level development test

All demonstration videos are stored under:

assets/videos/

These recordings document the practical behavior of the system and complement the source code by showing the hardware running in real conditions.


🖼️ Project Media

Project imagery is organized separately from source code to keep the repository clean and make the project easier to navigate.

Hero

The primary project image is available at:

assets/images/hero/sonu-hero.jpeg

Development

Development and build documentation images are stored under:

assets/images/development/

These include hardware assembly, wiring, setup, charging, and development-stage photographs.

Hardware

Hardware reference images are stored under:

assets/images/hardware/

This collection includes photographs of:

  • Raspberry Pi 5
  • Raspberry Pi GPIO
  • Camera module
  • Motor drivers
  • Chassis
  • Batteries
  • Charging hardware

Gallery

Finished robot photographs are stored under:

assets/images/gallery/

The gallery contains multiple views of the completed SONU platform, including front, rear, side, top, and cinematic views.


🧰 Development Environment

The project was developed and tested primarily using Python on Raspberry Pi hardware.

Software

The development stack includes:

  • Python
  • Raspberry Pi OS
  • VOSK
  • OpenCV
  • GPIO interfaces
  • pigpio
  • PWM motor control
  • Linux shell scripting

The exact runtime dependencies can evolve as the project progresses. Experimental implementations are intentionally kept separate from the current source tree to avoid mixing deprecated approaches with active components.


🚀 Running the Project

SONU is a hardware-dependent robotics project, so execution should be performed on the target Raspberry Pi environment with the required hardware connected.

Before running motor-related software:

  1. Verify the motor driver wiring.
  2. Verify the GPIO configuration.
  3. Confirm the motor power supply.
  4. Keep the robot's wheels lifted from the ground during initial testing.
  5. Confirm that the emergency-stop mechanism is functional.
  6. Start individual subsystem tests before attempting full-system execution.

For example, hardware validation programs can be found in:

tests/

The project also includes a runtime helper script:

scripts/run_monitor_and_controller.sh

The source tree is organized as a Python package, with subsystem implementations available under:

src/

Because SONU interacts directly with physical hardware, the appropriate execution method depends on the connected hardware configuration and the specific subsystem being tested.


⚠️ Safety Notice

SONU directly controls physical motors and hardware.

Use appropriate precautions when testing or modifying the system.

Always test motor-control code with the robot's wheels lifted off the ground during initial validation.

Additional precautions:

  • Do not leave motors running unattended.
  • Verify GPIO assignments before connecting hardware.
  • Use an appropriate power supply for the selected motor driver.
  • Confirm motor-driver wiring before applying power.
  • Test emergency-stop behavior before autonomous movement.
  • Keep hands and loose objects away from moving mechanisms.
  • Stop testing immediately if unexpected motor behavior occurs.

The safety subsystem exists specifically to reduce the impact of software or worker failures during operation.


📊 Chapter 1 Status

Chapter 1 establishes the first stable foundation of the SONU platform.

Subsystem Status
Voice interaction ✅ Implemented
Offline speech recognition ✅ Implemented
Text-to-speech ✅ Implemented
Camera integration ✅ Implemented
Vision pipeline ✅ Implemented
Motor control ✅ Implemented
PWM speed control ✅ Implemented
Smooth motor control ✅ Implemented
Emergency-stop mechanisms ✅ Implemented
Worker supervision ✅ Implemented
Failure-recovery foundation ✅ Implemented
Autonomy foundation 🧠 Initial implementation
Advanced autonomous reasoning 🔭 Future work

Chapter 1 should therefore be viewed as a foundation milestone, not the final form of SONU.


🔭 Future Direction

The architecture established in Chapter 1 is intended to support progressively more capable robotics systems.

Potential future development areas include:

  • Advanced computer vision
  • Object detection and tracking
  • Improved spatial perception
  • Autonomous navigation
  • Sensor fusion
  • Obstacle avoidance
  • Local AI / LLM integration
  • Context-aware voice interaction
  • Autonomous task planning
  • Higher-level decision making
  • Improved fault recovery
  • More sophisticated safety mechanisms
  • Hardware expansion
  • Remote monitoring and control

The goal is to evolve SONU incrementally while preserving the modular architecture established in this chapter.


📌 Design Philosophy

The central engineering principle behind Chapter 1 is:

Build a reliable foundation before adding intelligence.

SONU was not designed as a single monolithic robotics script.

Instead, the system was progressively separated into independently testable components:

Hardware
   ↓
Drivers
   ↓
Subsystems
   ↓
Workers
   ↓
Supervisor
   ↓
Safety
   ↓
Autonomy

This structure makes it easier to debug failures, replace individual components, experiment with new approaches, and extend the robot without destabilizing unrelated parts of the system.

The result is a foundation that can evolve from a manually controlled voice-vision robot into a more capable autonomous robotics platform.


📚 Development History

SONU's development involved numerous experiments, revisions, and hardware-level tests.

Instead of deleting unsuccessful or superseded implementations, selected versions have been preserved in:

archive/

This provides a practical development history showing how the project evolved from individual experiments into a structured robotics codebase.

The archive should not be considered the primary runtime implementation.

For current development, refer to:

src/
tests/
examples/
scripts/

This distinction keeps the active codebase clean while preserving useful historical implementations for reference and comparison.

📚 Repository Documentation

The repository is organized to separate active implementations, hardware experiments, demonstrations, and supporting resources.

Source Code

The src/ directory contains the current modular implementation of the robot.

src/
├── core/
│   ├── motor_drive.py
│   └── sonu_autonomy.py
│
├── motion/
│   ├── motor_cli.py
│   └── motor_control_toggle2_smooth.py
│
├── safety/
│   ├── emergency_monitor_flag_v2.py
│   └── emergency_stop_cb.py
│
├── supervisor/
│   └── supervisor_with_voice_worker.py
│
├── vision/
│   ├── camera_worker.py
│   ├── sonu_vision.py
│   └── sonu_vision_auto_action.py
│
└── voice/
    ├── sonu_voice.py
    ├── sonu_voice_personality_fixed.py
    └── voice_worker.py

The package structure reflects the main functional boundaries of the system:

  • Core — central robot and autonomy logic
  • Motion — motor-control implementations and interfaces
  • Safety — emergency handling and protective mechanisms
  • Supervisor — worker/process supervision
  • Vision — camera capture and visual processing
  • Voice — speech recognition, interaction, and voice workers

This separation makes individual subsystems easier to test, replace, and extend.


🧪 Testing & Validation

Testing was an important part of Chapter 1 because the robot interacts with physical hardware.

The tests/ directory contains dedicated experiments and validation programs for individual components.

tests/
├── cam_test.py
├── debug_sensor_print.py
├── motor_test_debug.py
├── motor_test_no_pigpio.py
├── motor_test_pigpio.py
├── test_cam_device.py
├── test_cam_opencv.py
├── test_gpio_pigpio.py
└── transcribe_wav.py

The testing workflow generally followed this progression:

Hardware Test
      ↓
Standalone Component Test
      ↓
Failure / Edge-Case Testing
      ↓
Safety Improvements
      ↓
Worker Integration
      ↓
Supervisor Integration
      ↓
System-Level Validation

This approach reduced the risk of debugging multiple interacting hardware and software components simultaneously.

It also allowed hardware-specific problems to be isolated before integrating the corresponding subsystem into the complete robot.


🎥 Demonstrations

Chapter 1 includes recorded demonstrations of the robot's major capabilities.

The demonstration videos are stored under:

assets/videos/

Available demonstrations include:

  • camera-llm-test.mp4 — camera and LLM-related experimentation
  • final-test-1.mp4 — integrated system testing
  • motor-control-voice.mp4 — voice-controlled motor interaction
  • sonu-inaction.mp4 — robot operation demonstration
  • ultrasonic-sensor-test.mp4 — ultrasonic sensor testing
  • voice-motor-test.mp4 — voice and motor-control integration testing

These recordings document the practical development and validation of the system rather than serving only as promotional material.


🖼️ Project Media

The repository contains development photographs, hardware references, and final robot views.

Development Documentation

Development-stage photographs are located in:

assets/images/development/

These include:

  • Initial construction
  • Hardware setup
  • Raspberry Pi operating-system setup
  • Motor-driver connections
  • Battery charging
  • Intermediate build stages

Hardware References

Hardware-related images are located in:

assets/images/hardware/

The collection documents major components used during development, including:

  • Raspberry Pi 5
  • Raspberry Pi GPIO interface
  • Camera module
  • Chassis
  • DC motor hardware
  • L298N motor driver
  • TB6612FNG motor driver
  • Battery pack
  • TP4056 charging module

Robot Gallery

Final and presentation-oriented photographs are stored under:

assets/images/gallery/

These provide multiple views of the completed SONU platform.

Hero Image

The primary project image is:

assets/images/hero/sonu-hero.jpeg

Mr.SONU Robot


⚙️ Hardware Platform

Chapter 1 was developed around a Raspberry Pi-based robotic platform.

Primary Components

Component Role
Raspberry Pi 5 Main computing platform
USB Microphone Voice input
5MP Camera Visual input
TB6612FNG Primary motor driver
DC Motors Robot locomotion
Chassis Mechanical platform
Battery Pack Mobile power source
TP4056 Module Battery charging
Ultrasonic Sensors Distance / obstacle sensing

The system was designed so that hardware-facing components remain separated from higher-level application logic wherever practical.

This allows future hardware changes to be introduced without requiring a complete rewrite of the software architecture.


🔊 Voice Subsystem

The voice subsystem provides the primary human-to-robot interaction interface.

Its development included:

  • Wake-word based interaction
  • Offline speech recognition using VOSK
  • Speech-to-text processing
  • Text-to-speech responses
  • Voice worker execution
  • Multiple personality experiments
  • Stabilized voice interaction logic

The current implementation is organized under:

src/voice/

The repository also preserves earlier voice implementations under:

archive/

This provides a development trail showing how the voice subsystem evolved through experimentation and refinement.


👁️ Vision Subsystem

The vision subsystem provides SONU with camera-based perception capabilities.

Chapter 1 explored:

  • Camera initialization
  • Device validation
  • Frame capture
  • YUYV / MJPEG camera formats
  • OpenCV-based testing
  • Vision worker execution
  • Automatic frame handling
  • Resilient camera operation
  • Image transmission for downstream processing

The active vision implementation is located under:

src/vision/

The architecture leaves room for future improvements such as more advanced perception, object recognition, scene understanding, and autonomous decision-making.


🛞 Motion Control

Motor control is implemented around the TB6612FNG motor-driver platform.

The motion subsystem includes:

  • PWM-based speed control
  • Direction control
  • Smooth acceleration behavior
  • Command-line motor interaction
  • Safe motor toggling
  • Emergency-stop integration
  • Hardware fallback testing

The active motion implementations are located under:

src/motion/

The lower-level motor logic is additionally represented in:

src/core/

The separation between motion interfaces and core motor functionality helps keep physical movement logic isolated from higher-level autonomy.


🛡️ Safety & Emergency Handling

Because SONU controls physical motors, safety was treated as a core engineering requirement rather than an optional feature.

The safety subsystem includes mechanisms for:

  • Emergency monitoring
  • Emergency-stop callbacks
  • Motor shutdown
  • Failure detection
  • Worker supervision
  • Safe recovery behavior

Relevant implementations are located under:

src/safety/

Safety-related historical experiments are also preserved in:

archive/

The guiding principle is that software failures should not unnecessarily result in uncontrolled physical movement.


🔄 Supervisor & Worker Model

SONU uses independent workers for major subsystems.

Conceptually:

                    ┌──────────────────────┐
                    │   Supervisor Layer   │
                    │  Process Management  │
                    └──────────┬───────────┘
                               │
          ┌────────────────────┼────────────────────┐
          │                    │                    │
          ▼                    ▼                    ▼
      Voice Worker        Vision Worker        Motor Worker
          │                    │                    │
          └────────────────────┼────────────────────┘
                               │
                               ▼
                       Safety / Monitoring
                               │
                               ▼
                         Robot Hardware

This model provides several advantages:

  • Individual workers can be tested independently.
  • A subsystem failure can be isolated.
  • The supervisor can manage worker processes.
  • Safety mechanisms remain separate from normal application logic.
  • Future subsystems can be introduced without redesigning the entire application.

🔬 Engineering Philosophy

Chapter 1 was built around an incremental engineering process:

Build → Test → Break → Analyze → Improve → Integrate

Instead of attempting to build the entire autonomous system at once, individual capabilities were developed and validated independently before being integrated.

This resulted in a codebase that documents both the working system and the engineering process that produced it.


📈 Chapter 1 Outcome

At the completion of this phase, SONU had established a foundation for:

  • Voice interaction
  • Camera-based perception
  • Motor control
  • Ultrasonic sensing
  • Safety monitoring
  • Worker supervision
  • Modular subsystem integration
  • Future autonomy development

The goal of Chapter 1 was not to create a finished autonomous robot.

The goal was to establish a reliable robotics foundation on which more advanced perception, reasoning, navigation, and autonomous behavior can be developed.

About

SONU Robot – Chapter 1 (Voice, Vision, Motor Control, Safety Systems)

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages