Summary
Migrate the Smart Cluster Sign firmware from its current predominantly sequential Arduino architecture to an explicit FreeRTOS-based architecture while retaining the Arduino framework.
The ESP32 already runs FreeRTOS underneath the Arduino framework. The current firmware, however, relies primarily on the Arduino setup()/loop() programming model and does not explicitly organize the application's subsystems into independent FreeRTOS tasks.
The goal of this task is to take advantage of the concurrency and synchronisation primitives provided by FreeRTOS while preserving the existing Arduino-based development environment and functionality.
This task does not include migrating from Arduino to ESP-IDF.
Motivation
The firmware has grown into a relatively complex system consisting of multiple independent subsystems, including:
- Wi-Fi connectivity
- Internet connectivity monitoring
- OTA update handling
- GitHub-based firmware retrieval
- Telegram functionality
- e-paper display management
- display update scheduling
- power/sleep management
- device state management
- various timers and periodic operations
The current architecture handles much of this logic through sequential execution.
As the firmware continues to grow, this can make the control flow increasingly difficult to understand and maintain. Slow operations, particularly e-paper display updates and network operations, can also interfere with unrelated parts of the application.
FreeRTOS provides a natural way to express the actual concurrency already present in the system.
Target Architecture
The target architecture should remain based on the Arduino framework:
Arduino Framework
│
▼
FreeRTOS
│
├── System / State Task
├── Network Task
├── Display Task
├── OTA Task
└── Telegram Task
The exact task structure should be determined during implementation. The above is an architectural direction rather than a strict requirement that every subsystem must become its own task.
The objective is clear ownership and well-defined communication between subsystems, rather than simply creating as many tasks as possible.
Important Architectural Boundary
This migration should use Arduino + explicit FreeRTOS. It should not migrate the project to ESP-IDF.
The existing Arduino ecosystem should remain available, including where appropriate:
Arduino.h- Arduino Wi-Fi APIs
- existing Arduino-compatible libraries
- existing peripheral libraries
- existing OTA infrastructure
- existing filesystem functionality
The project should continue to be buildable using the existing Arduino/arduino-cli toolchain.
A future ESP-IDF migration may be considered as a separate project, but it is explicitly outside the scope of this task.
Proposed Tasks
The following task structure should be investigated.
Display Task
Responsible for e-paper display operations.
Potential responsibilities:
- receiving display update requests
- preparing display content
- performing e-paper updates
- handling the display's BUSY state
- managing display-related timing
The display task should be able to block while waiting for the e-paper hardware without unnecessarily blocking unrelated system functionality.
Network Task
Responsible for network connectivity.
Potential responsibilities:
- Wi-Fi connection
- Wi-Fi reconnection
- connection state monitoring
- Internet availability
- network-related events
The network subsystem should expose its state to other tasks through a well-defined interface rather than requiring every subsystem to independently inspect Wi-Fi state.
OTA Task
Responsible for OTA update operations.
Potential responsibilities:
- periodically checking for updates
- retrieving the OTA manifest
- determining whether an update is available
- downloading firmware
- validating firmware integrity
- initiating OTA installation
- handling OTA failures
- coordinating reboot after successful installation
OTA operations should not unnecessarily block display or other independent application functionality.
Telegram Task
If Telegram functionality remains part of the firmware, investigate separating its processing into its own task.
Potential responsibilities:
- checking for incoming commands
- processing commands
- triggering system events
- communicating requested actions to other tasks
Telegram functionality should communicate with other subsystems through explicit interfaces rather than directly manipulating their internal state.
System / State Task
A central system-level task may be introduced to coordinate high-level device state.
Potential responsibilities:
- coordinating major system events
- managing device operating states
- coordinating sleep/wake transitions
- coordinating OTA and reboot states
- initiating display updates
- handling interactions between subsystems
This should not become another giant centralised function containing all application logic.
Its purpose should be coordination rather than accumulation of business logic.
Inter-Task Communication
The migration should make use of appropriate FreeRTOS synchronization and communication mechanisms.
Investigate the use of:
- Queues for passing data or requests between tasks.
- Task notifications for lightweight task-to-task signaling.
- Event groups for representing system-wide state and events.
- Mutexes for protecting shared resources.
- Semaphores where appropriate for synchronisation.
- Software timers for periodic operations.
The appropriate primitive should be selected according to the communication requirement rather than using one mechanism everywhere.
Resource Ownership
The architecture should establish clear ownership of shared resources.
Particular attention should be paid to:
- SPI
- e-paper display
- Wi-Fi
- filesystem
- network clients
- shared configuration/state
- serial output
Tasks should avoid directly manipulating resources owned by another task.
For example, instead of allowing arbitrary tasks to access the e-paper driver:
OTA Task ──────┐
│
Telegram Task ─┼──> Display Driver
│
System Task ───┘
prefer:
OTA Task ──────┐
│
Telegram Task ─┼──> Display Queue ──> Display Task ──> E-Paper
│
System Task ───┘
This provides a clear ownership model and reduces the need for excessive locking.
State Machines
Where appropriate, major subsystems should be represented as explicit state machines.
For example, the network subsystem could have states such as:
DISCONNECTED
│
▼
CONNECTING
│
▼
CONNECTED
│
▼
INTERNET_OK
Similarly, OTA could have states such as:
IDLE
│
▼
CHECKING
│
▼
DOWNLOADING
│
▼
VERIFYING
│
▼
INSTALLING
│
▼
REBOOT
The exact states should be determined during implementation.
The purpose is to make complex asynchronous behaviour explicit rather than allowing it to emerge from deeply nested conditional logic.
E-Paper Display Considerations
The e-paper display is particularly important in this migration because display operations can take a significant amount of time.
The new architecture should ensure that:
- a display update does not block unrelated tasks,
- the display's BUSY state is handled by the Display Task,
- other tasks can continue operating while the display task waits,
- multiple simultaneous display requests are handled deterministically,
- unnecessary concurrent access to the display hardware is prevented.
The existing e-paper performance issues should be considered during the redesign, but optimizing the display driver itself is not necessarily part of this issue unless required by the new architecture.
Scheduling and Timing
Review existing timing mechanisms and determine which should become FreeRTOS mechanisms.
Potential candidates include:
- periodic OTA checks
- Wi-Fi monitoring
- Telegram polling
- display update scheduling
- inactivity timers
- sleep scheduling
Avoid replacing every delay() mechanically with a FreeRTOS API.
The goal is to identify operations that genuinely benefit from independent scheduling and asynchronous execution.
Error Handling
The new architecture should define how errors propagate between tasks.
Examples:
- Wi-Fi failure
- Internet unavailable
- OTA download failure
- checksum mismatch
- display timeout
- filesystem error
- Telegram/network failure
Errors should not require unrelated tasks to inspect arbitrary global variables.
Where appropriate, errors should be represented as explicit events or messages.
Power Management
Review the existing sleep and power-management implementation as part of the migration.
Determine how FreeRTOS tasks should behave when:
- the device is preparing to sleep,
- the device is asleep,
- the device is waking,
- a wake-up event occurs.
Tasks should be able to terminate, suspend, or otherwise enter an appropriate low-activity state when the device does not need them.
Power consumption should not regress as a result of introducing additional tasks.
Task Priorities and Stack Sizes
Task priorities and stack sizes should be chosen deliberately.
Do not simply assign high priority to every task.
The implementation should consider:
- task execution frequency
- latency requirements
- blocking operations
- CPU usage
- memory consumption
- stack requirements
- interaction with ESP32 system tasks
Task stack usage should be monitored during development to avoid stack overflows and unnecessary RAM consumption.
Avoid Over-Engineering
The migration should not turn the firmware into a collection of unnecessarily tiny tasks.
Avoid creating separate tasks for every small operation, such as:
wifi_task
wifi_check_task
wifi_timeout_task
wifi_reconnect_task
wifi_monitor_task
unless there is a genuine architectural reason for doing so.
The goal is clear subsystem boundaries, not maximum task count.
Migration Strategy
The migration should be performed incrementally rather than rewriting the entire firmware in one step.
A possible progression:
- Identify the current major subsystems.
- Identify shared state and shared resources.
- Define ownership boundaries.
- Introduce the first FreeRTOS task.
- Move one subsystem into an explicit task.
- Introduce appropriate inter-task communication.
- Verify functionality.
- Migrate the next subsystem.
- Repeat until the desired architecture is reached.
- Remove obsolete sequential control flow.
- Perform a full regression test on real hardware.
The firmware should remain buildable and testable throughout the migration wherever practical.
Testing
The migrated firmware must be tested on real ESP32 hardware.
At minimum, verify:
- boot
- Wi-Fi connection
- Internet connectivity
- e-paper display updates
- Telegram functionality
- OTA update checks
- OTA firmware download
- SHA-256 verification
- firmware installation
- rollback behaviour
- sleep/wake behaviour
- power consumption
- recovery from network failures
- recovery from display failures
- long-running stability
Particular attention should be paid to race conditions and timing-dependent failures that may not occur during short manual tests.
Performance and Stability
Compare the new architecture against the current implementation.
Investigate:
- CPU utilization
- free heap
- task stack high-water marks
- boot time
- display responsiveness
- Wi-Fi responsiveness
- OTA behaviour
- power consumption
- long-term stability
The migration should not introduce unacceptable memory or CPU overhead.
Documentation
Document the resulting FreeRTOS architecture after implementation.
Documentation should explain:
- which tasks exist
- what each task owns
- task priorities
- major queues/events
- shared resources
- synchronization mechanisms
- important state machines
- task interactions
A high-level architecture diagram should be added to the project documentation.
Out of Scope
The following are explicitly outside the scope of this issue:
- Migrating from Arduino to ESP-IDF.
- Rewriting the firmware in a different programming language.
- Replacing all Arduino libraries.
- Replacing the existing OTA architecture.
- Reimplementing the e-paper driver solely for performance reasons.
- Creating tasks merely for the sake of increasing concurrency.
A future Arduino → ESP-IDF migration may be considered independently.
Acceptance Criteria
The migration is considered complete when:
- The firmware explicitly uses FreeRTOS tasks for its major independent subsystems.
- The Arduino framework remains the project's application framework.
- Inter-task communication uses appropriate FreeRTOS primitives.
- Shared resources have clearly defined ownership.
- Major asynchronous subsystems have explicit state management where appropriate.
- Slow e-paper operations no longer unnecessarily block unrelated application functionality.
- Existing firmware functionality remains operational.
- OTA functionality continues to work correctly.
- Rollback functionality continues to work correctly.
- Power consumption does not regress unacceptably.
- No task stack overflows or race conditions are observed during testing.
- The firmware demonstrates stable long-term operation on real hardware.
- The resulting architecture is documented.
Expected Result
The final firmware architecture should evolve from:
Arduino
│
└── FreeRTOS
│
└── Arduino loop()
└── most application logic
toward something closer to:
Arduino Framework
│
▼
FreeRTOS
│
├── System / State Task
│
├── Network Task
│
├── Display Task
│
├── OTA Task
│
└── Telegram Task
with explicit communication and synchronization between the subsystems.
The objective is not simply to "use FreeRTOS", but to make the firmware's concurrency, state, timing, and subsystem boundaries explicit, understandable, and maintainable.
Summary
Migrate the Smart Cluster Sign firmware from its current predominantly sequential Arduino architecture to an explicit FreeRTOS-based architecture while retaining the Arduino framework.
The ESP32 already runs FreeRTOS underneath the Arduino framework. The current firmware, however, relies primarily on the Arduino
setup()/loop()programming model and does not explicitly organize the application's subsystems into independent FreeRTOS tasks.The goal of this task is to take advantage of the concurrency and synchronisation primitives provided by FreeRTOS while preserving the existing Arduino-based development environment and functionality.
This task does not include migrating from Arduino to ESP-IDF.
Motivation
The firmware has grown into a relatively complex system consisting of multiple independent subsystems, including:
The current architecture handles much of this logic through sequential execution.
As the firmware continues to grow, this can make the control flow increasingly difficult to understand and maintain. Slow operations, particularly e-paper display updates and network operations, can also interfere with unrelated parts of the application.
FreeRTOS provides a natural way to express the actual concurrency already present in the system.
Target Architecture
The target architecture should remain based on the Arduino framework:
The exact task structure should be determined during implementation. The above is an architectural direction rather than a strict requirement that every subsystem must become its own task.
The objective is clear ownership and well-defined communication between subsystems, rather than simply creating as many tasks as possible.
Important Architectural Boundary
This migration should use Arduino + explicit FreeRTOS. It should not migrate the project to ESP-IDF.
The existing Arduino ecosystem should remain available, including where appropriate:
Arduino.hThe project should continue to be buildable using the existing Arduino/
arduino-clitoolchain.A future ESP-IDF migration may be considered as a separate project, but it is explicitly outside the scope of this task.
Proposed Tasks
The following task structure should be investigated.
Display Task
Responsible for e-paper display operations.
Potential responsibilities:
The display task should be able to block while waiting for the e-paper hardware without unnecessarily blocking unrelated system functionality.
Network Task
Responsible for network connectivity.
Potential responsibilities:
The network subsystem should expose its state to other tasks through a well-defined interface rather than requiring every subsystem to independently inspect Wi-Fi state.
OTA Task
Responsible for OTA update operations.
Potential responsibilities:
OTA operations should not unnecessarily block display or other independent application functionality.
Telegram Task
If Telegram functionality remains part of the firmware, investigate separating its processing into its own task.
Potential responsibilities:
Telegram functionality should communicate with other subsystems through explicit interfaces rather than directly manipulating their internal state.
System / State Task
A central system-level task may be introduced to coordinate high-level device state.
Potential responsibilities:
This should not become another giant centralised function containing all application logic.
Its purpose should be coordination rather than accumulation of business logic.
Inter-Task Communication
The migration should make use of appropriate FreeRTOS synchronization and communication mechanisms.
Investigate the use of:
The appropriate primitive should be selected according to the communication requirement rather than using one mechanism everywhere.
Resource Ownership
The architecture should establish clear ownership of shared resources.
Particular attention should be paid to:
Tasks should avoid directly manipulating resources owned by another task.
For example, instead of allowing arbitrary tasks to access the e-paper driver:
prefer:
This provides a clear ownership model and reduces the need for excessive locking.
State Machines
Where appropriate, major subsystems should be represented as explicit state machines.
For example, the network subsystem could have states such as:
Similarly, OTA could have states such as:
The exact states should be determined during implementation.
The purpose is to make complex asynchronous behaviour explicit rather than allowing it to emerge from deeply nested conditional logic.
E-Paper Display Considerations
The e-paper display is particularly important in this migration because display operations can take a significant amount of time.
The new architecture should ensure that:
The existing e-paper performance issues should be considered during the redesign, but optimizing the display driver itself is not necessarily part of this issue unless required by the new architecture.
Scheduling and Timing
Review existing timing mechanisms and determine which should become FreeRTOS mechanisms.
Potential candidates include:
Avoid replacing every
delay()mechanically with a FreeRTOS API.The goal is to identify operations that genuinely benefit from independent scheduling and asynchronous execution.
Error Handling
The new architecture should define how errors propagate between tasks.
Examples:
Errors should not require unrelated tasks to inspect arbitrary global variables.
Where appropriate, errors should be represented as explicit events or messages.
Power Management
Review the existing sleep and power-management implementation as part of the migration.
Determine how FreeRTOS tasks should behave when:
Tasks should be able to terminate, suspend, or otherwise enter an appropriate low-activity state when the device does not need them.
Power consumption should not regress as a result of introducing additional tasks.
Task Priorities and Stack Sizes
Task priorities and stack sizes should be chosen deliberately.
Do not simply assign high priority to every task.
The implementation should consider:
Task stack usage should be monitored during development to avoid stack overflows and unnecessary RAM consumption.
Avoid Over-Engineering
The migration should not turn the firmware into a collection of unnecessarily tiny tasks.
Avoid creating separate tasks for every small operation, such as:
unless there is a genuine architectural reason for doing so.
The goal is clear subsystem boundaries, not maximum task count.
Migration Strategy
The migration should be performed incrementally rather than rewriting the entire firmware in one step.
A possible progression:
The firmware should remain buildable and testable throughout the migration wherever practical.
Testing
The migrated firmware must be tested on real ESP32 hardware.
At minimum, verify:
Particular attention should be paid to race conditions and timing-dependent failures that may not occur during short manual tests.
Performance and Stability
Compare the new architecture against the current implementation.
Investigate:
The migration should not introduce unacceptable memory or CPU overhead.
Documentation
Document the resulting FreeRTOS architecture after implementation.
Documentation should explain:
A high-level architecture diagram should be added to the project documentation.
Out of Scope
The following are explicitly outside the scope of this issue:
A future Arduino → ESP-IDF migration may be considered independently.
Acceptance Criteria
The migration is considered complete when:
Expected Result
The final firmware architecture should evolve from:
toward something closer to:
with explicit communication and synchronization between the subsystems.
The objective is not simply to "use FreeRTOS", but to make the firmware's concurrency, state, timing, and subsystem boundaries explicit, understandable, and maintainable.