Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,14 @@
components: rustfmt, clippy

- name: Install protoc
uses: arduino/setup-protoc@v3

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'CI' step
Uses Step
uses 'arduino/setup-protoc' with ref 'v3', not a pinned commit hash
with:
version: '25.x'
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Install OpenSSL (required for quic-openssl feature in --all-features builds)
run: sudo apt-get update -qq && sudo apt-get install -y libssl-dev

- name: Cache dependencies
uses: actions/cache@v4
with:
Expand Down Expand Up @@ -118,6 +121,47 @@
timeout 30s cargo fuzz run fuzz_parser_funs || true
timeout 30s cargo fuzz run fuzz_mqtt_packet_symmetric || true

quic-build-modes:
name: QUIC build modes
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'CI' step
Uses Step
uses 'dtolnay/rust-toolchain' with ref 'stable', not a pinned commit hash

- name: Install OpenSSL (required for quic-openssl backend)
run: sudo apt-get update -qq && sudo apt-get install -y libssl-dev

- name: Install protoc
uses: arduino/setup-protoc@v3
with:
version: '25.x'
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Cache dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-quic-build-${{ hashFiles('**/Cargo.lock') }}

# Mode 1: default build — quinn not in the dep tree at all
- name: Build (default, no QUIC)
run: cargo build

# Mode 2: mainstream quinn 0.11 (crates.io) + ring crypto
- name: Build flowsdk_ffi --features quic (mainstream quinn)
run: cargo build -p flowsdk_ffi --features quic --no-default-features

# Mode 3: git fork + OpenSSL crypto, optimised for binary size
- name: Build flowsdk_ffi --features quic-openssl --profile release-small (OpenSSL fork)
run: cargo build -p flowsdk_ffi --features quic-openssl --no-default-features --profile release-small

security:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
name: Security Audit
runs-on: ubuntu-latest
Expand Down Expand Up @@ -287,6 +331,9 @@
- name: Install cargo-llvm-cov
run: cargo install cargo-llvm-cov

- name: Install OpenSSL (required for quic-openssl feature)
run: sudo apt-get update -qq && sudo apt-get install -y libssl-dev

- name: Generate coverage report
run: |
./scripts/generate_coverage.sh
Expand Down
44 changes: 44 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"configurations": [
{
"type": "swift",
"request": "launch",
"args": [],
"cwd": "${workspaceFolder:flowsdk-ffi}/swift",
"name": "Debug TcpClientExample (swift)",
"target": "TcpClientExample",
"configuration": "debug",
"preLaunchTask": "swift: Build Debug TcpClientExample (swift)"
},
{
"type": "swift",
"request": "launch",
"args": [],
"cwd": "${workspaceFolder:flowsdk-ffi}/swift",
"name": "Release TcpClientExample (swift)",
"target": "TcpClientExample",
"configuration": "release",
"preLaunchTask": "swift: Build Release TcpClientExample (swift)"
},
{
"type": "swift",
"request": "launch",
"args": [],
"cwd": "${workspaceFolder:flowsdk-ffi}/swift",
"name": "Debug QuicClientExample (swift)",
"target": "QuicClientExample",
"configuration": "debug",
"preLaunchTask": "swift: Build Debug QuicClientExample (swift)"
},
{
"type": "swift",
"request": "launch",
"args": [],
"cwd": "${workspaceFolder:flowsdk-ffi}/swift",
"name": "Release QuicClientExample (swift)",
"target": "QuicClientExample",
"configuration": "release",
"preLaunchTask": "swift: Build Release QuicClientExample (swift)"
}
]
}
6 changes: 6 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"chat.tools.terminal.autoApprove": {
"gradle wrapper": true,
"cargo build": true
}
}
61 changes: 46 additions & 15 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ categories = ["network-programming", "asynchronous"]

[workspace]
members = [".", "mqtt_grpc_duality", "flowsdk_ffi"]
exclude = ["fuzz"]
exclude = ["fuzz", "release-small"]

[dependencies]
tonic = "0.14.1"
prost = "0.14"
tokio = { version = "1.0", features = ["macros", "rt-multi-thread", "net", "io-util", "sync", "time"] }
tonic-prost = "0.14.1"
tonic = { version = "0.14.1", optional = true }
prost = { version = "0.14", optional = true }
tokio = { version = "1.0", features = ["sync", "time", "io-util"] }
tonic-prost = { version = "0.14.1", optional = true }
serde = { version = "1.0.218", features = ["derive"] }
hex = "0.4"
bytes = { version = "1", features = ["serde"] }
tokio-stream = "0.1"
tokio-stream = { version = "0.1", optional = true }
arbitrary = { version = "1", optional = true, features = ["derive"] }
serde_json = "1.0"
slab = "0.4.11"
Expand All @@ -33,27 +33,51 @@ ctrlc = "3.4"
tokio-native-tls = { version = "0.3", optional = true }
native-tls = { version = "0.2", optional = true }

# QUIC support (optional)
quinn = { version = "0.11", optional = true }
quinn-proto = { version = "0.11", optional = true }
rustls = { version = "0.23", optional = true, default-features = false, features = ["ring", "std"] }
# QUIC support — mainstream quinn with ring crypto (regular builds)
quinn = { version = "0.11", optional = true, default-features = false, features = ["runtime-tokio", "rustls", "ring"] }
quinn-proto = { version = "0.11", optional = true, default-features = false, features = ["rustls", "ring"] }
# QUIC support — OpenSSL crypto via fork (for release-small / size-optimised builds)
quinn-openssl = { package = "quinn", git = "https://github.com/qzhuyan/quinn.git", branch = "dev/william/ring-no-deps", optional = true, default-features = false, features = ["runtime-tokio", "rustls-openssl"] }
quinn-proto-openssl = { package = "quinn-proto", git = "https://github.com/qzhuyan/quinn.git", branch = "dev/william/ring-no-deps", optional = true, default-features = false, features = ["rustls-openssl"] }
rustls = { version = "0.23", optional = true, default-features = false, features = ["std"] }
rustls-openssl = { version = "0.3", optional = true }
rustls-native-certs = { version = "0.7", optional = true }
rustls-pki-types = { version = "1", optional = true }
tokio-rustls = { version = "0.26", optional = true }

[features]
default = ["strict-protocol-compliance", "tls"]
default = ["strict-protocol-compliance", "tls", "async-client"]
# TLS/SSL transport support
tls = ["dep:tokio-native-tls", "dep:native-tls"]
# QUIC transport support
quic = ["dep:quinn", "dep:quinn-proto", "dep:rustls", "dep:rustls-native-certs", "dep:rustls-pki-types"]
tls = ["dep:tokio-native-tls", "dep:native-tls", "async-client"]
# QUIC transport support (mainstream quinn + ring crypto)
quic = ["quic-proto", "dep:quinn", "async-client"]
# QUIC transport support (OpenSSL backend via fork, for release-small builds)
# Implies quic so all #[cfg(feature = "quic")] gates activate; the fork is aliased
# over mainstream quinn via extern crate in lib.rs. LTO strips unused mainstream
# quinn symbols from the final binary.
# NOTE: do not enable quic and quic-openssl at the same time.
quic-openssl = ["quic", "quic-proto-openssl", "dep:quinn-openssl"]
# Sans-I/O QUIC protocol engine — ring crypto (no tokio runtime required)
quic-proto = ["dep:quinn-proto", "dep:rustls", "dep:rustls-native-certs", "dep:rustls-pki-types"]
# Sans-I/O QUIC protocol engine — OpenSSL crypto (for release-small builds)
quic-proto-openssl = ["quic-proto", "dep:quinn-proto-openssl", "dep:rustls-openssl"]
# Rustls-based TLS over TCP (mqtts://) transport support
rustls-tls = [
"dep:tokio-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:rustls-pki-types",
"dep:tokio-rustls",
"async-client",
]
# Tokio-based async MQTT client (adds the high-throughput multi-threaded runtime)
async-client = [
"tokio/macros",
"tokio/rt-multi-thread",
"tokio/net",
"dep:tokio-stream",
]
# gRPC support (only needed for mqtt_grpc_duality proxy)
grpc = ["dep:tonic", "dep:prost", "dep:tonic-prost"]
strict-protocol-compliance = []
# ⚠️ DANGEROUS: Enable raw packet API for protocol compliance testing
# DO NOT enable in production builds
Expand All @@ -64,6 +88,13 @@ protocol-testing = []
[target.aarch64-unknown-linux-musl]
rustflags = ["-C", "target-feature=+crt-static"]

[profile.release-small]
inherits = "release"
opt-level = "z"
lto = true
codegen-units = 1
strip = true
panic = "abort"
[dev-dependencies]
futures = "0.3"

Expand Down
127 changes: 127 additions & 0 deletions docs/UNIFIED_CLIENT_CONSIDERATIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Unified Client Design Considerations

This document outlines design considerations and decisions made when implementing unified TCP/QUIC transport support in the Python async client.

## 1. Tick Timing Strategy

### Issue
TCP and QUIC transports use different tick scheduling strategies:

- **TCP**: Adaptive scheduling using `engine.next_tick_ms()` to calculate optimal delay
- **QUIC**: Fixed 10ms tick interval for responsiveness

### Decision
**Use transport-specific defaults** - Each protocol implements its own tick scheduling strategy that's optimized for the transport characteristics.

**Rationale:**
- QUIC requires more frequent ticking for connection handshake and keep-alive
- TCP can optimize battery/CPU usage with adaptive scheduling
- Users shouldn't need to tune timing parameters manually
- Future transports can implement their own optimal strategies

**Implementation:**
```python
# FlowMqttProtocol (TCP)
def _on_timer(self):
# Use next_tick_ms() for adaptive scheduling
next_tick_ms = self.engine.next_tick_ms()
delay = max(0, (next_tick_ms - now_ms) / 1000.0)

# FlowMqttDatagramProtocol (QUIC)
def _on_timer(self):
# Fixed 10ms for QUIC responsiveness
delay = 0.01
```

**Future Enhancement:**
Could add optional `tick_interval_ms` parameter to constructor for advanced users who want to tune performance.

---

## 2. Missing QUIC APIs

### Issue
`QuicMqttEngineFfi` doesn't implement all methods available in `MqttEngineFfi`:

**Missing Methods:**
- `handle_connection_lost()` - Notifies engine of connection loss
- `next_tick_ms()` - Returns optimal next tick time

### Current Approach
**Document limitations and provide fallback behavior:**

1. **handle_connection_lost()**: Not called for QUIC since UDP is connectionless
- QUIC engine detects connection loss through timeout internally
- No action needed in protocol's `connection_lost()`

2. **next_tick_ms()**: Not available for QUIC
- Use fixed 10ms interval (see Tick Timing Strategy above)
- QUIC's `handle_tick()` returns events directly without needing separate timing

### Rationale
These differences reflect fundamental protocol differences:
- UDP is connectionless, so "connection lost" is detected differently
- QUIC's internal state machine handles timing optimization

### Future Enhancement
Could add these methods to the Rust FFI if adaptive QUIC timing proves beneficial, but current fixed approach works well.

---

## 3. TLS Transport Support

### Current State
The unified client architecture now fully supports `TransportType.TLS` for explicit TLS over TCP transport. This allows for fine-grained control over TLS configuration, including certificate validation and SNI.

### FFI Support
`TlsMqttEngineFfi` is used to handle the MQTT protocol over an encrypted TCP stream. It follows a similar API to the standard TCP engine but requires explicit calls to `handle_socket_data` and `take_socket_data` for BIOS-like data pumping.

### Implementation Details

**Key Features:**
- **Unified Adapter**: Transparently handle different method names across engines (`handle_incoming` vs `handle_socket_data`).
- **SNI Support**: Uses `server_name` parameter for TLS Server Name Indication.
- **Certificate Validation**: Supports standard CA files, client certificates, and insecure mode (skip verify).

**Usage Example:**
```python
from flowsdk import FlowMqttClient, TransportType

client = FlowMqttClient(
"my_client",
transport=TransportType.TLS,
server_name="broker.emqx.io",
ca_cert_file="emqxsl-ca.crt"
)
await client.connect("broker.emqx.io", 8883)
```

### Decision
**Implement explicit TLS transport support** - To provide full flexibility for production environments where custom CA bundles or client certificates are required.

**Rationale:**
- Many industrial environments use internal PKI.
- Client certificate authentication is a common requirement for high-security IoT.
- Provides a consistent API across TCP, TLS, and QUIC.

---

## Summary of Decisions

| Consideration | Decision | Status |
|--------------|----------|--------|
| Tick Timing | Transport-specific defaults | ✅ Implemented |
| Missing QUIC APIs | Document limitations, use fallbacks | ✅ Implemented |
| TLS Transport | Explicit unified support | ✅ Implemented |

---

## Related Documentation

- [Python Async Client API](../python/package/README.md)
- [MQTT Session Management](MQTT_SESSION.md)
- [Async Client Architecture](ASYNC_CLIENT.md)

---

*Last Updated: February 3, 2026*
Loading
Loading