Microservices Simulator

The artifact supports the test of business logic of a microservices application designed on the concept of Domain-Driven Design Aggregate and using several transactional models.

The currently supported transactional models are:

• Eventual Consistency
  • Sagas applying the Orchestration variant
• Transactional Causal Consistency

The system allows testing the interleaving of functionalities execution in a deterministic context, such that it is possible to evaluate the resulting behavior.

The description of the examples for Transactional Causal Consistency are in Transactional Causal Consistent Microservices Simulator.

The simulator supports multiple execution modes to test different aspects of system behavior, ranging from simple local execution to full distributed deployment.

Mode	Description	Profiles	Infrastructure
Monolith	Runs as a single application. Supports local (internal), stream (RabbitMQ), or gRPC service calls.	sagas|tcc, local|stream|grpc	PostgreSQL, Jaeger, (RabbitMQ for stream)
Microservices	Fully distributed. Each domain service runs independently. Uses Eureka for discovery and RabbitMQ or gRPC.	Service-specific (e.g., answer,sagas|tcc,stream|grpc)	PostgreSQL (per service in Docker, centralized with multiple databases with Maven), Jaeger, Eureka, (RabbitMQ for stream)
Kubernetes	Distributed microservices orchestrated by Kubernetes (k8s/services-stream, k8s/services-grpc or k8s/services-azure to deploy in Microsoft Azure). Uses Spring Cloud Kubernetes for discovery.	kubernetes	K8s Cluster, PostgreSQL, Jaeger, (RabbitMQ for stream)

Run Using Docker

Technology Requirements

• Docker Compose V2

Build the Application

docker compose build

Or run the service with the flag --build

Running as a Monolith with Local Service Calls

# Sagas
docker compose up quizzes-sagas

# TCC
docker compose up quizzes-tcc

Running as a Monolith with Remote Service Calls With RabbitMQ

# Sagas
docker compose up quizzes-sagas-stream

# TCC
docker compose up quizzes-tcc-stream

Running as Microservices

# Sagas (default) with Stream (default)
docker compose build --with-dependencies gateway

# TCC with Stream (default)
TX_MODE=tcc docker compose build --with-dependencies gateway

# With gRPC instead of stream
COMM_LAYER=grpc docker compose build --with-dependencies gateway

# TCC + gRPC
TX_MODE=tcc COMM_LAYER=grpc docker compose build --with-dependencies gateway

This will build the gateway and all microservices.

docker compose up gateway -d

Starting the gateway will automatically start the entire microservices ecosystem, including:

Infrastructure:

• eureka-server: Service discovery
• rabbitmq: Message broker for async communication
• gateway: API Gateway (entry point)

Microservices:

• version-service
• answer-service
• course-service
• course-execution-service
• question-service
• quiz-service
• topic-service
• tournament-service
• user-service

Databases (One per service):

• version-db
• answer-db
• course-db
• execution-db
• question-db
• quiz-db
• topic-db
• tournament-db
• user-db

Running Local Tests

Note: Run build-simulator first before running tests.

docker compose up build-simulator

Simulator Sagas:

docker compose up test-simulator-sagas

# Quizzes Sagas:
docker compose up test-quizzes-sagas

# Quizzes TCC:
docker compose up test-quizzes-tcc

---

Run Using IntelliJ

Technology Requirements

• IntelliJ IDEA (Ultimate or Community Edition)

Pre-configured Run Configurations

The project includes ready-to-use IntelliJ run configurations in the .run/ directory. After opening the project in IntelliJ, these configurations will be automatically available in the Run/Debug dropdown.

1. Open the project in IntelliJ IDEA
2. Run the build-simulator configuration to install the simulator library
3. Select a run configuration from the dropdown (e.g., Quizzes)
4. Click the Run button

Running as a Monolith with Local Service Calls

• Run the sagas local or the tcc local configuration

Running as a Monolith with Remote Service Calls

• Run the quizzes-simulator folder (contains sagas-stream, sagas-grpc, tcc-stream, tcc-grpc configurations)
• Run one of the version-service folder configurations (version-stream or version-grpc) matching the communication layer

Running as Microservices

• Run one of the microservices folders to start all domain services:
  • microservices-sagas-stream — Sagas with RabbitMQ
  • microservices-sagas-grpc — Sagas with gRPC
  • microservices-tcc-stream — TCC with RabbitMQ
  • microservices-tcc-grpc — TCC with gRPC
• Run the matching version-service configuration (version-stream or version-grpc)
• Run the api-gateway configuration

---

Run Using Maven

Technology Requirements

• Maven 3.9.9
• Java 21+
• PSQL 14
• RabbitMQ 3.12+ (required for stream profile)
• Jaeger 1.75
• JMeter 5.6

Setting up the Database

There is two ways to set up the database:

Running in a Docker container

1. Start postgres container

   docker compose up postgres -d

   This will create all the necessary databases with user and password postgres

Running on a local machine

1. Start PostgreSQL:

   sudo service postgresql start

2. Create the databases:

   sudo su -l postgres
   dropdb msdb
   createdb msdb

3. Create microservice databases (required for microservices mode):

   psql -U postgres -d msdb -f data/init/init-databases.sh

4. Create user to access db:

   psql msdb
   CREATE USER your-username WITH SUPERUSER LOGIN PASSWORD 'yourpassword';
   \q
   exit

5. Configure application properties:

   • Fill in the placeholder fields with your database credentials in applications/quizzes/src/main/resources/application.yaml

Setting up Jaeger Tracing

docker compose up jaeger -d

---

Simulator

cd simulator

Install simulator library

mvn clean install

Run simulator tests

mvn clean -Ptest-sagas test

---

Quizzes Monolithic Simulation

cd applications/quizzes

Launch simulator for Sagas with local

mvn clean spring-boot:run -Psagas,local

Launch simulator for TCC

mvn clean spring-boot:run -Ptcc,local

Running Sagas Tests

mvn clean -Ptest-sagas test

Running TCC Tests

mvn clean -Ptest-tcc test

---

Quizzes Monolithic Simulation with Remote Service Calls

Additional Requirements:

1. Start RabbitMQ (for stream profile):

   docker compose up rabbitmq -d

Running with Stream

cd applications/quizzes
mvn spring-boot:run -Psagas,stream

Running with gRPC

cd applications/quizzes
mvn spring-boot:run -Psagas,grpc

---

Quizzes Microservices Simulation Deployment

Running the application as distributed microservices requires setting up individual databases for each service and running RabbitMQ for inter-service communication.

Prerequisites

1. Start PostgreSQL

2. Start RabbitMQ

3. Start Eureka service discovery (required for local microservices):

   docker compose up eureka-server -d

4. Install the simulator library (if not already done):

   cd simulator
   mvn clean install
   cd ..

Running the Microservices

1. Start the Version Service:

cd simulator
mvn spring-boot:run -Dspring-boot.run.profiles=version-service,stream

2. Start each Quizzes microservice (from applications/quizzes):

cd applications/quizzes

Service	Command
Answer Service	mvn spring-boot:run -Panswer,sagas|tcc,stream|grpc
Course Service	mvn spring-boot:run -Pcourse,sagas|tcc,stream|grpc
Course Execution Service	mvn spring-boot:run -Pcourse-execution,sagas|tcc,stream|grpc
Question Service	mvn spring-boot:run -Pquestion,sagas|tcc,stream|grpc
Quiz Service	mvn spring-boot:run -Pquiz,sagas|tcc,stream|grpc
Topic Service	mvn spring-boot:run -Ptopic,sagas|tcc,stream|grpc
Tournament Service	mvn spring-boot:run -Ptournament,sagas|tcc,stream|grpc
User Service	mvn spring-boot:run -Puser,sagas|tcc,stream|grpc

3. Start the Gateway (from applications/gateway):

cd applications/gateway
mvn spring-boot:run

---

Kubernetes Deployment

The application supports deployment on Kubernetes using Spring Cloud Kubernetes for service discovery.

Prerequisites

Install the following packages:

• Docker - Container runtime
• kubectl - Kubernetes CLI
• Kind (recommended) – Local Kubernetes cluster

Create a Kind cluster:

kind create cluster --name microservices

Build and Load Images

# Build all Docker images
docker compose build --with-dependencies gateway

# Load images into Kind cluster
for img in gateway simulator quizzes-answer quizzes-course quizzes-execution quizzes-question quizzes-quiz quizzes-topic quizzes-tournament quizzes-user; do
  kind load docker-image ${img}:latest --name microservices
done

Deploy to Kubernetes

# Create namespace and RBAC
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/rbac.yaml
kubectl apply -f k8s/configmap.yaml

# Deploy infrastructure
kubectl apply -f k8s/infrastructure/rabbitmq.yaml
kubectl apply -f k8s/infrastructure/jaeger.yaml

# Wait for infrastructure to be ready
kubectl wait --for=condition=ready pod -l app=rabbitmq -n microservices-simulator --timeout=120s
kubectl wait --for=condition=ready pod -l app=jaeger -n microservices-simulator --timeout=60s

# Deploy microservices (choose one)
# For stream communication
kubectl apply -f k8s/services-stream/

# For gRPC communication
# kubectl apply -f k8s/services-grpc/

# Check status
kubectl get pods -n microservices-simulator

Note: To change transactional model profile, edit k8s/services-stream/ or k8s/services-grpc/ and change the SPRING_PROFILES_ACTIVE environment variable of each service.

Access the Application

# Port-forward to gateway
kubectl port-forward svc/gateway 8080:8080 -n microservices-simulator

Access Jaeger UI

kubectl port-forward svc/jaeger 16686:16686 -n microservices-simulator

Then open http://localhost:16686 to view distributed traces.

Cleanup

kubectl delete namespace microservices-simulator

---

Azure Kubernetes Service (AKS) Deployment

Deploy the microservices to Azure Kubernetes Service for cloud-based deployments.

Prerequisites

• Azure CLI installed
• Active Azure subscription (e.g., Azure for Students)

Setup AKS Cluster

# Login to Azure
az login

# Create Resource Group
az group create --name simulator-rg-es --location spaincentral

# Create AKS Cluster (Free tier, minimal resources)
az aks create \
  --resource-group simulator-rg-es \
  --name simulator-cluster \
  --tier free \
  --node-count 1 \
  --node-vm-size Standard_B2s_v2 \
  --generate-ssh-keys

# Connect to the Cluster
az aks get-credentials --resource-group simulator-rg-es --name simulator-cluster

# Verify connection
kubectl get nodes

Register Azure Resource Providers (One-time setup)

# Register Container Registry provider (required for ACR)
az provider register --namespace Microsoft.ContainerRegistry

# Check registration status (wait until "Registered")
az provider show --namespace Microsoft.ContainerRegistry --query "registrationState"

Push Images to Azure Container Registry

# Run the push script (creates ACR, attaches to AKS, pushes images)
chmod +x scripts/push-to-acr.sh
./scripts/push-to-acr.sh

Deploy to Azure

# 1. Base setup
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/rbac.yaml

# 2. Infrastructure (Centralized PostgreSQL + RabbitMQ)
kubectl apply -f k8s/infrastructure/

# 3. Wait for infrastructure to be ready
kubectl wait --for=condition=ready pod -l app=postgres -n microservices-simulator --timeout=180s
kubectl wait --for=condition=ready pod -l app=rabbitmq -n microservices-simulator --timeout=120s

# 4. Deploy Azure-optimized microservices (uses ACR images + centralized DB)
kubectl apply -f k8s/services-azure/

# 5. Check status
kubectl get pods -n microservices-simulator

# 6. Access the gateway
kubectl get svc gateway -n microservices-simulator
# Or use port-forward
kubectl port-forward -n microservices-simulator svc/gateway 8080:8080

Save costs by stopping the cluster when not in use:

# Stop the cluster
az aks stop --name simulator-cluster --resource-group simulator-rg-es

# Start the cluster again
az aks start --name simulator-cluster --resource-group simulator-rg-es

Cleanup Azure Resources

# Delete the cluster
az aks delete --name simulator-cluster --resource-group simulator-rg-es

# Delete everything (including ACR)
az group delete --name simulator-rg-es

Test Cases

Sagas test cases:

• Workflow Test Plan (Simulator)
• Tournament Functionality Tests (Quizzes)

TCC test cases:

• Tournament Merge Tests (Quizzes)
• Tournament Functionality Tests (Quizzes)

---

Configuration

The application uses Spring Boot profiles and YAML configuration files to manage different deployment modes.

Jaeger Tracing

The project uses Jaeger for distributed tracing to monitor and visualize the flow of requests across microservices.

• Dashboard: Access the Jaeger UI at http://localhost:16686.
• Collector: The application sends traces to the Jaeger collector on http://localhost:4317 using the OTLP gRPC protocol.
• Instrumentation: Custom instrumentation is implemented in TraceManager using the OpenTelemetry SDK to trace functionalities and their steps.

Service Discovery

Local microservices use Eureka for service discovery. The gateway and each microservice register with the Eureka server at http://${EUREKA_HOST:localhost}:8761/eureka/. In Kubernetes, the kubernetes profile enables Spring Cloud Kubernetes discovery instead of Eureka.

Database Configuration

Database settings are defined in application.yaml:

Profile	Database	Description
Monolith	msdb	Single database for all aggregates
Microservices	Per-service DBs	Each service has its own database (e.g., tournamentdb, userdb)

Service-specific database URLs are configured in profile files like application-tournament-service.yaml.

Spring Cloud Stream Bindings

When running with the stream profile, inter-service communication uses RabbitMQ. Bindings are configured in application.yaml:

Binding Type	Example	Purpose
Command Channels	tournament-command-channel	Send commands to services
Command Consumers	tournamentServiceCommandChannel-in-0	Receive and process commands
Event Channel	event-channel	Broadcast events to subscribers
Event Subscribers	tournamentEventSubscriber-in-0	Receive events for processing
Response Channel	commandResponseChannel-in-0	Receive command responses

Service-specific bindings override only the channels relevant to that service, as shown in application-tournament-service.yaml.

gRPC Command Gateway

An alternative remote transport is available with the grpc profile. Each service exposes a gRPC endpoint for commands (see GrpcServerRunner), and callers use GrpcCommandGateway with Eureka-based discovery. Default and service-specific gRPC ports are configured in the application-*-service.yaml files (and exposed via Eureka metadata key grpcPort). Override the default client port with grpc.command.default-port or per-service with grpc.command.<service>.port when needed.

Service URLs and Ports

Each microservice runs on a dedicated port:

Service	Port	Profile File
Gateway	8080	application.yaml
Version Service	8081	-
Answer Service	8082	application-answer-service.yaml
Course Execution	8083	application-course-execution-service.yaml
Question Service	8084	application-question-service.yaml
Quiz Service	8085	application-quiz-service.yaml
Topic Service	8086	application-topic-service.yaml
Tournament Service	8087	application-tournament-service.yaml
User Service	8088	application-user-service.yaml

Every service port can be changed, including version-service port 8081, and gateway port 8080. Service Discovery will map the service name to the service port automatically.

API Gateway Configuration

The Gateway application.yaml configures:

1. Service discovery (lines 8-25): Eureka discovery for local deployments; Kubernetes discovery is enabled in the kubernetes profile.

2. Route definitions (lines 30-87): Map API paths to backend services using lb://<service>${gateway.service-suffix} URIs.

3. Version service URL (line 18): Base URL for the version service used by admin endpoints.

Code structure

Simulator

• The core concepts of Domain-Driven Design
• The core concepts for the distributed functionalities Coordination
• The core concepts for management of Sagas
• The core concepts for management of TCC

Quizzes Microservice System

• A case study for Quizzes Tutor
  • The transactional model independent Microservices
  • The Sagas implementation for
    • Aggregates (per microservice, e.g. Tournament)
    • Coordination (per microservice, e.g. Tournament)
  • The TCC implementation for
    • Aggregates (per microservice, e.g. Tournament)
    • Coordination (per microservice, e.g. Tournament)
• The tests of the Quizzes Tutor for
  • Sagas
  • TCC Aggregates
  • TCC Coordination

The code follows the structure in the simulator library and application decomposition figures, where the packages in blue and orange contain, respectively, the microservices domain specific code and the transactional causal consistency domain specific code.

The API Gateway is used when running the quizzes application as microservices to route API requests to the appropriate microservice.

How to implement and test your own business logic for Sagas and TCC (Illustrated with Quizzes Microservice System)

The figure shows the main classes to be extended for aggregates, their events and services.

Apply the following steps to define a domain-specific aggregate, its events and services, here illustrated with the Quizzes Tutor system and its Tournament aggregate.

For the transactional model independent part:

1. Define Aggregate: Each microservice is modeled as an aggregate. The first step is to define the aggregates. The simulator uses Spring-Boot and JPA, so the domain entities definition uses the JPA notation. In Tournament aggregate we can see the aggregate root entity and the reference to its internal entities.
2. Specify Invariants: The aggregate invariants are defined by overriding method verifyInvariants().
3. Define Events: Define the events published by upstream aggregates and subscribed by downstream aggregates, like UpdateStudentNameEvent.
4. Subscribe Events: The events published by upstream aggregates can be subscribed by overriding method getEventSubscriptions().
5. Define Event Subscriptions: Events can be subscribed depending on its data. Therefore, define subscription classes like TournamentSubscribesUpdateStudentName.
6. Define Event Handlers: For each subscribed event define an event handler that delegates the handling in a handling functionality, like UpdateStudentNameEventHandler and its handling functionality processUpdateStudentNameEvent(...).
7. Define Aggregate Services: Define the microservice API, whose implementation interact with the unit of work to register changes and publish events, like service updateExecutionStudentName(...).
8. Define Event Handling: Define the aggregates event handling, that periodically polls the event table to process events, like TournamentEventHandling.
9. Define Event Subscriber Service: Define the event subscriber service, that subscribes to events published by other microservices via Spring Cloud Stream, like TournamentEventSubscriberService.

For the transactional model dependent part:

1. Define Saga Aggregates: Extend aggregates with the information required for semantic locks, like SagaTournament and its Semantic Lock.
2. Define Causal Aggregates: Extend aggregates with the information required for causal consistency, like CausalTournament

To define the system functionalities, it is necessary to extend the simulator part for coordination.

For the functionalities:

1. Define Functionalities: Functionalities coordinate the execution of aggregate services using sagas, like functionality AddParticipantFunctionalitySagas(...) and AddParticipantFunctionalityTCC(...)
2. Define Commands: Define the commands to be executed by the functionalities, like AddParticipantCommand. Every method of the aggregate service should have a corresponding command.

For the inter-service communication:

1. Create the CommandHandlers of the aggregate: It receives commands from local or remote services' functionalities and calls the corresponding aggregate service method of that command, like TournamentCommandHandler for local calls, TournamentStreamCommandHandler for remote calls via messaging, and TournamentGrpcCommandHandler for remote calls via gRPC.
2. Configure Spring Cloud Stream Bindings (for stream profile): Define the command and event channels in application.yaml, like tournament-service bindings.
3. Configure gRPC Server Port (for grpc profile): Define the gRPC server port in the service profile file and expose it via Eureka metadata, like tournament-service gRPC config.
4. Configure API Gateway Routes: Define the route mappings in the Gateway application.yaml to route API requests to the new microservice.

To write tests:

1. Design Test Cases: Define tests cases for the concurrent execution of functionalities deterministically enforcing execution orders, like in the Concurrent Execution of Update Name and Add Participant. Directory coordination contains the test of more complex interleavings using the sagas transactional model.

Running JMeter tests

• After starting application with the tcc profile, either using Docker or Maven, and installing JMeter

cd applications/quizzes/jmeter/tournament/thesis-cases/
jmeter -n -t TEST.jmx

• Some test cases:
  • 5a-updateStudentName-addParticipant-processUpdateNameEvent.jmx
  • 5b-addParticipant-updateStudentName-processUpdateNameEvent.jmx
  • 5c-updateStudentName1-addParticipant-updateStudentName2-processUpdateNameEvent.jmx
  • 5d-addParticipant1-updateStudentName-processUpdateNameEvent1-addParticipant2-processUpdateNameEvent2.jmx
  • 8-5-update-tournament-concurrent-intention-pass.jmx
  • 8-6-add-participant-concurrent-update-execution-student-name-processing-ends-first.jmx
  • 8-7-add-participant-concurrent-anonymize-event-processing-processing-ends-last.jmx
  • 8-8-update-execution-student-add-participant-process-event-add-participant.jmx
  • 8-9-add-participant-concurrent-anonymize-event-processing-processing-ends-first.jmx
  • 8-10-concurrent-delete-tournament-add-participant.jmx

Viewing JMeter tests structure

cd applications/quizzes/jmeter/tournament/thesis-cases/
jmeter

• The command launches JMeter GUI. By clicking File > Open and selecting a test file it is possible to observe the test structure.
• Tests can also be run using the GUI, by clicking on the Start button.

Spock Tests in DAIS2023 paper - 23nd International Conference on Distributed Applications and Interoperable Systems

To reproduce the paper results follow the steps:

• Analyze a figure in the paper, fig3a-d and fig4;
• Read the test case code for the figure, including the final assertions that define the expected behavior (see below);
• Run the test case (see below);
• Read the logger INFO messages, they use UPPERCASE. They identify when a functionality and event processing starts and ends and what its version number is.
  • For instance, in test-fig4 both functionalities start with the same version number (they are concurrent), but addParticipant finishes with a higher number, because it finishes after updateName. It can be observed in the log that an exception was thrown, due to the invariant break.

Figure 3(a)

• Test code
• Run:

docker compose up test-fig3a

Figure 3(b)

• Test code
• Run:

docker compose up test-fig3b

Figure 3(c)

• Test code
• Run:

docker compose up test-fig3c

Figure 3(d)

• Test code
• Run:

docker compose up test-fig3d

Figure 4

• Test code
• Run:

docker compose up test-fig4