Skip to content

dsouzadilton/TestDataService

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🚀 TestDataService

A lightweight, Redis-powered Test Data Service for Performance & Automation Testing


🧠 Why This Exists

Managing test data in distributed performance tests is painful:

  • ❌ CSV files don’t scale
  • ❌ Data duplication across threads
  • ❌ No central coordination
  • ❌ VTS limitations

TestDataService solves this.


⚡ What It Does

  • 📤 Upload CSV datasets
  • 📊 View & manage data via UI
  • 🔁 Share data across threads & nodes
  • 🎯 Fetch unique records using POP (queue behavior)
  • 🧹 Clear / Drop / Export datasets

🏗️ Architecture

JMeter / API Clients

TestDataService API

Redis


🔑 Core Concepts

Concept Meaning
Instance Logical grouping (like VTS port)
Table Dataset
Row JSON record
POP Fetch + remove (queue)

🚀 Features

✅ Data Management

  • Upload CSV → auto-converted & mapped to JSON in backend
  • Append / Overwrite datasets
  • Export data back to CSV

✅ UI

  • Instance & table selection
  • Grid view (like VTS)
  • Sticky headers
  • Record count

✅ Data Consumer & Producer

  • GET by index
  • POP (queue-based unique data)
  • POST (append data to table)

✅ Controls

  • Clear table (keep structure)
  • Drop table (delete completely)

🔌 APIs

Get Data by Index

GET /tds/getByIndex?instance=demo&table=users&index=0

POP (🔥 Most Important)

GET /tds/pop?instance=demo&table=users

POST

POST /tds/push?instance=demo&table=users

Upload CSV

POST /tds/upload


🧪 JMeter Usage

  1. Add HTTP Sampler: GET /tds/pop?instance=demo&table=users

  2. Add JSON Extractor: id → $.id
    email → $.email

👉 Each thread gets unique data


🛠️ Tech Stack

  • Java (Spring Boot)
  • Redis
  • HTML + JS (lightweight UI)

⚠️ Notes

  • POP is destructive (data is removed)
  • Ensure proper dataset size for load tests
  • Redis must be running

🚀 Getting Started

1. Start Redis

redis-server

2. Run Application

mvn spring-boot:run

3. Open UI

http://localhost:8080


🔥 Future Enhancements

  • Pagination / virtual scrolling
  • Authentication
  • Multi-user support
  • Redis clustering
  • JMeter plugin

🤝 Contributing

PRs welcome. Keep it simple, clean, and scalable.


⭐ Final Thought

This started as a VTS replacement…

👉 It’s now a Test Data Platform

About

A lightweight, Redis-powered Test Data Service for Performance & Automation Testing

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors