Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
e500826
Fix Complex Method, context.WithValue and other errors in code
r4ulcl Feb 2, 2024
edf1e40
Add worker dev workflow
r4ulcl Feb 2, 2024
f7c8def
Fix DOCKERHUB_USERNAME error
r4ulcl Feb 2, 2024
091a288
Fix login error
r4ulcl Feb 3, 2024
0132c0c
SSH remove log.Fatal and loop to autoconnect always test
r4ulcl Feb 3, 2024
6654633
Fix filter task
r4ulcl Feb 3, 2024
0e085e9
Try fix get tasks filters
r4ulcl Feb 3, 2024
9baf94a
Add sleep between checks
r4ulcl Feb 3, 2024
1403360
Fix error name workflow
r4ulcl Feb 3, 2024
7536494
Log runModule
r4ulcl Feb 3, 2024
a555b45
Debug add1IddleThreads
r4ulcl Feb 3, 2024
6053a18
Remove idleThreads count in workers status
r4ulcl Feb 3, 2024
9cadd10
Remove AddWorkerIddleThreads1
r4ulcl Feb 3, 2024
ffa9297
Fix errors websocket
r4ulcl Feb 3, 2024
0bc2a08
Fix status websocket
r4ulcl Feb 3, 2024
8a0fad7
Add workingID on start module
r4ulcl Feb 3, 2024
a6e685d
Fix iddleThreads
r4ulcl Feb 3, 2024
3fe013a
Add again SubtractWorkerIddleThreads1
r4ulcl Feb 3, 2024
0a57fa5
AddWorkerIddleThreads1 on callback
r4ulcl Feb 3, 2024
ad83d78
Set version to v0.1.1
r4ulcl Feb 3, 2024
4a95ad2
Fix return IddleThreads
r4ulcl Feb 3, 2024
9fe9e18
Fix add config to function
r4ulcl Feb 3, 2024
e6b5955
Fixx status and add worker info and 5 second timeout in ping websocket
r4ulcl Feb 3, 2024
a3317f5
Add debug info RecreateConnection
r4ulcl Feb 4, 2024
d7822f6
Add CreateConnection function to better code
r4ulcl Feb 4, 2024
ea40eeb
add connectino ok debug
r4ulcl Feb 4, 2024
4035093
Add SetPongHandler manager
r4ulcl Feb 4, 2024
7f718b3
config.Conn with lock and v0.2
r4ulcl Feb 4, 2024
99e8030
Add build and release test
r4ulcl Feb 5, 2024
e73e3e4
Buffer 8k, change client socket manager
r4ulcl Feb 5, 2024
e272cad
Update worker info on addworker already exists
r4ulcl Feb 5, 2024
08ccf07
Timeout to 5 secs and close conn on timeout
r4ulcl Feb 5, 2024
3d90d1f
Add activeConnections to manager
r4ulcl Feb 5, 2024
43af2e4
Set pending when remove worker
r4ulcl Feb 6, 2024
c8ff8db
Add digital occean test
r4ulcl Feb 6, 2024
f5aa803
Change 10 secons after delete
r4ulcl Feb 6, 2024
bf75077
Add certs to scratch for API digitalocean
r4ulcl Feb 6, 2024
44e07e1
Fix dockerfile for CA
r4ulcl Feb 6, 2024
7694bf3
Add default in switch
r4ulcl Feb 6, 2024
abfa873
Not disconnect worker if cant send message
r4ulcl Feb 6, 2024
840e0ea
ProcessDigitalOcean goroutine, change tags droplet
r4ulcl Feb 7, 2024
abef4fb
Bump golang.org/x/crypto from 0.18.0 to 0.19.0
dependabot[bot] Feb 8, 2024
9d3622a
Add ?style=flat, update TODO
r4ulcl Feb 9, 2024
edd84d2
platforms: linux/amd64,linux/arm64 in workflow
r4ulcl Feb 9, 2024
95a5405
test Build and Release and add image to compose
r4ulcl Feb 26, 2024
15836f0
Set ${{ github.repository }} instead of actor + manual repo name
r4ulcl Mar 9, 2024
a3c0b2a
Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#7)
dependabot[bot] Mar 9, 2024
1280ccf
Test set_lowercase_repo
r4ulcl Mar 9, 2024
aebda3d
Set set_lowercase_repo to all workflows
r4ulcl Mar 9, 2024
5f39372
Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.0 (#9)
dependabot[bot] Mar 11, 2024
42ca32e
Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#8)
dependabot[bot] Mar 11, 2024
af4add5
Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.1 (#12)
dependabot[bot] Mar 31, 2024
1670bdc
Test build and push once only
r4ulcl Apr 10, 2024
25cd9f0
Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 (#20)
dependabot[bot] Nov 17, 2024
27b4406
Bump github.com/gorilla/websocket from 1.5.1 to 1.5.3 (#19)
dependabot[bot] Nov 17, 2024
0988ff9
Bump golang.org/x/crypto from 0.20.0 to 0.24.0 (#17)
dependabot[bot] Nov 17, 2024
fe59c56
Bump golang.org/x/net from 0.21.0 to 0.23.0 in the go_modules group (…
dependabot[bot] Nov 17, 2024
45cbe6c
Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.1 (#13)
dependabot[bot] Nov 17, 2024
2aedf52
Change task to have array of files in b64 like commands
r4ulcl Nov 17, 2024
713dcc3
Update cert and change expiration to 10 years
r4ulcl Nov 17, 2024
6852d1c
Remove kali-tools-fuzzing from worker docker
r4ulcl Nov 18, 2024
c3f25ef
Remove vscode folder update go.mod
r4ulcl Nov 30, 2024
e3166d2
Add 2 ports, HTTP and HTTPS at the same time
r4ulcl Dec 4, 2024
dafb914
Fix both servers at the same time
r4ulcl Dec 4, 2024
71802da
Port to int and set server timeouts
r4ulcl Dec 5, 2024
57c5c64
codacy medium problems
r4ulcl Dec 7, 2024
628a7a7
Remove Fprintln to clean response
r4ulcl Dec 7, 2024
bc70f50
add Run Gosec dev
r4ulcl Dec 7, 2024
26efde3
Update workflows template
r4ulcl Dec 7, 2024
fb277e4
Update README with Cloud settings
r4ulcl Dec 7, 2024
5c49294
Fix SQLi, secure docker-compose and update go version and mod
r4ulcl Dec 10, 2024
be58eee
Fix gosec minor changes
r4ulcl Dec 10, 2024
04f5992
Fix gosec minor changes 2
r4ulcl Dec 10, 2024
590b45f
Use restricted permissions (0600)
r4ulcl Dec 10, 2024
9e5b743
VersionTLS12 and fix SQLi 2
r4ulcl Dec 10, 2024
ab022f9
Comment on exported functions, use verbose var and comment unused fun…
r4ulcl Dec 11, 2024
706c1b2
Remove else block
r4ulcl Dec 11, 2024
993ecfb
Fix var-naming
r4ulcl Dec 11, 2024
89033a6
Complex Method GetWorkerMessage and runModule fix and remove gosec
r4ulcl Dec 12, 2024
3f6c728
Minor fixes documentation in package and export functions and fix com…
r4ulcl Dec 12, 2024
802cd33
Remove LoC and add codacy in README
r4ulcl Dec 12, 2024
00be209
refactor StartManager
r4ulcl Dec 12, 2024
39b7768
Refactor StartManager
r4ulcl Dec 13, 2024
3cc80b3
Fix unused params
r4ulcl Dec 13, 2024
3c3823a
Remove duplicate code and add worker2 to test.yml
r4ulcl Dec 14, 2024
2757e4f
Fix loadManagerConfig duplication code
r4ulcl Dec 14, 2024
2bfbd85
add HandleAddWorkerError
r4ulcl Dec 14, 2024
86c6b9f
Add notes and httpsServer timeouts in config file, add default value…
r4ulcl Dec 15, 2024
42ed49e
Remove duplicated code send message
r4ulcl Dec 15, 2024
6a24922
Remove duplicates
r4ulcl Dec 15, 2024
5999a4f
Remove duplicates in digitalocean.go
r4ulcl Dec 15, 2024
3911e26
Fix "Error UpdateTask: sql: expected 10 arguments, got 11", add OK;ad…
r4ulcl Dec 15, 2024
32f3833
Fix error in code, update task
r4ulcl Dec 15, 2024
928ac0c
INDEX idx_status and add APIReadTimeout, etc.
r4ulcl Dec 17, 2024
9d8f684
Add option to delety Entrieshistory with a max number of stored tasks…
r4ulcl Dec 17, 2024
d9c8e5e
Update certs 10 years
r4ulcl Dec 26, 2024
498dac5
Add timeout to nTask
r4ulcl Dec 28, 2024
d1cae30
Remove unused exported functions
r4ulcl Dec 29, 2024
fc824c8
Use bash instead of sh
r4ulcl Dec 30, 2024
925e88b
use --login to load bashrc
r4ulcl Dec 30, 2024
25f961b
add deleteFiles bool to delete send files in Task
r4ulcl Dec 31, 2024
5d02835
Update dockerfile to first go mod download to cache download
r4ulcl Jan 3, 2025
a1d2b82
Add duration to all task in seconds
r4ulcl Jan 13, 2025
dce8412
Reestrucure GetWorkerMessage to avoid infinite loop error - Update certs
r4ulcl Jul 8, 2025
04891dc
Set task to pending if the worker fails and fix mysql error becase wg
r4ulcl Jul 10, 2025
944fe6f
Big fix and optimization, add updatedAt, add semaphores , recreate we…
r4ulcl Jul 16, 2025
caa0a29
Add tags DEV dockers
r4ulcl Jul 16, 2025
b29c7c2
Remove TODO comment
r4ulcl Jul 16, 2025
3a17a19
Update compose files
r4ulcl Jul 17, 2025
db33845
Final version of optimization, update dispatcher
r4ulcl Jul 18, 2025
43d7181
Delete unused functions
r4ulcl Jul 18, 2025
bc49fd3
Update Dockerfile
r4ulcl Jul 18, 2025
58cc8cb
Merge branch 'main' into dev
r4ulcl Jan 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# .dockerignore: exclude files/directories from build context
.git
.gitignore
.dockerignore
Dockerfile
vendor/
*.md
*.exe
*.exe~
*.dll
*.so
*.dylib
*.swp
*~
.DS_Store
db/
*.conf
*.md
19 changes: 19 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Manager
MANAGER_IMAGE=ntask_manager:latest
MANAGER_HTTP_PORT=8080
MANAGER_HTTPS_PORT=8443

# Worker
WORKER_IMAGE=ntask_worker:latest
WORKER_REPLICAS=2
WORKER_REPLICAS_DEV=2
WORKER_REPLICAS_TEST=100

# Database
MYSQL_VERSION=8.0
DB_PORT=3306
MYSQL_ROOT_PASSWORD=your_password_root
MYSQL_USER=your_username
MYSQL_PASSWORD=your_password
MYSQL_DATABASE=manager
MYSQL_ROOT_HOST=0.0.0.0
67 changes: 67 additions & 0 deletions .github/workflows/build-and-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Build and Release

on:
push:
tags:
- 'v*'

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
go-version: [1.21.6] # Add more versions if needed
os: [windows, linux, darwin]
arch: [amd64, arm64, 386, arm]

steps:
- name: Checkout code
uses: actions/checkout@v2

- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
# Set other Go-related options as needed

- name: Build binary
run: |
if [ "${{ runner.os }}" = "Windows" ]; then
go build -o ${GITHUB_REPOSITORY}-${{ matrix.os }}-${{ matrix.arch }}.exe
else
go build -o ${GITHUB_REPOSITORY}-${{ matrix.os }}-${{ matrix.arch }}
fi

release:
needs: build
runs-on: ubuntu-latest

steps:
- name: Create Release Draft
if: startsWith(github.ref, 'refs/tags/v')
id: create_release
uses: actions/create-release@v1
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
draft: true
body: |
Release notes for version ${{ github.ref }}

- name: Upload Release Assets
if: startsWith(github.ref, 'refs/tags/v')
id: upload-release-assets
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: |
${GITHUB_REPOSITORY}-windows-amd64.exe
${GITHUB_REPOSITORY}-windows-386.exe
${GITHUB_REPOSITORY}-linux-amd64
${GITHUB_REPOSITORY}-linux-arm64
${GITHUB_REPOSITORY}-linux-386
${GITHUB_REPOSITORY}-darwin-amd64
${GITHUB_REPOSITORY}-darwin-arm64
asset_name: ${GITHUB_REPOSITORY}-${{ matrix.arch }}
37 changes: 19 additions & 18 deletions .github/workflows/build-manager-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,46 +10,47 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v3

- name: Checkout
uses: actions/checkout@v3

# manager docker
- name: Set lowercase repository name
id: set_lowercase_repo
run: |
LOWERCASE_REPO=$(echo "${GITHUB_REPOSITORY}" | tr '[:upper:]' '[:lower:]')
echo "::set-output name=lowercase_repo::${LOWERCASE_REPO}"

# Docker
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ github.actor }}/ntask-manager
images: ${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-manager

- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push docker
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./manager/Dockerfile
platforms: linux/amd64
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/ntask-manager:dev

- name: Login to GitHub Container Registry
run: echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Build and push docker
id: docker_build_ghcr
# Push
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
context: .
file: ./manager/Dockerfile
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.actor }}/ntask-manager:dev
tags: |
${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-manager:dev
ghcr.io/${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-manager:dev
labels: ${{ steps.meta.outputs.labels }}

40 changes: 19 additions & 21 deletions .github/workflows/build-manager.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ name: Docker Build and Publish Manager

on:
push:
branches:
- 'main'
tags:
- 'v*'

Expand All @@ -12,47 +10,47 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v3

- name: Checkout
uses: actions/checkout@v3

- name: Set lowercase repository name
id: set_lowercase_repo
run: |
LOWERCASE_REPO=$(echo "${GITHUB_REPOSITORY}" | tr '[:upper:]' '[:lower:]')
echo "::set-output name=lowercase_repo::${LOWERCASE_REPO}"

# manager docker
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ github.actor }}/ntask-manager
images: ${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-manager

- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push docker
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./manager/Dockerfile
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Login to GitHub Container Registry
run: echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Build and push docker
id: docker_build_ghcr
# Push
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
context: .
file: ./manager/Dockerfile
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.actor }}/ntask-manager:latest, ghcr.io/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
tags: |
${{ steps.meta.outputs.tags }}
ghcr.io/${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-manager:latest
ghcr.io/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
56 changes: 56 additions & 0 deletions .github/workflows/build-worker-dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Docker Build and Publish Worker DEV

on:
push:
branches:
- 'dev'

jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Checkout
uses: actions/checkout@v3

- name: Set lowercase repository name
id: set_lowercase_repo
run: |
LOWERCASE_REPO=$(echo "${GITHUB_REPOSITORY}" | tr '[:upper:]' '[:lower:]')
echo "::set-output name=lowercase_repo::${LOWERCASE_REPO}"

# Docker
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-worker

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Login to GitHub Container Registry
run: echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

# Push
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
context: .
file: ./worker/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: |
${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-worker:dev
ghcr.io/${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-worker:dev
labels: ${{ steps.meta.outputs.labels }}

40 changes: 19 additions & 21 deletions .github/workflows/build-worker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ name: Docker Build and Publish Worker

on:
push:
branches:
- 'main'
tags:
- 'v*'

Expand All @@ -12,47 +10,47 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v3

- name: Checkout
uses: actions/checkout@v3

- name: Set lowercase repository name
id: set_lowercase_repo
run: |
LOWERCASE_REPO=$(echo "${GITHUB_REPOSITORY}" | tr '[:upper:]' '[:lower:]')
echo "::set-output name=lowercase_repo::${LOWERCASE_REPO}"

# worker docker
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ github.actor }}/ntask-worker
images: ${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-worker

- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push docker
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./worker/Dockerfile
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Login to GitHub Container Registry
run: echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

- name: Build and push docker
id: docker_build_ghcr
# Push
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
context: .
file: ./worker/Dockerfile
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.actor }}/ntask-worker:latest, ghcr.io/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
tags: |
${{ steps.meta.outputs.tags }}
ghcr.io/${{ steps.set_lowercase_repo.outputs.lowercase_repo }}-worker:latest
ghcr.io/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,5 @@
go.work
db/*
output/*
nTask
.vscode/*
Loading
Loading