diff --git a/README.md b/README.md index 0a35a3f30..5cc0712e9 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,8 @@ Setup Instructions Server Requirements ------------------- -* 4 Gb Ram +* Ubuntu 20.04, 18.04 (confirmed working) +* 8 Gb Ram * 2 CPU Cores (with AES_NI) * 150 Gb SSD-Backed Storage - If you're doing a multi-server install, the leaf nodes do not need this much storage. They just need enough storage to hold the blockchain for your node. The pool comes configured to use up to 60Gb of storage for LMDB. Assuming you have the longRunner worker running, it should never get near this size, but be aware that it /can/ bloat readily if things error, so be ready for this! * Notably, this happens to be approximately the size of a 4Gb linode instance, which is where the majority of automated deployment testing happened! @@ -59,13 +60,14 @@ Deployment via Installer ```shell cd ~/nodejs-pool/ -pm2 start init.js --name=blockManager --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=blockManager -pm2 start init.js --name=worker --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=worker -pm2 start init.js --name=payments --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" --no-autorestart -- --module=payments -pm2 start init.js --name=remoteShare --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=remoteShare -pm2 start init.js --name=longRunner --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=longRunner -pm2 start init.js --name=pool --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=pool -pm2 start init.js --name=api --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=api +pm2 start init.js --name=blockManager --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=blockManager +pm2 start init.js --name=worker --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=worker +pm2 start init.js --name=pool_stats --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=pool_stats +pm2 start init.js --name=payments --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" --no-autorestart -- --module=payments +pm2 start init.js --name=remoteShare --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=remoteShare +pm2 start init.js --name=longRunner --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=longRunner +pm2 start init.js --name=pool --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=pool +pm2 start init.js --name=api --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=api pm2 restart api ``` @@ -87,8 +89,6 @@ The installer assumes that you will be running a single-node instance and using The following raw binaries **MUST BE AVAILABLE FOR IT TO BOOTSTRAP**: * sudo -I've confirmed that the default server 16.04 installation has these requirements. - The pool comes pre-configured with values for Monero (XMR), these may need to be changed depending on the exact requirements of your coin. Other coins will likely be added down the road, and most likely will have configuration.sqls provided to overwrite the base configurations for their needs, but can be configured within the frontend as well. The pool ALSO applies a series of patches: Fluffy Blocks, Additional Open P2P Connections, 128 Txn Bug Fix. If you don't like these, replace the auto-installed monerod fixes! @@ -256,19 +256,33 @@ For assistance, please contact MoneroOcean at support@moneroocean.stream. Developer Donations =================== If you'd like to make a one time donation, the addresses are as follows: -* XMR - ```499fS1Phq64hGeqV8p2AfXbf6Ax7gP6FybcMJq6Wbvg8Hw6xms8tCmdYpPsTLSaTNuLEtW4kF2DDiWCFcw4u7wSvFD8wFWE``` +* XMR - ```89TxfrUmqJJcb1V124WsUzA78Xa3UYHt7Bg8RGMhXVeZYPN8cE5CZEk58Y1m23ZMLHN7wYeJ9da5n5MXharEjrm41hSnWHL``` * AEON - ```WmsEg3RuUKCcEvFBtXcqRnGYfiqGJLP1FGBYiNMgrcdUjZ8iMcUn2tdcz59T89inWr9Vae4APBNf7Bg2DReFP5jr23SQqaDMT``` * ETN - ```etnkQMp3Hmsay2p7uxokuHRKANrMDNASwQjDUgFb5L2sDM3jqUkYQPKBkooQFHVWBzEaZVzfzrXoETX6RbMEvg4R4csxfRHLo1``` * SUMO - ```Sumoo1DGS7c9LEKZNipsiDEqRzaUB3ws7YHfUiiZpx9SQDhdYGEEbZjRET26ewuYEWAZ8uKrz6vpUZkEVY7mDCZyGnQhkLpxKmy``` * GRFT - ```GACadqdXj5eNLnyNxvQ56wcmsmVCFLkHQKgtaQXNEE5zjMDJkWcMVju2aYtxbTnZgBboWYmHovuiH1Ahm4g2N5a7LuMQrpT``` * MSR - ```5hnMXUKArLDRue5tWsNpbmGLsLQibt23MEsV3VGwY6MGStYwfTqHkff4BgvziprTitbcDYYpFXw2rEgXeipsABTtEmcmnCK``` -* ITNS - ```iz53aMEaKJ25zB8xku3FQK5VVvmu2v6DENnbGHRmn659jfrGWBH1beqAzEVYaKhTyMZcxLJAdaCW3Kof1DwTiTbp1DSqLae3e``` +* LTHN - ```iz53aMEaKJ25zB8xku3FQK5VVvmu2v6DENnbGHRmn659jfrGWBH1beqAzEVYaKhTyMZcxLJAdaCW3Kof1DwTiTbp1DSqLae3e``` * WOW - ```Wo3yjV8UkwvbJDCB1Jy7vvXv3aaQu3K8YMG6tbY3Jo2KApfyf5RByZiBXy95bzmoR3AvPgNq6rHzm98LoHTkzjiA2dY7sqQMJ``` -* XMV - ```4BDgQohRBqg2wFZ5ezYqCrNGjgECAttARdbh1fNkuAbd3HnNkSgas11QD9VFQMzbnvDD3Mfcky1LAFihkbEYph5oGAMLurw``` +* XMV - ```XvyVfpAYp3zSuvdtoHgnDzMUf7GAeiumeUgVC7RTq6SfgtzGEzy4dUgfEEfD5adk1kN4dfVZdT3zZdgSD2xmVBs627Vwt2C3Ey``` * RYO - ```RYoLsi22qnoKYhnv1DwHBXcGe9QK6P9zmekwQnHdUAak7adFBK4i32wFTszivQ9wEPeugbXr2UD7tMd6ogf1dbHh76G5UszE7k1``` -* XTL - ```Se3Qr5s83AxjCtYrkkqg6QXJagCVi8dELbHb5Cnemw4rMk3xZzEX3kQfWrbTZPpdAJSP3enA6ri3DcvdkERkGKE518vyPQTyi``` +* XLA - ```SvkpUizij25ZGRHGb1c8ZTAHp3VyNFU3NQuQR1PtMyCqdpoZpaYAGMfG99z5guuoktY13nrhEerqYNKXvoxD7cUM1xA6Z5rRY``` * XHV - ```hvxyEmtbqs5TEk9U2tCxyfGx2dyGD1g8EBspdr3GivhPchkvnMHtpCR2fGLc5oEY42UGHVBMBANPge5QJ7BDXSMu1Ga2KFspQR``` -* TUBE - ```bxcpZTr4C41NshmJM9Db7FBE5crarjaDXVUApRbsCxHHBf8Jkqjwjzz1zmWHhm9trWNhrY1m4RpcS7tmdG4ykdHG2kTgDcbKJ``` +* TUBE - ```TubedBNkgkTbd2CBmLQSwW58baJNghD9xdmctiRXjrW3dE8xpUcoXimY4J5UMrnUBrUDmfQrbxRYRX9s5tQe7pWYNF2QiAdH1Fh``` +* LOKI - ```L6XqN6JDedz5Ub8KxpMYRCUoQCuyEA8EegEmeQsdP5FCNuXJavcrxPvLhpqY6emphGTYVrmAUVECsE9drafvY2hXUTJz6rW``` +* TRTL - ```TRTLv2x2bac17cngo1r2wt3CaxN8ckoWHe2TX7dc8zW8Fc9dpmxAvhVX4u4zPjpv9WeALm2koBLF36REVvsLmeufZZ1Yx6uWkYG``` +* XTNC - ```XtazhSxz1bbJLpT2JuiD2UWFUJYSFty5SVWuF6sy2w9v8pn69smkUxkTVCQc8NKCd6CBMNDGzgdPRYBKaHdbgZ5SNptVH1yPCTQ``` +* IRD - ```ir3DHyB8Ub1aAHEewMeUxQ7b7tQdWa7VL8M5oXDPohS3Me4nhwvALXM4mym2kWg9VsceT75dm6XWiWF1K4zu8RVQ1HJD8Z3R9``` +* ARQ - ```ar4Ha6ZQCkKRhkKQLfexv7VZQM2MhUmMmU9hmzswCPK4T3o2rbPKZM1GxEoYg4AFQsh57PsEets7sbpU958FAvxo2RkkTQ1gE``` +* XWP - ```fh4MCJrakhWGoS6Meqp6UxGE1GNfAjKaRdPjW36rTffDiqvEq2HWEKZhrbYRw7XJb3CXxkjL3tcYGTT39m5qgjvk1ap4bVu1R``` +* XEQ - ```Tvzp9tTmdGP9X8hCEw1Qzn18divQajJYTjR5HuUzHPKyLK5fzRt2X73FKBDzcnHMDJKdgsPhUDVrKHVcDJQVmLBg33NbkdjQb``` +* XTA - ```ipN5cNhm7RXAGACP4ZXki4afT3iJ1A6Ka5U4cswE6fBPDcv8JpivurBj3vu1bXwPyb8KZEGsFUYMmToFG4N9V9G72X4WpAQ8L``` +* DERO - ```dERokvcrnuWH1ai1QmZQc9cgxrLwE3rX3TbhdrnLmi3BVZmf197qd5FaFqmPMp5dZ3igXfVQwUUMgTSjpVKDtUeb6DT2xp64XJ``` +* CCX - ```ccx7dmnBBoRPuVcpKJSAVZKdSDo9rc7HVijFbhG34jsXL3qiqfRwu7A5ecem44s2rngDd8y8N4QnYK6WR3mXAcAZ5iXun9BQBx``` +* BLOC - ```abLoc5iUG4a6oAb2dqygxkS5M2uHWx16zHb9fUWMzpSEDwm6T7PSq2MLdHonWZ16CGfnJKRomq75aZyviTo6ZjHeYQMzNAEkjMg``` +* RVN - ```RLVJv9rQNHzXS3Zn4JH8hfAHmm1LfECMxy``` +* RTM - ```RUCyaEZxQu3Eure73XPQ57si813RYAMQKC``` +* ERG - ```9fe533kUzAE57YfPP6o3nzsYMKN2W2uCxvg8KG8Vn5DDeJGetRw``` * BTC - ```3BzvMuLStA388kYZ9nudfm8L22937dSPS3``` * BCH - ```qrhww48p5s6zw9twhc7cujgwp7vym2k4vutem6f92p``` * ETH - ```0xCF8BABC074C487Ae17F9Ce0394eab492E6A35658``` diff --git a/block_notify.sh b/block_notify.sh new file mode 100755 index 000000000..dce9fe99d --- /dev/null +++ b/block_notify.sh @@ -0,0 +1,2 @@ +#!/bin/bash +/bin/echo 18081 | /bin/nc -N localhost 2223 \ No newline at end of file diff --git a/block_share_dumps/calc_mo_cvs.js b/block_share_dumps/calc_mo_cvs.js new file mode 100644 index 000000000..7d67e6b5c --- /dev/null +++ b/block_share_dumps/calc_mo_cvs.js @@ -0,0 +1,79 @@ +"use strict"; + +if (Boolean(process.stdin.isTTY) || process.argv.length !== 3) { + console.log("Usage: unxz -c .cvs.xz | node calc_mo_cvs.js "); + console.log(" wget -O - https://block-share-dumps.moneroocean.stream/.cvs.xz | unxz -c | node calc_mo_cvs.js "); + process.exit(1); +} + +const my_wallet = process.argv[2].slice(-16); + +let stdin = ""; + +process.stdin.on('data', function(data) { + stdin += data.toString(); +}); + +function human_hashrate(hashes) { + const power = Math.pow(10, 2 || 0); + if (hashes > 1000000000000) return String(Math.round((hashes / 1000000000000) * power) / power) + " TH/s"; + if (hashes > 1000000000) return String(Math.round((hashes / 1000000000) * power) / power) + " GH/s"; + if (hashes > 1000000) return String(Math.round((hashes / 1000000) * power) / power) + " MH/s"; + if (hashes > 1000) return String(Math.round((hashes / 1000) * power) / power) + " KH/s"; + return Math.floor( hashes || 0 ) + " H/s" +}; + +process.stdin.on('end', function() { + let pplns_window = 0; + let oldest_timestamp = 0; + let newest_timestamp = 0; + + let my_share_count = 0; + let my_xmr_diff = 0; + let my_xmr_diff_payed = 0; + let my_coin_raw_diff = {}; + let my_coin_xmr_diff = {}; + + for (let line of stdin.split("\n")) { + if (line.substring(0, 1) == "#") continue; + const items = line.split('\t'); + if (items.length < 7) { + console.error("Skipped invalid line: " + line); + continue; + } + const wallet = items[0]; + const timestamp = parseInt(items[1], 16); + const raw_diff = parseInt(items[2]); + const count = parseInt(items[3]); + const coin = items[4]; + const xmr_diff = parseInt(items[5]); + const xmr_diff_payed = items[6] == "" ? xmr_diff : parseInt(items[6]); + pplns_window += xmr_diff; + if (!oldest_timestamp || timestamp < oldest_timestamp) oldest_timestamp = timestamp; + if (newest_timestamp < timestamp) newest_timestamp = timestamp; + if (wallet === my_wallet) { + my_share_count += count; + my_xmr_diff += xmr_diff; + my_xmr_diff_payed += xmr_diff_payed; + if (!(coin in my_coin_raw_diff)) my_coin_raw_diff[coin] = 0; + my_coin_raw_diff[coin] += raw_diff; + if (!(coin in my_coin_xmr_diff)) my_coin_xmr_diff[coin] = 0; + my_coin_xmr_diff[coin] += xmr_diff; + } + } + + console.log("PPLNS window size: \t" + ((newest_timestamp - oldest_timestamp)/1000/60/60).toFixed(2) + " hours"); + console.log("PPLNS window size: \t" + pplns_window + " xmr hashes"); + console.log("Pool XMR normalized hashrate: \t" + human_hashrate(pplns_window / (newest_timestamp - oldest_timestamp) * 1000)); + console.log(""); + console.log("Your submitted shares: \t" + my_share_count); + console.log("Your payment: \t" + ((my_xmr_diff_payed / pplns_window) * 100).toFixed(6) + "% (" + my_xmr_diff_payed + " xmr hashes)"); + console.log("Your XMR normalized hashrate: \t" + human_hashrate(my_xmr_diff_payed / (newest_timestamp - oldest_timestamp) * 1000)); + console.log(""); + console.log("You mined these coins:"); + for (let coin of Object.keys(my_coin_raw_diff).sort()) { + console.log("\t" + coin + ": " + my_coin_raw_diff[coin] + " raw coin hashes (" + ((my_coin_xmr_diff[coin] / my_xmr_diff) * 100).toFixed(6) + "% of XMR normalized hashrate)"); + } + + process.exit(0); +}); \ No newline at end of file diff --git a/block_share_dumps/calc_mo_cvs_top.js b/block_share_dumps/calc_mo_cvs_top.js new file mode 100644 index 000000000..2cc85aca0 --- /dev/null +++ b/block_share_dumps/calc_mo_cvs_top.js @@ -0,0 +1,75 @@ +"use strict"; + +if (Boolean(process.stdin.isTTY) || process.argv.length !== 2) { + console.log("Usage: unxz -c .cvs.xz | node calc_mo_cvs_top.js"); + console.log(" wget -O - https://block-share-dumps.moneroocean.stream/.cvs.xz | unxz -c | node calc_mo_cvs_top.js"); + process.exit(1); +} + +let stdin = ""; + +process.stdin.on('data', function(data) { + stdin += data.toString(); +}); + +function human_hashrate(hashes) { + const power = Math.pow(10, 2 || 0); + if (hashes > 1000000000000) return String(Math.round((hashes / 1000000000000) * power) / power) + " TH/s"; + if (hashes > 1000000000) return String(Math.round((hashes / 1000000000) * power) / power) + " GH/s"; + if (hashes > 1000000) return String(Math.round((hashes / 1000000) * power) / power) + " MH/s"; + if (hashes > 1000) return String(Math.round((hashes / 1000) * power) / power) + " KH/s"; + return Math.floor( hashes || 0 ) + " H/s" +}; + +process.stdin.on('end', function() { + let pplns_window = 0; + let oldest_timestamp = 0; + let newest_timestamp = 0; + + let wallets = {}; + + let my_share_count = 0; + let my_xmr_diff = 0; + let my_xmr_diff_payed = 0; + let my_coin_raw_diff = {}; + let my_coin_xmr_diff = {}; + + for (let line of stdin.split("\n")) { + if (line.substring(0, 1) == "#") continue; + const items = line.split('\t'); + if (items.length < 7) { + console.error("Skipped invalid line: " + line); + continue; + } + const wallet = items[0]; + const timestamp = parseInt(items[1], 16); + const raw_diff = parseInt(items[2]); + const count = parseInt(items[3]); + const coin = items[4]; + const xmr_diff = parseInt(items[5]); + const xmr_diff_payed = items[6] == "" ? xmr_diff : parseInt(items[6]); + pplns_window += xmr_diff; + if (!oldest_timestamp || timestamp < oldest_timestamp) oldest_timestamp = timestamp; + if (newest_timestamp < timestamp) newest_timestamp = timestamp; + if (!(wallet in wallets)) wallets[wallet] = { + share_count: 0, + xmr_diff: 0, + xmr_diff_payed: 0, + coin_raw_diff: {}, + coin_xmr_diff: {}, + }; + wallets[wallet].share_count += count; + wallets[wallet].xmr_diff += xmr_diff; + wallets[wallet].xmr_diff_payed += xmr_diff_payed; + if (!(coin in wallets[wallet].coin_raw_diff)) wallets[wallet].coin_raw_diff[coin] = 0; + wallets[wallet].coin_raw_diff[coin] += raw_diff; + if (!(coin in wallets[wallet].coin_xmr_diff)) wallets[wallet].coin_xmr_diff[coin] = 0; + wallets[wallet].coin_xmr_diff[coin] += xmr_diff; + } + + for (let wallet of Object.keys(wallets).sort((a, b) => (wallets[a].xmr_diff < wallets[b].xmr_diff) ? 1 : -1)) { + console.log(wallet + ": " + wallets[wallet].xmr_diff); + } + + process.exit(0); +}); \ No newline at end of file diff --git a/config_example.json b/config_example.json index cf3ea8a5b..7c070264f 100644 --- a/config_example.json +++ b/config_example.json @@ -1,8 +1,10 @@ { "pool_id": 0, + "eth_pool_support": 1, "bind_ip": "127.0.0.1", "hostname": "testpool.com", "db_storage_path": "CHANGEME", + "verify_shares_host": null, "coin": "xmr", "mysql": { "connectionLimit": 20, diff --git a/deployment/caddyfile b/deployment/caddyfile index 9352fcfe5..5f075a99e 100644 --- a/deployment/caddyfile +++ b/deployment/caddyfile @@ -14,5 +14,4 @@ Access-Control-Allow-Headers "Content-Type, x-access-token" } gzip - mime .manifest text/cache-manifest } diff --git a/deployment/deploy.bash b/deployment/deploy.bash index c26427bc0..9bfa07da0 100644 --- a/deployment/deploy.bash +++ b/deployment/deploy.bash @@ -10,40 +10,29 @@ ROOT_SQL_PASS=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) CURUSER=$(whoami) sudo timedatectl set-timezone Etc/UTC sudo apt-get update -sudo DEBIAN_FRONTEND=noninteractive apt-get -y upgrade +DEBIAN_FRONTEND=noninteractive sudo --preserve-env=DEBIAN_FRONTEND apt-get -y upgrade sudo debconf-set-selections <<< "mysql-server mysql-server/root_password password $ROOT_SQL_PASS" sudo debconf-set-selections <<< "mysql-server mysql-server/root_password_again password $ROOT_SQL_PASS" echo -e "[client]\nuser=root\npassword=$ROOT_SQL_PASS" | sudo tee /root/.my.cnf -sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev mysql-server lmdb-utils libzmq3-dev -cd ~ -git clone https://github.com/MoneroOcean/nodejs-pool.git # Change this depending on how the deployment goes. -cd /usr/src/gtest -sudo cmake . -sudo make -sudo mv libg* /usr/lib/ +DEBIAN_FRONTEND=noninteractive sudo --preserve-env=DEBIAN_FRONTEND apt-get -y install libcap2-bin git python python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev mysql-server lmdb-utils libzmq3-dev libsodium-dev cd ~ +git clone https://github.com/MoneroOcean/nodejs-pool.git sudo systemctl enable ntp cd /usr/local/src -sudo git clone --recursive https://github.com/monero-project/monero.git +sudo git clone https://github.com/monero-project/monero.git cd monero -sudo git checkout v0.12.2.0 -curl https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v -sudo git submodule init -sudo git submodule update -sudo make -j$(nproc) +sudo git checkout v0.17.2.3 +sudo git submodule update --init +USE_SINGLE_BUILDDIR=1 sudo --preserve-env=USE_SINGLE_BUILDDIR make -j$(nproc) release || USE_SINGLE_BUILDDIR=1 sudo --preserve-env=USE_SINGLE_BUILDDIR make release || exit 0 sudo cp ~/nodejs-pool/deployment/monero.service /lib/systemd/system/ sudo useradd -m monerodaemon -d /home/monerodaemon -BLOCKCHAIN_DOWNLOAD_DIR=$(sudo -u monerodaemon mktemp -d) -sudo -u monerodaemon wget --limit-rate=50m -O $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw https://downloads.getmonero.org/blockchain.raw -sudo -u monerodaemon /usr/local/src/monero/build/release/bin/monero-blockchain-import --input-file $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw --batch-size 20000 --database lmdb#fastest --data-dir /home/monerodaemon/.bitmonero -sudo -u monerodaemon rm -rf $BLOCKCHAIN_DOWNLOAD_DIR sudo systemctl daemon-reload sudo systemctl enable monero sudo systemctl start monero curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash source ~/.nvm/nvm.sh -nvm install v8.11.3 -nvm alias default v8.11.3 +nvm install v14.17.3 +nvm alias default v14.17.3 cd ~/nodejs-pool npm install npm install -g pm2 @@ -51,16 +40,18 @@ openssl req -subj "/C=IT/ST=Pool/L=Daemon/O=Mining Pool/CN=mining.pool" -newkey mkdir ~/pool_db/ sed -r "s/(\"db_storage_path\": ).*/\1\"\/home\/$CURUSER\/pool_db\/\",/" config_example.json > config.json cd ~ -git clone https://github.com/mesh0000/poolui.git -cd poolui -npm install -./node_modules/bower/bin/bower update -./node_modules/gulp/bin/gulp.js build +git clone https://github.com/MoneroOcean/moneroocean-gui.git +cd moneroocean-gui +DEBIAN_FRONTEND=noninteractive sudo --preserve-env=DEBIAN_FRONTEND sudo apt install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils +apt install -y libx11-xcb1 libxcomposite-dev libxcursor-dev libxcursor-dev libxi-dev libxtst-dev libcups2-dev libxss-dev libxrandr-dev libatk1.0-0 libatk-bridge2.0-0 +npm install -g uglifycss uglify-js html-minifier +npm install -D critical@latest +./build.sh cd build sudo ln -s `pwd` /var/www CADDY_DOWNLOAD_DIR=$(mktemp -d) cd $CADDY_DOWNLOAD_DIR -curl -sL "https://snipanet.com/caddy.tar.gz" | tar -xz caddy init/linux-systemd/caddy.service +curl -sL "https://github.com/caddyserver/caddy/releases/download/v0.11.5/caddy_v0.11.5_linux_amd64.tar.gz" | tar -xz caddy init/linux-systemd/caddy.service sudo mv caddy /usr/local/bin sudo chown root:root /usr/local/bin/caddy sudo chmod 755 /usr/local/bin/caddy @@ -85,7 +76,7 @@ rm -rf $CADDY_DOWNLOAD_DIR cd ~ sudo env PATH=$PATH:`pwd`/.nvm/versions/node/v8.11.3/bin `pwd`/.nvm/versions/node/v8.11.3/lib/node_modules/pm2/bin/pm2 startup systemd -u $CURUSER --hp `pwd` cd ~/nodejs-pool -sudo chown -R $CURUSER. ~/.pm2 +sudo chown -R $CURUSER ~/.pm2 echo "Installing pm2-logrotate in the background!" pm2 install pm2-logrotate & mysql -u root --password=$ROOT_SQL_PASS < deployment/base.sql diff --git a/deployment/deploy_test.bash b/deployment/deploy_test.bash index 9d797b0fc..98e6aaab5 100644 --- a/deployment/deploy_test.bash +++ b/deployment/deploy_test.bash @@ -14,21 +14,15 @@ sudo DEBIAN_FRONTEND=noninteractive apt-get -y upgrade sudo debconf-set-selections <<< "mysql-server mysql-server/root_password password $ROOT_SQL_PASS" sudo debconf-set-selections <<< "mysql-server mysql-server/root_password_again password $ROOT_SQL_PASS" echo -e "[client]\nuser=root\npassword=$ROOT_SQL_PASS" | sudo tee /root/.my.cnf -sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev libgtest-dev mysql-server lmdb-utils libzmq3-dev -cd ~ -git clone https://github.com/MoneroOcean/nodejs-pool.git # Change this depending on how the deployment goes. -cd /usr/src/gtest -sudo cmake . -sudo make -sudo mv libg* /usr/lib/ +sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev libgtest-dev mysql-server lmdb-utils libzmq3-dev libssl-dev pkg-config cd ~ +git clone https://github.com/MoneroOcean/nodejs-pool.git sudo systemctl enable ntp cd /usr/local/src -sudo git clone https://github.com/monero-project/monero.git +sudo git clone --recursive https://github.com/monero-project/monero.git cd monero -sudo git checkout release-v0.12 -curl https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v -sudo make -j$(nproc) +sudo git checkout v0.17.2.3 +sudo USE_SINGLE_BUILDDIR=1 make -j$(nproc) release || sudo USE_SINGLE_BUILDDIR=1 make release || exit 0 sudo cp ~/nodejs-pool/deployment/monero_test.service /lib/systemd/system/monero.service sudo useradd -m monerodaemon -d /home/monerodaemon sudo systemctl daemon-reload @@ -36,8 +30,8 @@ sudo systemctl enable monero sudo systemctl start monero curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash source ~/.nvm/nvm.sh -nvm install v8.11.3 -nvm alias default v8.11.3 +nvm install v14.17.3 +nvm alias default v14.17.3 cd ~/nodejs-pool npm install npm install -g pm2 @@ -79,7 +73,7 @@ rm -rf $CADDY_DOWNLOAD_DIR cd ~ sudo env PATH=$PATH:`pwd`/.nvm/versions/node/v8.11.3/bin `pwd`/.nvm/versions/node/v8.11.3/lib/node_modules/pm2/bin/pm2 startup systemd -u $CURUSER --hp `pwd` cd ~/nodejs-pool -sudo chown -R $CURUSER. ~/.pm2 +sudo chown -R $CURUSER ~/.pm2 echo "Installing pm2-logrotate in the background!" pm2 install pm2-logrotate & mysql -u root --password=$ROOT_SQL_PASS < deployment/base.sql diff --git a/deployment/install_lmdb_tools.sh b/deployment/install_lmdb_tools.sh index 80b95c518..f072ffed8 100644 --- a/deployment/install_lmdb_tools.sh +++ b/deployment/install_lmdb_tools.sh @@ -1,9 +1,10 @@ #!/bin/bash cd ~ -git clone https://github.com/LMDB/lmdb -cd lmdb -git checkout 4d2154397afd90ca519bfa102b2aad515159bd50 -cd libraries/liblmdb/ +rm -rf node-lmdb +git clone https://github.com/Venemo/node-lmdb.git +cd node-lmdb +git checkout 5941c1e553de4ae1d57a67d355b7c2dd87feaea6 +cd dependencies/lmdb/libraries/liblmdb make -j `nproc` mkdir ~/.bin echo ' ' >> ~/.bashrc diff --git a/deployment/leaf.bash b/deployment/leaf.bash index be2402739..792cf895f 100644 --- a/deployment/leaf.bash +++ b/deployment/leaf.bash @@ -10,43 +10,32 @@ CURUSER=$(whoami) sudo timedatectl set-timezone Etc/UTC sudo apt-get update sudo DEBIAN_FRONTEND=noninteractive apt-get -y upgrade -sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev libzmq3-dev -cd ~ -git clone https://github.com/MoneroOcean/nodejs-pool.git # Change this depending on how the deployment goes. -cd /usr/src/gtest -sudo cmake . -sudo make -sudo mv libg* /usr/lib/ +sudo DEBIAN_FRONTEND=noninteractive apt-get -y install ntp build-essential cmake pkg-config libboost-all-dev libssl-dev libzmq3-dev libunbound-dev libsodium-dev libunwind8-dev liblzma-dev libreadline6-dev libldns-dev libexpat1-dev doxygen graphviz libpgm-dev cd ~ +git clone https://github.com/MoneroOcean/nodejs-pool.git sudo systemctl enable ntp cd /usr/local/src -sudo git clone --recursive https://github.com/monero-project/monero.git +sudo git clone https://github.com/monero-project/monero.git cd monero -sudo git checkout v0.12.2.0 -curl https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v -sudo git submodule init -sudo git submodule update -sudo make -j$(nproc) +sudo git checkout v0.17.2.3 +sudo git submodule update --init +sudo USE_SINGLE_BUILDDIR=1 make -j$(nproc) release || sudo USE_SINGLE_BUILDDIR=1 make release || exit 0 sudo cp ~/nodejs-pool/deployment/monero.service /lib/systemd/system/ sudo useradd -m monerodaemon -d /home/monerodaemon -BLOCKCHAIN_DOWNLOAD_DIR=$(sudo -u monerodaemon mktemp -d) -sudo -u monerodaemon wget --limit-rate=50m -O $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw https://downloads.getmonero.org/blockchain.raw -sudo -u monerodaemon /usr/local/src/monero/build/release/bin/monero-blockchain-import --input-file $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw --batch-size 20000 --database lmdb#fastest --data-dir /home/monerodaemon/.bitmonero -sudo -u monerodaemon rm -rf $BLOCKCHAIN_DOWNLOAD_DIR sudo systemctl daemon-reload sudo systemctl enable monero sudo systemctl start monero curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash source ~/.nvm/nvm.sh -nvm install v8.11.3 -nvm alias default v8.11.3 +nvm install v14.17.3 +nvm alias default v14.17.3 cd ~/nodejs-pool npm install npm install -g pm2 openssl req -subj "/C=IT/ST=Pool/L=Daemon/O=Mining Pool/CN=mining.pool" -newkey rsa:2048 -nodes -keyout cert.key -x509 -out cert.pem -days 36500 cd ~ sudo env PATH=$PATH:`pwd`/.nvm/versions/node/v8.11.3/bin `pwd`/.nvm/versions/node/v8.11.3/lib/node_modules/pm2/bin/pm2 startup systemd -u $CURUSER --hp `pwd` -sudo chown -R $CURUSER. ~/.pm2 +sudo chown -R $CURUSER ~/.pm2 echo "Installing pm2-logrotate in the background!" pm2 install pm2-logrotate echo "You're setup with a leaf node! Congrats" diff --git a/deployment/monero.service b/deployment/monero.service index a1ad70d6e..5778a19ee 100644 --- a/deployment/monero.service +++ b/deployment/monero.service @@ -5,7 +5,7 @@ After=network.target [Service] Type=forking GuessMainPID=no -ExecStart=/usr/local/src/monero/build/release/bin/monerod --rpc-bind-ip 127.0.0.1 --detach --restricted-rpc +ExecStart=/usr/local/src/monero/build/release/bin/monerod --rpc-bind-ip 127.0.0.1 --detach --restricted-rpc --prune-blockchain --block-notify '/bin/bash /home/user/nodejs-pool/block_notify.sh' Restart=always User=monerodaemon diff --git a/deployment/monero_daemon.patch b/deployment/monero_daemon.patch deleted file mode 100644 index e98c4b85f..000000000 --- a/deployment/monero_daemon.patch +++ /dev/null @@ -1,13 +0,0 @@ -diff --git a/src/cryptonote_core/tx_pool.cpp b/src/cryptonote_core/tx_pool.cpp -index 5dfbc1d..1e0487a 100644 ---- a/src/cryptonote_core/tx_pool.cpp -+++ b/src/cryptonote_core/tx_pool.cpp -@@ -1093,7 +1093,7 @@ namespace cryptonote - LockedTXN lock(m_blockchain); - - auto sorted_it = m_txs_by_fee_and_receive_time.begin(); -- while (sorted_it != m_txs_by_fee_and_receive_time.end()) -+ while (sorted_it != m_txs_by_fee_and_receive_time.end() && bl.tx_hashes.size() <= 120) - { - txpool_tx_meta_t meta; - if (!m_blockchain.get_txpool_tx_meta(sorted_it->second, meta)) diff --git a/deployment/monero_test.service b/deployment/monero_test.service index d6a20b1e4..e2f508f80 100644 --- a/deployment/monero_test.service +++ b/deployment/monero_test.service @@ -5,7 +5,7 @@ After=network.target [Service] Type=forking GuessMainPID=no -ExecStart=/usr/local/src/monero/build/release/bin/monerod --rpc-bind-ip 127.0.0.1 --detach --restricted-rpc --testnet +ExecStart=/usr/local/src/monero/build/release/bin/monerod --rpc-bind-ip 127.0.0.1 --detach --restricted-rpc --testnet --prune-blockchain Restart=always User=monerodaemon diff --git a/deployment/upgrade_monero.bash b/deployment/upgrade_monero.bash index 73e60f408..5068725c0 100755 --- a/deployment/upgrade_monero.bash +++ b/deployment/upgrade_monero.bash @@ -6,10 +6,9 @@ cd /usr/local/src/monero &&\ sudo git checkout . &&\ sudo git checkout master &&\ sudo git pull &&\ -sudo git checkout v0.12.2.0 &&\ -curl -L https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v &&\ +sudo git checkout v0.17.2.3 &&\ sudo git submodule init &&\ sudo git submodule update &&\ sudo rm -rf build &&\ -sudo nice make &&\ -echo "Done building the new Monero daemon! Please go ahead and reboot monero with: sudo systemctl restart monero as soon as the pool source is updated!" +sudo USE_SINGLE_BUILDDIR=1 nice make release &&\ +echo "Done building the new Monero daemon! Please go ahead and reboot monero with: sudo systemctl restart monero as soon as the pool source is updated!" diff --git a/ex_keys.example.json b/ex_keys.example.json new file mode 100644 index 000000000..12e71b284 --- /dev/null +++ b/ex_keys.example.json @@ -0,0 +1,10 @@ +{ + "CRYPTOPIA": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "CRYPTOPIA_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=", + "TRADEOGRE": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "TRADEOGRE_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "QRYPTOS": "NNNNNN", + "QRYPTOS_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX==", + "LIVECOIN": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "LIVECOIN_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" +} \ No newline at end of file diff --git a/init.js b/init.js index d4dc05d10..b1ec5a06b 100644 --- a/init.js +++ b/init.js @@ -81,7 +81,10 @@ global.mysql.query("SELECT * FROM config").then(function (rows) { require('./lib/blockManager.js'); break; case 'altblockManager': - require('./lib/altblockManager.js'); + require('./lib2/altblockManager.js'); + break; + case 'altblockExchange': + require('./lib2/altblockExchange.js'); break; case 'payments': require('./lib/payments.js'); @@ -95,6 +98,9 @@ global.mysql.query("SELECT * FROM config").then(function (rows) { case 'worker': require('./lib/worker.js'); break; + case 'pool_stats': + require('./lib/pool_stats.js'); + break; case 'longRunner': require('./lib/longRunner.js'); break; diff --git a/lib/api.js b/lib/api.js index d209aba5b..b4e3c5d53 100644 --- a/lib/api.js +++ b/lib/api.js @@ -14,7 +14,7 @@ const jwt = require('jsonwebtoken'); // used to create, sign, and verify tokens const crypto = require('crypto'); const cors = require('cors'); -let addressBase58Prefix = cnUtil.address_decode(new Buffer(global.config.pool.address)); +let addressBase58Prefix = cnUtil.address_decode(Buffer.from(global.config.pool.address)); let threadName = ""; if (cluster.isMaster) { @@ -28,7 +28,11 @@ if (global.config.pplns.enable === true) pool_list.push('pplns'); if (global.config.pps.enable === true) pool_list.push('pps'); if (global.config.solo.enable === true) pool_list.push('solo'); -app.use(cors({origin: true})); +//var whitelist = ['https://moneroocean.stream', 'https://test.moneroocean.stream']; +//app.use(cors({ +// origin: function (origin, callback) { return callback(null, whitelist.indexOf(origin) !== -1 || !origin); } +//})); +app.use(cors()); app.use(bodyParser.urlencoded({extended: false})); app.use(bodyParser.json()); @@ -54,11 +58,14 @@ function getAllWorkerStats(address, callback){ let globalStatsCache = global.database.getCache("stats:" + address); let returnData = { global: { - lts: globalStatsCache !== false ? Math.floor(globalStatsCache.lastHash / 1000) : false, - identifer: 'global', - hash: globalStatsCache !== false ? globalStatsCache.hash : false, - hash2: globalStatsCache !== false ? globalStatsCache.hash2 : false, - totalHash: globalCache !== false ? globalCache.totalHashes : false + lts: globalStatsCache !== false ? Math.floor(globalStatsCache.lastHash / 1000) : false, + identifer: 'global', + hash: globalStatsCache !== false ? globalStatsCache.hash : false, + hash2: globalStatsCache !== false ? globalStatsCache.hash2 : false, + totalHash: globalCache !== false ? globalCache.totalHashes : false, + validShares: globalCache !== false ? Number(globalCache.goodShares) : false, + invalidShares: globalCache !== false ? (globalCache.badShares ? Number(globalCache.badShares) : 0) : false + } }; if (identifiers === false || identifiers.length == 0) return callback(null, returnData); @@ -68,11 +75,13 @@ function getAllWorkerStats(address, callback){ let cachedData = global.database.getCache(id2); let cachedStatsData = global.database.getCache("stats:" + id2); returnData[identifier] = { - lts: cachedStatsData !== false ? Math.floor(cachedStatsData.lastHash / 1000) : false, - identifer: identifier, - hash: cachedStatsData !== false ? cachedStatsData.hash : false, - hash2: cachedStatsData !== false ? cachedStatsData.hash2 : false, - totalHash: cachedData !== false ? cachedData.totalHashes : false + lts: cachedStatsData !== false ? Math.floor(cachedStatsData.lastHash / 1000) : false, + identifer: identifier, + hash: cachedStatsData !== false ? cachedStatsData.hash : false, + hash2: cachedStatsData !== false ? cachedStatsData.hash2 : false, + totalHash: cachedData !== false ? cachedData.totalHashes : false, + validShares: cachedData !== false ? Number(cachedData.goodShares) : false, + invalidShares: cachedData !== false ? (cachedData.badShares ? Number(cachedData.badShares) : 0) : false }; if (++ intCounter === identifiers.length) return callback(null, returnData); }); @@ -96,13 +105,13 @@ function getAddressStats(address, extCallback){ function (callback) { debug(threadName + "Checking Influx for last 10min avg for /miner/address/stats"); return callback(null, { - hash: cachedStatsData.hash, - hash2: cachedStatsData.hash2, - identifier: 'global', - lastHash: Math.floor(cachedStatsData.lastHash / 1000), - totalHashes: cachedData.totalHashes, - validShares: Number(cachedData.goodShares), - invalidShares: Number(cachedData.badShares) + hash: cachedStatsData.hash, + hash2: cachedStatsData.hash2, + identifier: 'global', + lastHash: Math.floor(cachedStatsData.lastHash / 1000), + totalHashes: cachedData.totalHashes, + validShares: Number(cachedData.goodShares), + invalidShares: cachedData.badShares ? Number(cachedData.badShares) : 0 }); }, function (returnData, callback) { @@ -185,7 +194,7 @@ app.get('/config', cache('5 minutes'), function (req, res) { // Pool APIs app.get('/pool/address_type/:address', cache('10 seconds'), function (req, res) { let address = req.params.address; - if (addressBase58Prefix === cnUtil.address_decode(new Buffer(address))) { + if (addressBase58Prefix === cnUtil.address_decode(Buffer.from(address))) { res.json({valid: true, address_type: global.config.general.coinCode}); } else if (btcValidator.validate(this.address) && global.config.general.allowBitcoin) { res.json({valid: true, address_type: 'BTC'}); @@ -194,7 +203,12 @@ app.get('/pool/address_type/:address', cache('10 seconds'), function (req, res) } }); -app.get('/pool/stats', cache('10 seconds'), function (req, res) { +app.get('/pool/motd', cors(), cache('60 seconds'), function (req, res) { + const news = global.database.getCache('news'); + res.json({created: news.created, subject: news.subject, body: news.body}); +}); + +app.get('/pool/stats', cors(), cache('10 seconds'), function (req, res) { let localCache = global.database.getCache('pool_stats_global'); delete(localCache.minerHistory); delete(localCache.hashHistory); @@ -279,31 +293,31 @@ app.get('/pool/ports', cache('10 seconds'), function (req, res) { app.get('/pool/blocks/:pool_type', cache('10 seconds'), function (req, res) { let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getBlockList(req.params.pool_type).slice(page*limit, (page + 1) * limit)); + res.json(global.database.getBlockList(req.params.pool_type, page*limit, (page + 1) * limit)); }); app.get('/pool/altblocks/:pool_type', cache('10 seconds'), function (req, res) { let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getAltBlockList(req.params.pool_type).slice(page*limit, (page + 1) * limit)); + res.json(global.database.getAltBlockList(req.params.pool_type, null, page*limit, (page + 1) * limit)); }); app.get('/pool/blocks', cache('10 seconds'), function (req, res) { let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getBlockList().slice(page*limit, (page + 1) * limit)); + res.json(global.database.getBlockList(null, page*limit, (page + 1) * limit)); }); app.get('/pool/altblocks', cache('10 seconds'), function (req, res) { let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getAltBlockList().slice(page*limit, (page + 1) * limit)); + res.json(global.database.getAltBlockList(null, null, page*limit, (page + 1) * limit)); }); app.get('/pool/coin_altblocks/:coin_port', cache('10 seconds'), function (req, res) { let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getAltBlockList(null, parseInt(req.params.coin_port)).slice(page*limit, (page + 1) * limit)); + res.json(global.database.getAltBlockList(null, parseInt(req.params.coin_port), page*limit, (page + 1) * limit)); }); app.get('/pool/payments/:pool_type', cache('1 minute'), function (req, res) { @@ -455,6 +469,49 @@ app.get('/miner/:address/payments', cache('1 minute'), function (req, res) { }); }); +app.get('/miner/:address/block_payments', cache('1 minute'), function (req, res) { + const limit = typeof(req.query.limit) !== 'undefined' ? (Number(req.query.limit) > 100 ? 100 : Number(req.query.limit)) : 10; + const page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; + + const address_parts = req.params.address.split('.'); + const address = address_parts[0]; + const payment_id = address_parts[1]; + const where_str = typeof(payment_id) === 'undefined' + ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" + : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; + + global.mysql.query("SELECT * FROM paid_blocks WHERE paid_time > (NOW() - INTERVAL 7 DAY) ORDER BY id DESC LIMIT ? OFFSET ?", [limit, page * limit]).then(function (rows) { + if (rows.length === 0) return res.json([]); + let block_hexes = []; + rows.forEach(function (row) { block_hexes.push('"' + row.hex + '"'); }); + + global.mysql.query("SELECT hex, amount FROM block_balance WHERE " + where_str + " AND hex IN (" + block_hexes.join() + ")").then(function (rows2) { + let block_miner_shares = {}; + rows2.forEach(function (row2) { block_miner_shares[row2.hex] = row2.amount; }); + let response = []; + rows.forEach(function (row) { + const miner_payment_share = row.hex in block_miner_shares ? block_miner_shares[row.hex] : 0; + response.push({ + id: row.id, + ts: (new Date(row.paid_time)).getTime() / 1000, + ts_found: (new Date(row.found_time)).getTime() / 1000, + port: row.port, + hash: row.hex, + value_percent: miner_payment_share * 100.0, + value: miner_payment_share * row.amount / global.config.general.sigDivisor + }); + }); + return res.json(response.sort(global.support.tsCompare)); + }).catch(function (err) { + console.error(threadName + "Error getting block_balance miner block payments: " + JSON.stringify(err)); + return res.json({error: 'Issue getting block payments'}); + }); + }).catch(function (err) { + console.error(threadName + "Error getting paid_blocks miner block payments: " + JSON.stringify(err)); + return res.json({error: 'Issue getting block payments'}); + }); +}); + app.get('/miner/:address/stats/allWorkers', cache('10 seconds'), function (req, res) { getAllWorkerStats(req.params.address, function(err, data){ return res.json(data); @@ -473,13 +530,13 @@ app.get('/miner/:address/stats/:identifier', cache('10 seconds'), function (req, let cachedData = global.database.getCache(memcKey); let cachedStatsData = global.database.getCache("stats:" + memcKey); return res.json({ - lts: Math.floor(cachedStatsData.lastHash / 1000), - identifer: identifier, - hash: cachedStatsData.hash, - hash2: cachedStatsData.hash2, - totalHash: cachedData.totalHashes, - validShares: Number(cachedData.goodShares), - invalidShares: Number(cachedData.badShares) + lts: Math.floor(cachedStatsData.lastHash / 1000), + identifer: identifier, + hash: cachedStatsData.hash, + hash2: cachedStatsData.hash2, + totalHash: cachedData.totalHashes, + validShares: Number(cachedData.goodShares), + invalidShares: cachedData.badShares ? Number(cachedData.badShares) : 0 }); }); @@ -503,6 +560,85 @@ app.get('/miner/:address/stats', cache('1 minute'), function (req, res) { }); }); +app.get('/user/:address', function (req, res) { + global.mysql.query("SELECT payout_threshold, enable_email FROM users WHERE username = ? LIMIT 1", [req.params.address]).then(function(row){ + if (row.length == 1) { + return res.json({payout_threshold: row[0].payout_threshold, email_enabled: row[0].enable_email}); + } else { + return res.json({payout_threshold: global.support.decimalToCoin(global.config.payout.defaultPay), email_enabled: 0}); + } + }); +}); + +app.post('/user/subscribeEmail', function (req, res) { + const username = req.body.username; + if (!username) return res.status(401).send({'success': false, 'msg': "No \"username\" parameter was found"}); + if (!("enabled" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"enabled\" parameter was found"}); + if (!("from" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"from\" parameter was found"}); + if (!("to" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"to\" parameter was found"}); + const enabled = req.body.enabled; + const from = req.body.from; + const to = req.body.to; + if (from === "" && to === "") { + global.mysql.query("UPDATE users SET enable_email = ? WHERE username = ?", [enabled, username]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'This XMR address does not have email subscription'}); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }); + } else if (from === "") { + global.mysql.query("UPDATE users SET enable_email = ?, email = ? WHERE username = ? AND (email IS NULL OR email = '')", [enabled, to, username]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + if (global.database.getCache(username) === false) return res.status(401).send({'success': false, 'msg': "Can't set email for unknown user"}); + global.mysql.query("INSERT INTO users (username, enable_email, email) VALUES (?, ?, ?)", [username, enabled, to]).then(function () { + return res.json({'msg': 'Email preferences were updated'}); + }).catch(function(err) { + return res.status(401).json({'error': 'Please specify valid FROM email'}); + }); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }); + } else { + global.mysql.query("UPDATE users SET enable_email = ?, email = ? WHERE username = ? AND email = ?", [enabled, to, username, from]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'FROM email does not match'}); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }); + } +}); + +app.get('/user/:address/unsubscribeEmail', function (req, res) { + global.mysql.query("UPDATE users SET enable_email = 0 WHERE username = ?", [req.params.address]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'This XMR address does not have email subscription'}); + } else { + return res.json({'msg': 'Your email was unsubscribed from further notifications'}); + } + }); +}); + +app.post('/user/updateThreshold', function (req, res) { + let threshold = req.body.threshold; + if (!threshold) return res.status(401).send({'success': false, 'msg': "Can't set threshold to a wrong value"}); + if (threshold > 1000) threshold = 1000; + const username = req.body.username; + if (!username || global.database.getCache(username) === false) return res.status(401).send({'success': false, 'msg': "Can't set threshold for unknown user"}); + const threshold2 = global.support.decimalToCoin(threshold < global.config.payout.walletMin ? global.config.payout.walletMin : threshold); + global.mysql.query("SELECT * FROM users WHERE username = ? AND payout_threshold_lock = '1'", [username]).then(function (rows) { + if (rows.length === 0) { + global.mysql.query("INSERT INTO users (username, payout_threshold) VALUES (?, ?) ON DUPLICATE KEY UPDATE payout_threshold=?", [username, threshold2, threshold2]).then(function () { + return res.json({'msg': 'Threshold updated, set to: ' + global.support.coinToDecimal(threshold2)}); + }); + } else { + return res.status(401).send({'success': false, 'msg':"Can't update locked payment threshold"}); + } + }); +}); + // Authentication app.post('/authenticate', function (req, res) { let hmac; @@ -599,277 +735,8 @@ secureRoutes.post('/changePayoutThreshold', function (req, res) { }); }); -// Administrative routes/APIs - -/*adminRoutes.use(function (req, res, next) { - let token = req.body.token || req.query.token || req.headers['x-access-token']; - if (token) { - jwt.verify(token, global.config.api.secKey, function (err, decoded) { - if (decoded.admin !== 1) { - return res.status(403).send({ - success: false, - msg: 'You are not an admin.' - }); - } - if (err) { - return res.json({success: false, msg: 'Failed to authenticate token.'}); - } else { - req.decoded = decoded; - next(); - } - }); - - } else { - return res.status(403).send({ - success: false, - msg: 'No token provided.' - }); - } -}); - -adminRoutes.get('/stats', function (req, res) { - // Admin interface stats. - // For each pool type + global, we need the following: - // Total Owed, Total Paid, Total Mined, Total Blocks, Average Luck - let intCache = { - 'pplns': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'pps': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'solo': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'global': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'fees': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0} - }; - async.series([ - function (callback) { - global.mysql.query("select * from balance").then(function (rows) { - rows.forEach(function (row) { - intCache[row.pool_type].owed += row.amount; - intCache.global.owed += row.amount; - }); - }).then(function () { - return callback(null); - }); - }, - function (callback) { - global.mysql.query("select * from payments").then(function (rows) { - rows.forEach(function (row) { - intCache[row.pool_type].paid += row.amount; - intCache.global.paid += row.amount; - }); - }).then(function () { - return callback(null); - }); - }, - function (callback) { - global.database.getBlockList().forEach(function (block) { - intCache[block.pool_type].mined += block.value; - intCache.global.mined += block.value; - intCache[block.pool_type].shares += block.shares; - intCache.global.shares += block.shares; - intCache[block.pool_type].targetShares += block.diff; - intCache.global.targetShares += block.diff; - }); - return callback(null); - } - ], function () { - return res.json(intCache); - }); -}); - -adminRoutes.get('/wallet', function (req, res) { - // Stats for the admin interface. - // Load the wallet state from cache, NOTHING HAS DIRECT ACCESS. - // walletStateInfo - return res.json(global.database.getCache('walletStateInfo')); -}); - -adminRoutes.get('/wallet/history', function (req, res) { - // walletHistory - if (req.decoded.admin === 1) { - return res.json(global.database.getCache('walletHistory')); - } -}); - -adminRoutes.get('/ports', function (req, res) { - let retVal = []; - global.mysql.query("SELECT * FROM port_config").then(function (rows) { - rows.forEach(function (row) { - retVal.push({ - port: row.poolPort, - diff: row.difficulty, - desc: row.portDesc, - portType: row.portType, - hidden: row.hidden === 1, - ssl: row.ssl === 1 - }); - }); - }).then(function () { - return res.json(retVal); - }); -}); - -adminRoutes.post('/ports', function (req, res) { - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [req.body.port]).then(function (rows) { - if (rows.length !== 0) { - return "Port already exists with that port number."; - } - if (req.body.diff > global.config.pool.maxDifficulty || req.body.diff < global.config.pool.minDifficulty) { - return "Invalid difficulty."; - } - if (["pplns", "solo", "pps"].indexOf(req.body.portType) === -1) { - return "Invalid port type"; - } - global.mysql.query("INSERT INTO port_config (poolPort, difficulty, portDesc, portType, hidden, ssl) VALUES (?, ?, ?, ?, ?, ?)", - [req.body.port, req.body.diff, req.body.desc, req.body.portType, req.body.hidden === 1, req.body.ssl === 1]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Added port to database"}); - }); -}); - -adminRoutes.put('/ports', function (req, res) { - let portNumber = Number(req.body.portNum); - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [portNumber]).then(function (rows) { - if (rows.length === 0) { - return "Port doesn't exist in the database"; - } - if (req.body.diff > global.config.pool.maxDifficulty || req.body.diff < global.config.pool.minDifficulty) { - return "Invalid difficulty."; - } - if (["pplns", "solo", "pps"].indexOf(req.body.portType) === -1) { - return "Invalid port type"; - } - global.mysql.query("UPDATE port_config SET difficulty=?, portDesc=?, portType=?, hidden=?, ssl=? WHERE poolPort = ?", - [req.body.diff, req.body.desc, req.body.portType, req.body.hidden === 1, req.body.ssl === 1, portNumber]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Updated port in database"}); - }); -}); - -adminRoutes.delete('/ports', function (req, res) { - let portNumber = Number(req.body.portNum); - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [portNumber]).then(function (rows) { - if (rows.length === 0) { - return "Port doesn't exist in the database"; - } - global.mysql.query("DELETE FROM port_config WHERE poolPort = ?", [portNumber]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Added port to database"}); - }); -}); - -adminRoutes.get('/config', function (req, res) { - let retVal = []; - global.mysql.query("SELECT * FROM config").then(function (rows) { - rows.forEach(function (row) { - retVal.push({ - id: row.id, - module: row.module, - item: row.item, - value: row.item_value, - type: row.item_type, - desc: row.item_desc - }); - }); - }).then(function () { - return res.json(retVal); - }); -}); - -adminRoutes.put('/config', function (req, res) { - let configID = Number(req.body.id); - global.mysql.query("SELECT * FROM config WHERE id = ?", [configID]).then(function (rows) { - if (rows.length === 0) { - return "Config item doesn't exist in the database"; - } - global.mysql.query("UPDATE config SET item_value=? WHERE id = ?", [req.body.value, configID]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Updated port in database"}); - }); -}); - -adminRoutes.get('/userList', function (req, res) { - // List of all the users in the system. - // Might as well do it all, right? :3 - // Data Format to be documented. - let intCache = {}; - global.mysql.query("select sum(balance.amount) as amt_due, sum(payments.amount) as amt_paid," + - "balance.payment_address as address, balance.payment_id as payment_id from balance LEFT JOIN payments on " + - "payments.payment_address=balance.payment_address or payments.payment_id=balance.payment_id " + - "group by address, payment_id").then(function (rows) { - rows.forEach(function (row) { - let key = row.address; - if (row.payment_id !== null) { - key += '.' + row.payment_id; - } - intCache[key] = { - paid: row.amt_paid, - due: row.amt_due, - address: key, - workers: [], - lastHash: 0, - totalHashes: 0, - hashRate: 0, - goodShares: 0, - badShares: 0 - }; - }); - }).then(function () { - let minerList = global.database.getCache('minerList'); - if (minerList) { - minerList.forEach(function (miner) { - let minerData = miner.split('_'); - let minerCache = global.database.getCache(miner); - if (!minerCache.hasOwnProperty('goodShares')) { - minerCache.goodShares = 0; - minerCache.badShares = 0; - } - if (!intCache.hasOwnProperty(minerData[0])) { - intCache[minerData[0]] = {paid: 0, due: 0, address: minerData[0], workers: []}; - } - if (typeof(minerData[1]) !== 'undefined') { - intCache[minerData[0]].workers.push({ - worker: minerData[1], - hashRate: minerCache.hash, - lastHash: minerCache.lastHash, - totalHashes: minerCache.totalHashes, - goodShares: minerCache.goodShares, - badShares: minerCache.badShares - }); - } else { - intCache[minerData[0]].lastHash = minerCache.lastHash; - intCache[minerData[0]].totalHashes = minerCache.totalHashes; - intCache[minerData[0]].hashRate = minerCache.hash; - intCache[minerData[0]].goodShares = minerCache.goodShares; - intCache[minerData[0]].badShares = minerCache.badShares; - } - }); - let retList = []; - for (let minerId in intCache) { - if (intCache.hasOwnProperty(minerId)) { - let miner = intCache[minerId]; - retList.push(miner); - } - } - return res.json(retList); - } - return res.json([]); - }); -});*/ - // apply the routes to our application with the prefix /api app.use('/authed', secureRoutes); -//app.use('/admin', adminRoutes); // Authenticated routes diff --git a/lib/blockManager.js b/lib/blockManager.js index e6c36ab81..9586d23a2 100644 --- a/lib/blockManager.js +++ b/lib/blockManager.js @@ -2,53 +2,36 @@ const range = require("range"); const debug = require("debug")("blockManager"); const async = require("async"); +const fs = require('fs'); +const child_process = require('child_process'); // This file is for managing the block databases within the SQL database. // Primary Tasks: -// Sync the chain into the block_log database. - Scan on startup for missing data, starting from block 0 // Maintain a check for valid blocks in the system. (Only last number of blocks required for validation of payouts) - Perform every 2 minutes. Scan on the main blocks table as well for sanity sake. // Maintain the block_log database in order to ensure payments happen smoothly. - Scan every 1 second for a change in lastblockheader, if it changes, insert into the DB. -let blockIDCache = []; let paymentInProgress = false; -let scanInProgress = false; -let blockHexCache = {}; -let lastBlock = 0; let balanceIDCache = {}; -let blockScannerTask; -let blockQueue = async.queue(function (task, callback) { - global.coinFuncs.getBlockHeaderByID(task.blockID, (err, body) => { - if (err !== null) { - console.error("Can't get block with " + task.blockID + " height"); - return; - } - if (body.hash in blockHexCache) { - return callback(); + +let createBlockBalanceQueue = async.queue(function (task, callback) { + const sqlq = "REPLACE INTO block_balance (hex, payment_address, payment_id, amount) VALUES ?"; + let sqlp = []; + task.hexes.forEach(function(block_hex) { + sqlp.push([block_hex, task.payment_address, task.payment_id, task.amount]); + }); + global.mysql.query(sqlq, [sqlp]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows < task.hexes.length) { + console.error(JSON.stringify(result)); + console.error("Can't do SQL block balance replace: " + sqlq + " with " + JSON.stringify(sqlp)); + return callback(false); } - debug("Adding block to block_log, ID: " + task.blockID); - blockIDCache.push(task.blockID); - blockHexCache[body.hash] = null; - global.mysql.query("INSERT INTO block_log (id, orphan, hex, find_time, reward, difficulty, major_version, minor_version) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [task.blockID, body.orphan_status, body.hash, global.support.formatDate(body.timestamp * 1000), body.reward, body.difficulty, body.major_version, body.minor_version]).then(function () { - return calculatePPSPayments(body, callback); - }).catch(function (err) { - debug("BlockHexCache Check: " + body.hash in blockHexCache); - debug("BlockIDCache Check: " + blockIDCache.hasOwnProperty(task.blockID)); - debug("Hex: " + body.hash + " Height:" + task.blockID); - console.error("Tried to reprocess a block that'd already been processed"); - console.error(JSON.stringify(err)); - return callback(); - }); + return callback(true); + }).catch(function (err) { + console.error(err); + console.error("Can't do SQL block balance replace: " + sqlq + " with " + JSON.stringify(sqlp)); + return callback(false); }); -}, 16); - -blockQueue.drain = function () { - debug("blockQueue drained: unlocking remainder of blockManager functionality"); - scanInProgress = false; - if (typeof(blockScannerTask) === 'undefined'){ - blockScannerTask = setInterval(blockScanner, 1000); - } -}; +}, 1); let createBalanceQueue = async.queue(function (task, callback) { let pool_type = task.pool_type; @@ -77,53 +60,50 @@ let createBalanceQueue = async.queue(function (task, callback) { }, 1); let balanceQueue = async.queue(function (task, callback) { - let pool_type = task.pool_type; - let payment_address = task.payment_address; - let payment_id = null; - if (typeof(task.payment_id) !== 'undefined' && task.payment_id !== null && task.payment_id.length > 10){ - payment_id = task.payment_id; - } - task.payment_id = payment_id; - let bitcoin = task.bitcoin; - let amount = task.amount; - debug("Processing balance increment task: " + JSON.stringify(task)); - async.waterfall([ - function (intCallback) { - let cacheKey = payment_address + pool_type + bitcoin + payment_id; - if (cacheKey in balanceIDCache) { - return intCallback(null, balanceIDCache[cacheKey]); - } else { + const pool_type = task.pool_type; + const bitcoin = task.bitcoin; + const amount = task.amount; + const payment_address = task.payment_address; + let payment_id = null; + if (typeof(task.payment_id) !== 'undefined' && task.payment_id !== null && task.payment_id.length > 10) payment_id = task.payment_id; + task.payment_id = payment_id; + debug("Processing balance increment task: " + JSON.stringify(task)); + async.waterfall([ + function (intCallback) { + let cacheKey = payment_address + pool_type + bitcoin + payment_id; + if (cacheKey in balanceIDCache) { + return intCallback(null, balanceIDCache[cacheKey]); + } else { + createBalanceQueue.push(task, function () {}); + async.until(function (untilCB) { + return untilCB(null, cacheKey in balanceIDCache); + }, function (intCallback) { createBalanceQueue.push(task, function () { + return intCallback(null, balanceIDCache[cacheKey]); }); - async.until(function () { - return cacheKey in balanceIDCache; - }, function (intCallback) { - createBalanceQueue.push(task, function () { - return intCallback(null, balanceIDCache[cacheKey]); - }); - }, function () { - return intCallback(null, balanceIDCache[cacheKey]); - } - ); + }, function () { + return intCallback(null, balanceIDCache[cacheKey]); } - }, - function (balance_id, intCallback) { - debug("Made it to the point that I can update the balance for: " + balance_id + " for the amount: " + amount); - global.mysql.query("UPDATE balance SET amount = amount+? WHERE id = ?", [amount, balance_id]).then(function (result) { - if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { - console.error("Can't do SQL balance update: UPDATE balance SET amount = amount+" + amount + " WHERE id = " + balance_id + ";") - } - return intCallback(null); - }); - } - ], - function () { - return callback(); + ); } - ) - ; - }, 24 -); + }, + function (balance_id, intCallback) { + debug("Made it to the point that I can update the balance for: " + balance_id + " for the amount: " + amount); + global.mysql.query("UPDATE balance SET amount = amount+? WHERE id = ?", [amount, balance_id]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do SQL balance update: UPDATE balance SET amount = amount+" + amount + " WHERE id = " + balance_id + ";") + } + return intCallback(null); + }).catch(function (err) { + console.error(err); + console.error("Can't do SQL balance update: UPDATE balance SET amount = amount+" + amount + " WHERE id = " + balance_id + ";") + }); + } + ], + function () { + return callback(); + }); +}, 24); let is_full_stop = false; @@ -138,9 +118,9 @@ function full_stop(err) { } let block_unlock_callback = null; -let prev_balance_sum = null; +let prev_balance_sum = null; -balanceQueue.drain = function () { +balanceQueue.drain(function () { if (!paymentInProgress) { debug("balanceQueue.drain: paymentInProgress is false"); return; @@ -149,10 +129,6 @@ balanceQueue.drain = function () { debug("balanceQueue.drain: block_unlock_callback is not defined"); return; } - if (prev_balance_sum === null) { - debug("balanceQueue.drain: prev_balance_sum is not defined"); - return; - } console.log("balanceQueue drained: performing block unlocking"); global.mysql.query("SELECT SUM(amount) as amt FROM balance").then(function (rows) { if (typeof(rows[0]) === 'undefined' || typeof(rows[0].amt) === 'undefined') { @@ -164,7 +140,7 @@ balanceQueue.drain = function () { } let balance_sum = rows[0].amt; if (balance_sum !== prev_balance_sum) { - console.log("Total balance changed from " + global.support.coinToDecimal(prev_balance_sum) + " to " + global.support.coinToDecimal(balance_sum)); + console.log("Total balance changed from " + global.support.coinToDecimal(prev_balance_sum) + " to " + global.support.coinToDecimal(balance_sum) + "\n"); block_unlock_callback(); } else { full_stop("Total balance not changed from " + prev_balance_sum + " to " + balance_sum); @@ -173,125 +149,39 @@ balanceQueue.drain = function () { prev_balance_sum = null; paymentInProgress = false; }); -}; +}); -function calculatePPSPayments(blockHeader, callback) { - if (global.config.pps.enable === false) return callback(); - console.log("Performing PPS payout on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward)); - let paymentData = {}; - paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', - payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - let totalPayments = 0; - if (global.config.pps.enable === true) { - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - for (let found = (cursor.goToRange(blockHeader.height) === blockHeader.height); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, data) { // jshint ignore:line - let shareData; - try { - shareData = global.protos.Share.decode(data); - } catch (e) { - console.error(e); - return; - } - let blockDiff = blockHeader.difficulty; - let rewardTotal = blockHeader.reward; - if (shareData.poolType === global.protos.POOLTYPE.PPS) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } - if (!(userIdentifier in paymentData)) { - paymentData[userIdentifier] = { - pool_type: 'pps', - payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 - }; - } - let amountToPay = Math.floor((shareData.shares / blockDiff) * rewardTotal); - let feesToPay = Math.floor(amountToPay * (global.config.payout.ppsFee / 100)); - if (shareData.bitcoin === true) { - feesToPay += Math.floor(amountToPay * (global.config.payout.btcFee / 100)); - } - amountToPay -= feesToPay; - paymentData[userIdentifier].amount = paymentData[userIdentifier].amount + amountToPay; - let donations = 0; - if(global.config.payout.devDonation > 0){ - let devDonation = (feesToPay * (global.config.payout.devDonation / 100)); - donations += devDonation; - paymentData[global.coinFuncs.coinDevAddress].amount = paymentData[global.coinFuncs.coinDevAddress].amount + devDonation ; - } - if(global.config.payout.poolDevDonation > 0){ - let poolDevDonation = (feesToPay * (global.config.payout.poolDevDonation / 100)); - donations += poolDevDonation; - paymentData[global.coinFuncs.poolDevAddress].amount = paymentData[global.coinFuncs.poolDevAddress].amount + poolDevDonation; - } - paymentData[global.config.payout.feeAddress].amount = paymentData[global.config.payout.feeAddress].amount + feesToPay - donations; - } - }); - } - cursor.close(); - txn.abort(); - } - Object.keys(paymentData).forEach(function (key) { - balanceQueue.push(paymentData[key], function () { - }); - totalPayments += paymentData[key].amount; - }); - console.log("PPS payout cycle complete on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / blockHeader.reward) * 100 + "%"); - return callback(); -} +function preCalculatePPLNSPayments(block_hexes, block_height, block_difficulty, is_store_dump, done_callback) { + const rewardTotal = 1.0; + console.log("Performing PPLNS reward pre-calculations of block " + block_hexes.join(', ') + " on (anchor) height " + block_height); + const blockDiff = block_difficulty; + const windowPPLNS = blockDiff * global.config.pplns.shareMulti; -function calculatePPLNSPayments(block_height, block_reward, block_difficulty, unlock_callback) { - console.log("Performing PPLNS payout on block: " + block_height + " Block Value: " + global.support.coinToDecimal(block_reward)); - let rewardTotal = block_reward; - let blockDiff = block_difficulty; - let windowPPLNS = blockDiff * global.config.pplns.shareMulti; let blockCheckHeight = block_height; - let totalPaid = 0; - let totalShares = 0; - let paymentData = {}; + let totalPaid = 0; + let totalShares = 0; + let paymentData = {}; + paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + bitcoin: 0, + amount: 0 }; paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + bitcoin: 0, + amount: 0 }; paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + bitcoin: 0, + amount: 0 }; function addPayment(keyAdd, valueAdd) { @@ -299,28 +189,22 @@ function calculatePPLNSPayments(block_height, block_reward, block_difficulty, un if (totalPaid >= rewardTotal) return; totalShares += valueAdd; paymentData[keyAdd].amount += valueAdd; - let totalPaid2 = totalShares / windowPPLNS * rewardTotal; - if (totalPaid2 + 1 < rewardTotal) { // totalPaid can not overflow rewardTotal now + const totalPaid2 = totalShares / windowPPLNS * rewardTotal; + if (totalPaid2 > rewardTotal) { // totalPaid can not overflow rewardTotal now + //console.log("Value totalPaid " + totalPaid + " reached max " + rewardTotal); + const extra = (totalPaid2 - rewardTotal) / rewardTotal * windowPPLNS; + //console.log("Rewarded " + (valueAdd - extra) + " instead of " + valueAdd + " hashes for " + keyAdd); + paymentData[keyAdd].amount -= extra; + totalPaid = rewardTotal; + } else { totalPaid = totalPaid2; - } else { // we need recalculate totalPaid precisely now - totalPaid = 0; - Object.keys(paymentData).forEach(function (key) { - totalPaid += Math.floor(paymentData[key].amount / windowPPLNS * rewardTotal); - }); - console.log("Aproximate totalPaid " + totalPaid2 + " was reset to precise value " + totalPaid); - if (totalPaid >= rewardTotal) { - console.log("Precise value totalPaid " + totalPaid + " reached max " + rewardTotal); - let extra = (totalPaid - rewardTotal) / rewardTotal * windowPPLNS; - console.log("Rewarded " + (valueAdd - extra) + " instead of " + valueAdd + " hashes for " + keyAdd); - paymentData[keyAdd].amount -= extra; - totalPaid = rewardTotal; - } } }; let portShares = {}; let firstShareTime; let lastShareTime; + let shares4dump = []; async.doWhilst(function (callback) { let txn = global.database.env.beginTxn({readOnly: true}); @@ -334,37 +218,37 @@ function calculatePPLNSPayments(block_height, block_reward, block_difficulty, un console.error(e); return; } - let blockDiff = block_difficulty; - let rewardTotal = block_reward; if (shareData.poolType === global.protos.POOLTYPE.PPLNS) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } + const userIdentifier = shareData.paymentID ? shareData.paymentAddress + "." + shareData.paymentID : shareData.paymentAddress; if (!(userIdentifier in paymentData)) { paymentData[userIdentifier] = { - pool_type: 'pplns', + pool_type: 'pplns', payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 + payment_id: shareData.paymentID, + bitcoin: shareData.bitcoin, + amount: 0 }; } if (!firstShareTime) firstShareTime = shareData.timestamp; if (totalPaid < rewardTotal) lastShareTime = shareData.timestamp; - let amountToPay = shareData.shares2 ? shareData.shares2 : shareData.shares; - let feesToPay = amountToPay * (global.config.payout.pplnsFee / 100) + + const amountToPay = shareData.shares2; + const feesToPay = amountToPay * (global.config.payout.pplnsFee / 100) + (shareData.bitcoin === true ? amountToPay * (global.config.payout.btcFee / 100) : 0); - let devDonation = feesToPay * (global.config.payout.devDonation / 100); - let poolDevDonation = feesToPay * (global.config.payout.poolDevDonation / 100); + const devDonation = feesToPay * (global.config.payout.devDonation / 100); + const poolDevDonation = feesToPay * (global.config.payout.poolDevDonation / 100); + const amountToPay2 = amountToPay - feesToPay; + + shares4dump.push(userIdentifier.slice(-16) + "\t" + shareData.timestamp.toString(16) + "\t" + shareData.raw_shares + "\t" + shareData.share_num + "\t" + + global.coinFuncs.PORT2COIN_FULL(shareData.port) + "\t" + amountToPay + "\t" + (amountToPay === amountToPay2 ? "" : amountToPay2)); - addPayment(userIdentifier, amountToPay - feesToPay); + addPayment(userIdentifier, amountToPay2); addPayment(global.config.payout.feeAddress, feesToPay - devDonation - poolDevDonation); addPayment(global.coinFuncs.poolDevAddress, poolDevDonation); addPayment(global.coinFuncs.coinDevAddress, devDonation); + if (typeof(shareData.port) !== 'undefined') { if (shareData.port in portShares) { portShares[shareData.port] += amountToPay; @@ -378,346 +262,369 @@ function calculatePPLNSPayments(block_height, block_reward, block_difficulty, un cursor.close(); txn.abort(); setImmediate(callback, null, totalPaid); - }, function (totalPayment) { + }, function (totalPayment, whilstCB) { blockCheckHeight = blockCheckHeight - 1; debug("Decrementing the block chain check height to:" + blockCheckHeight); if (totalPayment >= rewardTotal) { debug("Loop 1: Total Payment: " + totalPayment + " Amount Paid: " + rewardTotal + " Amount Total: " + totalPaid); - return false; + return whilstCB(null, false); } else { debug("Loop 2: Total Payment: " + totalPayment + " Amount Paid: " + rewardTotal + " Amount Total: " + totalPaid); - return blockCheckHeight !== 0; + return whilstCB(null, blockCheckHeight !== 0); } }, function (err) { + let sumAllPorts = 0; for (let port in portShares) sumAllPorts += portShares[port]; let pplns_port_shares = {}; for (let port in portShares) { const port_share = portShares[port] / sumAllPorts; pplns_port_shares[port] = port_share; - console.log("Port " + port + ": " + (100.0 * port_share).toFixed(2) + "%"); + //console.log("Port " + port + ": " + (100.0 * port_share).toFixed(2) + "%"); } global.database.setCache('pplns_port_shares', pplns_port_shares); global.database.setCache('pplns_window_time', (firstShareTime - lastShareTime) / 1000); + let totalPayments = 0; + Object.keys(paymentData).forEach(function (key) { + totalPayments += paymentData[key].amount; + }); - global.mysql.query("SELECT SUM(amount) as amt FROM balance").then(function (rows) { - if (typeof(rows[0]) === 'undefined' || typeof(rows[0].amt) === 'undefined') { - console.error("SELECT SUM(amount) as amt FROM balance query returned undefined result"); - return; - } - prev_balance_sum = rows[0].amt; - block_unlock_callback = unlock_callback; - - Object.keys(paymentData).forEach(function (key) { - paymentData[key].amount = Math.floor((paymentData[key].amount / (blockDiff*global.config.pplns.shareMulti)) * rewardTotal); - balanceQueue.push(paymentData[key], function () {}); - //console.log("[PAYMENT] " + key + ": " + global.support.coinToDecimal(paymentData[key].amount)); - totalPayments += paymentData[key].amount; + let is_dump_done = false; + let is_ok = true; + let is_pay_done = false; + + if (totalPayments == 0) { + console.warn("PPLNS payout cycle for " + block_hexes.join(', ') + " block does not have any shares so will be redone using top height"); + global.support.sendEmail(global.config.general.adminEmail, + "FYI: No shares to pay block, so it was corrected by using the top height", + "PPLNS payout cycle for " + block_hexes.join(', ') + " block does not have any shares so will be redone using top height" + ); + global.coinFuncs.getLastBlockHeader(function(err, body){ + if (err !== null) { + console.error("Last block header request failed!"); + return done_callback(false); + } + const topBlockHeight = body.height; + return preCalculatePPLNSPayments(block_hexes, topBlockHeight, block_difficulty, is_store_dump, done_callback); }); + return; + } else { + if (is_store_dump && fs.existsSync("./block_share_dumps/process.sh")) { + shares4dump.sort(); + shares4dump.unshift("#last_16_chars_of_xmr_address\ttimestamp\traw_share_diff\tshare_count\tshare_coin\txmr_share_diff\txmr_share_diff_paid"); + const fn = "block_share_dumps/" + block_hexes[0] + ".cvs"; + fs.writeFile(fn, shares4dump.join("\n"), function(err) { + if (err) { + console.error("Error saving " + fn + " file"); + is_dump_done = true; + if (is_pay_done) return done_callback(is_ok); + return; + } + let fns = ""; + block_hexes.forEach(function(block_hex) { fns += " block_share_dumps/" + block_hex + ".cvs" }); + child_process.exec("./block_share_dumps/process.sh" + fns, function callback(error, stdout, stderr) { + if (error) console.error("./block_share_dumps/process.sh" + fns + ": returned error exit code: " + error.code + "\n" + stdout + "\n" + stderr); + else console.log("./block_share_dumps/process.sh" + fns + ": complete"); + is_dump_done = true; + if (is_pay_done) return done_callback(is_ok); + }); + }); + } else { + is_dump_done = true; + } + } - console.log("PPLNS payout cycle complete on block: " + block_height + " Block Value: " + global.support.coinToDecimal(block_reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / block_reward) * 100 + "% (precisely " + totalPayments + " / " + block_reward + ")"); - if (totalPayments != block_reward) { - global.support.sendEmail(global.config.general.adminEmail, - "Block was not payed completely!", - "PPLNS payout cycle complete on block: " + block_height + " Block Value: " + global.support.coinToDecimal(block_reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / block_reward) * 100 + "% (precisely " + totalPayments + " / " + block_reward + ")" - ); + const default_window = blockDiff*global.config.pplns.shareMulti; + const is_need_correction = Math.abs(totalPayments/default_window - 1) > 0.0001; + const pay_window = is_need_correction ? totalPayments : default_window; + + let add_count = 0; + + Object.keys(paymentData).forEach(function (key) { + const payment = paymentData[key]; + if (payment.amount) { + const paymentData2 = { + pool_type: 'pplns', + payment_address: payment.payment_address, + payment_id: payment.payment_id, + bitcoin: 0, + amount: payment.amount / pay_window, + hexes: block_hexes, + }; + ++ add_count; + createBlockBalanceQueue.push(paymentData2, function (status) { + if (status === false) is_ok = false; + if (--add_count == 0) { + is_pay_done = true; + if (is_dump_done) return done_callback(is_ok); + } + }); } }); + + console.log("PPLNS pre-payout cycle complete on block: " + block_height + " Payout Percentage: " + (totalPayments / pay_window) * 100 + "% (precisely " + totalPayments + " / " + pay_window + ")"); + if (is_need_correction) { + console.warn("(This PPLNS payout cycle complete on block was corrected: " + block_height + " Payout Percentage: " + (totalPayments / default_window) * 100 + "% (precisely " + totalPayments + " / " + default_window + "))"); + global.support.sendEmail(global.config.general.adminEmail, + "Warning: Not enought shares to pay block correctly, so it was corrected by upscaling miner rewards!", + "PPLNS payout cycle complete on block: " + block_height + " Payout Percentage: " + (totalPayments / pay_window) * 100 + "% (precisely " + totalPayments + " / " + pay_window + ")\n" + + "(This PPLNS payout cycle complete on block was corrected: " + block_height + " Payout Percentage: " + (totalPayments / default_window) * 100 + "% (precisely " + totalPayments + " / " + default_window + "))" + ); + } }); -}; +} -function calculateSoloPayments(blockHeader) { - console.log("Performing Solo payout on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward)); - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - let paymentData = {}; - paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', - payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - let totalPayments = 0; - for (let found = (cursor.goToRange(blockHeader.height) === blockHeader.height); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, data) { // jshint ignore:line - let shareData; - try { - shareData = global.protos.Share.decode(data); - } catch (e) { - console.error(e); - return; - } - let rewardTotal = blockHeader.reward; - if (shareData.poolType === global.protos.POOLTYPE.SOLO && shareData.foundBlock === true) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } - if (!(userIdentifier in paymentData)) { - paymentData[userIdentifier] = { - pool_type: 'solo', - payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 - }; - } - let feesToPay = Math.floor(rewardTotal * (global.config.payout.soloFee / 100)); - if (shareData.bitcoin === true) { - feesToPay += Math.floor(rewardTotal * (global.config.payout.btcFee / 100)); - } - rewardTotal -= feesToPay; - paymentData[userIdentifier].amount = rewardTotal; - let donations = 0; - if(global.config.payout.devDonation > 0){ - let devDonation = (feesToPay * (global.config.payout.devDonation / 100)); - donations += devDonation; - paymentData[global.coinFuncs.coinDevAddress].amount = paymentData[global.coinFuncs.coinDevAddress].amount + devDonation ; - } - if(global.config.payout.poolDevDonation > 0){ - let poolDevDonation = (feesToPay * (global.config.payout.poolDevDonation / 100)); - donations += poolDevDonation; - paymentData[global.coinFuncs.poolDevAddress].amount = paymentData[global.coinFuncs.poolDevAddress].amount + poolDevDonation; - } - paymentData[global.config.payout.feeAddress].amount = feesToPay - donations; +function doPPLNSPayments(block_hex, block_reward, block_port, block_timestamp, unlock_callback) { + console.log("Performing PPLNS payout of block " + block_hex + " with value " + global.support.coinToDecimal(block_reward)); + global.mysql.query("SELECT SUM(amount) as amt FROM balance").then(function (rows) { + if (typeof(rows[0]) === 'undefined' || typeof(rows[0].amt) === 'undefined') { + console.error("SELECT SUM(amount) as amt FROM balance query returned undefined result"); + return; + } + prev_balance_sum = rows[0].amt; + + global.mysql.query("SELECT payment_address, payment_id, amount FROM block_balance WHERE hex = ?", [block_hex]).then(function (rows) { + if (rows.length) { + global.mysql.query("INSERT INTO paid_blocks (hex, amount, port, found_time) VALUES (?,?,?,?)", [block_hex, block_reward, parseInt(block_port), global.support.formatDate(block_timestamp)]).then(function () { + console.log("Adding total due to " + rows.length + " miners"); + block_unlock_callback = unlock_callback; + rows.forEach(function (row) { + row.amount = Math.floor(row.amount * block_reward); + row.pool_type = "pplns"; + row.bitcoin = 0; + balanceQueue.push(row, function () {}); + }); + }).catch(function (error) { + console.error("Block " + block_hex + " can not be inserted into paid_blocks table"); + }); + } else { + console.error("Block " + block_hex + " has no payments in SQL"); } }); - } - cursor.close(); - txn.abort(); - Object.keys(paymentData).forEach(function (key) { - balanceQueue.push(paymentData[key], function () { - }); - totalPayments += paymentData[key].amount; }); - console.log("Solo payout cycle complete on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / blockHeader.reward) * 100 + "%"); } -function blockUnlocker() { +let payReadyBlockHashCalc = {}; + +function blockUnlocker(blockUnlockerCB) { if (is_full_stop) { debug("Dropping all block unlocks"); - return; - } - if (scanInProgress) { - debug("Skipping block unlocker run as there's a scan in progress"); - return; + return blockUnlockerCB(); } if (paymentInProgress) { - debug("Skipping block unlocker run as there's a payment in progress"); - return; + console.error("Skipping block unlocker run as there's a payment in progress"); + return blockUnlockerCB(); } console.log("Running block unlocker"); let blockList = global.database.getValidLockedBlocks(); global.coinFuncs.getLastBlockHeader(function(err, body){ if (err !== null) { console.error("Last block header request failed!"); - return; + return blockUnlockerCB(); } - let topBlockHeight = body.height; - blockList.forEach(function (block) { + const topBlockHeight = body.height; + async.eachSeries(blockList, function(block, next) { global.coinFuncs.getBlockHeaderByID(block.height, (err, body) => { if (err !== null) { console.error("Can't get block with " + block.height + " height"); - return; + return next(); } + if (topBlockHeight - block.height <= 5) return next(); + const is_pplns_block = block.poolType == global.protos.POOLTYPE.PPLNS; if (body.hash !== block.hash) { global.database.invalidateBlock(block.height); - global.mysql.query("UPDATE block_log SET orphan = true WHERE hex = ?", [block.hash]); - blockIDCache.splice(blockIDCache.indexOf(block.height)); console.log("Invalidating block " + block.height + " due to being an orphan block"); + return next(); + } else if (is_pplns_block && !(block.hash in payReadyBlockHashCalc) && block.pay_ready !== true) { + payReadyBlockHashCalc[block.hash] = 1; + preCalculatePPLNSPayments( [ block.hash ], block.height, block.difficulty, true, function(status) { + if (status) { + console.log("Completed PPLNS reward pre-calculations of block " + block.hash + " on height " + block.height); + global.database.payReadyBlock(block.hash); + } + return next(); + }); + } else if (topBlockHeight - block.height > global.config.payout.blocksRequired && (!is_pplns_block || block.pay_ready === true)) { + blockPayments(block, function() { return next(); } ); } else { - if (topBlockHeight - block.height > global.config.payout.blocksRequired) { - blockPayments(block); - } + return next(); } }); - + }, function() { + return blockUnlockerCB(); }); }); } -function altblockUnlocker() { +function altblockUnlocker(altblockUnlockerCB) { if (is_full_stop) { debug("Dropping all altblock unlocks"); - return; - } - if (scanInProgress) { - debug("Skipping altblock unlocker run as there's a scan in progress"); - return; + return altblockUnlockerCB(); } if (paymentInProgress) { - debug("Skipping altblock unlocker run as there's a payment in progress"); - return; + console.error("Skipping altblock unlocker run as there's a payment in progress"); + return altblockUnlockerCB(); } - debug("Running altblock unlocker"); let blockList = global.database.getValidLockedAltBlocks(); - blockList.forEach(function (block) { - global.coinFuncs.getPortBlockHeaderByID(block.port, block.height, (err, body) => { - if (err !== null) { - console.error("Can't get altblock of " + block.port + " port with " + block.height + " height"); - return; - } - if (body.hash !== block.hash) { - global.database.invalidateAltBlock(block.id); - console.log("Invalidating altblock from " + block.port + " port for " + block.height + " due to being an orphan block"); - } else { + console.log("Running altblock unlocker for " + blockList.length + " blocks"); + let blockHeightWait = {}; + global.coinFuncs.getLastBlockHeader(function(err, body){ + if (err !== null) { + console.error("Last block header request failed!"); + return altblockUnlockerCB(); + } + const topBlockHeight = body.height; + let preCalcAnchorBlockHashes = {}; + async.eachSeries(blockList, function(block, next) { + if (topBlockHeight - block.anchor_height <= 60) return next(); + const is_pplns_block = block.poolType == global.protos.POOLTYPE.PPLNS; + if (is_pplns_block && !(block.hash in payReadyBlockHashCalc) && block.pay_ready !== true) { + if (block.value) { + const anchor_height = block.anchor_height - (block.anchor_height % global.config.payout.anchorRound); + if (!(anchor_height in preCalcAnchorBlockHashes)) preCalcAnchorBlockHashes[anchor_height] = []; + preCalcAnchorBlockHashes[anchor_height].push(block.hash); + } else global.support.sendEmail(global.config.general.adminEmail, "FYI: blockManager saw zero value locked block", + "Hello,\r\nThe blockManager saw zero value locked block " + block.hash.toString('hex') + ); + return next(); + } else if (!is_pplns_block || block.pay_ready === true) { if (block.pay_value !== 0) { - altblockPayments(block); + console.log(block.port + ": " + block.hash); + global.coinFuncs.getPortBlockHeaderByHash(block.port, block.hash, (err, body) => { + if ( ( body.topoheight && body.topoheight === -1) || + body.confirmations === -1 || + ( body.error instanceof Object && body.error.message === "The requested hash could not be found." ) + ) { + global.database.invalidateAltBlock(block.id); + console.log("Invalidating altblock from " + block.port + " port for " + block.height + " due to being an orphan block"); + return next(); + } else if (err !== null && block.port != 8545) { + console.error("Can't get altblock of " + block.port + " port with " + block.height + " height"); + global.coinFuncs.getPortBlockHeaderByID(block.port, block.height, (err, body) => { + if (err === null && body.hash !== block.hash) { + global.database.invalidateAltBlock(block.id); + console.log("Invalidating altblock from " + block.port + " port for " + block.height + " due to being an orphan block"); + } + return next(); + }); + } else { + altblockPayments(block, function() { return next(); } ); + } + }); } else { - console.log("Waiting for altblock with " + block.port + " port and " + block.height + " height pay value"); + if (!(block.port in blockHeightWait)) blockHeightWait[block.port] = []; + blockHeightWait[block.port].push(block.height); + return next(); } + } else { + return next(); } + + }, function() { + console.log("Running altblock pre-payment for " + Object.keys(preCalcAnchorBlockHashes).length + " anchor heights"); + let maxPreCount = 10; + async.eachSeries(Object.keys(preCalcAnchorBlockHashes), function(anchor_height, next) { + if (--maxPreCount < 0) return next(); + global.coinFuncs.getBlockHeaderByID(anchor_height, function (anchor_err, anchor_header) { + if (anchor_err === null){ + const block_hexes = preCalcAnchorBlockHashes[anchor_height]; + block_hexes.forEach(function (block_hex) { + payReadyBlockHashCalc[block_hex] = 1; + }); + preCalculatePPLNSPayments(block_hexes, parseInt(anchor_height), anchor_header.difficulty, true, function(status) { + if (status) { + console.log("Completed PPLNS reward pre-calculations on altblock " + block_hexes.join(", ") + " on anchor height " + anchor_height + "\n"); + block_hexes.forEach(function (block_hex) { + global.database.payReadyAltBlock(block_hex); + }); + } + return next(); + }); + } else { + console.error("Can't get correct anchor block header by height " + anchor_height); + return next(); + } + }); + }, function() { + for (let port in blockHeightWait) { + console.log("Waiting for altblock with " + port + " port and " + blockHeightWait[port].join(", ") + " height(s) pay value"); + } + return altblockUnlockerCB(); + }); }); - }); } -function blockPayments(block) { +function blockPayments(block, cb) { + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } switch (block.poolType) { - case global.protos.POOLTYPE.PPS: - // PPS is paid out per share find per block, so this is handled in the main block-find loop. - global.database.unlockBlock(block.hash); - break; case global.protos.POOLTYPE.PPLNS: global.coinFuncs.getBlockHeaderByHash(block.hash, function (err, header) { if (err === null && block.height === header.height && block.value === header.reward && block.difficulty === header.difficulty){ if (paymentInProgress) { - debug("Skipping payment as there's a payment in progress"); - return; + console.error("Skipping payment as there's a payment in progress"); + return cb(); } paymentInProgress = true; - calculatePPLNSPayments(block.height, block.value, block.difficulty, function() { + doPPLNSPayments(block.hash, block.value, global.config.daemon.port, block.timestamp, function() { console.log("Unlocking main block on " + block.height + " height with " + block.hash.toString('hex') + " hash"); global.database.unlockBlock(block.hash); + return cb(); }); } else { console.error("Can't get correct block header by hash " + block.hash.toString('hex')); + global.support.sendEmail(global.config.general.adminEmail, "blockManager unable to make blockPayments", + "Hello,\r\nThe blockManager has hit an issue making blockPayments with block " + block.hash.toString('hex')); + return cb(); } }); break; - case global.protos.POOLTYPE.SOLO: - global.coinFuncs.getBlockHeaderByHash(block.hash, function (err, header) { - if (err === null){ - calculateSoloPayments(header); - global.database.unlockBlock(block.hash); - } - }); - break; + default: - console.log("Unknown payment type. FREAKOUT"); - global.database.unlockBlock(block.hash); - break; + console.error("Unknown payment type. FREAKOUT"); + return cb(); } } -function altblockPayments(block) { +function altblockPayments(block, cb) { + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } switch (block.poolType) { case global.protos.POOLTYPE.PPLNS: global.coinFuncs.getPortBlockHeaderByHash(block.port, block.hash, function (err, header) { - if (err === null && block.height === header.height && block.value === header.reward && block.difficulty === header.difficulty){ - global.coinFuncs.getBlockHeaderByID(block.anchor_height, function (anchor_err, anchor_header) { - if (anchor_err === null){ - if (paymentInProgress) { - debug("Skipping payment as there's a payment in progress"); - return; - } - paymentInProgress = true; - calculatePPLNSPayments(block.anchor_height, block.pay_value, anchor_header.difficulty, function() { - console.log("Unlocking " + block.port + " port block on " + block.height + " height with " + block.hash.toString('hex') + " hash"); - global.database.unlockAltBlock(block.hash); - }); - } else { - console.error("Can't get correct block header by height " + block.anchor_height.toString()); - } + if (err === null && block.height === header.height && block.value >= header.reward){ + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } + paymentInProgress = true; + doPPLNSPayments(block.hash, block.pay_value, block.port, block.timestamp, function() { + console.log("Unlocking " + block.port + " port block on " + block.height + " height with " + block.hash.toString('hex') + " hash"); + global.database.unlockAltBlock(block.hash); + return cb(); }); } else { console.error("Can't get correct altblock header of " + block.port.toString() + " port by hash " + block.hash.toString('hex')); + return cb(); } }); break; default: - console.log("Unknown payment type. FREAKOUT"); - global.database.unlockAltBlock(block.hash); - break; + console.error("Unknown payment type. FREAKOUT"); + return cb(); } } -function blockScanner() { - let inc_check = 0; - if (scanInProgress) { - debug("Skipping scan as there's one in progress."); - return; - } - scanInProgress = true; - global.coinFuncs.getLastBlockHeader(function (err, blockHeader) { - if (err === null){ - if (lastBlock === blockHeader.height) { - //debug("No new work to be performed, block header matches last block"); - scanInProgress = false; - return; - } - debug("Parsing data for new blocks"); - lastBlock = blockHeader.height; - range.range(0, (blockHeader.height - Math.floor(global.config.payout.blocksRequired/2))).forEach(function (blockID) { - if (!blockIDCache.hasOwnProperty(blockID)) { - ++ inc_check; - blockQueue.push({blockID: blockID}, function (err) { - debug("Completed block scan on " + blockID); - if (err) { - console.error("Error processing " + blockID); - } - }); - } - }); - if (inc_check === 0) { - debug("No new work to be performed, initial scan complete"); - scanInProgress = false; - blockScannerTask = setInterval(blockScanner, 1000); - } - } else { - console.error(`Upstream error from the block daemon. Resetting scanner due to: ${JSON.stringify(blockHeader)}`); - scanInProgress = false; - blockScannerTask = setInterval(blockScanner, 1000); - } - }); -} - -function initial_sync() { - console.log("Performing boot-sync"); - global.mysql.query("SELECT id, hex FROM block_log WHERE orphan = 0").then(function (rows) { - let intCount = 0; - rows.forEach(function (row) { - ++ intCount; - blockIDCache.push(row.id); - blockHexCache[row.hex] = null; - }); - }).then(function () { - // Enable block scanning for 1 seconds to update the block log. - blockScanner(); - // Scan every 120 seconds for invalidated blocks - setInterval(blockUnlocker, 2*60*1000); - blockUnlocker(); - setInterval(altblockUnlocker, 2*60*1000); - altblockUnlocker(); - debug("Blocks loaded from SQL: " + blockIDCache.length); - console.log("Boot-sync from SQL complete: pending completion of queued jobs to get back to work."); - }); +function blockUnlockerNext() { + altblockUnlocker(function() { + setTimeout(blockUnlocker, 2*60*1000, blockUnlockerNext); + }); } -initial_sync(); +blockUnlocker(blockUnlockerNext); diff --git a/lib/coins/xmr.js b/lib/coins/xmr.js index f5de82184..48cc187a7 100644 --- a/lib/coins/xmr.js +++ b/lib/coins/xmr.js @@ -5,21 +5,281 @@ const multiHashing = require('cryptonight-hashing'); const crypto = require('crypto'); const debug = require('debug')('coinFuncs'); const process = require('process'); +const fs = require('fs'); +const net = require('net'); +const async = require('async'); +const child_process = require('child_process'); -let hexChars = new RegExp("[0-9a-f]+"); +const reXMRig = /XMRig(?:-[a-zA-Z]+)?\/(\d+)\.(\d+)\./; // 2.8.0 +const reXMRSTAKRX = /\w+-stak-rx\/(\d+)\.(\d+)\.(\d+)/; // 1.0.1 +const reXMRSTAK = /\w+-stak(?:-[a-zA-Z]+)?\/(\d+)\.(\d+)\.(\d+)/; // 2.5.0 +const reXNP = /xmr-node-proxy\/(\d+)\.(\d+)\.(\d+)/; // 0.3.2 +const reCAST = /cast_xmr\/(\d+)\.(\d+)\.(\d+)/; // 1.5.0 +const reSRB = /SRBMiner Cryptonight AMD GPU miner\/(\d+)\.(\d+)\.(\d+)/; // 1.6.8 +const reSRBMULTI = /SRBMiner-MULTI\/(\d+)\.(\d+)\.(\d+)/; // 0.1.5 + +const pool_nonce_size = 16+1; // 1 extra byte for old XMR and new TRTL daemon bugs +const port2coin = { +// "11181": "AEON", + "11898": "TRTL", + "12211": "RYO", + "17750": "XHV", + "18081": "", + "18981": "GRFT", + "11812": "XLA", + "25182": "TUBE", +// "34568": "WOW", + "38081": "MSR", + "48782": "LTHN", + "19734": "SUMO", + "13007": "IRD", + "13102": "XTA", + "19994": "ARQ", + "33124": "XTNC", + "19281": "XMV", + "19950": "XWP", + "9231" : "XEQ", + "20206": "DERO", +// "18181": "XMC", + "16000": "CCX", + "8766" : "RVN", + "8545" : "ETH", + "2086" : "BLOC", + "9053" : "ERG", + "9998" : "RTM", +}; +const port2blob_num = { +// "11181": 7, // AEON + "11898": 2, // TRTL + "12211": 4, // RYO + "17750": 11, // XHV + "18081": 0, // XMR + "18981": 0, // GRFT + "11812": 0, // XLA + "25182": 10, // TUBE +// "34568": 0, // WOW + "38081": 6, // MSR + "48782": 0, // LTHN + "19734": 0, // SUMO + "13007": 2, // IRD + "13102": 12, // XTA + "19994": 0, // ARQ + "19281": 8, // XMV + "33124": 9, // XTNC + "19950": 8, // XWP + "9231" : 5, // XEQ +// "18181": 0, // XMC + "16000": 0, // CCX + "20206": 100, // DERO + "8766" : 101, // RVN + "8545" : 102, // ETH + "2086" : 1, // BLOC + "9053" : 103, // ERG + "9998" : 104, // RTM +}; + +const port2algo = { +// "11181": "k12", // Aeon + "11898": "argon2/chukwav2", // TRTL + "12211": "cn/gpu", // RYO + "13007": "cn-pico/trtl", // IRD + "13102": "c29i", // XTA + "17750": "cn-heavy/xhv", // Haven + "18081": "rx/0", // XMR + "18981": "rx/graft", // Graft + "19281": "c29v", // MoneroV + "19734": "cn/r", // SUMO + "19950": "c29s", // Swap + "19994": "rx/arq", // ArqMa + "11812": "panthera", // Scala + "25182": "c29b", // BitTube + "33124": "c29s", // XtendCash +// "34568": "rx/wow", // Wownero + "38081": "cn/half", // MSR + "48782": "cn/r", // Lethean + "9231" : "cn/gpu", // XEQ + "20206": "astrobwt", // DERO +// "18181": "cn/0", // XMC + "16000": "cn/gpu", // CCX + "8766" : "kawpow", // RVN + "8545" : "ethash", // ETH + "2086" : "cn-heavy/xhv", // BLOC + "9053" : "autolykos2", // ERG + "9998" : "ghostrider", // RTM +}; + +const mm_nonce_size = cnUtil.get_merged_mining_nonce_size(); +const mm_port_set = { }; + +const fix_daemon_sh = "./fix_daemon.sh"; + +const extra_nonce_template_hex = "02" + (pool_nonce_size + 0x100).toString(16).substr(-2) + "00".repeat(pool_nonce_size); +const extra_nonce_mm_template_hex = "02" + (mm_nonce_size + pool_nonce_size + 0x100).toString(16).substr(-2) + "00".repeat(mm_nonce_size + pool_nonce_size); + +function get_coin2port(port2coin) { + let coin2port = {}; + for (let port in port2coin) coin2port[port2coin[port]] = parseInt(port); + return coin2port; +} +const coin2port = get_coin2port(port2coin); +function get_coins(port2coin) { + let coins = []; + for (let port in port2coin) if (port2coin[port] != "") coins.push(port2coin[port]); + return coins; +} +const ports = Object.keys(port2coin); +const coins = get_coins(port2coin); +function get_mm_child_port_set(mm_port_set) { + let mm_child_port_set = {}; + for (let port in mm_port_set) { + const child_port = mm_port_set[port]; + if (!(child_port in mm_child_port_set)) mm_child_port_set[child_port] = {}; + mm_child_port_set[child_port][port] = 1; + } + return mm_child_port_set; +} +function get_algos() { + let algos = {}; + for (let port in port2algo) algos[port2algo[port]] = 1; + return algos; +} +const all_algos = get_algos(); +const mm_child_port_set = get_mm_child_port_set(mm_port_set); + +let shareVerifyQueue = []; +let shareVerifyQueueErrorTime = []; +let shareVerifyQueueErrorCount = []; + +if (global.config.verify_shares_host) global.config.verify_shares_host.forEach(function(verify_shares_host, index) { + shareVerifyQueueErrorTime[index] = 0; + shareVerifyQueueErrorCount[index] = 0; + shareVerifyQueue[index] = async.queue(function (task, queueCB) { + const cb = task.cb; + if (Date.now() - task.time > 1*60*1000) { + cb(null); + return queueCB(); + } + + const jsonInput = task.jsonInput; + + let socket = new net.Socket(); + let is_cb = false; + let return_cb = function(result) { + if (is_cb) return; + is_cb = true; + cb(result); + return queueCB(); + } + let timer = setTimeout(function() { + socket.destroy(); + if (shareVerifyQueueErrorCount[index] > 10) { + const err_str = "Server " + global.config.hostname + " timeouted share verification to " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + }, 60*1000); + socket.connect(2222, verify_shares_host, function () { + socket.write(JSON.stringify(jsonInput) + "\n"); + }); + + let buff = ""; + socket.on('data', function (buff1) { + buff += buff1; + }); + + socket.on("end", function () { + clearTimeout(timer); + timer = null; + try { + const jsonOutput = JSON.parse(buff.toString()); + if (!("result" in jsonOutput)) return return_cb(false); + shareVerifyQueueErrorCount[index] = 0; + return return_cb(jsonOutput.result); + } catch (e) { + if (shareVerifyQueueErrorCount[index] > 10) { + const err_str = "Server " + global.config.hostname + " got wrong JSON from " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + } + }); + + socket.on('error', function() { + socket.destroy(); + if (shareVerifyQueueErrorCount[index] > 10) { + const err_str = "Server " + global.config.hostname + " got socket error from " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + }); + }, 16); + + setInterval(function(queue_obj, index){ + if (queue_obj.length() >= 1000) { + let miner_address = {}; + queue_obj.remove(function(task) { + const d = task.data; + if (!(d.miner_address in miner_address)) miner_address[d.miner_address] = 1; + else ++ miner_address[d.miner_address]; + if (Date.now() - d.time > 1*60*1000) { + d.cb(null); + return true; + } + return false; + }); + console.error(global.database.thread_id + "Share verify queue " + index + " state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); + Object.keys(miner_address).forEach(function(key) { + const value = miner_address[key]; + if (value > 100) console.error("Too many shares from " + key + ": " + value); + }); + } + }, 30*1000, shareVerifyQueue[index], index); +}); + +const ETH_BASE_REWARD = 2; +const ETH_MULTIPLIER = 1000000000000000000; + +function calcEthReward(block, tx_reciepts) { + let gas_prices = {}; + block.transactions.forEach(function(tx) { + gas_prices[tx.hash] = parseInt(tx.gasPrice); + }); + let fee = 0; + tx_reciepts.forEach(function(tx) { + fee += parseInt(tx.gasUsed) * gas_prices[tx.transactionHash]; + }); + fee -= parseInt(block.baseFeePerGas) * parseInt(block.gasUsed); + return (ETH_BASE_REWARD + ETH_BASE_REWARD * (block.uncles.length / 32)) * ETH_MULTIPLIER + fee; +} + +function calcErgReward(height, block_tx) { + let reward = 0; + if (block_tx.length && block_tx[0].outputs.length == 2 && block_tx[0].outputs[1].creationHeight == height) { + reward += block_tx[0].outputs[1].value; + } + if (block_tx.length > 1) { + const last_tx = block_tx[block_tx.length - 1]; + if (last_tx.outputs.length == 1 && last_tx.outputs[0].creationHeight == height) { + reward += last_tx.outputs[0].value; + } + } + return reward; +} -var reXMRig = /XMRig\/(\d+)\.(\d+)\./; -var reXMRSTAK = /xmr-stak(?:-[a-z]+)\/(\d+)\.(\d+)/; -var reXMRSTAK2 = /xmr-stak(?:-[a-z]+)\/(\d+)\.(\d+)\.(\d+)/; -var reXNP = /xmr-node-proxy\/(\d+)\.(\d+)\.(\d+)/; -var reCCMINER = /ccminer-cryptonight\/(\d+)\.(\d+)/; - function Coin(data){ this.bestExchange = global.config.payout.bestExchange; this.data = data; - //let instanceId = crypto.randomBytes(4); - let instanceId = new Buffer(4); - instanceId.writeUInt32LE( ((global.config.pool_id % (1<<16)) << 16) + (process.pid % (1<<16)) ); + let instanceId = Buffer.alloc(4); + instanceId.writeUInt32LE( (((global.config.pool_id % (1<<10)) << 22) + (process.pid % (1<<22))) >>> 0 ); console.log("Generated instanceId: " + instanceId.toString('hex')); this.coinDevAddress = "44AFFq5kSiGBoZ4NMDwYtN18obc8AemS33DBLWs3H7otXft3XjrpDtQGv7SqSsaBYBb98uNbr2VBBEt7f2wfn3RVGQBEP3A"; // Monero Developers Address this.poolDevAddress = "499fS1Phq64hGeqV8p2AfXbf6Ax7gP6FybcMJq6Wbvg8Hw6xms8tCmdYpPsTLSaTNuLEtW4kF2DDiWCFcw4u7wSvFD8wFWE"; // MoneroOcean Address @@ -42,9 +302,11 @@ function Coin(data){ "43c2ykU9i2KZHjV8dWff9HKurYYRkckLueYK96Qh4p1EDoEvdo8mpgNJJpPuods53PM6wNzmj4K2D1V11wvXsy9LMiaYc86", // Changelly.com "45rTtwU6mHqSEMduDm5EvUEmFNx2Z6gQhGBJGqXAPHGyFm9qRfZFDNgDm3drL6wLTVHfVhbfHpCtwKVvDLbQDMH88jx2N6w", // ? "4ALcw9nTAStZSshoWVUJakZ6tLwTDhixhQUQNJkCn4t3fG3MMK19WZM44HnQRvjqmz4LkkA8t565v7iBwQXx2r34HNroSAZ", // Cryptopia.co.nz - "4BCeEPhodgPMbPWFN1dPwhWXdRX8q4mhhdZdA1dtSMLTLCEYvAj9QXjXAfF7CugEbmfBhgkqHbdgK9b2wKA6nqRZQCgvCDm", // ? - "41xeYWWKwtSiHju5AdyF8y5xeptuRY3j5X1XYHuB1g6ke4eRexA1iygjXqrT3anyZ22j7DEE74GkbVcQFyH2nNiC3gJqjM9", // HitBTC - "44rouyxW44oMc1yTGXBUsL6qo9AWWeHETFiimWC3TMQEizSqqZZPnw1UXCaJrCtUC9QT25L5MZvkoGKRxZttvbkmFXA3TMG" // BTC-Alpha + "4BCeEPhodgPMbPWFN1dPwhWXdRX8q4mhhdZdA1dtSMLTLCEYvAj9QXjXAfF7CugEbmfBhgkqHbdgK9b2wKA6nqRZQCgvCDm", // Bitfinex + "41xeYWWKwtSiHju5AdyF8y5xeptuRY3j5X1XYHuB1g6ke4eRexA1iygjXqrT3anyZ22j7DEE74GkbVcQFyH2nNiC3gJqjM9", // HitBTC 1 + "43Kg3mcpvaDhHpv8C4UWf7Kw2DAexn2NoRMqqM5cpAtuRgkedDZWjBQjXqrT3anyZ22j7DEE74GkbVcQFyH2nNiC3dx22mZ", // HitBTC 2 + "44rouyxW44oMc1yTGXBUsL6qo9AWWeHETFiimWC3TMQEizSqqZZPnw1UXCaJrCtUC9QT25L5MZvkoGKRxZttvbkmFXA3TMG", // BTC-Alpha + "45SLfxvu355SpjjzibLKaChA4NGoTrQAwZmSopAXQa9UXBT63BvreEoYyczTcfXow6eL8VaEG2X6NcTG67XZFTNPLgdR9iM", // some web wallet ]; // These are addresses that MUST have a paymentID to perform logins with. this.prefix = 18; @@ -62,70 +324,312 @@ function Coin(data){ this.niceHashDiff = 400000; this.getPortBlockHeaderByID = function(port, blockId, callback){ - global.support.rpcPortDaemon(port, 'getblockheaderbyheight', {"height": blockId}, function (body) { - if (body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/' + blockId, null, function (body) { + if (body) { + return callback(null, body); + } else { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + }); + } else if (port == 8766 || port == 9998) { + global.support.rpcPortDaemon2(port, '', { method: 'getblockhash', params: [ blockId ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + return global.coinFuncs.getPortAnyBlockHeaderByHash(port, body.result, false, callback); + }); + } else if (port == 8545) { + const blockId2 = blockId === "latest" ? blockId : "0x" + blockId.toString(16); + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getBlockByNumber', params: [ blockId2, true ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.result.height = parseInt(body.result.number); + if (blockId === "latest") return callback(null, body.result); // do not need rewards for the latest block + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'parity_getBlockReceipts', params: [ blockId2 ] }, function (body2) { + if (!body2 || !body2.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body2)); + return callback(true, body2); + } + body.result.reward = calcEthReward(body.result, body2.result); + return callback(null, body.result); + }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'blocks/at/' + blockId, null, function (body) { + if (!body || !(body instanceof Array) || body.length != 1) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + return global.coinFuncs.getPortAnyBlockHeaderByHash(port, body[0], false, callback); + }); + } else { + global.support.rpcPortDaemon(port, 'getblockheaderbyheight', {"height": blockId}, function (body) { + if (body && body.hasOwnProperty('result')) { + return callback(null, body.result.block_header); + } else { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + }); + + } }; this.getBlockHeaderByID = function(blockId, callback){ return this.getPortBlockHeaderByID(global.config.daemon.port, blockId, callback); }; - this.getPortBlockHeaderByHash = function(port, blockHash, callback){ - global.support.rpcPortDaemon(port, 'getblockheaderbyhash', {"hash": blockHash}, function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ + this.getPortAnyBlockHeaderByHash = function(port, blockHash, is_our_block, callback){ + // TRTL/IRD does not get getblock LTHN / AEON / DERO have composite tx + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/' + blockHash, null, function (body) { + if (typeof(body) === 'undefined') { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + return callback(null, body); + }); + } else if (port == 8766 || port == 9998) { + global.support.rpcPortDaemon2(port, '', { method: 'getblock', params: [ blockHash ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.result.reward = (port == 8766 ? 5000 : 3750) * 100000000; // TODO: Change to 2500 on (~January 2022) at block 2,100,000 + if (port == 9998) body.result.difficulty *= 0xFFFFFFFF; + return callback(null, body.result); + }); + } else if (port == 8545) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getBlockByHash', params: [ "0x" + blockHash, true ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.result.height = parseInt(body.result.number); + global.coinFuncs.getPortBlockHeaderByID(port, body.result.height, function(err, body_height) { + if (err) return callback(true, body); + if (body.result.hash === body_height.hash) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'parity_getBlockReceipts', params: [ body.result.number ] }, function (body2) { + if (!body2 || !body2.result) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body2)); + return callback(true, body2); + } + body.result.reward = calcEthReward(body.result, body2.result); + return callback(null, body.result); + }); + + // uncle block? + } else async.eachSeries(Array(16).fill().map((element, index) => body.result.height + index - 7), function(block_height, next) { + global.coinFuncs.getPortBlockHeaderByID(port, block_height, function(err, body_height) { + if (err) { + if (is_our_block) return next(false); // need to wait for more blocks before it will be reported as uncle + return next(); + } + const uncleIndex = body_height.uncles.indexOf("0x" + blockHash); + if (uncleIndex === -1) return next(); + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getUncleByBlockNumberAndIndex', params: [ "0x" + block_height.toString(16), "0x" + uncleIndex.toString(16) ] }, function (body_uncle) { + if (!body_uncle || !body_uncle.result) { + console.error("eth_getUncleByBlockNumberAndIndex(0x" + block_height.toString(16) + ", 0x" + uncleIndex.toString(16) + "): " + JSON.stringify(body_uncle)); + return next(); + } + return next((ETH_BASE_REWARD * (8 - (parseInt(body_height.number) - parseInt(body_uncle.result.number))) / 8) * ETH_MULTIPLIER); + }); + }); + }, function(uncleReward) { + if (uncleReward === false) return callback(true, body); + body.result.reward = uncleReward ? uncleReward : null; + return callback(null, body.result); + }); + }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'blocks/' + blockHash, null, function (body) { + if (!body || !body.header) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.header.reward = calcErgReward(body.header.height, body.blockTransactions.transactions); + return callback(null, body.header); + }); + } else if (port == 13007 || port == 2086 || port == 48782 || port == 11181 || port == 20206 || port == 16000) { + global.support.rpcPortDaemon(port, 'getblockheaderbyhash', {"hash": blockHash}, function (body) { + if ( typeof(body) === 'undefined' || !body.hasOwnProperty('result') ) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); + }); + } else global.support.rpcPortDaemon(port, 'getblock', {"hash": blockHash}, function (body) { + if (typeof(body) === 'undefined' || !body.hasOwnProperty('result')) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); return callback(true, body); } - }); + + body.result.block_header.reward = 0; + + let reward_check = 0; + const blockJson = JSON.parse(body.result.json); + const minerTx = blockJson.miner_tx; + + if (port == 17750 || port == 33124 || port == 25182 || port == 13102 || port == 18181) { // XHV / XtendCash / TUBE / Italocoin / XMC has reward as zero transaction + reward_check = minerTx.vout[0].amount; + } else { + for (var i=0; i reward_check) { + reward_check = minerTx.vout[i].amount; + } + } + } + const miner_tx_hash = body.result.miner_tx_hash == "" ? body.result.block_header.miner_tx_hash : body.result.miner_tx_hash; + + if (is_our_block && body.result.hasOwnProperty('miner_tx_hash')) global.support.rpcPortWalletShort(port + 1, "get_transfer_by_txid", {"txid": miner_tx_hash}, function (body2) { + if (typeof(body2) === 'undefined' || body2.hasOwnProperty('error') || !body2.hasOwnProperty('result') || !body2.result.hasOwnProperty('transfer') || !body2.result.transfer.hasOwnProperty('amount')) { + console.error(port + ": block hash: " + blockHash + ": txid " + miner_tx_hash + ": " + JSON.stringify(body2)); + return callback(true, body.result.block_header); + } + let reward = body2.result.transfer.amount; + if (port == 17750) { + body2.result.transfers.forEach(function(transfer) { if (transfer.asset_type === "XHV") reward = transfer.amount; }); + } + + if (reward !== reward_check || reward == 0) { + if (port == 38081 && reward < reward_check /*&& reward != 0*/) { // MSR can have uncle block reward here + } else { + console.error(port + ": block reward does not match wallet reward: " + JSON.stringify(body) + "\n" + JSON.stringify(body2)); + return callback(true, body); + } + } + + body.result.block_header.reward = reward; + return callback(null, body.result.block_header); + + }); else { + body.result.block_header.reward = reward_check; + return callback(null, body.result.block_header); + } + }); + }; + + this.getPortBlockHeaderByHash = function(port, blockHash, callback){ + return this.getPortAnyBlockHeaderByHash(port, blockHash, true, callback); }; this.getBlockHeaderByHash = function(blockHash, callback){ return this.getPortBlockHeaderByHash(global.config.daemon.port, blockHash, callback); }; - this.getPortLastBlockHeader = function(port, callback){ - global.support.rpcPortDaemon(port, 'getlastblockheader', [], function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); + this.getPortLastBlockHeader = function(port, callback, no_error_report) { + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/last', null, function (body) { + if (typeof(body) === 'object') { + return callback(null, body); + } else { + if (!no_error_report) console.error("Last block header invalid: " + JSON.stringify(body)); + return callback(true, body); + } + }); + } else if (port == 8766 || port == 9998) { + global.support.rpcPortDaemon2(port, '', { method: 'getbestblockhash' }, function (body) { + if (!body || !body.result) { + console.error("getPortLastBlockHeader(" + port + "): " + JSON.stringify(body)); + return callback(true, body); + } + if (global.coinFuncs.lastRavenBlockHash === body.result) return callback(null, global.coinFuncs.lastRavenBlock); + global.coinFuncs.getPortAnyBlockHeaderByHash(port, body.result, false, function (err, body2) { + if (err === null) { + global.coinFuncs.lastRavenBlockHash = body.result; + global.coinFuncs.lastRavenBlock = body2; + } + return callback(err, body2); + }); + }); + } else if (port == 8545) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: "eth_getWork", "params": [] }, function(body) { + if (!body || !body.result || !(body.result instanceof Array)) return callback(true, body); + const bt = cnUtil.EthBlockTemplate(body.result); + return callback(null, { hash: bt.hash, timestamp: Date.now() / 1000, difficulty: bt.difficulty, height: bt.height, seed_hash: bt.seed_hash }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'mining/candidate', null, function(body) { + if (!body || !body.pk) return callback(true, body); + const bt = cnUtil.ErgBlockTemplate(body); + return callback(null, { hash: bt.hash, timestamp: Date.now() / 1000, difficulty: bt.difficulty, height: bt.height, hash2: bt.hash2 }); + }); + } else { + global.support.rpcPortDaemon(port, 'getlastblockheader', [], function (body) { + if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ + return callback(null, body.result.block_header); + } else { + if (!no_error_report) console.error("Last block header invalid: " + JSON.stringify(body)); + return callback(true, body); + } + }); + } }; - this.getLastBlockHeader = function(callback){ + this.getLastBlockHeader = function(callback) { return this.getPortLastBlockHeader(global.config.daemon.port, callback); }; - this.getPortBlockTemplate = function(port, callback){ - global.support.rpcPortDaemon(port, 'getblocktemplate', { - reserve_size: 17, - wallet_address: global.config.pool[port == global.config.daemon.port ? "address" : "address_" + port.toString()] - }, function(body){ - return callback(body); - }); + this.getPortBlockTemplate = function(port, callback) { + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/template', { + address: global.config.pool["address_" + port.toString()], + reserveSize: port in mm_port_set ? mm_nonce_size + pool_nonce_size : pool_nonce_size + }, function(body) { + return callback(body ? body : null); + }); + + } else if (port == 8766 || port == 9998) { + global.support.rpcPortDaemon2(port, '', { + method: 'getblocktemplate', + params: [{ + capabilities: [ "coinbasetxn", "workid", "coinbase/append" ], + rules: [ "segwit" ] + }] + }, function(body) { + if (body && body.result) switch (parseInt(port)) { + case 8766: return callback(cnUtil.RavenBlockTemplate(body.result, global.config.pool["address_" + port.toString()])); + case 9998: return callback(cnUtil.RtmBlockTemplate(body.result, global.config.pool["address_" + port.toString()])); + } else return callback(null); + }); + + } else if (port == 8545) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: "eth_getWork", "params": [] }, function(body) { + return callback(body && body.result ? cnUtil.EthBlockTemplate(body.result) : null); + }); + + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'mining/candidate', null, function(body) { + return callback(body && body.pk ? cnUtil.ErgBlockTemplate(body) : null); + }); + + + } else { + global.support.rpcPortDaemon(port, 'getblocktemplate', { + reserve_size: port in mm_port_set ? mm_nonce_size + pool_nonce_size : pool_nonce_size, + wallet_address: global.config.pool[port == global.config.daemon.port ? "address" : "address_" + port.toString()] + }, function(body){ + return callback(body && body.result ? body.result : null); + }); + } }; this.getBlockTemplate = function(callback){ return this.getPortBlockTemplate(global.config.daemon.port, callback); }; - this.baseDiff = function(){ - return bignum('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF', 16); - }; + this.baseDiff = cnUtil.baseDiff; + this.baseRavenDiff = cnUtil.baseRavenDiff; this.validatePlainAddress = function(address){ // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. - address = new Buffer(address); + address = Buffer.from(address); let code = cnUtil.address_decode(address); return code === this.prefix || code === this.subPrefix; }; @@ -133,190 +637,548 @@ function Coin(data){ this.validateAddress = function(address){ if (this.validatePlainAddress(address)) return true; // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. - address = new Buffer(address); + address = Buffer.from(address); return cnUtil.address_decode_integrated(address) === this.intPrefix; }; - this.portBlobType = function(port) { - switch (port) { - case 38081: return 3; // MSR - default: return 0; + this.portBlobType = function(port, version) { return port2blob_num[port]; } + + this.blobTypeGrin = function(blob_type_num) { + switch (blob_type_num) { + case 8: + case 9: + case 10: + case 12: return true; + default: return false; + } + } + + this.c29ProofSize = function(blob_type_num) { + switch (blob_type_num) { + case 10: return 40; + case 12: return 48; + default: return 32; + } + } + + this.nonceSize = function(blob_type_num) { + switch (blob_type_num) { + case 7: + case 101: // RVN + case 102: // ETH + case 103: return 8; // ERG + default: return 4; } } - this.convertBlob = function(blobBuffer, port){ - return cnUtil.convert_blob(blobBuffer, this.portBlobType(port)); + this.blobTypeDero = function(blob_type_num) { return blob_type_num == 100; } + + this.blobTypeRvn = function(blob_type_num) { return blob_type_num == 101; } + + this.blobTypeEth = function(blob_type_num) { return blob_type_num == 102; } + + this.blobTypeErg = function(blob_type_num) { return blob_type_num == 103; } + + this.blobTypeRtm = function(blob_type_num) { return blob_type_num == 104; } + + this.convertBlob = function(blobBuffer, port) { + const blob_type_num = this.portBlobType(port, blobBuffer[0]); + if (this.blobTypeDero(blob_type_num)) return blobBuffer; + let blob; + try { + if (this.blobTypeRvn(blob_type_num)) { + blob = cnUtil.convertRavenBlob(blobBuffer); + } else if (this.blobTypeRtm(blob_type_num)) { + blob = cnUtil.convertRtmBlob(blobBuffer); + } else { + blob = cnUtil.convert_blob(blobBuffer, blob_type_num); + } + } catch (e) { + const err_str = "Can't do port " + port + " convert_blob " + blobBuffer.toString('hex') + " with blob type " + blob_type_num + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't convert_blob", err_str); + return null; + } + return blob; + }; + + this.constructNewBlob = function(blockTemplate, params, port) { + const blob_type_num = this.portBlobType(port, blockTemplate[0]); + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + return cnUtil.construct_block_blob(blockTemplate, + bignum(params.nonce, 10).toBuffer({endian: 'little', size: 4}), + blob_type_num, params.pow + ); + } else if (global.coinFuncs.blobTypeDero(blob_type_num)) { + return cnUtil.constructNewDeroBlob(blockTemplate, Buffer.from(params.nonce, 'hex')); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + return cnUtil.constructNewRavenBlob(blockTemplate, + bignum(params.nonce, 16).toBuffer({endian: 'little', size: 8}), + bignum(params.mixhash, 16).toBuffer({endian: 'little', size: 32}) + ); + } else if (global.coinFuncs.blobTypeRtm(blob_type_num)) { + return cnUtil.constructNewRtmBlob(blockTemplate, Buffer.from(params.nonce, 'hex')); + } else { + return cnUtil.construct_block_blob(blockTemplate, Buffer.from(params.nonce, 'hex'), blob_type_num); + } + }; + + this.constructMMParentBlockBlob = function(parentTemplateBuffer, port, childTemplateBuffer) { + //console.log("MERGED MINING: constructMMParentBlockBlob"); + return cnUtil.construct_mm_parent_block_blob(parentTemplateBuffer, this.portBlobType(port, parentTemplateBuffer[0]), childTemplateBuffer); }; - this.constructNewBlob = function(blockTemplate, NonceBuffer, port){ - return cnUtil.construct_block_blob(blockTemplate, NonceBuffer, this.portBlobType(port)); + this.constructMMChildBlockBlob = function(shareBuffer, port, childTemplateBuffer) { + console.log("MERGED MINING: constructMMChildBlockBlob"); + return cnUtil.construct_mm_child_block_blob(shareBuffer, this.portBlobType(port, shareBuffer[0]), childTemplateBuffer); }; this.getBlockID = function(blockBuffer, port){ - return cnUtil.get_block_id(blockBuffer, this.portBlobType(port)); + const blob_type_num = this.portBlobType(port, blockBuffer[0]); + if (global.coinFuncs.blobTypeRtm(blob_type_num)) { + return cnUtil.blockHashBuff(cnUtil.convertRtmBlob(blockBuffer)); + } else { + return cnUtil.get_block_id(blockBuffer, blob_type_num); + } }; this.BlockTemplate = function(template) { - /* - Generating a block template is a simple thing. Ask for a boatload of information, and go from there. - Important things to consider. - The reserved space is 16 bytes long now in the following format: - Assuming that the extraNonce starts at byte 130: - |130-133|134-137|138-141|142-145| - |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| - This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) - Each with 4 billion clients. (clientNonce) - While being unique to this particular pool thread (instanceId) - With up to 4 billion clients (minerNonce/extraNonce) - Overkill? Sure. But that's what we do here. Overkill. - */ - - // Set this.blob equal to the BT blob that we get from upstream. - this.blob = template.blocktemplate_blob; - this.idHash = crypto.createHash('md5').update(template.blocktemplate_blob).digest('hex'); - // Set this.diff equal to the known diff for this block. - this.difficulty = template.difficulty; - // Set this.height equal to the known height for this block. - this.height = template.height; - // Set this.reserveOffset to the byte location of the reserved offset. - this.reserveOffset = template.reserved_offset; - // Set this.buffer to the binary decoded version of the BT blob. - this.buffer = new Buffer(this.blob, 'hex'); + // Generating a block template is a simple thing. Ask for a boatload of information, and go from there. + // Important things to consider. + // The reserved space is 16 bytes long now in the following format: + // Assuming that the extraNonce starts at byte 130: + // |130-133|134-137|138-141|142-145| + // |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| + // This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) + // Each with 4 billion clients. (clientNonce) + // While being unique to this particular pool thread (instanceId) + // With up to 4 billion clients (minerNonce/extraNonce) + // Overkill? Sure. But that's what we do here. Overkill. + + // Set these params equal to values we get from upstream (if they are set) + this.difficulty = template.difficulty; + this.height = template.height; + this.bits = template.bits; + this.seed_hash = template.seed_hash; + this.coin = template.coin; + this.port = template.port; + + const port_blob_num = port2blob_num[this.port]; + + if (template.blocktemplate_blob) { + this.blocktemplate_blob = template.blocktemplate_blob; + } else if (template.blob) { + this.blocktemplate_blob = template.blob; + } else { + const isExtraNonceBT = global.coinFuncs.blobTypeEth(port_blob_num) || global.coinFuncs.blobTypeErg(port_blob_num); + if (isExtraNonceBT) { + const hash = template.hash; + this.hash = this.idHash = this.prev_hash = hash; + this.hash2 = template.hash2; + this.block_version = 0; + this.nextBlobHex = function () { return hash; }; + return; + } else { + console.error("INTERNAL ERROR: No blob in " + this.port + " port block template: " + JSON.stringify(template)); + this.blocktemplate_blob = extra_nonce_mm_template_hex; // to avoid hard crash + } + } + + const is_mm = "child_template" in template; + + if (is_mm) { + this.child_template = template.child_template; + this.child_template_buffer = template.child_template_buffer; + } + + const is_dero = global.coinFuncs.blobTypeDero(port_blob_num); + const blob = is_dero ? template.blockhashing_blob : (is_mm ? template.parent_blocktemplate_blob : this.blocktemplate_blob); + + this.idHash = crypto.createHash('md5').update(blob).digest('hex'); + + // Set this.buffer to the binary decoded version of the BT blob + this.buffer = Buffer.from(blob, 'hex'); + this.block_version = this.buffer[0]; + + if (global.coinFuncs.blobTypeRvn(port_blob_num) || global.coinFuncs.blobTypeRtm(port_blob_num)) { + this.reserved_offset = template.reserved_offset; + } else if (is_dero) { // exception for DERO + this.reserved_offset = template.reserved_offset + 1; + } else { + const template_hex = (template.port in mm_port_set && !is_mm) ? extra_nonce_mm_template_hex : extra_nonce_template_hex; + const found_reserved_offset_template = blob.indexOf(template_hex); + + if (found_reserved_offset_template !== -1) { + const found_reserved_offset = (found_reserved_offset_template >> 1) + 2; + if (is_mm) { + this.reserved_offset = found_reserved_offset; + } else { + if (template.reserved_offset) { + // here we are OK with +1 difference because we put extra byte into pool_nonce_size + if (found_reserved_offset != template.reserved_offset && found_reserved_offset + 1 != template.reserved_offset) { + console.error("INTERNAL ERROR: Found reserved offset " + found_reserved_offset + " do not match " + template.reserved_offset + " reported by daemon in " + this.port + " block " + ": " + blob); + } + this.reserved_offset = template.reserved_offset; + } else if (template.reservedOffset) { + // here we are OK with +1 difference because we put extra byte into pool_nonce_size + if (found_reserved_offset != template.reservedOffset && found_reserved_offset + 1 != template.reservedOffset) { + console.error("INTERNAL ERROR: Found reserved offset " + found_reserved_offset + " do not match " + template.reservedOffset + " reported by daemon in " + this.port + " block " + ": " + blob); + } + this.reserved_offset = template.reservedOffset; + } else { + this.reserved_offset = found_reserved_offset; + } + } + } else { + //console.error("INTERNAL ERROR: Can not find reserved offset template '" + template_hex + "' in " + this.port + " block " + ": " + blob); + this.reserved_offset = template.reserved_offset ? template.reserved_offset : template.reservedOffset; + } + } + + if (!this.reserved_offset) { + console.error("INTERNAL ERROR: No reserved offset in " + this.port + " port block template: " + JSON.stringify(template)); + this.reserved_offset = 0; // to avoid hard crash + } + + if (!("prev_hash" in template)) { // Get prev_hash from blob + let prev_hash = Buffer.alloc(32); + const prev_hash_start = global.coinFuncs.blobTypeRvn(port2blob_num[this.port]) ? 4 : 7; + this.buffer.copy(prev_hash, 0, prev_hash_start, prev_hash_start + 32); + this.prev_hash = prev_hash.toString('hex'); + } else { + this.prev_hash = template.prev_hash; + } + // Copy the Instance ID to the reserve offset + 4 bytes deeper. Copy in 4 bytes. - instanceId.copy(this.buffer, this.reserveOffset + 4, 0, 4); - // Generate a clean, shiny new buffer. - this.previous_hash = new Buffer(32); - // Copy in bytes 7 through 39 to this.previous_hash from the current BT. - this.buffer.copy(this.previous_hash, 0, 7, 39); - // Reset the Nonce. - This is the per-miner/pool nonce + instanceId.copy(this.buffer, this.reserved_offset + 4, 0, 4); + // Reset the Nonce - this is the per-miner/pool nonce this.extraNonce = 0; // The clientNonceLocation is the location at which the client pools should set the nonces for each of their clients. - this.clientNonceLocation = this.reserveOffset + 12; + this.clientNonceLocation = this.reserved_offset + 12; // The clientPoolLocation is for multi-thread/multi-server pools to handle the nonce for each of their tiers. - this.clientPoolLocation = this.reserveOffset + 8; - // this is current algo type - this.algo = template.algo; - // this is current daemon port - this.port = template.port; - this.nextBlob = function () { + this.clientPoolLocation = this.reserved_offset + 8; + + this.nextBlobHex = function () { // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Convert the blob into something hashable. - return global.coinFuncs.convertBlob(this.buffer, this.port).toString('hex'); + this.buffer.writeUInt32BE(++this.extraNonce, this.reserved_offset); + // Convert the buffer into something hashable. + const blob = global.coinFuncs.convertBlob(this.buffer, this.port); + return blob ? blob.toString('hex') : null; }; - // Make it so you can get the raw block blob out. - this.nextBlobWithChildNonce = function () { + // Make it so you can get the raw block buffer out. + this.nextBlobWithChildNonceHex = function () { // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Don't convert the blob to something hashable. You bad. + this.buffer.writeUInt32BE(++this.extraNonce, this.reserved_offset); + // Don't convert the buffer to something hashable. You bad. return this.buffer.toString('hex'); }; }; - this.cryptoNight = function(convertedBlob, port) { - switch (port) { - case 11181: return multiHashing.cryptonight_light(convertedBlob, 1); // Aeon - case 12211: return multiHashing.cryptonight_heavy(convertedBlob, 0); // RYO - case 17750: return multiHashing.cryptonight_heavy(convertedBlob, 1); // Haven - case 20189: return multiHashing.cryptonight(convertedBlob, 3); // Stellite - case 24182: return multiHashing.cryptonight_heavy(convertedBlob, 2); // BitTube - case 38081: return multiHashing.cryptonight(convertedBlob, 4); // MSR - default: return multiHashing.cryptonight(convertedBlob, 1); + this.getPORTS = function() { return ports; } + this.getCOINS = function() { return coins; } + this.PORT2COIN = function(port) { return port2coin[port]; } + this.PORT2COIN_FULL = function(port) { const coin = port2coin[port]; return coin == "" ? "XMR" : coin; } + this.COIN2PORT = function(coin) { return coin2port[coin]; } + this.getMM_PORTS = function() { return mm_port_set; } + this.getMM_CHILD_PORTS = function() { return mm_child_port_set; } + + this.getDefaultAlgos = function() { + return [ "rx/0" ]; + } + + this.getDefaultAlgosPerf = function() { + return { "rx/0": 1 }; + } + + this.getPrevAlgosPerf = function() { + return { "cn/r": 1, "cn/half": 1.9, "cn/rwz": 1.3, "cn/zls": 1.3, "cn/double": 0.5 }; + } + + this.convertAlgosToCoinPerf = function(algos_perf) { + let coin_perf = {}; + + if ("rx/0" in algos_perf) coin_perf[""] = algos_perf["rx/0"]; + + if ("cn/r" in algos_perf) coin_perf["SUMO"] = coin_perf["LTHN"] = algos_perf["cn/r"]; + + if ("cn/half" in algos_perf) coin_perf["MSR"] = algos_perf["cn/half"]; + else if ("cn/fast2" in algos_perf) coin_perf["MSR"] = algos_perf["cn/fast2"]; + + if ("panthera" in algos_perf) coin_perf["XLA"] = algos_perf["panthera"]; + + if ("cn/gpu" in algos_perf) coin_perf["RYO"] = coin_perf["CCX"] = coin_perf["XEQ"] = algos_perf["cn/gpu"]; + +// if ("rx/wow" in algos_perf) coin_perf["WOW"] = algos_perf["rx/wow"]; + + if ("kawpow" in algos_perf) coin_perf["RVN"] = algos_perf["kawpow"]; + + if ("ghostrider" in algos_perf) coin_perf["RTM"] = algos_perf["ghostrider"]; + + if ("ethash" in algos_perf) coin_perf["ETH"] = algos_perf["ethash"]; + + if ("autolykos2" in algos_perf) coin_perf["ERG"] = algos_perf["autolykos2"]; + + if ("rx/graft" in algos_perf) coin_perf["GRFT"] = algos_perf["rx/graft"]; + + if ("cn-heavy/xhv" in algos_perf) coin_perf["XHV"] = coin_perf["BLOC"] = algos_perf["cn-heavy/xhv"]; + + //if ("k12" in algos_perf) coin_perf["AEON"] = algos_perf["k12"]; + + if ("cn-pico" in algos_perf) coin_perf["IRD"] = algos_perf["cn-pico"]; + else if ("cn-pico/trtl" in algos_perf) coin_perf["IRD"] = algos_perf["cn-pico/trtl"]; + + if ("rx/arq" in algos_perf) coin_perf["ARQ"] = algos_perf["rx/arq"]; + + if ("c29s" in algos_perf) coin_perf["XTNC"] = coin_perf["XWP"] = algos_perf["c29s"]; + if ("c29v" in algos_perf) coin_perf["XMV"] = algos_perf["c29v"]; + if ("c29b" in algos_perf) coin_perf["TUBE"] = algos_perf["c29b"]; + if ("c29i" in algos_perf) coin_perf["XTA"] = algos_perf["c29i"]; + + if ("astrobwt" in algos_perf) coin_perf["DERO"] = algos_perf["astrobwt"]; + + //if ("cn/0" in algos_perf) coin_perf["XMC"] = algos_perf["cn/0"]; + + if ("argon2/chukwav2" in algos_perf) coin_perf["TRTL"] = algos_perf["argon2/chukwav2"]; + else if ("chukwav2" in algos_perf) coin_perf["TRTL"] = algos_perf["chukwav2"]; + + return coin_perf; + } + + // returns true if algo set reported by miner is for main algo + this.algoMainCheck = function(algos) { + if ("rx/0" in algos) return true; + return false; + } + // returns true if algo set reported by miner is one of previous main algos + this.algoPrevMainCheck = function(algos) { + if ("cn/r" in algos) return true; + return false; + } + // returns true if algo set reported by miner is OK or error string otherwise + this.algoCheck = function(algos) { + if (this.algoMainCheck(algos)) return true; + for (let algo in all_algos) if (algo in algos) return true; + return "algo array must include at least one supported pool algo: [" + Object.keys(algos).join(", ") + "]"; + } + + this.slowHashBuff = function(convertedBlob, blockTemplate, nonce, mixhash) { + switch (blockTemplate.port) { + case 2086: return multiHashing.cryptonight_heavy(convertedBlob, 1); // BLOC + case 8766: return multiHashing.kawpow(convertedBlob, Buffer.from(nonce, 'hex'), Buffer.from(mixhash, 'hex')); // RVN + case 9998: return multiHashing.cryptonight(convertedBlob, 18); // RTM + case 8545: return multiHashing.ethash(convertedBlob, Buffer.from(nonce, 'hex'), blockTemplate.height); // ETH + case 9053: return multiHashing.autolykos2_hashes(convertedBlob, blockTemplate.height); // ERG + case 9231 : return multiHashing.cryptonight(convertedBlob, 11); // XEQ + //case 11181: return multiHashing.k12(convertedBlob); // Aeon + case 11898: return multiHashing.argon2(convertedBlob, 2); // TRTL + case 12211: return multiHashing.cryptonight(convertedBlob, 11); // RYO + case 13007: return multiHashing.cryptonight_pico(convertedBlob, 0); // Iridium + case 16000: return multiHashing.cryptonight(convertedBlob, 11); // CCX + case 17750: return multiHashing.cryptonight_heavy(convertedBlob, 1); // Haven + case 18081: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 0); // XMR + //case 18181: return multiHashing.cryptonight(convertedBlob, 0); // XMC + case 18981: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 20); // Graft + case 19734: return multiHashing.cryptonight(convertedBlob, 13, blockTemplate.height); // SUMO + case 19994: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 2); // ArqMa + case 11812: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 3); // Scala + case 20206: return multiHashing.astrobwt(convertedBlob, 0); // Dero +// case 34568: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 17); // Wownero + case 38081: return multiHashing.cryptonight(convertedBlob, 9); // MSR + case 48782: return multiHashing.cryptonight(convertedBlob, 13, blockTemplate.height); // Lethean + default: + console.error("Unknown " + blockTemplate.port + " port for Cryptonight PoW type"); + return multiHashing.cryptonight(convertedBlob, 13, blockTemplate.height); } } - this.blobTypeStr = function(port) { - switch (port) { - case 38081: return "cryptonote2"; // MSR - default: return "cryptonote"; + this.slowHash = function(convertedBlob, blockTemplate, nonce, mixhash) { + return this.slowHashBuff(convertedBlob, blockTemplate, nonce, mixhash).toString("hex"); + } + + this.verify_share_host_index = 0; + + this.slowHashAsync = function(convertedBlob, blockTemplate, miner_address, cb) { + if (!global.config.verify_shares_host) return cb(this.slowHash(convertedBlob, blockTemplate)); + let jsonInput; + switch (blockTemplate.port) { + case 18081: + case 18981: + case 19994: + case 11812: + case 22023: +// case 34568: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex'), "seed_hash": blockTemplate.seed_hash }; + break; + case 19734: + case 48782: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex'), "height": blockTemplate.height }; + break; + //case 11181: + // return cb(this.slowHash(convertedBlob, blockTemplate)); // AEON K12 is too fast + default: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex') }; } + const time_now = Date.now(); + let best_index = null; + let min_queue_size = null; + let max_noerr_time = null; + shareVerifyQueue.forEach(function(queue_obj, index) { + if (time_now - shareVerifyQueueErrorTime[index] < 1*60*1000 && shareVerifyQueueErrorCount[index] > 10) return; + const qlength = queue_obj.length() + queue_obj.running(); + if (min_queue_size === null || qlength < min_queue_size) { + best_index = index; + min_queue_size = qlength; + } + }); + if (best_index === null) shareVerifyQueueErrorTime.forEach(function(last_error_time, index) { + const noerr_time = time_now - last_error_time; + if (max_noerr_time === null || noerr_time > max_noerr_time) { + best_index = index; + max_noerr_time = noerr_time; + } + }); + return shareVerifyQueue[best_index].unshift({ + jsonInput: jsonInput, + cb: cb, + time: time_now, + miner_address: miner_address + }); } - this.algoTypeStr = function(port) { + this.c29 = function(header, ring, port) { switch (port) { - case 11181: return "cryptonight-lite/1"; // Aeon - case 12211: return "cryptonight-heavy/0"; // RYO - case 17750: return "cryptonight-heavy/xhv"; // Haven - case 20189: return "cryptonight/xtl"; // Stellite - case 24182: return "cryptonight-heavy/tube"; // BitTube - case 38081: return "cryptonight/msr"; // MSR - default: return "cryptonight/1"; + case 13102: return multiHashing.c29i(header, ring); // MoneroV + case 19281: return multiHashing.c29v(header, ring); // MoneroV + case 19950: return multiHashing.c29s(header, ring); // Swap + case 25182: return multiHashing.c29b(header, ring); // TUBE + case 33124: return multiHashing.c29s(header, ring); // XtendCash + default: + console.error("Unknown " + port + " port for Cuckaroo PoW type"); + return multiHashing.c29s(header, ring); } } - this.algoShortTypeStr = function(port) { - switch (port) { - case 11181: return "cn-lite/1"; // Aeon - case 12211: return "cn-heavy/0"; // RYO - case 17750: return "cn-heavy/xhv"; // Haven - case 20189: return "cn/xtl"; // Stellite - case 24182: return "cn-heavy/tube"; // BitTube - case 38081: return "cn/msr"; // MSR - default: return "cn/1"; + this.c29_cycle_hash = function(ring, blob_type_num) { + switch (blob_type_num) { + case 10: return multiHashing.c29b_cycle_hash(ring); + case 12: return multiHashing.c29i_cycle_hash(ring); + default: return multiHashing.c29_cycle_hash(ring); } } - this.variantValue = function(port) { + this.blobTypeStr = function(port, version) { switch (port) { - case 12211: return "0"; // RYO - case 17750: return "xhv"; // Haven - case 20189: return "xtl"; // Stellite - case 24182: return "tube"; // BitTube - case 38081: return "msr"; // MSR - default: return "1"; + case 2086: return "forknote1"; // BLOC + case 8545: return "eth"; // ETH + case 8766: return "raven"; // RVN + case 9053: return "erg"; // ERG + case 9231: return "cryptonote_loki"; // XEQ + case 9998: return "raptoreum"; // RTM + //case 11181: return "aeon"; // Aeon + case 11898: return "forknote2"; // TRTL + case 13007: return "forknote2"; // Iridium + case 12211: return "cryptonote_ryo"; // RYO + case 13102: return "cryptonote_xta"; // Italocoin + case 17750: return "cryptonote_xhv"; // XHV + case 19281: return "cuckaroo"; // MoneroV + case 19950: return "cuckaroo"; // Swap + case 20206: return "cryptonote_dero"; // Dero + case 22023: return "cryptonote_loki"; // LOKI + case 25182: return "cryptonote_tube"; // TUBE + case 33124: return "cryptonote_xtnc"; // XtendCash + case 38081: return "cryptonote3"; // MSR + default: return "cryptonote"; } } - this.get_miner_agent_notification = function(agent) { + this.algoShortTypeStr = function(port, version) { + if (port in port2algo) return port2algo[port]; + console.error("Unknown " + port + " port for PoW type on " + version + " version"); + return "rx/0"; + } + + this.isMinerSupportAlgo = function(algo, algos) { + if (algo in algos) return true; + if (algo === "cn-heavy/0" && "cn-heavy" in algos) return true; + return false; + } + + this.get_miner_agent_warning_notification = function(agent) { let m; if (m = reXMRig.exec(agent)) { - let majorv = parseInt(m[1]) * 100; - let minorv = parseInt(m[2]); - if (majorv + minorv < 205) { - return "Please update your XMRig miner (" + agent + ") to v2.6.1+"; + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + if (majorv + minorv < 30200) { + return "Please update your XMRig miner (" + agent + ") to v3.2.0+ to support new rx/0 Monero algo"; } - } else if (m = reXMRSTAK.exec(agent)) { - let majorv = parseInt(m[1]) * 100; - let minorv = parseInt(m[2]); - if (majorv + minorv < 203) { - return "Please update your xmr-stak miner (" + agent + ") to v2.4.3+ (and use cryptonight_v7 in config)"; + if (majorv + minorv >= 40000 && majorv + minorv < 40200) { + return "Please update your XMRig miner (" + agent + ") to v4.2.0+ to support new rx/0 Monero algo"; } + } else if (m = reXMRSTAKRX.exec(agent)) { + return false; + } else if (m = reXMRSTAK.exec(agent)) { + return "Please update your xmr-stak miner (" + agent + ") to xmr-stak-rx miner to support new rx/0 Monero algo"; } else if (m = reXNP.exec(agent)) { - let majorv = parseInt(m[1]) * 10000; - let minorv = parseInt(m[2]) * 100; - let minorv2 = parseInt(m[3]); - if (majorv + minorv + minorv2 < 2) { - return "Please update your xmr-node-proxy (" + agent + ") to version v0.1.3+ (from https://github.com/MoneroOcean/xmr-node-proxy repo)"; + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + const minorv2 = parseInt(m[3]); + const version = majorv + minorv + minorv2; + if (version < 1400) { + return "Please update your xmr-node-proxy (" + agent + ") to version v0.14.0+ by doing 'cd xmr-node-proxy && ./update.sh' (or check https://github.com/MoneroOcean/xmr-node-proxy repo) to support new rx/0 Monero algo"; } - } else if (m = reCCMINER.exec(agent)) { - let majorv = parseInt(m[1]) * 100; - let minorv = parseInt(m[2]); - if (majorv + minorv < 300) { - return "Please update ccminer-cryptonight miner to v3.02+"; + } else if (m = reSRBMULTI.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + const minorv2 = parseInt(m[3]); + if (majorv + minorv + minorv2 < 105) { + return "Please update your SRBminer-MULTI (" + agent + ") to version v0.1.5+ to support new rx/0 Monero algo"; } } return false; }; - - this.get_miner_agent_warning_notification = function(agent) { + + this.is_miner_agent_no_haven_support = function(agent) { let m; - if (m = reXMRSTAK2.exec(agent)) { - let majorv = parseInt(m[1]) * 10000; - let minorv = parseInt(m[2]) * 100; - let minorv2 = parseInt(m[3]); - if (majorv + minorv + minorv2 < 20403) { - return "Please update your xmr-stak miner (" + agent + ") to v2.4.3+ (and use cryptonight_v7 in config)"; - } - } else if (m = reXNP.exec(agent)) { - let majorv = parseInt(m[1]) * 10000; - let minorv = parseInt(m[2]) * 100; - let minorv2 = parseInt(m[3]); - if (majorv + minorv + minorv2 < 103) { - return "Please update your xmr-node-proxy (" + agent + ") to version v0.1.3+ by doing 'cd xmr-node-proxy && ./update.sh' (or check https://github.com/MoneroOcean/xmr-node-proxy repo)"; + if (m = reXMRig.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + if (majorv + minorv < 60300) { + return true; } } return false; }; -} + + this.get_miner_agent_not_supported_algo = function(agent) { + let m; + if (m = reXMRSTAKRX.exec(agent)) { + return "rx/0"; + } else if (m = reXMRSTAK.exec(agent)) { + return "cn/r"; + } + return false; + }; + + this.fixDaemonIssue = function(height, top_height, port) { + global.support.sendEmail(global.config.general.adminEmail, + "Pool server " + global.config.hostname + " has stuck block template", + "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " with current block height " + + height + " is stuck compared to top height (" + top_height + ") amongst other leaf nodes for " + + port + " port\nAttempting to fix..." + ); + if (fs.existsSync(fix_daemon_sh)) { + child_process.exec(fix_daemon_sh + " " + port, function callback(error, stdout, stderr) { + console.log("> " + fix_daemon_sh + " " + port); + console.log(stdout); + console.error(stderr); + if (error) console.error(fix_daemon_sh + " script returned error exit code: " + error.code); + }); + } else { + console.error("No " + fix_daemon_sh + " script was found to fix stuff"); + } + } +}; + + module.exports = Coin; diff --git a/lib/data.proto b/lib/data.proto index f2b6fba1a..18f9b8e14 100644 --- a/lib/data.proto +++ b/lib/data.proto @@ -23,10 +23,11 @@ message InvalidShare{ required string paymentAddress = 1; optional string paymentID = 2; required string identifier = 3; + optional int32 count = 4; } message Share { - required int32 shares = 1; + optional int64 shares = 1; required string paymentAddress = 2; required bool foundBlock = 3; optional string paymentID = 4; @@ -39,8 +40,9 @@ message Share { required int64 timestamp = 11; required string identifier = 12; optional int32 port = 13; - optional int32 shares2 = 14; - optional int32 share_num = 15; + optional int64 shares2 = 14; + optional int64 share_num = 15; + optional float raw_shares = 16; } message Block { @@ -52,6 +54,7 @@ message Block { required bool unlocked = 6; required bool valid = 7; optional int64 value = 8; + optional bool pay_ready = 9; } message AltBlock { @@ -65,8 +68,9 @@ message AltBlock { required int32 port = 8; required int32 height = 9; required int32 anchor_height = 10; - optional int64 value = 11; - optional int64 pay_value = 12; + optional uint64 value = 11; + optional uint64 pay_value = 12; optional string pay_stage = 13; optional string pay_status = 14; + optional bool pay_ready = 15; } diff --git a/lib/local_comms.js b/lib/local_comms.js index d7ea169eb..43148f7d3 100644 --- a/lib/local_comms.js +++ b/lib/local_comms.js @@ -2,6 +2,8 @@ let range = require('range'); let debug = require('debug')('db'); let async = require('async'); +let cleanShareInProgress = false; +let cleanShareStuckCount = 0; function poolTypeStr(poolType) { switch (poolType) { @@ -22,18 +24,14 @@ function Database(){ this.altblockDB = null; this.cacheDB = null; - this.dirtyenv = false; this.initEnv = function(){ global.database.env = new this.lmdb.Env(); global.database.env.open({ path: global.config.db_storage_path, maxDbs: 10, - mapSize: 24 * 1024 * 1024 * 1024, - noSync: false, - mapAsync: false, - useWritemap: false, - noMetaSync: true, + mapSize: global.config.general.dbSizeGB * 1024 * 1024 * 1024, + useWritemap: true, maxReaders: 512 }); global.database.shareDB = this.env.openDbi({ @@ -61,15 +59,13 @@ function Database(){ name: 'cache', create: true }); - global.database.intervalID = setInterval(function(){ - global.database.env.sync(function(){}); - }, 60000); // Sync the DB every 60 seconds - global.database.dirtyenv = false; + //global.database.intervalID = setInterval(function(){ + // global.database.env.sync(function(){}); + //}, 60000); // Sync the DB every 60 seconds console.log("Database Worker: LMDB Env Initialized."); }; this.incrementCacheData = function(key, data){ - this.refreshEnv(); let txn = this.env.beginTxn(); let cached = txn.getString(this.cacheDB, key); if (cached !== null){ @@ -88,7 +84,7 @@ function Database(){ } }; - this.getBlockList = function(pool_type){ + this.getBlockList = function(pool_type, first, last) { debug("Getting block list"); switch (pool_type) { case 'pplns': @@ -105,23 +101,15 @@ function Database(){ } let response = []; try{ - this.refreshEnv(); let txn = global.database.env.beginTxn({readOnly: true}); let cursor = new global.database.lmdb.Cursor(txn, global.database.blockDB); - for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { - /* - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - */ + for (let found = cursor.goToLast(), i = 0; found; found = cursor.goToPrev()) { + if (typeof last !== 'undefined' && i >= last) break; cursor.getCurrentBinary(function (key, data) { // jshint ignore:line let blockData = global.protos.Block.decode(data); let poolType = poolTypeStr(blockData.poolType); - if (blockData.poolType === pool_type || pool_type === false) { + if (pool_type === false || blockData.poolType === pool_type) { + if (typeof first !== 'undefined' && i++ < first) return; response.push({ ts: blockData.timestamp, hash: blockData.hash, @@ -138,13 +126,13 @@ function Database(){ } cursor.close(); txn.abort(); - return response.sort(global.support.blockCompare); + return response; //.sort(global.support.blockCompare); } catch (e){ return response; } }; - this.getAltBlockList = function(pool_type, coin_port){ + this.getAltBlockList = function(pool_type, coin_port, first, last) { debug("Getting altblock list"); switch (pool_type) { case 'pplns': @@ -161,23 +149,15 @@ function Database(){ } let response = []; try{ - this.refreshEnv(); let txn = global.database.env.beginTxn({readOnly: true}); let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); - for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { - /* - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - */ + for (let found = cursor.goToLast(), i = 0; found; found = cursor.goToPrev()) { + if (typeof last !== 'undefined' && i >= last) break; cursor.getCurrentBinary(function (key, data) { // jshint ignore:line let blockData = global.protos.AltBlock.decode(data); let poolType = poolTypeStr(blockData.poolType); - if (blockData.poolType === pool_type || pool_type === false && (!coin_port || blockData.port === coin_port)) { + if ((pool_type === false || blockData.poolType === pool_type) && (!coin_port || blockData.port === coin_port)) { + if (typeof first !== 'undefined' && i++ < first) return; response.push({ ts: blockData.timestamp, hash: blockData.hash, @@ -198,7 +178,7 @@ function Database(){ } cursor.close(); txn.abort(); - return response.sort(global.support.tsCompare); + return response; //.sort(global.support.tsCompare); } catch (e){ return response; } @@ -214,7 +194,7 @@ function Database(){ shareObject.forEach(function(share){ //Data is the share object at this point. ++ shareCount; - if (typeof(share.shares) === "number") { + if (typeof(share.raw_shares) === "number") { if (!shares.hasOwnProperty(share.blockHeight)) { shares[share.blockHeight] = []; } @@ -231,21 +211,21 @@ function Database(){ cachedData[stats_type1] = { totalHashes: 0, roundHashes: 0 }; } if (port_suffix === "") { - cachedData[global_stats1].totalHashes += share.shares; - cachedData[global_stats1].roundHashes += share.shares; - cachedData[stats_type1].totalHashes += share.shares; - cachedData[stats_type1].roundHashes += share.shares; + cachedData[global_stats1].totalHashes += share.raw_shares; + cachedData[global_stats1].roundHashes += share.raw_shares; + cachedData[stats_type1].totalHashes += share.raw_shares; + cachedData[stats_type1].roundHashes += share.raw_shares; } else { let global_stats2 = global_stats1 + port_suffix; let stats_type2 = stats_type1 + port_suffix; if (!(global_stats2 in cachedData)) cachedData[global_stats2] = { totalHashes: 0, roundHashes: 0 }; if (!(stats_type2 in cachedData)) cachedData[stats_type2] = { totalHashes: 0, roundHashes: 0 }; - cachedData[global_stats1].totalHashes += share.shares; - cachedData[global_stats2].totalHashes += share.shares; - cachedData[global_stats2].roundHashes += share.shares; - cachedData[stats_type1].totalHashes += share.shares; - cachedData[stats_type2].totalHashes += share.shares; - cachedData[stats_type2].roundHashes += share.shares; + cachedData[global_stats1].totalHashes += share.raw_shares; + cachedData[global_stats2].totalHashes += share.raw_shares; + cachedData[global_stats2].roundHashes += share.raw_shares; + cachedData[stats_type1].totalHashes += share.raw_shares; + cachedData[stats_type2].totalHashes += share.raw_shares; + cachedData[stats_type2].roundHashes += share.raw_shares; } if (!cachedData.hasOwnProperty(minerID)) { cachedData[minerID] = {totalHashes: 0, goodShares: 0}; @@ -253,8 +233,8 @@ function Database(){ if (!cachedData.hasOwnProperty(minerIDWithIdentifier)) { cachedData[minerIDWithIdentifier] = {totalHashes: 0, goodShares: 0}; } - cachedData[minerIDWithIdentifier].totalHashes += share.shares; - cachedData[minerID].totalHashes += share.shares; + cachedData[minerIDWithIdentifier].totalHashes += share.raw_shares; + cachedData[minerID].totalHashes += share.raw_shares; const share_num = typeof(share.share_num) !== 'undefined' && share.share_num ? share.share_num : 1; cachedData[minerIDWithIdentifier].goodShares += share_num; cachedData[minerID].goodShares += share_num; @@ -333,8 +313,8 @@ function Database(){ minerID = minerID + '.' + share.paymentID; } let minerIDWithIdentifier = minerID + "_" + share.identifier; - this.incrementCacheData(minerIDWithIdentifier, [{location: 'badShares', value: 1}]); - this.incrementCacheData(minerID, [{location: 'badShares', value: 1}]); + this.incrementCacheData(minerIDWithIdentifier, [{location: 'badShares', value: share.count ? share.count : 1}]); + this.incrementCacheData(minerID, [{location: 'badShares', value: share.count ? share.count : 1}]); callback(true); } catch (e){ console.error("Ran into an error storing an invalid share. Damn!"); @@ -343,7 +323,6 @@ function Database(){ }; this.getBlockByID = function(blockID){ - this.refreshEnv(); debug("Getting the data for blockID: " + blockID); let txn = this.env.beginTxn({readOnly: true}); let data = txn.getBinary(this.blockDB, blockID); @@ -361,30 +340,36 @@ function Database(){ let orphanBlocks = {}; this.storeBlock = function(blockId, blockData, callback){ - this.refreshEnv(); try{ let blockDataDecoded = global.protos.Block.decode(blockData); global.coinFuncs.getBlockHeaderByHash(blockDataDecoded.hash, function(err, header){ // after 5 minutes of submit attempts finally cosider this block as orphan - if (err && header && header.error && typeof(header.error.message) === 'string' && - (header.error.message.indexOf("can't get block by hash") > -1 || header.error.message.indexOf("hash wasn't found") > -1)) { - let time_now = Date.now(); - if (blockDataDecoded.hash in orphanBlocks) { - if (time_now - orphanBlocks[blockDataDecoded.hash] > 5*60*1000) { - console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); - err = false; - header = {}; - header.reward = 0; - blockDataDecoded.valid = false; - blockDataDecoded.unlocked = true; + if (err && header) { + const is_orphan1 = header.orphan_status && header.orphan_status === true; + const is_orphan2 = header.error && typeof(header.error.message) === 'string' && ( + header.error.message.indexOf("can't get block by hash") > -1 || + header.error.message.indexOf("hash wasn't found") > -1 || + header.error.message.indexOf("Transaction not found") > -1 + ); + if (is_orphan1 || is_orphan2) { + let time_now = Date.now(); + if (blockDataDecoded.hash in orphanBlocks) { + if (time_now - orphanBlocks[blockDataDecoded.hash] > 10*60*1000) { + console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); + err = false; + header = {}; + header.reward = 0; + blockDataDecoded.valid = false; + blockDataDecoded.unlocked = true; + } + } else { + console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); + orphanBlocks[blockDataDecoded.hash] = time_now; } - } else { - console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); - orphanBlocks[blockDataDecoded.hash] = time_now; } } if (err || typeof(header) === 'undefined' || !header){ - setTimeout(function () { return callback(false) }, 1000); + setTimeout(function () { return callback(false) }, 30*1000); return; } blockDataDecoded.value = header.reward; @@ -411,32 +396,74 @@ function Database(){ } }; + let potentiallyBadBlocks = {}; // port -> block hash that has issues that will move into badBlocks after we will find good block for same port + let badBlocks = {}; // block hashes that we just start ignore (can't find incoming wallet tx) + let busyPorts = {}; // ports that are alredy have active getPortBlockHeaderByHash request + this.storeAltBlock = function(blockId, blockData, callback){ - this.refreshEnv(); try{ let blockDataDecoded = global.protos.AltBlock.decode(blockData); + if (blockDataDecoded.port in busyPorts) { + console.error("Pausing altblock with " + blockDataDecoded.port.toString() + " port and " + blockDataDecoded.height.toString() + " height processing"); + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + busyPorts[blockDataDecoded.port] = 1; global.coinFuncs.getPortBlockHeaderByHash(blockDataDecoded.port, blockDataDecoded.hash, function(err, header){ + delete busyPorts[blockDataDecoded.port]; // after 5 minutes of submit attempts finally cosider this block as orphan - if (err && header && header.error && typeof(header.error.message) === 'string' && header.error.message.indexOf("can't get block by hash") > -1) { - let time_now = Date.now(); - if (blockDataDecoded.hash in orphanBlocks) { - if (time_now - orphanBlocks[blockDataDecoded.hash] > 5*60*1000) { - console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); - err = false; - header = {}; - header.reward = 0; - blockDataDecoded.valid = false; - blockDataDecoded.unlocked = true; + let is_orphan = false; + if (err && header) { + const is_orphan1 = header.orphan_status && header.orphan_status === true; + const is_orphan2 = header.error && typeof(header.error.message) === 'string' && ( + header.error.message.indexOf("can't get block by hash") > -1 || + header.error.message.indexOf("Requested hash wasn't found in main blockchain") > -1 + ); + const is_orphan3 = header.topoheight && header.topoheight === -1; + if (is_orphan1 || is_orphan2 || is_orphan3) { + let time_now = Date.now(); + if (blockDataDecoded.hash in orphanBlocks) { + if (time_now - orphanBlocks[blockDataDecoded.hash] > 10*60*1000) { + is_orphan = true; + console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); + err = false; + header = {}; + header.reward = 0; + blockDataDecoded.valid = false; + blockDataDecoded.unlocked = true; + } + } else { + console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); + orphanBlocks[blockDataDecoded.hash] = time_now; + setTimeout(function () { return callback(false) }, 30*1000); + return; } - } else { - console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); - orphanBlocks[blockDataDecoded.hash] = time_now; } } - if (err || typeof(header) === 'undefined' || !header){ - setTimeout(function () { return callback(false) }, 1000); + if (blockDataDecoded.port == 20206 && header.depth < 30) { + console.log("Delaying " + blockDataDecoded.port + " port block hash " + blockDataDecoded.hash); + setTimeout(function () { return callback(false) }, 30*1000); return; } + if (err || typeof(header) === 'undefined' || !header || !header.reward) { // bad block and not orphan + if (blockDataDecoded.hash in badBlocks) { + console.error("Invalidating " + blockDataDecoded.port + " port block hash " + blockDataDecoded.hash); + return callback(true); + } + if (!(blockDataDecoded.port in potentiallyBadBlocks)) potentiallyBadBlocks[blockDataDecoded.port] = {}; + potentiallyBadBlocks[blockDataDecoded.port][blockDataDecoded.hash] = 1; + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + if (!is_orphan) { // now we found good block (not orphan) and we can move potentiallyBadBlocks to badBlocks + if (blockDataDecoded.port in potentiallyBadBlocks) { + for (let hash in potentiallyBadBlocks[blockDataDecoded.port]) { + console.log("Allowing bad " + blockDataDecoded.port + " port block hash " + hash); + badBlocks[hash] = 1; + } + delete potentiallyBadBlocks[blockDataDecoded.port]; + } + } blockDataDecoded.value = header.reward; blockDataDecoded.pay_value = 0; //blockData = global.database.calculateShares(blockDataDecoded, blockId); @@ -449,9 +476,16 @@ function Database(){ return callback(true); } let txn = global.database.env.beginTxn(); - while (txn.getBinary(global.database.altblockDB, blockId) !== null) { - console.error("Can't store altblock with " + blockId.toString() + " key, trying to increment it"); - ++ blockId; + let blockData2; + while ((blockData2 = txn.getBinary(global.database.altblockDB, blockId)) !== null) { + const blockDataDecoded2 = global.protos.AltBlock.decode(blockData2); + if (blockDataDecoded2.hash === blockDataDecoded.hash) { + txn.abort(); + console.error("Can't store already stored altblock with " + blockDataDecoded.hash.toString() + " hash: " + JSON.stringify(blockDataDecoded)); + return callback(true); + } + console.error("Can't store altblock with " + blockId.toString() + " key, trying to increment it"); + ++ blockId; } txn.putBinary(global.database.altblockDB, blockId, blockData); txn.commit(); @@ -466,7 +500,6 @@ function Database(){ }; this.invalidateBlock = function(blockId){ - this.refreshEnv(); let txn = this.env.beginTxn(); let blockData = global.protos.Block.decode(txn.getBinary(this.blockDB, blockId)); blockData.valid = false; @@ -476,7 +509,6 @@ function Database(){ }; this.invalidateAltBlock = function(blockId){ - this.refreshEnv(); let txn = this.env.beginTxn(); let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); blockData.valid = false; @@ -486,7 +518,6 @@ function Database(){ }; this.changeAltBlockPayStageStatus = function(blockId, pay_stage, pay_status){ - this.refreshEnv(); let txn = this.env.beginTxn(); let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); blockData.pay_stage = pay_stage; @@ -495,8 +526,21 @@ function Database(){ txn.commit(); }; + this.moveAltBlockReward = function(srcBlockId, dstBlockId, srcAmount){ + let txn = this.env.beginTxn(); + let srcBlockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, srcBlockId)); + let dstBlockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, dstBlockId)); + dstBlockData.value += srcAmount; + srcBlockData.value = 0; + srcBlockData.pay_stage = "Paid by other block"; + srcBlockData.pay_status = "Will be paid by block " + dstBlockData.hash + " on " + dstBlockData.height + " height"; + srcBlockData.unlocked = true; + txn.putBinary(this.altblockDB, srcBlockId, global.protos.AltBlock.encode(srcBlockData)); + txn.putBinary(this.altblockDB, dstBlockId, global.protos.AltBlock.encode(dstBlockData)); + txn.commit(); + }; + this.changeAltBlockPayValue = function(blockId, pay_value){ - this.refreshEnv(); let txn = this.env.beginTxn(); let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); blockData.pay_value = pay_value; @@ -505,7 +549,6 @@ function Database(){ }; this.getValidLockedBlocks = function(){ - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cursor = new this.lmdb.Cursor(txn, this.blockDB); let blockList = []; @@ -524,7 +567,6 @@ function Database(){ }; this.getValidLockedAltBlocks = function(){ - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cursor = new this.lmdb.Cursor(txn, this.altblockDB); let blockList = []; @@ -543,7 +585,6 @@ function Database(){ }; this.isAltBlockInDB = function(port, height){ - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cursor = new this.lmdb.Cursor(txn, this.altblockDB); let isBlockFound = false; @@ -561,7 +602,6 @@ function Database(){ }; this.unlockBlock = function(blockHex){ - this.refreshEnv(); let txn = this.env.beginTxn(); let cursor = new this.lmdb.Cursor(txn, this.blockDB); for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { @@ -580,7 +620,6 @@ function Database(){ }; this.unlockAltBlock = function(blockHex){ - this.refreshEnv(); let txn = this.env.beginTxn(); let cursor = new this.lmdb.Cursor(txn, this.altblockDB); for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { @@ -598,10 +637,45 @@ function Database(){ txn.commit(); }; + this.payReadyBlock = function(blockHex){ + let txn = this.env.beginTxn(); + let cursor = new this.lmdb.Cursor(txn, this.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + let blockDB = this.blockDB; + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.Block.decode(data); + if (blockData.hash === blockHex){ + blockData.pay_ready = true; + txn.putBinary(blockDB, key, global.protos.Block.encode(blockData)); + } + }); + blockDB = null; + } + cursor.close(); + txn.commit(); + }; + + this.payReadyAltBlock = function(blockHex){ + let txn = this.env.beginTxn(); + let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + let altblockDB = this.altblockDB; + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === blockHex){ + blockData.pay_ready = true; + txn.putBinary(altblockDB, key, global.protos.AltBlock.encode(blockData)); + } + }); + altblockDB = null; + } + cursor.close(); + txn.commit(); + }; + this.getCache = function(cacheKey){ debug("Getting Key: "+cacheKey); try { - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cached = txn.getString(this.cacheDB, cacheKey); txn.abort(); @@ -617,7 +691,6 @@ function Database(){ this.setCache = function(cacheKey, cacheData){ debug("Setting Key: "+cacheKey+ " Data: " + JSON.stringify(cacheData)); - this.refreshEnv(); let txn = this.env.beginTxn(); txn.putString(this.cacheDB, cacheKey, JSON.stringify(cacheData)); txn.commit(); @@ -627,14 +700,18 @@ function Database(){ let txn = this.env.beginTxn(); txn.putString(this.cacheDB, 'cacheUpdate', 'cacheUpdate'); txn.commit(); + //let size = 0; txn = this.env.beginTxn(); - for (let key in cacheUpdates){ - if (cacheUpdates.hasOwnProperty(key)){ - txn.putString(this.cacheDB, key, JSON.stringify(cacheUpdates[key])); - } + for (const [key, value] of Object.entries(cacheUpdates)) { + const value_str = JSON.stringify(value); + txn.putString(this.cacheDB, key, value_str); + //size += key.length + value_str.length; } txn.del(this.cacheDB, 'cacheUpdate'); txn.commit(); + //this.env.sync(function() { + //console.log("Wrote " + size + " bytes to LMDB"); + //}); }; this.getOldestLockedBlockHeight = function(){ @@ -643,7 +720,6 @@ function Database(){ This function returns a decompressed block proto for the first locked block in the system as part of the share depth functions. DO NOT BLINDLY REPLACE getLastBlock WITH THIS FUNCTION. */ - this.refreshEnv(); debug("Getting the oldest locked block in the system"); let oldestLockedBlockHeight = null; @@ -654,7 +730,7 @@ function Database(){ for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { cursor.getCurrentBinary(function(key, data){ // jshint ignore:line let blockData = global.protos.AltBlock.decode(data); - if (blockData.unlocked === false){ + if (blockData.unlocked === false && blockData.pay_ready !== true){ if (oldestLockedBlockHeight === null || oldestLockedBlockHeight > blockData.anchor_height) { oldestLockedBlockHeight = blockData.anchor_height; } @@ -669,7 +745,7 @@ function Database(){ cursor.getCurrentBinary(function(key, data){ // jshint ignore:line if (oldestLockedBlockHeight !== null && oldestLockedBlockHeight <= key) return; let blockData = global.protos.Block.decode(data); - if (blockData.unlocked === false) { + if (blockData.unlocked === false && blockData.pay_ready !== true) { oldestLockedBlockHeight = key; } }); @@ -712,6 +788,13 @@ function Database(){ where there's unlocked blocks. A find on the current block will have enough depth as long as the saves are correct. This will cause the system to clean up shares massively when there are no unlocked blocks. */ + if (cleanShareInProgress) { + console.error("CleanShareDB already running"); + ++cleanShareStuckCount; + if (cleanShareStuckCount > 5) global.support.sendEmail(global.config.general.adminEmail,"LongRunner stuck",cleanShareStuckCount); + return; // already running + } + cleanShareInProgress = true; let oldestLockedBlockHeight = this.getOldestLockedBlockHeight(); async.waterfall([ function(callback){ @@ -721,7 +804,7 @@ function Database(){ global.coinFuncs.getBlockHeaderByID(oldestLockedBlockHeight, (err, result) => { if (err !== null) { console.error("Can't get block with " + oldestLockedBlockHeight + " height"); - return; + return callback(true); } callback(null, oldestLockedBlockHeight, result.difficulty); }); @@ -731,7 +814,7 @@ function Database(){ global.coinFuncs.getLastBlockHeader(function(err, body){ if (err !== null) { console.error("Last block header request failed!"); - return; + return callback(true); } if (oldestLockedBlockHeight === null){ /* @@ -749,6 +832,10 @@ function Database(){ } } else { + console.log("Block depth to keep is " + (body.height - oldestLockedBlockHeight)); + if (body.height - oldestLockedBlockHeight > global.config.general.blockCleanWarning) { + global.support.sendEmail(global.config.general.adminEmail, "longRunner module can not clean DB good enough", "longRunner can not clean " + (body.height - oldestLockedBlockHeight) + " block from DB!"); + } /* Otherwise, start the scan from the oldest locked block downwards. This protects against the blockManager being messed up and not unlocking blocks. @@ -783,7 +870,7 @@ function Database(){ try{ let shareData = global.protos.Share.decode(data); if (shareData.poolType === global.protos.POOLTYPE.PPLNS){ - shareCount += shareData.shares; + shareCount += shareData.shares2; } } catch(e){ console.error("Invalid share"); @@ -803,9 +890,13 @@ function Database(){ callback(null, Array.from(Object.keys(blockSet))); } ], function(err, data){ + if (err !== null) { + console.error("ERROR with cleaning up because of daemon stuck"); + cleanShareInProgress = false; + return; + } if (global.config.general.blockCleaner === true){ if(data.length > 0){ - global.database.refreshEnv(); let totalDeleted = 0; let totalDeleted2 = 0; console.log("Block cleaning started: removing " + data.length + " block share records"); @@ -829,15 +920,17 @@ function Database(){ } else { console.log("Block cleaning disabled. Would have removed: " + JSON.stringify(data)); } + cleanShareInProgress = false; + cleanShareStuckCount = 0; console.log("Done cleaning up the share DB"); }); }; +} - this.refreshEnv = function(){}; +process.on('SIGINT', function() { + console.log("Closing DB"); + global.database.env.close(); +}); - setInterval(function(){ - global.database.dirtyenv = true; - }, 900000); // Set DB env reload for every 15 minutes. -} module.exports = Database; diff --git a/lib/longRunner.js b/lib/longRunner.js index 1bbdb1fc9..c6bdcfb2c 100644 --- a/lib/longRunner.js +++ b/lib/longRunner.js @@ -1,13 +1,16 @@ "use strict"; +const async = require("async"); + function cleanCacheDB() { - console.log("Cleaning up the cache DB"); - let count = 0; + console.log("Cleaning up the cache DB. Searching for items to delete/update"); let txn = global.database.env.beginTxn({readOnly: true}); let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + let updated = {}; + let deleted = []; for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { cursor.getCurrentString(function(key, data){ // jshint ignore:line - if (key.length < 95) return; // min XMR address length + if (key.length < global.config.pool.address.length) return; // min XMR address length if (key.includes("identifiers:")) { // remove frozen worker names after 24h let parts = key.split(/:(.+)/); @@ -22,10 +25,7 @@ function cleanCacheDB() { if (stats && Date.now() - stats.lastHash <= 24*60*60*1000) isAlive = true; } if (!isAlive) { - data2 = []; - let txn2 = global.database.env.beginTxn(); - txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); - txn2.commit(); + updated[key] = JSON.stringify([]); } } catch (e) { @@ -37,12 +37,9 @@ function cleanCacheDB() { if (!stats) return; if (!global.database.getCache("history:" + key)) return; if (Date.now() - stats.lastHash > 7*24*60*60*1000) { - let txn2 = global.database.env.beginTxn(); - txn2.del(global.database.cacheDB, key); - txn2.del(global.database.cacheDB, "history:" + key); - txn2.del(global.database.cacheDB, "stats:" + key); - txn2.commit(); - ++ count; + deleted.push(key); + deleted.push("history:" + key); + deleted.push("stats:" + key); } } else if (!key.includes("_") && key.includes("stats:")) { // zero frozen account hashrate after 24h @@ -50,9 +47,7 @@ function cleanCacheDB() { let data2 = JSON.parse(data); if ((data2.hash || data2.hash2) && Date.now() - data2.lastHash > 24*60*60*1000) { data2.hash = data2.hash2 = 0; - let txn2 = global.database.env.beginTxn(); - txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); - txn2.commit(); + updated[key] = JSON.stringify(data2); } } catch (e) { console.error("Bad cache data with " + key + " key"); @@ -64,12 +59,74 @@ function cleanCacheDB() { cursor.close(); txn.commit(); - console.log("Deleted cache items: " + count); + + console.log("Deleting cache items: " + deleted.length); + + let chunkSize = 0; + txn = global.database.env.beginTxn(); + deleted.forEach(function(key) { + ++ chunkSize; + txn.del(global.database.cacheDB, key); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + }); + txn.commit(); + + console.log("Updating cache items: " + Object.keys(updated).length); + + chunkSize = 0; + txn = global.database.env.beginTxn(); + for (const [key, value] of Object.entries(updated)) { + ++ chunkSize; + txn.putString(global.database.cacheDB, key, value); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + } + txn.commit(); +} + +let saw_block_hash_before = {}; + +let cleanBlockBalanceTableQueue = async.queue(function (task, callback) { + global.mysql.query("DELETE FROM block_balance WHERE hex = ?", [task.hex]).then(function () { return callback(true); }); +}, 10); + +function cleanBlockBalanceTable() { + console.log("Cleaning up the block balance table"); + + let locked_block_hashes = {}; + global.database.getValidLockedBlocks().forEach(function (block) { locked_block_hashes[block.hash] = 1; }); + global.database.getValidLockedAltBlocks().forEach(function (block) { locked_block_hashes[block.hash] = 1; }); + + global.mysql.query("SELECT hex FROM paid_blocks WHERE paid_time > (NOW() - INTERVAL 7 DAY)").then(function (rows_keep) { + rows_keep.forEach(function (row) { locked_block_hashes[row.hex] = 1; }); + let deleted_row_count = 0; + global.mysql.query("SELECT DISTINCT hex FROM block_balance").then(function (rows) { + rows.forEach(function (row) { + if (row.hex in locked_block_hashes) return; + if (row.hex in saw_block_hash_before) { + cleanBlockBalanceTableQueue.push(row, function () {}); + delete saw_block_hash_before[row.hex]; + ++ deleted_row_count; + } else { + saw_block_hash_before[row.hex] = 1; + } + }); + console.log("Finished cleaning the block balance table. Removed " + deleted_row_count + " rows."); + }); + }); } console.log("Cleaning up the share DB"); global.database.cleanShareDB(); cleanCacheDB(); +cleanBlockBalanceTable(); setInterval(function(){ console.log("Cleaning up the share DB"); @@ -80,3 +137,8 @@ setInterval(function(){ setInterval(function(){ cleanCacheDB(); }, 24*60*60*1000); + +// clean block balance table +setInterval(function(){ + cleanBlockBalanceTable(); +}, 24*60*60*1000); diff --git a/lib/payment_systems/xmr.js b/lib/payment_systems/xmr.js index 3373095a4..0bc204717 100644 --- a/lib/payment_systems/xmr.js +++ b/lib/payment_systems/xmr.js @@ -243,8 +243,8 @@ let xmrToQueue = async.queue(function (task, callback) { } }); }, - function (xmrCallback) { - return xmrCallback !== "TO_BE_CREATED"; + function (xmrCallback, untilCB) { + return untilCB(null, xmrCallback !== "TO_BE_CREATED"); }, function () { intCallback(null, txnID); @@ -343,7 +343,13 @@ let paymentQueue = async.queue(function (paymentDetails, callback) { global.support.rpcWallet(transferFunc, paymentDetails, function (body) { debug("Payment made: " + JSON.stringify(body)); if (body.hasOwnProperty('error') || !body.hasOwnProperty('result')) { - if (typeof(body.error) !== 'undefined' && body.error.hasOwnProperty('message') && (body.error.message === "not enough money" || body.error.message === "not enough unlocked money")){ + if ( typeof(body.error) !== 'undefined' && + body.error.hasOwnProperty('message') && + ( body.error.message === "not enough money" || + body.error.message === "not enough unlocked money" || + body.error.message === "transaction was rejected by daemon" + ) + ) { console.error("Issue making payments, not enough money, will try later"); setTimeout(getbalance, 10*60*1000); } else { @@ -360,10 +366,10 @@ let paymentQueue = async.queue(function (paymentDetails, callback) { }, 1); -paymentQueue.drain = function(){ +paymentQueue.drain(function(){ console.log("Payment queue drained"); global.database.setCache('lastPaymentCycle', Math.floor(Date.now()/1000)); -}; +}); function updateShapeshiftCompletion() { global.mysql.query("SELECT * FROM shapeshiftTxn WHERE txnStatus NOT IN ('complete', 'error')").then(function (rows) { @@ -699,16 +705,13 @@ function makePayments() { console.log("Loaded all payees into the system for processing"); let paymentDestinations = []; let totalAmount = 0; - let roundCount = 0; - let payeeList = []; let payeeObjects = {}; - rows.forEach(function (row) { + async.eachSeries(rows, function(row, next) { //debug("Starting round for: " + JSON.stringify(row)); + if ((row.payment_address + (row.payment_id ? ('.' + row.payment_id) : '')) in payeeObjects) return next(); // avoid doing payment for different pool types at the same time let payee = new Payee(row.amount, row.payment_address, row.payment_id, row.bitcoin); - payeeObjects[row.payment_address] = payee; global.mysql.query("SELECT payout_threshold FROM users WHERE username = ?", [payee.id]).then(function (userRow) { - ++ roundCount; - let threshold = global.support.decimalToCoin(0.3); + let threshold = global.support.decimalToCoin(global.config.payout.defaultPay); let custom_threshold = false; if (userRow.length !== 0 && userRow[0].payout_threshold != 0) { threshold = userRow[0].payout_threshold; @@ -730,73 +733,75 @@ function makePayments() { if (payee.amount >= threshold) { payee.setFeeAmount(); if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length !== 106) { + payeeObjects[payee.id] = payee; console.log("[++] " + payee.id + " miner to bulk payment. Amount: " + global.support.coinToDecimal(payee.amount)); paymentDestinations.push({amount: payee.amount - payee.fee, address: payee.address}); totalAmount += payee.amount; - payeeList.push(payee); } else if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length === 106 && (payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && custom_threshold))) { // Special code to handle integrated payment addresses. What a pain in the rear. // These are exchange addresses though, so they need to hit the exchange payout amount. + payeeObjects[payee.id] = payee; console.log("[+] " + payee.id + " as separate payment to integrated address. Amount: " + global.support.coinToDecimal(payee.amount)); payee.makePaymentAsIntegrated(); - } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && custom_threshold)) && payee.bitcoin === 0) { - console.log("[+] " + payee.id + " as separate payment to payment ID address. Amount: " + global.support.coinToDecimal(payee.amount)); - payee.makePaymentWithID(); + //} else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && custom_threshold)) && payee.bitcoin === 0) { + // payeeObjects[payee.id] = payee; + // console.log("[+] " + payee.id + " as separate payment to payment ID address. Amount: " + global.support.coinToDecimal(payee.amount)); + // payee.makePaymentWithID(); } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && custom_threshold)) && payee.bitcoin === 1) { + payeeObjects[payee.id] = payee; console.log("[+] " + payee.id + " as separate payment to bitcoin. Amount: " + global.support.coinToDecimal(payee.amount)); payee.makeBitcoinPayment(); } } - //debug("Went: " + roundCount + " With: " + paymentDestinations.length + " Possible destinations and: " + rows.length + " Rows"); - if (roundCount === rows.length && paymentDestinations.length > 0) { - while (paymentDestinations.length > 0) { - let paymentDetails = { - destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn - }; - console.log("Adding payment for " + paymentDetails.destinations.length + " miners"); - paymentQueue.unshift(paymentDetails, function (body) { //jshint ignore:line - // This is the only section that could potentially contain multiple txns. Lets do this safely eh? - if (body.fee && body.fee > 10) { - let totalAmount = 0; - let totalFee = 0; + return next(); + }); + }, function() { + while (paymentDestinations.length > 0) { + let paymentDetails = { + destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), + priority: global.config.payout.priority, + mixin: global.config.payout.mixIn + }; + console.log("Adding payment for " + paymentDetails.destinations.length + " miners"); + paymentQueue.unshift(paymentDetails, function (body) { //jshint ignore:line + // This is the only section that could potentially contain multiple txns. Lets do this safely eh? + if (body.fee && body.fee > 10) { + let totalAmount = 0; + let totalFee = 0; + paymentDetails.destinations.forEach(function (payeeItem) { + totalAmount += payeeObjects[payeeItem.address].amount; + totalFee += payeeObjects[payeeItem.address].fee; + console.log("[**] Successful payment to " + payeeItem.address + " for " + global.support.coinToDecimal(payeeObjects[payeeItem.address].amount) + " XMR (fee " + global.support.coinToDecimal(payeeObjects[payeeItem.address].fee) + ")"); + }); + console.log("[*] Successful payment to multiple miners of " + global.support.coinToDecimal(totalAmount) + " XMR (fee " + global.support.coinToDecimal(totalFee) + " - " + global.support.coinToDecimal(body.fee) + " = " + global.support.coinToDecimal(totalFee - body.fee) + ") with tx_hash " + body.tx_hash.match(hexChars)[0] + " and tx_key " + body.tx_key); + global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", + [0, null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, paymentDetails.destinations.length]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do: INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (0, null, null, " + + totalAmount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", " + paymentDetails.destinations.length + ")" + ); paymentDetails.destinations.forEach(function (payeeItem) { - totalAmount += payeeObjects[payeeItem.address].amount; - totalFee += payeeObjects[payeeItem.address].fee; - console.log("[**] Successful payment to " + payeeItem.address + " for " + global.support.coinToDecimal(payeeObjects[payeeItem.address].amount) + " XMR (fee " + global.support.coinToDecimal(payeeObjects[payeeItem.address].fee) + ")"); + let payee = payeeObjects[payeeItem.address]; + payee.transactionID = 0; + payee.manualPaymentShow(); }); - console.log("[*] Successful payment to multiple miners of " + global.support.coinToDecimal(totalAmount) + " XMR (fee " + global.support.coinToDecimal(totalFee) + " - " + global.support.coinToDecimal(body.fee) + " = " + global.support.coinToDecimal(totalFee - body.fee) + ") with tx_hash " + body.tx_hash.match(hexChars)[0] + " and tx_key " + body.tx_key); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, paymentDetails.destinations.length]).then(function (result) { - if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { - console.error("Can't do: INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (0, null, null, " - + totalAmount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", " + paymentDetails.destinations.length + ")" - ); - paymentDetails.destinations.forEach(function (payeeItem) { - payee = payeeObjects[payeeItem.address]; - payee.transactionID = 0; - payee.manualPaymentShow(); - }); - full_stop(result); - return; - } - paymentDetails.destinations.forEach(function (payeeItem) { - payee = payeeObjects[payeeItem.address]; - payee.transactionID = result.insertId; - payee.tx_hash = body.tx_hash.match(hexChars)[0]; - payee.tx_key = body.tx_key; - payee.trackPayment(); - }); - }); - } else { - console.error("Unknown error from the wallet: " + JSON.stringify(body)); + full_stop(result); + return; } + paymentDetails.destinations.forEach(function (payeeItem) { + let payee = payeeObjects[payeeItem.address]; + payee.transactionID = result.insertId; + payee.tx_hash = body.tx_hash.match(hexChars)[0]; + payee.tx_key = body.tx_key; + payee.trackPayment(); + }); }); + } else { + console.error("Unknown error from the wallet: " + JSON.stringify(body)); } - } - if (roundCount === rows.length) debug("Finished processing payments for now"); - }); + }); + } + debug("Finished processing payments for now"); }); }); debug("Finished makePayments"); @@ -828,4 +833,4 @@ if (global.config.payout.timer > 35791) { console.error("Payout timer is too high. Please use a value under 35791 to avoid overflows."); } else { init(); -} \ No newline at end of file +} diff --git a/lib/pool.js b/lib/pool.js index 3e4161891..2599cc359 100644 --- a/lib/pool.js +++ b/lib/pool.js @@ -1,6 +1,5 @@ "use strict"; const debug = require('debug')('pool'); -const uuidV4 = require('uuid/v4'); const crypto = require('crypto'); const bignum = require('bignum'); const cluster = require('cluster'); @@ -11,40 +10,80 @@ const tls = require('tls'); const fs = require('fs'); const child_process = require('child_process'); -let nonceCheck = new RegExp("^[0-9a-f]{8}$"); -let bannedIPs = {}; -let bannedAddresses = {}; -let notifyAddresses = {}; -let baseDiff = global.coinFuncs.baseDiff(); +const httpResponse = ' 200 OK\nContent-Type: text/plain\nContent-Length: 18\n\nMining Pool Online'; +const nonceCheck32 = new RegExp("^[0-9a-f]{8}$"); +const nonceCheck64 = new RegExp("^[0-9a-f]{16}$"); +const hashCheck32 = new RegExp("^[0-9a-f]{64}$"); +const hexMatch = new RegExp("^(?:[0-9a-f][0-9a-f])+$"); +const localhostCheck = new RegExp(/127\.0\.0\.1$/); +const baseDiff = global.coinFuncs.baseDiff(); +const baseRavenDiff = global.coinFuncs.baseRavenDiff(); + +const BLOCK_NOTIFY_PORT = 2223; +const DAEMON_POLL_MS = 500; + +let decId = 0; +function get_new_id() { + if (++decId > 999999999999999) decId = 0; + return decId.toString(10); +}; + +function pad_hex(str, bytes) { + const bytes2 = bytes * 2; + return ("00".repeat(bytes) + str.substr(0, bytes2)).substr(-bytes2); +} + +let ethJobId = 0; + +function get_new_eth_job_id() { + if (++ethJobId > 0xFFFF) ethJobId = 0; + return pad_hex(ethJobId.toString(16), 2); +}; -let activeMiners = {}; -let activeSmartMiners = {}; // miners with algos/algos-perf +let uniqueWorkerId; +let uniqueWorkerIdBits; +let freeEthExtranonces = []; + +function get_new_eth_extranonce_id() { + if (!freeEthExtranonces.length) { + const err_str = threadName + "Pool server " + global.config.hostname + " has overlow extranonce of " + (16 - uniqueWorkerIdBits) + " bits"; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Pool node has extranonce overflow", err_str); + return null; + } + return freeEthExtranonces.pop(); +}; + +function eth_extranonce(id) { + return id === null ? null : pad_hex(((id << uniqueWorkerIdBits) + uniqueWorkerId).toString(16), 2); +}; -let lastBlockHash = {}; // algo key -let lastAlgoHashFactor = {}; // algo key -let activeBlockTemplate = {}; // algo key +let bannedTmpIPs = {}; // ip banned for short time +let bannedTmpWallets = {}; // wallets banned for short time +let bannedBigTmpWallets = {}; // wallets banned for a long time +let bannedAddresses = {}; // forever banned wallets +let notifyAddresses = {}; // wallet notifications -let pastBlockTemplates = global.support.circularBuffer(10); +let activeMiners = new Map(); -let lastPortErrorTime = {}; // main algo port +let lastBlockHash = {}; // coin key +let activeBlockTemplates = {}; // coin key +let pastBlockTemplates = {}; // coin key -> global.support.circularBuffer -> activeBlockTemplates + +let newCoinHashFactor = {}; // coin key, current individual coin hash factor, set in updateCoinHashFactor +let lastCoinHashFactor = {}; // coin key, last set individual coin hash factor, set in setNewCoinHashFactor +let lastCoinHashFactorMM = {}; // coin key, current individual coin hash factor that includes merged mining factor, set in setNewCoinHashFactor -const fix_daemon_sh = "./fix_daemon.sh"; let lastBlockFixTime = {}; // time when blocks were checked to be in line with other nodes or when fix_daemon_sh was attempted let lastBlockFixCount = {}; // number of times fix_daemon_sh was run -let workerList = []; -let httpResponse = ' 200 OK\nContent-Type: text/plain\nContent-Length: 18\n\nMining Pool Online'; let threadName; let minerCount = []; -let BlockTemplate = global.coinFuncs.BlockTemplate; -let hexMatch = new RegExp("^[0-9a-f]+$"); let totalShares = 0, trustedShares = 0, normalShares = 0, invalidShares = 0, outdatedShares = 0, throttledShares = 0; // wallet -> { connectTime, count (miner), hashes, last_ver_shares } // this is need to thottle down some high share count miners let minerWallets = {}; -const MAX_VER_SHARES_PER_SEC = 10; // per thread -const VER_SHARES_PERIOD = 5; Buffer.prototype.toByteArray = function () { return Array.prototype.slice.call(this, 0); @@ -67,42 +106,32 @@ if (cluster.isMaster) { throttledShares = 0; }, 30*1000); } else { - threadName = "(Worker " + cluster.worker.id + " - " + process.pid + ") "; - // reset last verified share counters every VER_SHARES_PERIOD seconds + threadName = "(Worker " + process.env['WORKER_ID'] + " - " + process.pid + ") "; + // reset last verified share counters every global.config.pool.minerThrottleShareWindow seconds setInterval(function () { for (let wallet in minerWallets) { minerWallets[wallet].last_ver_shares = 0; } - }, VER_SHARES_PERIOD*1000); + }, global.config.pool.minerThrottleShareWindow*1000); } global.database.thread_id = threadName; -// algo can be "", "Heavy", "Light", "Fast" -const ALGOS = [ "Heavy", "Light", "Fast", "XTL" ]; +const COINS = global.coinFuncs.getCOINS(); function registerPool() { - global.mysql.query("SELECT * FROM pools WHERE id = ?", [global.config.pool_id]).then(function (rows) { - rows.forEach(function (row) { - if (row.ip !== global.config.bind_ip) { - console.error("Pool ID in use already for a different IP. Update MySQL or change pool ID."); - process.exit(1); + global.mysql.query("INSERT INTO pools (id, ip, last_checkin, active, hostname) VALUES (?, ?, now(), ?, ?) ON DUPLICATE KEY UPDATE last_checkin=now(), active=?", + [global.config.pool_id, global.config.bind_ip, true, global.config.hostname, true]); + global.mysql.query("DELETE FROM ports WHERE pool_id = ?", [global.config.pool_id]).then(function () { + global.config.ports.forEach(function (port) { + if ('ssl' in port && port.ssl === true) { + global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 1)", + [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); + } else { + global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 0)", + [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); } }); - }).then(function () { - global.mysql.query("INSERT INTO pools (id, ip, last_checkin, active, hostname) VALUES (?, ?, now(), ?, ?) ON DUPLICATE KEY UPDATE last_checkin=now(), active=?", - [global.config.pool_id, global.config.bind_ip, true, global.config.hostname, true]); - global.mysql.query("DELETE FROM ports WHERE pool_id = ?", [global.config.pool_id]).then(function () { - global.config.ports.forEach(function (port) { - if ('ssl' in port && port.ssl === true) { - global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 1)", - [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); - } else { - global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 0)", - [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); - } - }); - }); }); } @@ -114,22 +143,20 @@ function messageHandler(message) { if (cluster.isMaster) { sendToWorkers(message); } else { - bannedIPs[message.data] = 1; + if (!localhostCheck.test(message.data)) bannedTmpIPs[message.data] = 1; + else if (message.wallet) bannedTmpWallets[message.wallet] = 1; } break; case 'newBlockTemplate': debug(threadName + "Received new block template"); setNewBlockTemplate(message.data); break; - case 'newAlgoHashFactor': - debug(threadName + "Received new algo hash factor"); - setNewAlgoHashFactor(message.data.algo, message.data.algoHashFactor); - break; - case 'removeMiner': - if (cluster.isMaster) -- minerCount[message.data]; + case 'newCoinHashFactor': + debug(threadName + "Received new coin hash factor"); + setNewCoinHashFactor(true, message.data.coin, message.data.coinHashFactor); break; - case 'newMiner': - if (cluster.isMaster) ++ minerCount[message.data]; + case 'minerPortCount': + if (cluster.isMaster) minerCount[message.data.worker_id] = message.data.ports; break; case 'sendRemote': if (cluster.isMaster) { @@ -162,44 +189,37 @@ function messageHandler(message) { process.on('message', messageHandler); function sendToWorkers(data) { - workerList.forEach(function (worker) { - worker.send(data); + Object.keys(cluster.workers).forEach(function(key) { + cluster.workers[key].send(data); }); } -function retargetMiners() { - debug(threadName + "Performing difficulty check on miners"); - - function retargetMiner(miner) { - if (miner.fixed_diff) { - const newDiff = miner.calcNewDiff(); - if (miner.difficulty * 10 < newDiff) { - console.log("Dropped low fixed diff " + miner.difficulty + " for " + miner.logString + " miner to " + newDiff + " dynamic diff"); - miner.fixed_diff = false; - if (miner.setNewDiff(newDiff)) miner.sendNewJob(); - } - } else { - miner.updateDifficulty(); +function adjustMinerDiff(miner) { + if (miner.fixed_diff) { + const newDiff = miner.calcNewDiff(); + if (miner.difficulty * 10 < newDiff) { + console.log("Dropped low fixed diff " + miner.difficulty + " for " + miner.logString + " miner to " + newDiff + " dynamic diff"); + miner.fixed_diff = false; + if (miner.setNewDiff(newDiff)) return true; } + } else if (miner.setNewDiff(miner.calcNewDiff())) { + return true; } + return false; +} - let miner_count = 0; - const time_before = Date.now(); +function retargetMiners() { + debug(threadName + "Performing difficulty check on miners"); - for (let minerId in activeSmartMiners) { - if (activeSmartMiners.hasOwnProperty(minerId)) { - retargetMiner(activeSmartMiners[minerId]); - } - ++ miner_count; - } - for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - retargetMiner(activeMiners[minerId]); - } - ++ miner_count; + global.config.ports.forEach(function (portData) { minerCount[portData.port] = 0; }); + const time_before = Date.now(); + for (var [minerId, miner] of activeMiners) { + if (adjustMinerDiff(miner)) miner.sendSameCoinJob(); + ++ minerCount[miner.port]; } const elapsed = Date.now() - time_before; - if (elapsed > 500) console.error("retargetMiners() consumed " + elapsed + " ms for " + miner_count + " miners"); + if (elapsed > 50) console.error(threadName + "retargetMiners() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); + process.send({type: 'minerPortCount', data: { worker_id: process.env['WORKER_ID'], ports: minerCount } }); } // wallet " " proxy miner name -> { connectTime, count (miner), hashes } @@ -207,7 +227,12 @@ function retargetMiners() { let proxyMiners = {}; function addProxyMiner(miner) { - let proxyMinerName = miner.payout + ":" + miner.identifier; + if (miner.proxyMinerName && miner.proxyMinerName in proxyMiners) return; + + const wallet = miner.payout; + const proxyMinerName = wallet; //+ ":" + miner.identifier; + miner.proxyMinerName = proxyMinerName; + if (!(proxyMinerName in proxyMiners)) { proxyMiners[proxyMinerName] = {}; proxyMiners[proxyMinerName].connectTime = Date.now(); @@ -215,172 +240,148 @@ function addProxyMiner(miner) { proxyMiners[proxyMinerName].hashes = 0; console.log("Starting to calculate high diff for " + proxyMinerName + " proxy"); } else { - ++ proxyMiners[proxyMinerName].count; + if (++ proxyMiners[proxyMinerName].count > global.config.pool.workerMax && !miner.xmrig_proxy) { + console.error(threadName + "Starting to long ban " + wallet + " miner address"); + bannedBigTmpWallets[wallet] = 1; + for (var [minerId2, miner2] of activeMiners) if (miner2.payout === wallet) removeMiner(miner2); + return false; + } } + return true; } function removeMiner(miner) { - process.send({type: 'removeMiner', data: miner.port}); - let proxyMinerName = miner.payout + ":" + miner.identifier; - if (proxyMinerName in proxyMiners && --proxyMiners[proxyMinerName].count <= 0) delete proxyMiners[proxyMinerName]; + if (!miner || miner.removed_miner) return; + const proxyMinerName = miner.proxyMinerName; + if (proxyMinerName && proxyMinerName in proxyMiners && --proxyMiners[proxyMinerName].count <= 0) delete proxyMiners[proxyMinerName]; if (miner.payout in minerWallets && --minerWallets[miner.payout].count <= 0) delete minerWallets[miner.payout]; - if (miner.algos) { - delete activeSmartMiners[miner.id]; - } else { - delete activeMiners[miner.id]; - } + activeMiners.delete(miner.id); + miner.removed_miner = true; } function checkAliveMiners() { debug(threadName + "Verifying if miners are still alive"); const time_before = Date.now(); const deadline = time_before - global.config.pool.minerTimeout * 1000; - let miner_count = 0; - for (let minerId in activeSmartMiners) { - if (activeSmartMiners.hasOwnProperty(minerId)) { - let miner = activeSmartMiners[minerId]; - if (miner.lastContact < deadline) removeMiner(miner); - } - ++ miner_count; - } - for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - let miner = activeMiners[minerId]; - if (miner.lastContact < deadline) removeMiner(miner); - } - ++ miner_count; - } + for (var [minerId, miner] of activeMiners) if (miner.lastContact < deadline) removeMiner(miner); const elapsed = Date.now() - time_before; - if (elapsed > 500) console.error("checkAliveMiners() consumed " + elapsed + " ms for " + miner_count + " miners"); + if (elapsed > 50) console.error(threadName + "checkAliveMiners() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); } -// global.config.daemon["activePort" + algo] is only updated in master thread -function updateActivePort(algo) { - global.support.getActivePort(algo, function (newActivePort) { - const oldActivePort = global.config.daemon["activePort" + algo]; - if (newActivePort === null) { - if (algo === "" && oldActivePort != global.config.daemon.port) { - console.error("Error getting activePort, so rolling back to main port"); - global.config.daemon.activePort = global.config.daemon.port; - } else { - console.error("Error getting " + "activePort" + algo); - global.config.daemon["algoHashFactor" + algo] = 0.0; - } +// coin hash factor is only updated in master thread +function updateCoinHashFactor(coin) { + global.support.getCoinHashFactor(coin, function (coinHashFactor) { + if (coinHashFactor === null) { + console.error("Error getting coinHashFactor for " + coin + " coin"); + coinHashFactorUpdate(coin, newCoinHashFactor[coin] = 0); + } else if (!coinHashFactor) { + coinHashFactorUpdate(coin, newCoinHashFactor[coin] = 0); } else { - if (algo !== "") { - global.support.getAlgoHashFactor(algo, function (newAlgoHashFactor) { - if (newAlgoHashFactor === null) { - console.error("Error getting " + "algoHashFactor" + algo); - global.config.daemon["algoHashFactor" + algo] = 0.0; - } else { - if (newAlgoHashFactor == 0) debug("Got zero " + "algoHashFactor" + algo); - global.config.daemon["algoHashFactor" + algo] = newAlgoHashFactor; - if (oldActivePort !== newActivePort) { - console.log("Changing " + "activePort" + algo + " from " + oldActivePort + " to " + newActivePort); - global.config.daemon["activePort" + algo] = newActivePort; - } - } - }); - } else if (oldActivePort !== newActivePort) { - if (!(newActivePort in lastPortErrorTime) || Date.now() - lastPortErrorTime[newActivePort] > 30*60*1000) { - console.log("Changing " + "activePort" + algo + " from " + oldActivePort + " to " + newActivePort); - global.config.daemon["activePort" + algo] = newActivePort; - } else if ((Date.now() - lastPortErrorTime[newActivePort]) % 60*1000 < 6*1000) { // print every 10th message - console.warn("Avoiding changing recently problem " + "activePort" + algo + " from " + oldActivePort + " to " + newActivePort); - } - } + newCoinHashFactor[coin] = coinHashFactor; } }); } +function process_rpc_template(rpc_template, coin, port, coinHashFactor, isHashFactorChange) { + let template = Object.assign({}, rpc_template); + + template.coin = coin; + template.port = parseInt(port); + template.coinHashFactor = coinHashFactor; + template.isHashFactorChange = isHashFactorChange; + + if (port in global.coinFuncs.getMM_PORTS()) { + const child_coin = global.coinFuncs.PORT2COIN(global.coinFuncs.getMM_PORTS()[port]); + if (child_coin in activeBlockTemplates) { + template.child_template = activeBlockTemplates[child_coin]; + template.child_template_buffer = template.child_template.buffer; + template.parent_blocktemplate_blob = global.coinFuncs.constructMMParentBlockBlob( + Buffer.from(rpc_template.blocktemplate_blob, 'hex'), port, template.child_template_buffer + ).toString('hex'); + } + } -function setProblemPort(port) { - console.warn("Returning to " + global.config.daemon.port + " port."); - lastPortErrorTime[port] = Date.now(); - global.config.daemon.activePort = global.config.daemon.port; - global.support.sendEmail(global.config.general.adminEmail, - "FYI: Block template request failed for " + port + " port.", - "On pool server " + global.config.hostname + " block template request failed for " + port + " port.\n" + - "Returning to " + global.config.daemon.port + " port." - ); + return template; } -// templateUpdateReal is only called in master thread (except the beginning of a worker thread) -function templateUpdateReal(algo, activePort, algoHashFactor) { - global.coinFuncs.getPortBlockTemplate(activePort, function (rpcResponse) { - if (activePort !== global.config.daemon["activePort" + algo]) { - console.log("Aborting " + activePort + " last block template request because " + "activePort" + algo + " was already changed to " + global.config.daemon["activePort" + algo] + " port"); +// templateUpdate3 is only called in master thread (except the beginning of a worker thread) +function templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_bt) { + const template = process_rpc_template(body_bt, coin, port, coinHashFactor, isHashFactorChange); + debug(threadName + "New block template found at " + template.height + " height"); + if (cluster.isMaster) { + sendToWorkers({type: 'newBlockTemplate', data: template}); + setNewBlockTemplate(template); + // update parent coins if current coin was updated now + if (port in global.coinFuncs.getMM_CHILD_PORTS()) { + const parent_ports = global.coinFuncs.getMM_CHILD_PORTS()[port]; + for (let parent_port in parent_ports) { + const parent_coin = global.coinFuncs.PORT2COIN(parent_port); + if (parent_coin in activeBlockTemplates) { + const parent_template = process_rpc_template(activeBlockTemplates[parent_coin], parent_coin, parent_port, lastCoinHashFactor[parent_coin], false); + sendToWorkers({type: 'newBlockTemplate', data: parent_template}); + setNewBlockTemplate(parent_template); + } + } + } + } else { + setNewBlockTemplate(template); + } +} + +// templateUpdate2 is only called in master thread (except the beginning of a worker thread) +function templateUpdate2(coin, port, coinHashFactor, isHashFactorChange, body_header) { + if (port == 8545) { + return templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_header); + } else global.coinFuncs.getPortBlockTemplate(port, function (body_bt) { + if (!newCoinHashFactor[coin]) { + console.log("Aborting " + port + " last block template request because " + coin + " already has zero hash factor"); return; } - if (rpcResponse && typeof rpcResponse.result !== 'undefined') { - rpcResponse = rpcResponse.result; - rpcResponse.algo = algo; - rpcResponse.port = activePort; - rpcResponse.algoHashFactor = algoHashFactor; - debug(threadName + "New block template found at " + rpcResponse.height + " height"); - if (cluster.isMaster) { - sendToWorkers({type: 'newBlockTemplate', data: rpcResponse}); - setNewBlockTemplate(rpcResponse); - } else { - setNewBlockTemplate(rpcResponse); - } - } else { - console.error("Block template request failed for " + activePort + " port."); - if (algo === "") { - if (activePort != global.config.daemon.port) setProblemPort(activePort); - } else { - algoHashFactorUpdate(algo, 0); - } - setTimeout(templateUpdateReal, 3000, algo, activePort); + if (!body_bt) { + console.error("Block template request failed for " + port + " port"); + coinHashFactorUpdate(coin, 0); + return setTimeout(templateUpdate2, 3000, coin, port, coinHashFactor, isHashFactorChange); } + return templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_bt); }); } -function algoHashFactorUpdate(algo, algoHashFactor) { - if (algo === "") return; +function coinHashFactorUpdate(coin, coinHashFactor) { + if (coin === "") return; + if (coinHashFactor === 0 && lastCoinHashFactor[coin] === 0) return; if (cluster.isMaster) { - let data = { algo: algo, algoHashFactor: algoHashFactor }; - sendToWorkers({type: 'newAlgoHashFactor', data: data}); - setNewAlgoHashFactor(algo, algoHashFactor); - console.log('[*] New ' + algo + ' algo hash factor is set to ' + algoHashFactor); - } else { - setNewAlgoHashFactor(algo, algoHashFactor); + //console.log('[*] New ' + coin + ' coin hash factor is set from ' + newCoinHashFactor[coin] + ' to ' + coinHashFactor); + let data = { coin: coin, coinHashFactor: coinHashFactor }; + sendToWorkers({type: 'newCoinHashFactor', data: data}); } + setNewCoinHashFactor(true, coin, coinHashFactor); } // templateUpdate is only called in master thread (except the beginning of a worker thread) -function templateUpdate(algo, repeating) { - let activePort = global.config.daemon["activePort" + algo]; - if (activePort) global.coinFuncs.getPortLastBlockHeader(activePort, function (err, body) { - if (activePort !== global.config.daemon["activePort" + algo]) { - console.log("Aborting " + activePort + " last block header request because " + "activePort" + algo + " was already changed to " + global.config.daemon["activePort" + algo] + " port"); - if (repeating === true) setTimeout(templateUpdate, 50, algo, repeating); - return; - } - if (err === null) { - const algoHashFactor = algo === "" ? 1.0 : global.config.daemon["algoHashFactor" + algo]; - if (!(algo in lastBlockHash) || body.hash !== lastBlockHash[algo]) { - lastBlockHash[algo] = body.hash; - lastAlgoHashFactor[algo] = algoHashFactor; - templateUpdateReal(algo, activePort, algoHashFactor); - } else if ( !(algo in lastAlgoHashFactor) || (algoHashFactor == 0 && lastAlgoHashFactor[algo] != 0) || - (algoHashFactor != 0 && Math.abs(lastAlgoHashFactor[algo] - algoHashFactor) / algoHashFactor > 0.05) - ) { - lastAlgoHashFactor[algo] = algoHashFactor; - algoHashFactorUpdate(algo, algoHashFactor); - } - if (repeating === true) setTimeout(templateUpdate, 50, algo, repeating); - } else { - console.error("Last block header request for " + global.config.daemon["activePort" + algo] + " port failed!"); - if (algo === "") { - if (activePort != global.config.daemon.port) setProblemPort(activePort); - } else { - algoHashFactorUpdate(algo, 0); +function templateUpdate(coin, repeating) { + const port = global.coinFuncs.COIN2PORT(coin); + const coinHashFactor = newCoinHashFactor[coin]; + if (coinHashFactor) global.coinFuncs.getPortLastBlockHeader(port, function (err, body) { + if (!newCoinHashFactor[coin]) { + console.log(threadName + "Aborting " + port + " last block header request because " + coin + " already has zero hash factor"); + if (repeating === true) setTimeout(templateUpdate, DAEMON_POLL_MS, coin, repeating); + } else if (err === null && body.hash) { + const isHashFactorChange = Math.abs(lastCoinHashFactor[coin] - coinHashFactor) / coinHashFactor > 0.05; + if (!(coin in lastBlockHash) || body.hash !== lastBlockHash[coin]) { + lastBlockHash[coin] = body.hash; + templateUpdate2(coin, port, coinHashFactor, isHashFactorChange, body); + } else if (isHashFactorChange) { + coinHashFactorUpdate(coin, coinHashFactor); } - setTimeout(templateUpdate, 1000, algo, repeating); + if (repeating === true) setTimeout(templateUpdate, DAEMON_POLL_MS, coin, repeating); + } else { + console.error(threadName + "Last block header request for " + port + " port failed!"); + coinHashFactorUpdate(coin, 0); + if (repeating !== false) setTimeout(templateUpdate, global.config.daemon.pollInterval, coin, repeating); } - }); else setTimeout(templateUpdate, 1000, algo, repeating); - + }); else if (cluster.isMaster) { + if (repeating !== false) setTimeout(templateUpdate, global.config.daemon.pollInterval, coin, repeating); + } } // main chain anchor block height for alt chain block @@ -390,7 +391,7 @@ let anchorBlockPrevHeight; // update main chain anchor block height for alt chain block // anchorBlockUpdate is only called in worker threads function anchorBlockUpdate() { - if (("" in activeBlockTemplate) && global.config.daemon.port == activeBlockTemplate[""].port) return; + if (("" in activeBlockTemplates) && global.config.daemon.port == activeBlockTemplates[""].port) return; // only need to do that separately if we mine alt chain global.coinFuncs.getLastBlockHeader(function (err, body) { if (err === null) { @@ -405,67 +406,135 @@ function anchorBlockUpdate() { }); } -function setNewAlgoHashFactor(algo, algoHashFactor, check_height) { - let miner_count = 0; +function getCoinJobParams(coin) { + let params = {}; + params.bt = activeBlockTemplates[coin]; + params.coinHashFactor = lastCoinHashFactorMM[coin]; + params.algo_name = global.coinFuncs.algoShortTypeStr(params.bt.port, params.bt.block_version); + //params.variant_name = params.algo_name.split('/')[1]; + return params; +}; + +function setNewCoinHashFactor(isHashFactorChange, coin, coinHashFactor, check_height) { + if (isHashFactorChange) lastCoinHashFactor[coin] = coinHashFactor; + const prevCoinHashFactorMM = lastCoinHashFactorMM[coin]; + lastCoinHashFactorMM[coin] = coinHashFactor; // used in miner.selectBestCoin + + const port = global.coinFuncs.COIN2PORT(coin); + const is_mm = port in global.coinFuncs.getMM_PORTS(); + if (is_mm) { + const child_coin = global.coinFuncs.PORT2COIN(global.coinFuncs.getMM_PORTS()[port]); + lastCoinHashFactorMM[coin] += lastCoinHashFactor[child_coin]; + } + + if (cluster.isMaster && coin !== "" && prevCoinHashFactorMM != lastCoinHashFactorMM[coin]) { + console.log('[*] New ' + coin + ' coin hash factor is set from ' + prevCoinHashFactorMM + ' to ' + coinHashFactor + (is_mm ? ' (MM: ' + lastCoinHashFactorMM[coin] + ')' : "")); + } + if (!(coin in activeBlockTemplates)) return; + + // update parent coins if current coin was updated now + if (isHashFactorChange) if (port in global.coinFuncs.getMM_CHILD_PORTS()) { + const parent_ports = global.coinFuncs.getMM_CHILD_PORTS()[port]; + for (let parent_port in parent_ports) { + const parent_coin = global.coinFuncs.PORT2COIN(parent_port); + setNewCoinHashFactor(true, parent_coin, lastCoinHashFactor[parent_coin], 0); + } + } + const time_before = Date.now(); + let strLogPrefix; + + if (isHashFactorChange) { + const port = activeBlockTemplates[coin].port; + const block_version = activeBlockTemplates[coin].block_version; + const algo = global.coinFuncs.algoShortTypeStr(port, block_version); + + strLogPrefix = "Full BT update for coin " + coin; + if (cluster.isMaster) console.log(threadName + strLogPrefix + " with hash factor changed to " + lastCoinHashFactorMM[coin]); + + if (check_height) { + for (var [minerId, miner] of activeMiners) { + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) continue; + miner.trust.check_height = check_height; + miner.sendBestCoinJob(); + } + } else { + for (var [minerId, miner] of activeMiners) { + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) continue; + miner.sendBestCoinJob(); + } + } - if (algo !== "") global.config.daemon["algoHashFactor" + algo] = algoHashFactor; // used in miner.selectBestAlgo + } else { - for (let minerId in activeSmartMiners) { - if (activeSmartMiners.hasOwnProperty(minerId)) { - let miner = activeSmartMiners[minerId]; - if (check_height) miner.trust.check_height = check_height; - miner.sendNewJob(); + strLogPrefix = "Fast BT update for coin " + coin; + if (cluster.isMaster) console.log(threadName + strLogPrefix + " with the same " + lastCoinHashFactorMM[coin] + " hash factor"); + + const params = getCoinJobParams(coin); + if (check_height) { + for (var [minerId, miner] of activeMiners) { + //if (typeof(miner.curr_coin) === 'undefined') console.error("[INTERNAL ERROR]: " + miner.logString + ": undefined curr_coin"); + if (miner.curr_coin !== coin) continue; + //if (!(coin in miner.coin_perf)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported coin " + coin + " in miner " + JSON.stringify(miner.coin_perf) + " coin_perf"); + //if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported algo " + algo + " in miner " + JSON.stringify(miner.algos) + " algos"); + miner.trust.check_height = check_height; + miner.sendCoinJob(coin, params); + } + } else { + for (var [minerId, miner] of activeMiners) { + //if (typeof(miner.curr_coin) === 'undefined') console.error("[INTERNAL ERROR]: " + miner.logString + ": undefined curr_coin"); + if (miner.curr_coin !== coin) continue; + //if (!(coin in miner.coin_perf)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported coin " + coin + " in miner " + JSON.stringify(miner.coin_perf) + " coin_perf"); + //if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported algo " + algo + " in miner " + JSON.stringify(miner.algos) + " algos"); + miner.sendCoinJob(coin, params); + } } - ++ miner_count; } const elapsed = Date.now() - time_before; - if (elapsed > 500) console.error("setNewAlgoHashFactor() consumed " + elapsed + " ms for " + miner_count + " miners"); + if (elapsed > 50) console.error(threadName + strLogPrefix + " setNewCoinHashFactor() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); } function setNewBlockTemplate(template) { - const algo = template.algo; + const coin = template.coin; let isExtraCheck = false; - if (algo in activeBlockTemplate) { - if (activeBlockTemplate[algo].previous_hash.toString('hex') === template.prev_hash) { - console.log(threadName + 'Ignoring duplicate block template update at height: ' + template.height + '. Difficulty: ' + template.difficulty); - return; + if (coin in activeBlockTemplates) { + if (activeBlockTemplates[coin].prev_hash === template.prev_hash) { + if ("child_template" in template) { + if ("child_template" in activeBlockTemplates[coin] && activeBlockTemplates[coin].child_template.prev_hash === template.child_template.prev_hash) { + console.log(threadName + 'Ignoring duplicate parent block template update at height: ' + template.height + '. Difficulty: ' + template.difficulty); + return; + } + } else { + console.log(threadName + 'Ignoring duplicate block template update at height: ' + template.height + '. Difficulty: ' + template.difficulty); + return; + } + } + if (coin in pastBlockTemplates) { + pastBlockTemplates[coin].get(0).timeoutTime = Date.now() + 4*1000; + } else { + pastBlockTemplates[coin] = global.support.circularBuffer(10); } - activeBlockTemplate[algo].timeOutdate = Date.now() + 4*1000; - pastBlockTemplates.enq(activeBlockTemplate[algo]); - if (activeBlockTemplate[algo].port != template.port && global.config.pool.trustedMiners) isExtraCheck = true; + pastBlockTemplates[coin].enq(activeBlockTemplates[coin]); + if (activeBlockTemplates[coin].port != template.port && global.config.pool.trustedMiners) isExtraCheck = true; } if (cluster.isMaster) { - const algo_str = algo === "" ? "" : algo + " "; - console.log('[*] New ' + algo_str + 'block to mine at ' + template.height + ' height with ' + template.difficulty + ' difficulty and ' + template.port + ' port (with algo hash factor ' + template.algoHashFactor + ")"); + const coin_str = coin === "" ? "" : coin + " "; + console.log('[*] New ' + coin_str + 'block to mine at ' + template.height + ' height with ' + template.difficulty + ' difficulty and ' + template.port + ' port (with coin hash factor ' + template.coinHashFactor + ")"); } else { debug(threadName + 'New block to mine at ' + template.height + ' height with ' + template.difficulty + ' difficulty and ' + template.port + ' port'); } - activeBlockTemplate[algo] = new BlockTemplate(template); - const height = activeBlockTemplate[algo].height; - - if (algo === "" && global.config.daemon.port == activeBlockTemplate[""].port) { - anchorBlockHeight = height; - } + activeBlockTemplates[coin] = new global.coinFuncs.BlockTemplate(template); + activeBlockTemplates[coin].timeCreated = Date.now(); - setNewAlgoHashFactor(algo, template.algoHashFactor, isExtraCheck ? height : 0); + const height = activeBlockTemplates[coin].height; - let miner_count = 0; - const time_before = Date.now(); - - if (algo === "") for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - let miner = activeMiners[minerId]; - if (isExtraCheck) miner.trust.check_height = height; - miner.sendNewJob(); - } - ++ miner_count; + if (coin === "" && global.config.daemon.port == activeBlockTemplates[""].port) { + anchorBlockHeight = height; } - const elapsed = Date.now() - time_before; - if (elapsed > 500) console.error("setNewBlockTemplate() consumed " + elapsed + " ms for " + miner_count + " miners"); + setNewCoinHashFactor(template.isHashFactorChange, coin, template.coinHashFactor, isExtraCheck ? height : 0); } // here we keep verified share number of a specific wallet (miner.payout) @@ -476,20 +545,35 @@ let walletTrust = {}; // wallet last seen time (all wallets that are not detected for more than 1 day are removed) let walletLastSeeTime = {}; +// miner agent strings (for process.env['WORKER_ID'] == 1) +let minerAgents = {}; + var reEmail = /^\S+@\S+\.\S+$/; // wallet password last check time let walletLastCheckTime = {}; -function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVersion, portType, port, agent, algos, algos_perf) { +let wallet_debug = {}; + +function getTargetHex(diff, size) { + return pad_hex(baseDiff.div(diff).toBuffer({endian: 'little', size: size}).toString('hex'), size); +}; + +function getRavenTargetHex(diff) { + return pad_hex((baseRavenDiff / diff).toString(16), 32); +}; + +function Miner(id, login, pass, rigid, ipAddress, startingDiff, pushMessage, protoVersion, portType, port, agent, algos, algos_perf, algo_min_time) { // Username Layout -
. // Password Layout - .. // Default function is to use the password so they can login. Identifiers can be unique, payment ID is last. // If there is no miner identifier, then the miner identifier is set to the password // If the password is x, aka, old-logins, we're not going to allow detailed review of miners. - let diffSplit = login.split("+"); - let addressSplit = diffSplit[0].split('.'); - let pass_split = pass.split(":"); + const login_diff_split = login.split("+"); + const login_div_split = login_diff_split[0].split("%"); + const login_paymentid_split = login_div_split[0].split("."); + const pass_algo_split = pass.split("~"); + let pass_split = pass_algo_split[0].split(":"); // Workaround for a common mistake to put email without : before it // and also security measure to hide emails used as worker names @@ -500,42 +584,125 @@ function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVer // 1) set payout, identifier, email and logString - this.payout = this.address = addressSplit[0]; + this.payout = this.address = login_paymentid_split[0]; this.paymentID = null; - this.identifier = agent && agent.includes('MinerGate') ? "MinerGate" : pass_split[0].substring(0, 64); - if (typeof(addressSplit[1]) !== 'undefined') { - if (addressSplit[1].length === 64 && hexMatch.test(addressSplit[1]) && global.coinFuncs.validatePlainAddress(this.address)) { - this.paymentID = addressSplit[1]; + this.identifier = agent && agent.includes('MinerGate') ? "MinerGate" : (rigid ? rigid : pass_split[0]).substring(0, 64); + if (typeof(login_paymentid_split[1]) !== 'undefined') { + if (login_paymentid_split[1].length === 64 && hexMatch.test(login_paymentid_split[1]) && global.coinFuncs.validatePlainAddress(this.address)) { + this.paymentID = login_paymentid_split[1]; this.payout += "." + this.paymentID; - if (typeof(addressSplit[2]) !== 'undefined' && this.identifier === 'x') { - this.identifier = addressSplit[2].substring(0, 64); + if (typeof(login_paymentid_split[2]) !== 'undefined' && this.identifier === 'x') { + this.identifier = login_paymentid_split[2].substring(0, 64); } } else if (this.identifier === 'x') { - this.identifier = addressSplit[1].substring(0, 64); + this.identifier = login_paymentid_split[1].substring(0, 64); } } - this.email = pass_split.length === 2 ? pass_split[1] : ""; - this.logString = this.payout.substr(this.payout.length - 10) + ":" + this.identifier + " (" + ipAddress + ")"; + this.debugMiner = this.payout in wallet_debug; + this.email = pass_split.length === 2 ? pass_split[1] : ""; + this.logString = this.payout.substr(this.payout.length - 10) + ":" + this.identifier + " (" + ipAddress + ")"; + this.agent = agent; // 2) check stuff - if (diffSplit.length > 2) { - this.error = "Too many options in the login field"; + if (login_diff_split.length > 2) { + this.error = "Please use monero_address[.payment_id][+difficulty_number] login/user format"; this.valid_miner = false; return; } + if (Math.abs(login_div_split.length % 2) == 0 || login_div_split.length > 5) { + this.error = "Please use monero_address[.payment_id][%N%95_char_long_monero_wallet_address]+[+difficulty_number] login/user format"; + this.valid_miner = false; + return; + } + + this.payout_div = {}; + + let payout_percent_left = 100; + for (let index = 1; index < login_div_split.length - 1; index += 2) { + const percent = parseFloat(login_div_split[index]); + if (isNaN(percent) || percent < 0.1) { + this.error = "Your payment divide split " + percent + " is below 0.1% and can't be processed"; + this.valid_miner = false; + return; + } + if (percent > 99.9) { + this.error = "Your payment divide split " + percent + " is above 99.9% and can't be processed"; + this.valid_miner = false; + return; + } + payout_percent_left -= percent; + if (payout_percent_left < 0.1) { + this.error = "Your summary payment divide split exceeds 99.9% and can't be processed"; + this.valid_miner = false; + return; + } + const address = login_div_split[index + 1]; + if (address.length != 95 || !global.coinFuncs.validateAddress(address)) { + this.error = "Invalid payment address provided: " + address + ". Please use 95_char_long_monero_wallet_address format"; + this.valid_miner = false; + return; + } + if (address in bannedAddresses) { // Banned Address + this.error = "Permanently banned payment address " + address + " provided: " + bannedAddresses[address]; + this.valid_miner = false; + return; + } + if (address in bannedTmpWallets) { + this.error = "Temporary (10 minutes max) banned payment address " + address; + this.valid_miner = false; + return; + } + if (address in bannedBigTmpWallets) { + this.error = "Temporary (one hour max) ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)"; + this.valid_miner = false; + this.delay_reply = 600; + return; + } + if (address in this.payout_div) { + this.error = "You can't repeat payment split address " + address; + this.valid_miner = false; + return; + } + this.payout_div[address] = percent; + } + + if (payout_percent_left === 100) { + this.payout_div = null; + } else { + if (this.payout in this.payout_div) { + this.error = "You can't repeat payment split address " + this.payout; + this.valid_miner = false; + return; + } + this.payout_div[this.payout] = payout_percent_left; + } + if (pass_split.length > 2) { - this.error = "Too many options in the password field"; + this.error = "Please use worker_name[:email] password format"; this.valid_miner = false; return; } if (this.payout in bannedAddresses) { // Banned Address - this.error = "Banned payment address provided: " + bannedAddresses[this.payout]; + this.error = "Permanently banned payment address " + this.payout + " provided: " + bannedAddresses[this.payout]; + this.valid_miner = false; + return; + } + + if (this.payout in bannedTmpWallets) { + this.error = "Temporary (10 minutes max) banned payment address " + this.payout; + this.valid_miner = false; + return; + } + + if (this.payout in bannedBigTmpWallets) { + this.error = "Temporary (one hour max) ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)"; this.valid_miner = false; + this.delay_reply = 600; return; } @@ -547,71 +714,83 @@ function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVer if (global.coinFuncs.validateAddress(this.address)) { this.bitcoin = 0; - } else if (btcValidator.validate(this.address)) { - if (global.config.general.allowBitcoin && global.coinFuncs.supportsAutoExchange) { - this.bitcoin = 1; - } else { - this.error = "This pool does not allow payouts to bitcoin"; - this.valid_miner = false; - return; - } + } else if (global.config.general.allowBitcoin && global.coinFuncs.supportsAutoExchange && btcValidator.validate(this.address)) { + this.bitcoin = 1; } else { - this.error = "Invalid payment address provided: " + this.address; + this.error = "Invalid payment address provided: " + this.address + ". Please use 95_char_long_monero_wallet_address format"; this.valid_miner = false; return; } - if (!("" in activeBlockTemplate)) { + if (!("" in activeBlockTemplates)) { this.error = "No active block template"; this.valid_miner = false; return; } - this.setAlgos = function(algos, algos_perf) { - if (algos && (algos instanceof Array) && (algos.includes("cn/1") || algos.includes("cryptonight/1"))) { - this.algos = {}; - for (let i in algos) this.algos[algos[i]] = 1; - } else { - return "algo array should include cn/1 or cryptonight/1"; + this.setAlgos = function(algos, algos_perf, algo_min_time) { + this.algos = {}; + for (let i in algos) this.algos[algos[i]] = 1; + if (global.coinFuncs.is_miner_agent_no_haven_support(this.agent)) delete this.algos["cn-heavy/xhv"]; + const check = global.coinFuncs.algoCheck(this.algos); + if (check !== true) return check; + if ("cn-pico" in this.algos) this.algos["cn-pico/trtl"] = 1; + + if (!(algos_perf && algos_perf instanceof Object)) { + if (global.coinFuncs.algoMainCheck(this.algos)) algos_perf = global.coinFuncs.getDefaultAlgosPerf(); + else algos_perf = global.coinFuncs.getPrevAlgosPerf(); } - if (algos_perf && (algos_perf instanceof Object)) { - this.algos_perf = {}; - for (let algo in algos_perf) { - if (algo.includes("heavy")) this.algos_perf["Heavy"] = algos_perf[algo]; - else if (algo.includes("lite")) this.algos_perf["Light"] = algos_perf[algo]; - else if (algo.includes("fast")) this.algos_perf["Fast"] = algos_perf[algo]; - else this.algos_perf[""] = this.algos_perf["XTL"] = algos_perf[algo]; - } - if (!this.algos_perf[""]) return "algo_perf set should include non heavy/lite/fast hashrate"; + + let coin_perf = global.coinFuncs.convertAlgosToCoinPerf(algos_perf); + if (coin_perf instanceof Object) { + if (!("" in coin_perf && global.coinFuncs.algoMainCheck(this.algos))) coin_perf[""] = -1; + this.coin_perf = coin_perf; } else { - return "algo_perf set should be present"; + return coin_perf; } + this.algo_min_time = algo_min_time ? algo_min_time : 0; return ""; }; - if (algos) { - if (!algos_perf) algos_perf = { "cn/1": 1 }; - //console.error(JSON.stringify(algos)); - //console.error(JSON.stringify(algos_perf)); - const status = this.setAlgos(algos, algos_perf); - if (status != "") { - this.error = status; - this.valid_miner = false; - return; - } + if (pass_algo_split.length == 2) { + const algo_name = pass_algo_split[1]; + algos = [ algo_name ]; + algos_perf = {}; + algos_perf[algo_name] = 1; + algo_min_time = 0; + + } else if (!(algos && algos instanceof Array && global.config.daemon.enableAlgoSwitching)) { + const agent_algo = global.coinFuncs.get_miner_agent_not_supported_algo(agent); + if (agent_algo) { + algos = [ agent_algo ]; + } else { + algos = global.coinFuncs.getDefaultAlgos(); + } + algos_perf = global.coinFuncs.getDefaultAlgosPerf(); + algo_min_time = 0; } + const status = this.setAlgos(algos, algos_perf, algo_min_time); + if (status != "") { + this.error = status; + this.valid_miner = false; + return; + } + + // 3) setup valid miner stuff // 3a) misc stuff this.error = ""; this.valid_miner = true; + this.removed_miner = false; this.proxy = agent && agent.includes('xmr-node-proxy'); + this.xmrig_proxy = agent && agent.includes('xmrig-proxy'); this.id = id; this.ipAddress = ipAddress; - this.messageSender = messageSender; + this.pushMessage = pushMessage; this.connectTime = Date.now(); this.heartbeat = function () { this.lastContact = Date.now(); }; this.heartbeat(); @@ -629,31 +808,15 @@ function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVer this.poolTypeEnum = global.protos.POOLTYPE.PPLNS; } - // 3c) diff stuff + this.wallet_key = this.payout + " " + this.bitcoin + " " + this.poolTypeEnum + " " + JSON.stringify(this.payout_div) + " "; + + // 3c) diff calc stuff this.lastShareTime = Math.floor(Date.now() / 1000); this.validShares = 0; this.invalidShares = 0; this.hashes = 0; - this.fixed_diff = false; - this.difficulty = startingDiff; - - if (agent && agent.includes('NiceHash')) { - this.fixed_diff = true; - this.difficulty = global.coinFuncs.niceHashDiff; - } - if (diffSplit.length === 2) { - this.fixed_diff = true; - this.difficulty = Number(diffSplit[1]); - if (this.difficulty < global.config.pool.minDifficulty) { - this.difficulty = global.config.pool.minDifficulty; - } - if (this.difficulty > global.config.pool.maxDifficulty) { - this.difficulty = global.config.pool.maxDifficulty; - } - } - // 3d) trust stuff if (global.config.pool.trustedMiners) { @@ -661,18 +824,15 @@ function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVer walletTrust[this.payout] = 0; walletLastSeeTime[this.payout] = Date.now(); } - let is_trusted_wallet = this.difficulty <= 16001 && this.payout in walletTrust && walletTrust[this.payout] > global.config.pool.trustThreshold * 20; this.trust = { - threshold: is_trusted_wallet ? 1 : global.config.pool.trustThreshold, - probability: is_trusted_wallet ? global.config.pool.trustMin : 256, - penalty: 0, + trust: 0, check_height: 0 }; } // 3e) password setup stuff - let email = this.email; + let email = this.email.trim(); if (email != "") { // Need to do an initial registration call here. Might as well do it right... let payoutAddress = this.payout; @@ -691,255 +851,468 @@ function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVer this.validJobs = global.support.circularBuffer(10); this.cachedJob = null; - this.invalidShareProto = global.protos.InvalidShare.encode({ - paymentAddress: this.address, - paymentID: this.paymentID, - identifier: this.identifier - }); + this.storeInvalidShare = function() { + const time_now = Date.now(); + if (this.invalidShareCount) ++ this.invalidShareCount; + else this.invalidShareCount = 1; + if (!this.lastInvalidShareTime || time_now - this.lastInvalidShareTime > 10*60*1000) { + let _this = this; + global.database.storeInvalidShare(global.protos.InvalidShare.encode({ + paymentAddress: _this.address, + paymentID: _this.paymentID, + identifier: _this.identifier, + count: _this.invalidShareCount + })); + this.lastInvalidShareTime = time_now; + this.invalidShareCount = 0; + } + }; + + this.setNewDiff = function (difficulty) { + if (this.fixed_diff) return false; + const newDiff = difficulty; + this.newDiffRecommendation = newDiff; + const ratio = Math.abs(newDiff - this.difficulty) / this.difficulty; + if (ratio < 0.2) return false; + this.newDiffToSet = newDiff; + + debug(threadName + "Difficulty change to: " + this.newDiffToSet + " For: " + this.logString); + if (this.hashes > 0) { + debug(threadName + "Hashes: " + this.hashes + " in: " + Math.floor((Date.now() - this.connectTime) / 1000) + " seconds gives: " + + Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) + " hashes/second or: " + + Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) * global.config.pool.targetTime + " difficulty versus: " + this.newDiffToSet); + } + return true; + }; - this.selectBestAlgo = function() { - if (!this.algos) return ""; - if (typeof(this.curr_algo) !== 'undefined' && this.curr_algo_time && this.algos_perf[this.curr_algo] && Date.now() - this.curr_algo_time < 5*60*1000) { - return this.curr_algo; + this.selectBestCoin = function() { + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] current coin is " + this.curr_coin); + if (typeof(this.curr_coin) !== 'undefined' && this.curr_coin_time && lastCoinHashFactorMM[this.curr_coin] && + Date.now() - this.curr_coin_time < this.algo_min_time*1000 + ) { + return this.curr_coin; } - let best_algo = ""; - let best_algo_perf = this.algos_perf[""]; + let best_coin = ""; + let best_coin_perf = this.coin_perf[""] * 1.1; let miner = this; - ALGOS.forEach(function(algo) { - if (!(algo in activeBlockTemplate)) return; - const port = activeBlockTemplate[algo].port; - if (!(global.coinFuncs.algoTypeStr(port) in miner.algos) && !(global.coinFuncs.algoShortTypeStr(port) in miner.algos)) return; - const algoHashFactor = global.config.daemon["algoHashFactor" + algo]; - if (algo in miner.algos_perf && miner.algos_perf[algo] * algoHashFactor > best_algo_perf) { - debug(miner.logString + ": " + algo + ": " + miner.algos_perf[algo] * algoHashFactor); - best_algo = algo; - best_algo_perf = miner.algos_perf[algo] * algoHashFactor; + COINS.forEach(function(coin) { + if (!(coin in miner.coin_perf)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no coin_perf"); + return; + } + if (!(coin in activeBlockTemplates)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no activeBlockTemplates"); + return; + } + const coinHashFactor = lastCoinHashFactorMM[coin]; + if (!coinHashFactor) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no coinHashFactor"); + return; + } + const bt = activeBlockTemplates[coin]; + const port = bt.port; + const block_version = bt.block_version; + const algo = global.coinFuncs.algoShortTypeStr(port, block_version); + + if (miner.difficulty / coinHashFactor > bt.difficulty * 3) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] Rejected best " + coin + " coin due to high diff " + miner.difficulty + " " + coinHashFactor + " " + bt.difficulty); + return; + } + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no algo support"); + return; + } + let coin_perf = miner.coin_perf[coin] * coinHashFactor; + if (miner.curr_coin === coin) coin_perf *= 1.05; + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": " + coin_perf); + if (coin_perf > best_coin_perf) { + best_coin = coin; + best_coin_perf = coin_perf; } }); - if (typeof(this.curr_algo) === 'undefined' || this.curr_algo != best_algo) { - this.curr_algo = best_algo; - this.curr_algo_time = Date.now(); + if (best_coin_perf < 0) return false; + if (typeof(this.curr_coin) === 'undefined' || this.curr_coin != best_coin) { + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(best_coin)); + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + this.curr_coin_min_diff = 1; + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + this.curr_coin_min_diff = 0.01; + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + this.curr_coin_min_diff = 0.01 * 0x100000000; + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + this.curr_coin_min_diff = 0.01 * 0x100000000; + } else { + this.curr_coin_min_diff = global.config.pool.minDifficulty; + } + this.curr_coin = best_coin; + this.curr_coin_hash_factor = lastCoinHashFactorMM[best_coin]; + this.curr_coin_time = Date.now(); + if (global.config.pool.trustedMiners) this.trust.check_height = activeBlockTemplates[best_coin].height; + } + return best_coin; + }; + + // 3e) set diff stuff + + this.fixed_diff = false; + this.difficulty = startingDiff; + + if (login_diff_split.length === 2) { + this.fixed_diff = true; + this.difficulty = Number(login_diff_split[1]); + if (this.difficulty < global.config.pool.minDifficulty) { + this.difficulty = global.config.pool.minDifficulty; + } + if (this.difficulty > global.config.pool.maxDifficulty) { + this.difficulty = global.config.pool.maxDifficulty; } - return best_algo; + } + + this.curr_coin_hash_factor = 1; + this.curr_coin_min_diff = global.config.pool.minDifficulty; + this.curr_coin = this.selectBestCoin(); + + if (agent && agent.includes('NiceHash')) { + this.fixed_diff = true; + let minNiceHashDiff; + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(this.curr_coin)); + if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { + minNiceHashDiff = global.coinFuncs.niceHashDiff * 50; + } else { + minNiceHashDiff = global.coinFuncs.niceHashDiff; + } + + if (this.difficulty < minNiceHashDiff) this.difficulty = minNiceHashDiff; } this.calcNewDiff = function () { - const proxyMinerName = this.payout + ":" + this.identifier; let miner; let target; let min_diff; - if (proxyMinerName in proxyMiners) { - miner = proxyMiners[proxyMinerName]; - target = 5; - min_diff = 10*global.config.pool.minDifficulty; - } else if (this.payout in minerWallets && minerWallets[this.payout].last_ver_shares >= MAX_VER_SHARES_PER_SEC * VER_SHARES_PERIOD) { + let history_time; + const time_now = Date.now(); + const proxyMinerName = this.payout; // + ":" + this.identifier; + let proxyMiner = proxyMiners[proxyMinerName]; + if (proxyMiner && proxyMiner.hashes / (time_now - proxyMiner.connectTime) > this.difficulty) { + miner = proxyMiner; + target = 15; + min_diff = 10 * global.config.pool.minDifficulty; + history_time = 5; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc proxy miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); + } else if (this.payout in minerWallets && minerWallets[this.payout].last_ver_shares >= global.config.pool.minerThrottleSharePerSec * global.config.pool.minerThrottleShareWindow) { miner = minerWallets[this.payout]; - target = 5; - min_diff = 10*global.config.pool.minDifficulty; + target = 15; + min_diff = 10 * global.config.pool.minDifficulty; + history_time = 5; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc throttled miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); } else { miner = this; - target = this.proxy ? 5 : global.config.pool.targetTime; - min_diff = this.proxy ? 10*global.config.pool.minDifficulty : global.config.pool.minDifficulty; + target = this.proxy ? 15 : global.config.pool.targetTime; + min_diff = this.proxy ? 10 * global.config.pool.minDifficulty : global.config.pool.minDifficulty; + history_time = 60; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); } if (miner.connectTimeShift) { - if (Date.now() - miner.connectTimeShift > 60*60*1000) { + const timeSinceLastShift = time_now - miner.connectTimeShift; + const timeWindow = history_time * 60 * 1000; + if (timeSinceLastShift > timeWindow) { + if (timeSinceLastShift > 2 * timeWindow) { // forget all + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] forget diff"); + miner.hashes = 0; + } else { + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] diff window shift from " + miner.connectTimeShift + " and " + miner.hashesShift + " hashes"); + miner.hashes -= miner.hashesShift; + } miner.connectTime = miner.connectTimeShift; - miner.hashes -= miner.hashesShift; - miner.connectTimeShift = Date.now(); + miner.connectTimeShift = time_now; miner.hashesShift = miner.hashes; } } else { - miner.connectTimeShift = Date.now(); + miner.connectTimeShift = time_now; miner.hashesShift = miner.hashes; } let hashes = miner.hashes; - let period = (Date.now() - miner.connectTime) / 1000; + let period = (time_now - miner.connectTime) / 1000; if (hashes === 0) { hashes = this.difficulty; target = 2 * global.config.pool.retargetTime; if (period < target) period = target; } - const diff = Math.floor(hashes * target / period); + const diff = hashes * target / period; return diff < min_diff ? min_diff : diff; }; - this.updateDifficulty = function () { - if (this.fixed_diff) return; - if (this.setNewDiff(this.calcNewDiff())) this.sendNewJob(); - }; - - this.setNewDiff = function (difficulty) { - if (this.fixed_diff) return false; - - this.newDiff = Math.round(difficulty); - if (this.newDiff > global.config.pool.maxDifficulty && !this.proxy) { - this.newDiff = global.config.pool.maxDifficulty; - } - if (this.newDiff < global.config.pool.minDifficulty) { - this.newDiff = global.config.pool.minDifficulty; - } - - const ratio = Math.abs(this.newDiff - this.difficulty) / this.difficulty; - if (ratio < 0.05) return false; - - debug(threadName + "Difficulty change to: " + this.newDiff + " For: " + this.logString); - if (this.hashes > 0) { - debug(threadName + "Hashes: " + this.hashes + " in: " + Math.floor((Date.now() - this.connectTime) / 1000) + " seconds gives: " + - Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) + " hashes/second or: " + - Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) * global.config.pool.targetTime + " difficulty versus: " + this.newDiff); - } - return true; - }; - this.checkBan = function (validShare) { - if (!global.config.pool.banEnabled) { - return; - } + if (!global.config.pool.banEnabled) return; // Valid stats are stored by the pool. if (validShare) { - ++ this.validShares; + ++ this.validShares; } else { - ++ this.invalidShares; + ++ this.invalidShares; + if (this.validShares === 0) { + console.error(threadName + "Suspended miner IP for submitting bad share with zero trust " + this.logString); + removeMiner(this); + process.send({type: 'banIP', data: this.ipAddress, wallet: this.payout}); + return; + } } - if (this.validShares + this.invalidShares >= global.config.pool.banThreshold) { - if (this.invalidShares / this.validShares >= global.config.pool.banPercent / 100) { + + const shareCount = this.validShares + this.invalidShares; + if (shareCount >= global.config.pool.banThreshold) { + if (100 * this.invalidShares / shareCount >= global.config.pool.banPercent) { + console.error(threadName + "Suspended miner IP for submitting too many bad shares recently " + this.logString); removeMiner(this); - process.send({type: 'banIP', data: this.ipAddress}); + process.send({type: 'banIP', data: this.ipAddress, wallet: this.payout}); } else { this.invalidShares = 0; - this.validShares = 0; + this.validShares = 0; } } }; if (protoVersion === 1) { - this.getTargetHex = function () { - let padded = new Buffer(32); - padded.fill(0); - let diffBuff = baseDiff.div(this.difficulty).toBuffer(); - diffBuff.copy(padded, 32 - diffBuff.length); - - let buff = padded.slice(0, 4); - let buffArray = buff.toByteArray().reverse(); - let buffReversed = new Buffer(buffArray); - this.target = buffReversed.readUInt32BE(0); - return buffReversed.toString('hex'); - }; - this.getJob = function () { - const algo = this.selectBestAlgo(); - let bt = activeBlockTemplate[algo]; - if (this.jobLastBlockHash === bt.idHash && !this.newDiff && this.cachedJob !== null) return null; + this.getCoinJob = function (coin, params) { + const bt = params.bt; + if (this.jobLastBlockHash === bt.idHash && !this.newDiffToSet && this.cachedJob !== null) return null; this.jobLastBlockHash = bt.idHash; - if (this.newDiff) { - this.difficulty = this.newDiff; - this.newDiff = null; - } - const algoHashFactor = algo === "" ? 1.0 : global.config.daemon["algoHashFactor" + algo]; - if (!this.proxy) { - let blob = bt.nextBlob(); - let target = this.getTargetHex(); - let newJob = { - id: crypto.pseudoRandomBytes(21).toString('base64'), - algo_type: algo, - blockHash: bt.idHash, - extraNonce: bt.extraNonce, - height: bt.height, - difficulty: this.difficulty, - diffHex: this.diffHex, - algoHashFactor: algoHashFactor, - submissions: {} + + if (this.newDiffToSet) { + this.difficulty = this.newDiffToSet; + this.newDiffToSet = null; + this.newDiffRecommendation = null; + } else if (this.newDiffRecommendation) { + this.difficulty = this.newDiffRecommendation; + this.newDiffRecommendation = null; + } + + let coin_diff = this.difficulty / this.curr_coin_hash_factor; + if (coin_diff < this.curr_coin_min_diff) coin_diff = this.curr_coin_min_diff; + if (coin_diff > bt.difficulty) coin_diff = bt.difficulty; + + const blob_type_num = global.coinFuncs.portBlobType(bt.port); + const isEth = global.coinFuncs.blobTypeEth(blob_type_num); + const isErg = global.coinFuncs.blobTypeErg(blob_type_num); + const isExtraNonceBT = isEth || isErg; + + if (!this.proxy || isExtraNonceBT) { + const blob_hex = bt.nextBlobHex(); + if (!blob_hex) return null; + const isGrin = global.coinFuncs.blobTypeGrin(blob_type_num); + const isRvn = global.coinFuncs.blobTypeRvn(blob_type_num); + const newJob = { + id: isRvn ? get_new_eth_job_id() : get_new_id(), + coin: coin, + blob_type_num: blob_type_num, + blockHash: bt.idHash, + extraNonce: isExtraNonceBT ? this.eth_extranonce : bt.extraNonce, + height: bt.height, + seed_hash: bt.seed_hash, + difficulty: coin_diff, + norm_diff: coin_diff * this.curr_coin_hash_factor, + coinHashFactor: params.coinHashFactor, + submissions: {} }; this.validJobs.enq(newJob); - this.cachedJob = { - blob: blob, - algo: global.coinFuncs.algoShortTypeStr(bt.port), - variant: global.coinFuncs.variantValue(bt.port), - job_id: newJob.id, - target: target, - id: this.id + if (isGrin) this.cachedJob = { + pre_pow: blob_hex, + algo: this.protocol === "grin" ? "cuckaroo" : params.algo_name, + edgebits: 29, + proofsize: global.coinFuncs.c29ProofSize(blob_type_num), + noncebytes: 4, + height: bt.height, + job_id: newJob.id, + difficulty: coin_diff, + id: this.id + }; else if (isRvn) this.cachedJob = [ + newJob.id, + blob_hex, + bt.seed_hash, + getRavenTargetHex(coin_diff), + true, + bt.height, + bt.bits + ]; else if (isEth) this.cachedJob = [ + newJob.id, + bt.seed_hash, + blob_hex, + true, + coin_diff // this will be popped and used for separate mining.set_difficulty message + ]; else if (isErg) this.cachedJob = [ + newJob.id, + bt.height, + bt.hash, + "", + "", + 2, // curl http://localhost:9053/info: parameters.blockVersion + baseDiff.div(coin_diff).toString(), + "", + true + ]; else this.cachedJob = { + blob: blob_hex, + algo: params.algo_name, + height: bt.height, + seed_hash: bt.seed_hash, + job_id: newJob.id, + target: getTargetHex(coin_diff, global.coinFuncs.nonceSize(blob_type_num)), + id: this.id }; } else { - let blob = bt.nextBlobWithChildNonce(); - let newJob = { - id: crypto.pseudoRandomBytes(21).toString('base64'), - algo_type: algo, - blockHash: bt.idHash, - extraNonce: bt.extraNonce, - height: bt.height, - difficulty: this.difficulty, - diffHex: this.diffHex, - clientPoolLocation: bt.clientPoolLocation, + const blob_hex = bt.nextBlobWithChildNonceHex(); + const newJob = { + id: get_new_id(), + coin: coin, + blob_type_num: blob_type_num, + blockHash: bt.idHash, + extraNonce: bt.extraNonce, + height: bt.height, + seed_hash: bt.seed_hash, + difficulty: coin_diff, + norm_diff: coin_diff * this.curr_coin_hash_factor, + clientPoolLocation: bt.clientPoolLocation, clientNonceLocation: bt.clientNonceLocation, - algoHashFactor: algoHashFactor, - submissions: {} + coinHashFactor: params.coinHashFactor, + submissions: {} }; this.validJobs.enq(newJob); this.cachedJob = { - blocktemplate_blob: blob, - blob_type: global.coinFuncs.blobTypeStr(bt.port), - algo: global.coinFuncs.algoShortTypeStr(bt.port), - variant: global.coinFuncs.variantValue(bt.port), - difficulty: bt.difficulty, - height: bt.height, - reserved_offset: bt.reserveOffset, + blocktemplate_blob: blob_hex, + blob_type: global.coinFuncs.blobTypeStr(bt.port, bt.block_version), + algo: params.algo_name, + difficulty: bt.difficulty, + height: bt.height, + seed_hash: bt.seed_hash, + reserved_offset: bt.reserved_offset, client_nonce_offset: bt.clientNonceLocation, - client_pool_offset: bt.clientPoolLocation, - target_diff: this.difficulty, - target_diff_hex: this.diffHex, - job_id: newJob.id, - id: this.id + client_pool_offset: bt.clientPoolLocation, + target_diff: coin_diff, + job_id: newJob.id, + id: this.id }; } return this.cachedJob; }; - this.sendNewJob = function() { - let job = this.getJob(); + this.sendCoinJob = function(coin, params) { + const job = this.getCoinJob(coin, params); if (job === null) return; - return this.messageSender('job', job); + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(coin)); + if (this.protocol == "grin") { + return this.pushMessage({method: "getjobtemplate", result: job}); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + const target = job[3]; + if (!this.last_target || this.last_target !== target) { + this.pushMessage({method: "mining.set_target", params: [ target ], id:null}); + this.last_target = target; + } + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name, id:null}); + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + const diff = job.pop() / 0x100000000; + if (!this.last_diff || this.last_diff !== diff) { + this.pushMessage({method: "mining.set_difficulty", params: [ diff ]}); + this.last_diff = diff; + } + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name}); + + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name}); + + } else { + return this.pushMessage({method: "job", params: job}); + } + }; + + this.sendSameCoinJob = function () { + const coin = typeof(this.curr_coin) !== 'undefined' ? this.curr_coin : this.selectBestCoin(); + if (coin !== false) return this.sendCoinJob(coin, getCoinJobParams(coin)); + }; + + this.getBestCoinJob = function() { + const coin = this.selectBestCoin(); + if (coin !== false) return this.getCoinJob(coin, getCoinJobParams(coin)); + }; + + this.sendBestCoinJob = function() { + const coin = this.selectBestCoin(); + if (coin !== false) return this.sendCoinJob(coin, getCoinJobParams(coin)); }; } } -// store wallet_key (address, paymentID, bitcoin, poolTypeEnum, port) -> worker_name -> shareType -> (height, difficulty, time, acc, acc2) +// store wallet_key (address, paymentID, bitcoin, poolTypeEnum, port) -> worker_name -> isTrustedShare -> (height, difficulty, time, acc, acc2) let walletAcc = {}; // number of worker_name for wallet_key (so we do not count them by iteration) let walletWorkerCount = {}; // is share finalizer function for dead worker_name is active let is_walletAccFinalizer = {}; -function walletAccFinalizer(wallet_key, miner_address, miner_paymentID, miner_bitcoin, miner_poolTypeEnum, miner_port) { +function storeShareDiv(miner, share_reward, share_reward2, share_num, worker_name, bt_port, bt_height, bt_difficulty, isBlockCandidate, isTrustedShare) { + const time_now = Date.now(); + if (miner.payout_div === null) { + global.database.storeShare(bt_height, global.protos.Share.encode({ + paymentAddress: miner.address, + paymentID: miner.paymentID, + raw_shares: share_reward, + shares2: share_reward2, + share_num: share_num, + identifier: worker_name, + port: bt_port, + blockHeight: bt_height, + blockDiff: bt_difficulty, + poolType: miner.poolTypeEnum, + bitcoin: miner.bitcoin, + foundBlock: isBlockCandidate, + trustedShare: isTrustedShare, + poolID: global.config.pool_id, + timestamp: time_now + })); + } else { + for (let payout in miner.payout_div) { + const payout_split = payout.split("."); + const paymentAddress = payout_split[0]; + const paymentID = payout_split.length === 2 ? payout_split[1] : null; + const payoutPercent = miner.payout_div[payout]; + const shares = share_reward * payoutPercent / 100; + const shares2 = Math.floor(share_reward2 * payoutPercent / 100); + global.database.storeShare(bt_height, global.protos.Share.encode({ + paymentAddress: paymentAddress, + paymentID: paymentID, + raw_shares: shares, + shares2: shares2, + share_num: share_num, + identifier: worker_name, + port: bt_port, + blockHeight: bt_height, + blockDiff: bt_difficulty, + poolType: miner.poolTypeEnum, + bitcoin: miner.bitcoin, + foundBlock: isBlockCandidate, + trustedShare: isTrustedShare, + poolID: global.config.pool_id, + timestamp: time_now + })); + } + } +} + +function walletAccFinalizer(wallet_key, miner, bt_port) { debug("!!! " + wallet_key + ": scanning for old worker names"); let wallet = walletAcc[wallet_key]; let is_something_left = false; let time_now = Date.now(); for (let worker_name in wallet) { let worker = wallet[worker_name]; - if (time_now - worker.time > 60*1000) { + if (time_now - worker.time > global.config.pool.shareAccTime*1000) { let acc = worker.acc; if (acc != 0) { let height = worker.height; debug("!!! " + wallet_key + " / " + worker_name + ": storing old worker share " + height + " " + worker.difficulty + " " + time_now + " " + acc); - global.database.storeShare(height, global.protos.Share.encode({ - shares: acc, - shares2: worker.acc2, - paymentAddress: miner_address, - paymentID: miner_paymentID, - foundBlock: false, - trustedShare: true, - poolType: miner_poolTypeEnum, - poolID: global.config.pool_id, - blockDiff: worker.difficulty, - bitcoin: miner_bitcoin, - blockHeight: height, - timestamp: time_now, - identifier: worker_name, - port: miner_port, - share_num: worker.share_num - })); + storeShareDiv(miner, acc, worker.acc2, worker.share_num, worker_name, bt_port, height, worker.difficulty, false, true); } debug("!!! " + wallet_key + ": removing old worker " + worker_name); if (worker_name !== "all_other_workers") -- walletWorkerCount[wallet_key]; @@ -950,19 +1323,19 @@ function walletAccFinalizer(wallet_key, miner_address, miner_paymentID, miner_bi } if (is_something_left) { - setTimeout(walletAccFinalizer, 60*1000, wallet_key, miner_address, miner_paymentID, miner_bitcoin, miner_poolTypeEnum, miner_port); + setTimeout(walletAccFinalizer, global.config.pool.shareAccTime*1000, wallet_key, miner, bt_port); } else { is_walletAccFinalizer[wallet_key] = false; } } -function recordShareData(miner, job, shareDiff, blockCandidate, hashHex, shareType, blockTemplate) { - miner.hashes += job.difficulty; - let proxyMinerName = miner.payout + ":" + miner.identifier; - if (proxyMinerName in proxyMiners) proxyMiners[proxyMinerName].hashes += job.difficulty; +function recordShareData(miner, job, isTrustedShare, blockTemplate) { + miner.hashes += job.norm_diff; + let proxyMinerName = miner.payout; // + ":" + miner.identifier; + if (proxyMinerName in proxyMiners) proxyMiners[proxyMinerName].hashes += job.norm_diff; - let time_now = Date.now(); - let wallet_key = miner.address + " " + miner.paymentID + " " + miner.bitcoin + " " + miner.poolTypeEnum + " " + blockTemplate.port; + const time_now = Date.now(); + let wallet_key = miner.wallet_key + blockTemplate.port; if (!(wallet_key in walletAcc)) { walletAcc[wallet_key] = {}; @@ -970,299 +1343,479 @@ function recordShareData(miner, job, shareDiff, blockCandidate, hashHex, shareTy is_walletAccFinalizer[wallet_key] = false; } - let db_job_height = global.config.daemon.port == blockTemplate.port ? job.height : anchorBlockHeight; - - if (job.difficulty >= 1000000 || blockCandidate) { + const db_job_height = global.config.daemon.port == blockTemplate.port ? blockTemplate.height : anchorBlockHeight; - global.database.storeShare(db_job_height, global.protos.Share.encode({ - shares: job.rewarded_difficulty, - shares2: job.rewarded_difficulty2, - paymentAddress: miner.address, - paymentID: miner.paymentID, - foundBlock: blockCandidate, - trustedShare: shareType, - poolType: miner.poolTypeEnum, - poolID: global.config.pool_id, - blockDiff: blockTemplate.difficulty, - bitcoin: miner.bitcoin, - blockHeight: db_job_height, - timestamp: time_now, - identifier: miner.identifier, - port: blockTemplate.port, - share_num: 1 - })); - - } else { - - let wallet = walletAcc[wallet_key]; - - let worker_name = miner.identifier in wallet || walletWorkerCount[wallet_key] < 50 ? miner.identifier : "all_other_workers"; - - if (!(worker_name in wallet)) { - if (worker_name !== "all_other_workers") ++ walletWorkerCount[wallet_key]; - debug("!!! " + wallet_key + ": adding new worker " + worker_name + " (num " + walletWorkerCount[wallet_key] + ")"); - wallet[worker_name] = {}; - let worker = wallet[worker_name]; - worker.height = db_job_height; - worker.difficulty = blockTemplate.difficulty; - worker.time = time_now; - worker.acc = 0; - worker.acc2 = 0; - worker.share_num = 0; - } + let wallet = walletAcc[wallet_key]; + const worker_name = miner.identifier in wallet || walletWorkerCount[wallet_key] < 50 ? miner.identifier : "all_other_workers"; + if (!(worker_name in wallet)) { + if (worker_name !== "all_other_workers") ++ walletWorkerCount[wallet_key]; + debug("!!! " + wallet_key + ": adding new worker " + worker_name + " (num " + walletWorkerCount[wallet_key] + ")"); + wallet[worker_name] = {}; let worker = wallet[worker_name]; + worker.height = db_job_height; + worker.difficulty = blockTemplate.difficulty; + worker.time = time_now; + worker.acc = 0; + worker.acc2 = 0; + worker.share_num = 0; + } - let height = worker.height; - let difficulty = worker.difficulty; - let acc = worker.acc; - let acc2 = worker.acc2; - let share_num = worker.share_num; - - if (height !== db_job_height || difficulty !== blockTemplate.difficulty || time_now - worker.time > 60*1000 || acc >= 1000000) { - if (acc != 0) { - debug("!!! " + wallet_key + " / " + worker_name + ": storing share " + height + " " + difficulty + " " + time_now + " " + acc); - global.database.storeShare(height, global.protos.Share.encode({ - shares: acc, - shares2: acc2, - paymentAddress: miner.address, - paymentID: miner.paymentID, - foundBlock: false, - trustedShare: shareType, - poolType: miner.poolTypeEnum, - poolID: global.config.pool_id, - blockDiff: difficulty, - bitcoin: miner.bitcoin, - blockHeight: height, - timestamp: time_now, - identifier: worker_name, - port: blockTemplate.port, - share_num: share_num - })); - } + let worker = wallet[worker_name]; - worker.height = db_job_height; - worker.difficulty = blockTemplate.difficulty; - worker.time = time_now; - worker.acc = job.rewarded_difficulty; - worker.acc2 = job.rewarded_difficulty2; - worker.share_num = 1; + let height = worker.height; + let difficulty = worker.difficulty; + let acc = worker.acc; + let acc2 = worker.acc2; + let share_num = worker.share_num; - } else { - worker.acc += job.rewarded_difficulty; - worker.acc2 += job.rewarded_difficulty2; - ++ worker.share_num; + if (time_now - worker.time > global.config.pool.shareAccTime*1000 || acc >= 100000000) { + if (acc != 0) { + debug("!!! " + wallet_key + " / " + worker_name + ": storing share " + height + " " + difficulty + " " + time_now + " " + acc); + storeShareDiv(miner, acc, acc2, share_num, worker_name, blockTemplate.port, height, difficulty, false, isTrustedShare); } - debug("!!! " + wallet_key + " / " + worker_name + ": accumulating share " + db_job_height + " " + blockTemplate.difficulty + " " + worker.time + " " + worker.acc + " (+" + job.rewarded_difficulty + ")"); + worker.height = db_job_height; + worker.difficulty = blockTemplate.difficulty; + worker.time = time_now; + worker.acc = job.rewarded_difficulty; + worker.acc2 = job.rewarded_difficulty2; + worker.share_num = 1; + + } else { + worker.acc += job.rewarded_difficulty; + worker.acc2 += job.rewarded_difficulty2; + ++ worker.share_num; } + + debug("!!! " + wallet_key + " / " + worker_name + ": accumulating share " + db_job_height + " " + blockTemplate.difficulty + " " + worker.time + " " + worker.acc + " (+" + job.rewarded_difficulty + ")"); if (is_walletAccFinalizer[wallet_key] === false) { is_walletAccFinalizer[wallet_key] = true; - setTimeout(walletAccFinalizer, 60*1000, wallet_key, miner.address, miner.paymentID, miner.bitcoin, miner.poolTypeEnum, blockTemplate.port); + setTimeout(walletAccFinalizer, global.config.pool.shareAccTime*1000, wallet_key, miner, blockTemplate.port); } - if (blockCandidate) { - if (global.config.daemon.port == blockTemplate.port) { - global.database.storeBlock(job.height, global.protos.Block.encode({ - hash: hashHex, - difficulty: blockTemplate.difficulty, - shares: 0, - timestamp: time_now, - poolType: miner.poolTypeEnum, - unlocked: false, - valid: true - })); - } else { - global.database.storeAltBlock(Math.floor(time_now / 1000), global.protos.AltBlock.encode({ - hash: hashHex, - difficulty: blockTemplate.difficulty, - shares: 0, - timestamp: time_now, - poolType: miner.poolTypeEnum, - unlocked: false, - valid: true, - port: blockTemplate.port, - height: job.height, - anchor_height: anchorBlockHeight - })); - } - } - if (shareType) { + if (isTrustedShare) { process.send({type: 'trustedShare'}); - debug(threadName + "Accepted trusted share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + "/" + shareDiff + " from: " + miner.logString); + debug(threadName + "Accepted trusted share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + " from: " + miner.logString); } else { process.send({type: 'normalShare'}); - debug(threadName + "Accepted valid share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + "/" + shareDiff + " from: " + miner.logString); + debug(threadName + "Accepted valid share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + " from: " + miner.logString); } - if (activeBlockTemplate[blockTemplate.algo].idHash !== job.blockHash) { + if (activeBlockTemplates[job.coin].idHash !== blockTemplate.idHash) { process.send({type: 'outdatedShare'}); } } function getShareBuffer(miner, job, blockTemplate, params) { - let nonce = params.nonce; - let resultHash = params.result; - let template = new Buffer(blockTemplate.buffer.length); - if (!miner.proxy) { - blockTemplate.buffer.copy(template); - template.writeUInt32BE(job.extraNonce, blockTemplate.reserveOffset); - } else { - blockTemplate.buffer.copy(template); - template.writeUInt32BE(job.extraNonce, blockTemplate.reserveOffset); - template.writeUInt32BE(params.poolNonce, job.clientPoolLocation); - template.writeUInt32BE(params.workerNonce, job.clientNonceLocation); - } try { - let shareBuffer = global.coinFuncs.constructNewBlob(template, new Buffer(nonce, 'hex'), blockTemplate.port); - return shareBuffer; + let template = Buffer.alloc(blockTemplate.buffer.length); + blockTemplate.buffer.copy(template); + template.writeUInt32BE(job.extraNonce, blockTemplate.reserved_offset); + if (miner.proxy) { + template.writeUInt32BE(params.poolNonce, job.clientPoolLocation); + template.writeUInt32BE(params.workerNonce, job.clientNonceLocation); + } + return global.coinFuncs.constructNewBlob(template, params, blockTemplate.port); } catch (e) { - console.error("Can't constructNewBlob with " + nonce + " nonce from " + miner.logString + ": " + e); - global.support.sendEmail(global.config.general.adminEmail, - "FYI: Can't constructNewBlob", - "Can't constructNewBlob with " + nonce + " nonce from " + miner.logString + ": " + e - ); + const err_str = "Can't constructNewBlob of " + blockTemplate.port + " port with " + JSON.stringify(params) + " params from " + miner.logString + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't constructNewBlob", err_str); return null; } } -function processShare(miner, job, blockTemplate, params) { - let hash; - let shareType; - let shareBuffer; - const resultHash = params.result; - if (miner.payout in minerWallets) minerWallets[miner.payout].hashes += job.difficulty; +function invalid_share(miner) { + process.send({type: 'invalidShare'}); + miner.sendSameCoinJob(); + walletTrust[miner.payout] = 0; + return false; +} + +function submit_block(miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, isParentBlock, isRetrySubmitBlock, submit_blockCB) { + let reply_fn = function (rpcResult, rpcStatus) { + const blockDataStr = Buffer.isBuffer(blockData) ? blockData.toString('hex') : JSON.stringify(blockData); + const blob_type_num = global.coinFuncs.portBlobType(blockTemplate.port, blockTemplate.block_version); + if (rpcResult && (rpcResult.error || rpcResult.result === "high-hash")) { // did not manage to submit a block + let isNotifyAdmin = true; + if (isParentBlock && isTrustedShare) { + const convertedBlob = global.coinFuncs.convertBlob(blockData, blockTemplate.port); + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) isNotifyAdmin = false; + } + + console.error(threadName + "Error submitting " + blockTemplate.coin + " (port " + blockTemplate.port + ") block at height " + + blockTemplate.height + " (active block template height: " + activeBlockTemplates[blockTemplate.coin].height + ") from " + + miner.logString + ", isTrustedShare: " + isTrustedShare + ", valid: " + isNotifyAdmin + ", rpcStatus: " + rpcStatus + + ", error: " + JSON.stringify(rpcResult) + ", block hex: \n" + blockDataStr + ); + + if (isNotifyAdmin) setTimeout(function() { // only alert if block height is not changed in the nearest time + global.coinFuncs.getPortLastBlockHeader(blockTemplate.port, function(err, body) { + if (err !== null) { + console.error("Last block header request failed for " + blockTemplate.port + " port!"); + return; + } + if (blockTemplate.height == body.height + 1) global.support.sendEmail(global.config.general.adminEmail, + "FYI: Can't submit " + blockTemplate.coin + " block to deamon on " + blockTemplate.port + " port", + "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + + "Input: " + blockDataStr + "\n" + + threadName + "Error submitting " + blockTemplate.coin + " block at " + blockTemplate.height + " height from " + miner.logString + + ", isTrustedShare: " + isTrustedShare + " error ): " + JSON.stringify(rpcResult) + ); + }); + }, 2*1000); + + if (global.config.pool.trustedMiners) { + debug(threadName + "Share trust broken by " + miner.logString); + miner.trust.trust = 0; + walletTrust[miner.payout] = 0; + } + + if (submit_blockCB) submit_blockCB(false); + + // Success! Submitted a block without an issue. + } else if ( rpcResult && ( + ( typeof(rpcResult.result) !== 'undefined' ) || + ( rpcResult.response !== 'rejected' && global.coinFuncs.blobTypeErg(blob_type_num) ) || // ERG + ( typeof rpcResult === 'string' && rpcStatus == 202 && blockTemplate.port == 11898 ) // TRTL + ) + ) { + + const get_block_id = function(cb) { + if (global.coinFuncs.blobTypeDero(blob_type_num)) { + return cb(rpcResult.result.blid); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + return cb(resultBuff.toString('hex')); + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + setTimeout(global.coinFuncs.getPortBlockHeaderByID, 10*1000, blockTemplate.port, blockTemplate.height, function(err, body) { + if (err === null && body.powSolutions.pk === blockTemplate.hash2) return cb(body.id); + return cb("0000000000000000000000000000000000000000000000000000000000000000"); + }); + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + return cb(rpcResult.result.substr(2)); + } else { + return cb(global.coinFuncs.getBlockID(blockData, blockTemplate.port).toString('hex')); + } + }; + + get_block_id(function(newBlockHash) { + console.log(threadName + "New " + blockTemplate.coin + " (port " + blockTemplate.port + ") block " + newBlockHash + " found at height " + blockTemplate.height + " by " + miner.logString + + ", isTrustedShare: " + isTrustedShare + " - submit result: " + JSON.stringify(rpcResult) + + ", block hex: \n" + blockDataStr + ); + + const time_now = Date.now(); + if (global.config.daemon.port == blockTemplate.port) { + global.database.storeBlock(blockTemplate.height, global.protos.Block.encode({ + hash: newBlockHash, + difficulty: blockTemplate.difficulty, + shares: 0, + timestamp: time_now, + poolType: miner.poolTypeEnum, + unlocked: false, + valid: true + })); + } else { + global.database.storeAltBlock(Math.floor(time_now / 1000), global.protos.AltBlock.encode({ + hash: newBlockHash, + difficulty: blockTemplate.difficulty, + shares: 0, + timestamp: time_now, + poolType: miner.poolTypeEnum, + unlocked: false, + valid: true, + port: blockTemplate.port, + height: blockTemplate.height, + anchor_height: anchorBlockHeight + })); + } + + if (submit_blockCB) submit_blockCB(true); + }); + + } else { // something not expected happened + if (isRetrySubmitBlock) { + console.error(threadName + "Unknown error submitting " + blockTemplate.coin + " (port " + blockTemplate.port + ") block at height " + + blockTemplate.height + " (active block template height: " + activeBlockTemplates[blockTemplate.coin].height + ") from " + + miner.logString + ", isTrustedShare: " + isTrustedShare + ", rpcStatus: " + rpcStatus + ", error (" + (typeof rpcResult) + "): " + JSON.stringify(rpcResult) + + ", block hex: \n" + blockDataStr + ); + setTimeout(submit_block, 500, miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, isParentBlock, false, submit_blockCB); + } else { + // RPC bombed out massively. + console.error(threadName + "RPC Error. Please check logs for details"); + global.support.sendEmail(global.config.general.adminEmail, + "FYI: Can't submit block to deamon on " + blockTemplate.port + " port", + "Input: " + blockDataStr + "\n" + + "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + + "RPC Error. Please check logs for details" + ); + if (submit_blockCB) submit_blockCB(false); + } + } + }; + + if (blockTemplate.port == 11898) { + global.support.rpcPortDaemon2(blockTemplate.port, "block", blockData.toString('hex'), reply_fn); + } else if (global.coinFuncs.blobTypeRvn(job.blob_type_num) || global.coinFuncs.blobTypeRtm(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "", { method: "submitblock", params: [ blockData.toString('hex') ] }, reply_fn); + } else if (global.coinFuncs.blobTypeEth(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "", { method: "parity_submitWorkDetail", params: blockData, jsonrpc: "2.0", id: 0 }, reply_fn); + } else if (global.coinFuncs.blobTypeErg(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "mining/solution", {"n": blockData}, reply_fn); + } else if (global.coinFuncs.blobTypeDero(job.blob_type_num)) { + global.support.rpcPortDaemon(blockTemplate.port, "submitblock", [ blockTemplate.blocktemplate_blob, blockData.toString('hex') ], reply_fn); + } else { + global.support.rpcPortDaemon(blockTemplate.port, "submitblock", [ blockData.toString('hex') ], reply_fn); + } +} + +// wallets that need extra share verification +let extra_wallet_verify = {}; +let extra_verify_wallet_hashes = []; + +function is_safe_to_trust(reward_diff, miner_wallet, miner_trust) { + const reward_diff2 = reward_diff * global.config.pool.trustThreshold; + return reward_diff < 400000 && miner_trust != 0 && ( + ( miner_wallet in walletTrust && + reward_diff2 * global.config.pool.trustThreshold < walletTrust[miner_wallet] && + crypto.randomBytes(1).readUIntBE(0, 1) > global.config.pool.trustMin + ) || ( + reward_diff2 < miner_trust && + crypto.randomBytes(1).readUIntBE(0, 1) > Math.max(256 - miner_trust / reward_diff / 2, global.config.pool.trustMin) + ) + ); +} + +function hashBuffDiff(hash) { // bignum as result + return baseDiff.div(bignum.fromBuffer(hash, {endian: 'little', size: 32})); +} + +function hashRavenBuffDiff(hash) { // float as result + return baseRavenDiff / bignum.fromBuffer(hash).toNumber(); +} + +function hashEthBuffDiff(hash) { // bignum as result + return baseDiff.div(bignum.fromBuffer(hash)); +} + +// will work for numbers and bignum +function ge(l, r) { + if (typeof l === 'object') return l.ge(r); + if (typeof r === 'object') return !r.lt(l); + return l >= r; +} + +function report_miner_share(miner, job) { + const time_now = Date.now(); + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { + console.error(threadName + "Bad share from miner (diff " + job.difficulty + ") " + miner.logString); + lastMinerLogTime[miner.payout] = time_now; + } +} + +function processShare(miner, job, blockTemplate, params, processShareCB) { + const port = blockTemplate.port; + const blob_type_num = job.blob_type_num; + + if (miner.payout in minerWallets) minerWallets[miner.payout].hashes += job.norm_diff; walletLastSeeTime[miner.payout] = Date.now(); - if (global.config.pool.trustedMiners && miner.difficulty < 400000 && miner.trust.threshold <= 0 && miner.trust.penalty <= 0 && - crypto.randomBytes(1).readUIntBE(0, 1) > miner.trust.probability && miner.trust.check_height !== job.height) { - try { - hash = new Buffer(resultHash, 'hex'); - } catch (err) { - process.send({type: 'invalidShare'}); - miner.sendNewJob(); - walletTrust[miner.payout] = 0; - return false; - } - shareType = true; - } else { // verify share - if (miner.payout in minerWallets && ++minerWallets[miner.payout].last_ver_shares >= MAX_VER_SHARES_PER_SEC * VER_SHARES_PERIOD) { - if (minerWallets[miner.payout].last_ver_shares === MAX_VER_SHARES_PER_SEC * VER_SHARES_PERIOD) { - console.error(threadName + "Throttled down miner share (diff " + job.difficulty + ") submission from " + miner.logString); + let shareThrottled = function() { + if (miner.payout in minerWallets && ++minerWallets[miner.payout].last_ver_shares >= global.config.pool.minerThrottleSharePerSec * global.config.pool.minerThrottleShareWindow) { + if (minerWallets[miner.payout].last_ver_shares === global.config.pool.minerThrottleSharePerSec * global.config.pool.minerThrottleShareWindow) { + console.error(threadName + "Throttled down miner share (diff " + job.rewarded_difficulty2 + ") submission from " + miner.logString); } process.send({type: 'throttledShare'}); - addProxyMiner(miner); - miner.updateDifficulty(); - return null; - } - shareBuffer = getShareBuffer(miner, job, blockTemplate, params); - if (shareBuffer === null) { - process.send({type: 'invalidShare'}); - miner.sendNewJob(); - walletTrust[miner.payout] = 0; - return false; + if (addProxyMiner(miner)) { + const proxyMinerName = miner.payout; // + ":" + miner.identifier; + proxyMiners[proxyMinerName].hashes += job.norm_diff; + adjustMinerDiff(miner); + } + return true; } - let convertedBlob = global.coinFuncs.convertBlob(shareBuffer, blockTemplate.port); - hash = global.coinFuncs.cryptoNight(convertedBlob, blockTemplate.port); + return false; + } - if (hash.toString('hex') !== resultHash) { - let time_now = Date.now(); - if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { - console.error(threadName + "Bad share from miner (diff " + job.difficulty + ") " + miner.logString + (miner.trust.probability == 256 ? " [banned]" : "")); - lastMinerLogTime[miner.payout] = time_now; + let verifyShare = function(verifyShareCB) { + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const header = Buffer.concat([global.coinFuncs.convertBlob(blockData, port), bignum(params.nonce, 10).toBuffer({endian: 'big', size: 4})]); + if (global.coinFuncs.c29(header, params.pow, port)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); } - process.send({type: 'invalidShare'}); - miner.sendNewJob(); - walletTrust[miner.payout] = 0; - return false; + const resultBuff = global.coinFuncs.c29_cycle_hash(params.pow, blob_type_num); + return verifyShareCB(hashBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + if (params.header_hash !== convertedBlob.toString("hex")) { + console.error("Wrong header hash:" + params.header_hash + " " + convertedBlob.toString("hex")); + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + const resultBuff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate, params.nonce, params.mixhash); + return verifyShareCB(hashRavenBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + if (shareThrottled()) return processShareCB(null); + const hashes = global.coinFuncs.slowHashBuff(Buffer.from(blockTemplate.hash, 'hex'), blockTemplate, params.nonce); + const resultBuff = hashes[0]; + const blockData = [ "0x" + params.nonce, "0x" + blockTemplate.hash, "0x" + hashes[1].toString('hex') ]; + return verifyShareCB(hashEthBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + if (shareThrottled()) return processShareCB(null); + const coinbaseBuffer = Buffer.concat([Buffer.from(blockTemplate.hash, 'hex'), Buffer.from(params.nonce, 'hex')]); + const hashes = global.coinFuncs.slowHashBuff(coinbaseBuffer, blockTemplate); + return verifyShareCB(hashEthBuffDiff(hashes[1]), null, params.nonce, false, true); } - ++ walletTrust[miner.payout]; - shareType = false; - } + const resultHash = params.result; + let resultBuff; + try { + resultBuff = Buffer.from(resultHash, 'hex'); + } catch(e) { + return processShareCB(invalid_share(miner)); + } + const hashDiff = hashBuffDiff(resultBuff); + + if ( global.config.pool.trustedMiners && + is_safe_to_trust(job.rewarded_difficulty2, miner.payout, miner.trust.trust) && + miner.trust.check_height !== job.height + ) { + let blockData = null; + if (miner.payout in extra_wallet_verify) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData !== null) { + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + global.coinFuncs.slowHashAsync(convertedBlob, blockTemplate, miner.payout, function(hash) { + if (hash === null || hash === false) { + console.error(threadName + "[EXTRA CHECK] Can't verify share remotely!"); + } else if (hash !== resultHash) { + console.error(threadName + miner.logString + " [EXTRA CHECK] INVALID SHARE OF " + job.rewarded_difficulty2 + " REWARD HASHES"); + } else { + extra_verify_wallet_hashes.push(miner.payout + " " + convertedBlob.toString('hex') + " " + resultHash + " " + global.coinFuncs.algoShortTypeStr(port) + " " + blockTemplate.height + " " + blockTemplate.seed_hash); + } + }); + } else { + console.error(threadName + miner.logString + " [EXTRA CHECK] CAN'T MAKE SHARE BUFFER"); + } + } + if (miner.lastSlowHashAsyncDelay) { + setTimeout(function() { return verifyShareCB(hashDiff, resultBuff, blockData, true, true); }, miner.lastSlowHashAsyncDelay); + debug("[MINER] Delay " + miner.lastSlowHashAsyncDelay); + } else { + return verifyShareCB(hashDiff, resultBuff, blockData, true, true); + } - let hashArray = hash.toByteArray().reverse(); - let hashNum = bignum.fromBuffer(new Buffer(hashArray)); - let hashDiff = baseDiff.div(hashNum); - - if (hashDiff.ge(blockTemplate.difficulty)) { - // Submit block to the RPC Daemon. - // Todo: Implement within the coins/.js file. - if (!shareBuffer) shareBuffer = getShareBuffer(miner, job, blockTemplate, params); - function submit_block(retry) { - global.support.rpcPortDaemon(blockTemplate.port, 'submitblock', [shareBuffer.toString('hex')], function (rpcResult) { - if (rpcResult.error) { - // Did not manage to submit a block. Log and continue on. - recordShareData(miner, job, hashDiff.toString(), false, null, shareType, blockTemplate); - let isNotifyAdmin = true; - if (shareType) { - let convertedBlob = global.coinFuncs.convertBlob(shareBuffer, blockTemplate.port); - hash = global.coinFuncs.cryptoNight(convertedBlob); - if (hash.toString('hex') !== resultHash) isNotifyAdmin = false; - } - console.error(threadName + "Error submitting block at height " + job.height + " (active block template height: " + activeBlockTemplate[blockTemplate.algo].height + ") from " + miner.logString + ", share type: " + shareType + ", valid: " + isNotifyAdmin + " error: " + JSON.stringify(rpcResult.error)); - if (isNotifyAdmin) setTimeout(function() { // only alert if block height is not changed in the nearest time - global.coinFuncs.getPortLastBlockHeader(blockTemplate.port, function(err, body){ - if (err !== null) { - console.error("Last block header request failed for " + blockTemplate.port + " port!"); - return; + } else { // verify share + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] verify share"); + if (shareThrottled()) return processShareCB(null); + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + + const isBlockDiffMatched = ge(hashDiff, blockTemplate.difficulty); + if (isBlockDiffMatched) { + if (miner.validShares || (miner.payout in minerWallets && minerWallets[miner.payout].hashes)) { + submit_block(miner, job, blockTemplate, blockData, resultBuff, true, true, true, function(block_submit_result) { + if (!block_submit_result) { + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); } - if (job.height == body.height + 1) global.support.sendEmail(global.config.general.adminEmail, - "FYI: Can't submit block to deamon on " + blockTemplate.port + " port", - "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + - "Input: " + shareBuffer.toString('hex') + "\n" + - threadName + "Error submitting block at " + job.height + " height from " + miner.logString + ", share type: " + shareType + " error: " + JSON.stringify(rpcResult.error) - ); - }); - }, 2*1000); - if (global.config.pool.trustedMiners) { - debug(threadName + "Share trust broken by " + miner.logString); - miner.trust.probability = 256; - miner.trust.penalty = global.config.pool.trustPenalty; - miner.trust.threshold = global.config.pool.trustThreshold; - walletTrust[miner.payout] = 0; + } + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, false); + }); + } else { + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); } - } else if (rpcResult && typeof(rpcResult.result) !== 'undefined') { - //Success! Submitted a block without an issue. - let blockFastHash = global.coinFuncs.getBlockID(shareBuffer, blockTemplate.port).toString('hex'); - console.log(threadName + "Block " + blockFastHash.substr(0, 6) + " found at height " + job.height + " by " + miner.logString + - ", share type: " + shareType + " - submit result: " + JSON.stringify(rpcResult.result) - ); - recordShareData(miner, job, hashDiff.toString(), true, blockFastHash, shareType, blockTemplate); - } else { - if (retry) { - setTimeout(submit_block, 500, false); - } else { - // RPC bombed out massively. - console.error(threadName + "RPC Error. Please check logs for details"); - global.support.sendEmail(global.config.general.adminEmail, - "FYI: Can't submit block to deamon on " + blockTemplate.port + " port", - "Input: " + shareBuffer.toString('hex') + "\n" + - "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + - "RPC Error. Please check logs for details" - ); + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, true); + } + } else { + const time_now = Date.now(); + global.coinFuncs.slowHashAsync(convertedBlob, blockTemplate, miner.payout, function(hash) { + if (hash === null) { + return processShareCB(null); + } else if (hash === false) { + console.error(threadName + "Processed share locally instead of remotely!"); + hash = global.coinFuncs.slowHash(convertedBlob, blockTemplate); } - } - }); + if (hash !== resultHash) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + miner.lastSlowHashAsyncDelay = Date.now() - time_now; + if (miner.lastSlowHashAsyncDelay > 1000) miner.lastSlowHashAsyncDelay = 1000; + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, false); + }); + } } - if (shareBuffer) submit_block(true); + }; - } else if (hashDiff.lt(job.difficulty)) { - let time_now = Date.now(); - if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { - console.warn(threadName + "Rejected low diff (" + hashDiff.toString() + " < " + job.difficulty + ") share from miner " + miner.logString + (miner.trust.probability == 256 ? " [banned]" : "")); - lastMinerLogTime[miner.payout] = time_now; + verifyShare(function(hashDiff, resultBuff, blockData, isTrustedShare, isNeedCheckBlockDiff) { + if (isNeedCheckBlockDiff && ge(hashDiff, blockTemplate.difficulty)) { // Submit block to the RPC Daemon. + if (!blockData) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (!blockData) return processShareCB(invalid_share(miner)); + } + submit_block(miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, true, true); } - process.send({type: 'invalidShare'}); - return false; - - } else { - recordShareData(miner, job, hashDiff.toString(), false, null, shareType, blockTemplate); - } - - return true; + + const is_mm = "child_template" in blockTemplate; + if (is_mm && ge(hashDiff, blockTemplate.child_template.difficulty)) { // Submit child block to the RPC Daemon. + if (!blockData) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (!blockData) return processShareCB(invalid_share(miner)); + } + // need to properly restore child template buffer here since it went via message string and was restored not correctly + blockTemplate.child_template_buffer = Buffer.from(blockTemplate.child_template_buffer); + let shareBuffer2 = null; + try { + shareBuffer2 = global.coinFuncs.constructMMChildBlockBlob(blockData, port, blockTemplate.child_template_buffer); + } catch (e) { + const err_str = "Can't construct_mm_child_block_blob with " + blockData.toString('hex') + " parent block and " + blockTemplate.child_template_buffer.toString('hex') + " child block share buffers from " + miner.logString + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't construct_mm_child_block_blob", err_str); + return processShareCB(invalid_share(miner)); + } + if (shareBuffer2 === null) return processShareCB(invalid_share(miner)); + submit_block(miner, job, blockTemplate.child_template, shareBuffer2, resultBuff, isTrustedShare, false, true); + } + + if (!ge(hashDiff, job.difficulty)) { + let time_now = Date.now(); + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { + console.warn(threadName + "Rejected low diff (" + hashDiff + " < " + job.difficulty + ") share from miner " + miner.logString); + lastMinerLogTime[miner.payout] = time_now; + } + return processShareCB(invalid_share(miner)); + + } else { + recordShareData(miner, job, isTrustedShare, blockTemplate); + // record child proc share for rewarded_difficulty effort calcs status but with 0 rewards (all included in parent share) + if (is_mm) { + job.rewarded_difficulty2 = 0; + recordShareData(miner, job, isTrustedShare, blockTemplate.child_template); + } + return processShareCB(true); + } + }); } // Message times for different miner addresses @@ -1270,82 +1823,90 @@ let lastMinerLogTime = {}; // Miner notification times let lastMinerNotifyTime = {}; -// Share times of miners (payout:identifier:ipAddress) that never submitted any good share -let badMinerLastShareTime = {}; - function get_miner_notification(payout) { if (payout in notifyAddresses) return notifyAddresses[payout]; return false; } -function handleMinerData(method, params, ip, portData, sendReply, pushMessage) { - // Check for ban here, so preconnected attackers can't continue to screw you - if (ip in bannedIPs) { - // Handle IP ban off clip. - sendReply("IP Address currently banned"); - return; - } - let miner; +function handleMinerData(socket, id, method, params, ip, portData, sendReply, sendReplyFinal, pushMessage) { switch (method) { - case 'login': + case 'mining.authorize': // Eth/Raven/Erg only + if (!params || !(params instanceof Array)) { + sendReplyFinal("No array params specified"); + return; + } + params = { + login: params[0], + pass: params[1], + agent: socket.eth_agent ? socket.eth_agent : "[generic_ethminer]", + algo: [ "kawpow" ], + "algo-perf": { "kawpow": 1 }, + }; + // continue to normal login + + case 'login': { // Grin and default + if (ip in bannedTmpIPs) { + sendReplyFinal("New connections from this IP address are temporarily suspended from mining (10 minutes max)"); + return; + } + if (!params) { + sendReplyFinal("No params specified"); + return; + } if (!params.login) { - sendReply("No login specified"); + sendReplyFinal("No login specified"); return; } if (!params.pass) params.pass = "x"; - let difficulty = portData.difficulty; - let minerId = uuidV4(); - miner = new Miner(minerId, params.login, params.pass, ip, difficulty, pushMessage, 1, portData.portType, portData.port, params.agent, params.algo, params["algo-perf"]); + const difficulty = portData.difficulty; + const minerId = get_new_id(); + let miner = new Miner( + minerId, params.login, params.pass, params.rigid, ip, difficulty, pushMessage, 1, portData.portType, portData.port, params.agent, + params.algo, params["algo-perf"], params["algo-min-time"] + ); + if (miner.debugMiner) socket.debugMiner = 1; //console.log(threadName + miner.logString + " [WALLET DEBUG] " + method + ": " + JSON.stringify(params)); + if (method === 'mining.authorize') { + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + miner.eth_extranonce = eth_extranonce(new_id); + } else { + miner.valid_miner = false; + miner.error = "Not enough extranoces. Switch to other pool node."; + } + } + if (params.agent && process.env['WORKER_ID'] == 1) minerAgents[params.agent] = 1; let time_now = Date.now(); if (!miner.valid_miner) { if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 10*60*1000) { console.log("Invalid miner " + miner.logString + " [" + miner.email + "], disconnecting due to: " + miner.error); lastMinerLogTime[miner.payout] = time_now; } - sendReply(miner.error); - return; - } - let miner_id = miner.payout + ":" + miner.identifier + ":" + miner.ipAddress; - if (miner_id in badMinerLastShareTime) { - let ban_time_left = 3*60*1000 - (time_now - badMinerLastShareTime[miner_id]); - if (ban_time_left > 0) { - sendReply("You miner " + miner.identifier + " is currently banned for submitting wrong result for " + (ban_time_left / 1000) + " seconds"); - return; - } else { - debug(threadName + "Removed miner " + miner.logString + " from ban"); - delete badMinerLastShareTime[miner_id]; - } + return sendReplyFinal(miner.error, miner.delay_reply); } - let miner_agent_notification = params.agent ? global.coinFuncs.get_miner_agent_notification(params.agent) : false; - let miner_notification = miner_agent_notification ? miner_agent_notification : global.coinFuncs.get_miner_agent_warning_notification(params.agent); - miner_notification = miner_notification ? miner_notification : get_miner_notification(miner.payout); + + const miner_agent_notification = !global.coinFuncs.algoMainCheck(miner.algos) && global.coinFuncs.algoPrevMainCheck(miner.algos) ? + global.coinFuncs.get_miner_agent_warning_notification(params.agent) : false; + const miner_notification = miner_agent_notification ? miner_agent_notification : get_miner_notification(miner.payout); if (miner_notification) { if (!(miner.payout in lastMinerNotifyTime) || time_now - lastMinerNotifyTime[miner.payout] > 60*60*1000) { lastMinerNotifyTime[miner.payout] = time_now; console.error("Sent notification to " + miner.logString + ": " + miner_notification); - sendReply(miner_notification + " (miner will connect after several attempts)"); - return; - } - } - if (miner_agent_notification) { - if (!(miner.payout in lastMinerNotifyTime) || time_now - lastMinerNotifyTime[miner.payout] > 60*60*1000) { - lastMinerNotifyTime[miner.payout] = time_now; - console.error("Sent notification to " + miner.logString + ": " + miner_agent_notification); + return sendReplyFinal(miner_notification + " (miner will connect after several attempts)"); } - sendReply(miner_agent_notification); - return; - } - process.send({type: 'newMiner', data: miner.port}); - if (miner.algos) { - activeSmartMiners[minerId] = miner; - } else { - activeMiners[minerId] = miner; } + + if (!socket.miner_ids) socket.miner_ids = []; + socket.miner_ids.push(minerId); + activeMiners.set(minerId, miner); + if (!miner.proxy) { - let proxyMinerName = miner.payout + ":" + miner.identifier; + let proxyMinerName = miner.payout; // + ":" + miner.identifier; if ((params.agent && params.agent.includes('proxy')) || (proxyMinerName in proxyMiners)) { - addProxyMiner(miner); - if (proxyMiners[proxyMinerName].hashes) miner.setNewDiff(miner.calcNewDiff()); + if (!addProxyMiner(miner)) { + return sendReplyFinal("Temporary (one hour max) ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)", 600); + } + if (proxyMiners[proxyMinerName].hashes) adjustMinerDiff(miner); } else { if (!(miner.payout in minerWallets)) { minerWallets[miner.payout] = {}; @@ -1358,210 +1919,444 @@ function handleMinerData(method, params, ip, portData, sendReply, pushMessage) { } } } - sendReply(null, { - id: minerId, - job: miner.getJob(), - status: 'OK' - }); + if (id === "Stratum") { // if grin miner is connected directly to the pool + sendReply(null, "ok"); + miner.protocol = "grin"; + } else if (method === 'mining.authorize') { // if raven/eth miner is connected directly to the pool + sendReply(null, true); + miner.protocol = "eth"; // technically equivalent to "default" + miner.sendBestCoinJob(); + } else { // if meta-miner or xmrig or something else connected + const coin = miner.selectBestCoin(); + if (coin !== false) { + const params = getCoinJobParams(coin); + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(coin)); + if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { // xmrig specifics + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + miner.eth_extranonce = eth_extranonce(new_id); + sendReply(null, { id: minerId, algo: params.algo_name, extra_nonce: miner.eth_extranonce }); + miner.sendCoinJob(coin, params); + } else { + sendReplyFinal("Not enough extranoces. Switch to other pool node."); + } + } else { + sendReply(null, { id: minerId, job: miner.getCoinJob(coin, params), status: 'OK' }); + } + } else { + sendReplyFinal("No block template yet. Please wait."); + } + miner.protocol = "default"; + } + break; + } + + case 'mining.subscribe': { // Raven/Eth/Erg only + if (params && (params instanceof Array) && params.length >= 1) socket.eth_agent = params[0]; + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + // extranonce is not really needed for Raven (extraonce is specificed as part of coinbase tx) + sendReply(null, [ [ "mining.notify", get_new_id(), "EthereumStratum/1.0.0" ], eth_extranonce(new_id), 6 ]); + } else { + sendReplyFinal("Not enough extranoces. Switch to other pool node."); + } + break; + } + + case 'mining.extranonce.subscribe': { // Raven/Eth only + sendReply(null, true); + break; + } + + case 'getjobtemplate': { // grin-mode miner only + const minerId = socket.miner_ids && socket.miner_ids.length == 1 ? socket.miner_ids[0] : ""; + let miner = activeMiners.get(minerId); + if (!miner) { + sendReplyFinal("Unauthenticated"); + return; + } + miner.heartbeat(); + sendReply(null, miner.getBestCoinJob()); break; - case 'getjob': - miner = activeSmartMiners[params.id]; - if (!miner) miner = activeMiners[params.id]; + } + + case 'getjob': { + if (!params) { + sendReplyFinal("No params specified"); + return; + } + let miner = activeMiners.get(params.id); if (!miner) { - sendReply('Unauthenticated'); + sendReplyFinal("Unauthenticated"); return; } miner.heartbeat(); - if (miner.algos && params.algo && params["algo-perf"]) { - const status = miner.setAlgos(params.algo, params["algo-perf"]); + if (params.algo && params.algo instanceof Array && params["algo-perf"] && params["algo-perf"] instanceof Object) { + const status = miner.setAlgos(params.algo, params["algo-perf"], params["algo-min-time"]); if (status != "") { sendReply(status); return; } } - miner.sendNewJob(); + sendReply(null, miner.getBestCoinJob()); break; - case 'submit': - miner = activeSmartMiners[params.id]; - if (!miner) miner = activeMiners[params.id]; + } + + case 'mining.submit': + if (!params || !(params instanceof Array)) { + sendReply("No array params specified"); + return; + } + + for (const param of params) if (typeof param !== 'string') { + sendReply("Not correct params specified"); + return; + } + + if (params.length >= 3) params = { + job_id: params[1], + raw_params: params + }; else { + sendReply("Not correct params specified"); + return; + } + + // continue to normal login + + case 'submit': { // grin and default + if (!params) { + sendReplyFinal("No params specified"); + return; + } + const minerId = params.id ? params.id : (socket.miner_ids && socket.miner_ids.length == 1 ? socket.miner_ids[0] : ""); + let miner = activeMiners.get(minerId); if (!miner) { - sendReply('Unauthenticated'); + sendReplyFinal("Unauthenticated"); return; } + //if (miner.debugMiner) console.log("SUBMIT"); miner.heartbeat(); + if (typeof (params.job_id) === 'number') params.job_id = params.job_id.toString(); // for grin miner let job = miner.validJobs.toarray().filter(function (job) { return job.id === params.job_id; })[0]; if (!job) { - sendReply('Invalid job id'); + sendReply("Invalid job id"); return; } - params.nonce = (typeof params.nonce === 'string' ? params.nonce.substr(0, 8).toLowerCase() : ""); - if (!nonceCheck.test(params.nonce)) { + const blob_type_num = job.blob_type_num; + + if (method === 'mining.submit') { + if (global.coinFuncs.blobTypeEth(blob_type_num) || global.coinFuncs.blobTypeErg(blob_type_num)) { + params.nonce = params.raw_params[2]; + } else if (global.coinFuncs.blobTypeRvn(blob_type_num) && params.raw_params.length >= 5) { + params.nonce = params.raw_params[2].substr(2); + params.header_hash = params.raw_params[3].substr(2); + params.mixhash = params.raw_params[4].substr(2); + } else { + sendReply("Invalid job params"); + return; + } + } + + const nonce_sanity_check = function(blob_type_num, params) { + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + if (typeof params.nonce !== 'number') return false; + if (!(params.pow instanceof Array)) return false; + if (params.pow.length != global.coinFuncs.c29ProofSize(blob_type_num)) return false; + } else { + if (typeof params.nonce !== 'string') return false; + if (global.coinFuncs.nonceSize(blob_type_num) == 8) { + const isExtraNonceBT = global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num); + if (isExtraNonceBT) params.nonce = job.extraNonce + params.nonce; + if (!nonceCheck64.test(params.nonce)) return false; + if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + if (!hashCheck32.test(params.mixhash)) return false; + if (!hashCheck32.test(params.header_hash)) return false; + } else if (!isExtraNonceBT) { + if (!hashCheck32.test(params.result)) return false; + } + } else { + if (!nonceCheck32.test(params.nonce)) return false; + if (!hashCheck32.test(params.result)) return false; + } + } + return true; + }; + if (!nonce_sanity_check(blob_type_num, params)) { console.warn(threadName + 'Malformed nonce: ' + JSON.stringify(params) + ' from ' + miner.logString); miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + sendReply("Duplicate share"); + miner.storeInvalidShare(); return; } - if (!miner.proxy) { - if (params.nonce in job.submissions) { - console.warn(threadName + 'Duplicate share with ' + params.nonce.toString() + ' nonce from ' + miner.logString); - miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); - return; - } - job.submissions[params.nonce] = 1; - } else { + + let nonce_test; + + if (miner.proxy) { if (!Number.isInteger(params.poolNonce) || !Number.isInteger(params.workerNonce)) { console.warn(threadName + 'Malformed nonce: ' + JSON.stringify(params) + ' from ' + miner.logString); miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); - return; - } - let nonce_test = `${params.nonce}_${params.poolNonce}_${params.workerNonce}`; - if (nonce_test in job.submissions) { - console.warn(threadName + 'Duplicate proxy share with ' + params.nonce_test.toString() + ' nonce from ' + miner.logString); - miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + sendReply("Duplicate share"); + miner.storeInvalidShare(); return; } - job.submissions[nonce_test] = 1; + nonce_test = global.coinFuncs.blobTypeGrin(blob_type_num) ? + params.pow.join(':') + `_${params.poolNonce}_${params.workerNonce}` : + `${params.nonce}_${params.poolNonce}_${params.workerNonce}`; + } else { + nonce_test = global.coinFuncs.blobTypeGrin(blob_type_num) ? params.pow.join(':') : params.nonce; + } + + if (nonce_test in job.submissions) { + console.warn(threadName + 'Duplicate miner share with ' + nonce_test + ' nonce from ' + miner.logString); + miner.checkBan(false); + sendReply("Duplicate share"); + miner.storeInvalidShare(); + return; } + job.submissions[nonce_test] = 1; let blockTemplate; job.rewarded_difficulty = job.difficulty; - if (activeBlockTemplate[job.algo_type].idHash !== job.blockHash) { - blockTemplate = pastBlockTemplates.toarray().filter(function (t) { + if (activeBlockTemplates[job.coin].idHash !== job.blockHash) { + blockTemplate = pastBlockTemplates[job.coin].toarray().filter(function (t) { return t.idHash === job.blockHash; })[0]; let is_outdated = false; - if (blockTemplate) { - let late_time = Date.now() - blockTemplate.timeOutdate; + if (blockTemplate && blockTemplate.timeoutTime) { + const late_time = Date.now() - blockTemplate.timeoutTime; if (late_time > 0) { - let max_late_time = global.config.pool.targetTime*1000; + const max_late_time = global.config.pool.targetTime * 1000; if (late_time < max_late_time) { let factor = (max_late_time - late_time) / max_late_time; - job.rewarded_difficulty = Math.floor(job.difficulty * Math.pow(factor, 6)); - if (job.rewarded_difficulty === 0) job.rewarded_difficulty = 1; + job.rewarded_difficulty = job.difficulty * Math.pow(factor, 6); //Math.floor(job.difficulty * Math.pow(factor, 6)); + //if (job.rewarded_difficulty === 0) job.rewarded_difficulty = 1; } else { is_outdated = true; } - } + } } if (!blockTemplate || is_outdated) { - let err_str = is_outdated ? "Block outdated" : "Block expired"; - let time_now = Date.now(); + const err_str = blockTemplate ? "Block outdated" : "Block expired"; + const time_now = Date.now(); if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { console.warn(threadName + err_str + ', Height: ' + job.height + ' (diff ' + job.difficulty + ') from ' + miner.logString); lastMinerLogTime[miner.payout] = time_now; } - miner.sendNewJob(); + miner.sendSameCoinJob(); sendReply(err_str); - global.database.storeInvalidShare(miner.invalidShareProto); + miner.storeInvalidShare(); return; } } else { - blockTemplate = activeBlockTemplate[job.algo_type]; + blockTemplate = activeBlockTemplates[job.coin]; + // kill miner if it mines block template for disabled coin for more than some time + if (!lastCoinHashFactorMM[job.coin] && Date.now() - blockTemplate.timeCreated > 60*60*1000) { + sendReplyFinal("This algo was temporary disabled due to coin daemon issues. Consider using https://github.com/MoneroOcean/meta-miner to allow your miner auto algo switch in this case."); + return; + } } - job.rewarded_difficulty2 = job.rewarded_difficulty * job.algoHashFactor; + job.rewarded_difficulty2 = job.rewarded_difficulty * job.coinHashFactor; + //job.rewarded_difficulty = Math.floor(job.rewarded_difficulty); + //if (job.rewarded_difficulty === 0) job.rewarded_difficulty = 1; - let shareAccepted = processShare(miner, job, blockTemplate, params); - if (shareAccepted === null) { - sendReply('Throttled down share submission (please use high fixed diff or use xmr-node-proxy)'); - return; - } - miner.checkBan(shareAccepted); + processShare(miner, job, blockTemplate, params, function(shareAccepted) { + if (miner.removed_miner) return; + if (shareAccepted === null) { + sendReply('Throttled down share submission (please increase difficulty)'); + return; + } + miner.checkBan(shareAccepted); - if (global.config.pool.trustedMiners) { - if (shareAccepted) { - miner.trust.probability -= global.config.pool.trustChange; - if (miner.trust.probability < (global.config.pool.trustMin)) { - miner.trust.probability = global.config.pool.trustMin; + if (global.config.pool.trustedMiners) { + if (shareAccepted) { + miner.trust.trust += job.rewarded_difficulty2; + miner.trust.check_height = 0; + } else { + debug(threadName + "Share trust broken by " + miner.logString); + miner.storeInvalidShare(); + miner.trust.trust = 0; } - miner.trust.penalty--; - miner.trust.threshold--; - miner.trust.check_height = 0; + } + + if (!shareAccepted) { + sendReply("Low difficulty share"); + return; + } + + miner.lastShareTime = Date.now() / 1000 || 0; + if (miner.protocol === "grin") { + sendReply(null, "ok"); + } else if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { + sendReply(null, true); } else { - if (miner.trust.probability == 256) { - badMinerLastShareTime[miner.payout + ":" + miner.identifier + ":" + miner.ipAddress] = Date.now(); - debug(threadName + "Banned miner for some time " + miner.logString); - removeMiner(miner); - sendReply('Low difficulty share'); - return; - } - debug(threadName + "Share trust broken by " + miner.logString); - global.database.storeInvalidShare(miner.invalidShareProto); - miner.trust.probability = 256; - miner.trust.penalty = global.config.pool.trustPenalty; - miner.trust.threshold = global.config.pool.trustThreshold; + sendReply(null, { status: 'OK' }); } - } + //if (miner.debugMiner) console.log("SUBMIT OK"); + }); + break; + } - if (!shareAccepted) { - sendReply('Low difficulty share'); + case 'keepalive': + case 'keepalived': { + if (!params) { + sendReplyFinal("No params specified"); return; } - - miner.lastShareTime = Date.now() / 1000 || 0; - - sendReply(null, {status: 'OK'}); - break; - case 'keepalived': - miner = activeSmartMiners[params.id]; - if (!miner) miner = activeMiners[params.id]; + const minerId = params.id ? params.id : (socket.miner_ids && socket.miner_ids.length == 1 ? socket.miner_ids[0] : ""); + let miner = activeMiners.get(minerId); if (!miner) { - sendReply('Unauthenticated'); + sendReplyFinal("Unauthenticated"); return; } miner.heartbeat(); - sendReply(null, { - status: 'KEEPALIVED' - }); + sendReply(null, { status: 'KEEPALIVED' }); break; + } } } if (global.config.general.allowStuckPoolKill && fs.existsSync("block_template_is_stuck")) { console.error("Stuck block template was detected on previous run. Please fix monerod and remove block_template_is_stuck file after that. Exiting..."); - process.exit(); + setTimeout(function() { process.exit(); }, 5*1000); + return; } -if (cluster.isMaster) { - let numWorkers = require('os').cpus().length; - global.config.ports.forEach(function (portData) { - minerCount[portData.port] = 0; +setInterval(function dump_vars() { + const fn = "dump" + (cluster.isMaster ? "" : "_" + process.env['WORKER_ID'].toString()); + fs.access(fn, fs.F_OK, function(err) { + if (!err) return; + console.log("DUMPING VARS TO " + fn + " FILE"); + let s = fs.createWriteStream(fn, {'flags': 'a'}); + + s.write("activeMiners:\n"); + for (var [minerId, miner] of activeMiners) s.write(minerId + ": " + JSON.stringify(miner, null, '\t') + "\n"); + + s.write("\n\n\npastBlockTemplates:\n"); + s.write(JSON.stringify(pastBlockTemplates, null, '\t') + "\n"); + + s.write("\n\n\nlastBlockHash:\n"); + s.write(JSON.stringify(lastBlockHash, null, '\t') + "\n"); + + s.write("\n\n\nlastCoinHashFactor:\n"); + s.write(JSON.stringify(lastCoinHashFactor, null, '\t') + "\n"); + + s.write("\n\n\nnewCoinHashFactor:\n"); + s.write(JSON.stringify(newCoinHashFactor, null, '\t') + "\n"); + + s.write("\n\n\nlastCoinHashFactorMM:\n"); + s.write(JSON.stringify(lastCoinHashFactorMM, null, '\t') + "\n"); + + s.write("\n\n\nactiveBlockTemplates:\n"); + s.write(JSON.stringify(activeBlockTemplates, null, '\t') + "\n"); + + s.write("\n\n\nproxyMiners:\n"); + s.write(JSON.stringify(proxyMiners, null, '\t') + "\n"); + + s.write("\n\n\nanchorBlockHeight: " + anchorBlockHeight + "\n"); + s.write("\n\n\nanchorBlockPrevHeight: " + anchorBlockPrevHeight + "\n"); + + s.write("\n\n\nwalletTrust:\n"); + s.write(JSON.stringify(walletTrust, null, '\t') + "\n"); + + s.write("\n\n\nwalletLastSeeTime:\n"); + s.write(JSON.stringify(walletLastSeeTime, null, '\t') + "\n"); + + s.write("\n\n\nwalletAcc:\n"); + s.write(JSON.stringify(walletAcc, null, '\t') + "\n"); + + s.write("\n\n\nwalletWorkerCount:\n"); + s.write(JSON.stringify(walletWorkerCount, null, '\t') + "\n"); + + s.write("\n\n\nis_walletAccFinalizer:\n"); + s.write(JSON.stringify(is_walletAccFinalizer, null, '\t') + "\n"); + + s.write("\n\n\nbannedTmpIPs:\n"); + s.write(JSON.stringify(bannedTmpIPs, null, '\t') + "\n"); + + s.write("\n\n\nbannedTmpWallets:\n"); + s.write(JSON.stringify(bannedTmpWallets, null, '\t') + "\n"); + + s.write("\n\n\nbannedBigTmpWallets:\n"); + s.write(JSON.stringify(bannedBigTmpWallets, null, '\t') + "\n"); + + s.end(); + }); +}, 60*1000); + +let master_cluster_worker_id_map = {}; + +function getUniqueWorkerID(cb) { + if (!global.config.eth_pool_support) return cb(0, 1); + global.mysql.query("SELECT id FROM pool_workers WHERE pool_id = ? AND worker_id = ?", [global.config.pool_id, process.env['WORKER_ID']]).then(function (rows) { + if (rows.length === 0) { + global.mysql.query("INSERT INTO pool_workers (pool_id, worker_id) VALUES (?, ?) ON DUPLICATE KEY UPDATE id=id", [global.config.pool_id, process.env['WORKER_ID']]).then(function() { + return getUniqueWorkerID(cb); + }).catch(function(err) { + console.error("Can't register unique pool worker for " + global.config.pool_id + " pool_id and " + process.env['WORKER_ID'] + " worker_id"); + process.exit(1); + }); + } else if (rows.length !== 1) { + console.error("Can't get unique pool worker for " + global.config.pool_id + " pool_id and " + process.env['WORKER_ID'] + " worker_id"); + process.exit(1); + } + global.mysql.query("SELECT MAX(id) as maxId FROM pool_workers").then(function (rows_max) { + if (rows_max.length !== 1) { + console.error("Can't get max id from pool_workers table"); + process.exit(1); + } + return cb(rows[0].id, rows_max[0].maxId); + }); }); +} + +if (cluster.isMaster) { + const numWorkers = require('os').cpus().length; + for (let i = 1; i <= numWorkers; ++ i) { + minerCount[i] = []; + global.config.ports.forEach(function (portData) { + minerCount[i][portData.port] = 0; + }); + } registerPool(); setInterval(function () { - if ("" in activeBlockTemplate) { - global.mysql.query("UPDATE pools SET last_checkin = ?, active = ?, blockIDTime = now(), blockID = ?, port = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, activeBlockTemplate[""].height, activeBlockTemplate[""].port, global.config.pool_id]); + if ("" in activeBlockTemplates) { + global.mysql.query("UPDATE pools SET last_checkin = ?, active = ?, blockIDTime = now(), blockID = ?, port = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, activeBlockTemplates[""].height, activeBlockTemplates[""].port, global.config.pool_id]); } else { global.mysql.query("UPDATE pools SET last_checkin = ?, active = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, global.config.pool_id]); } global.config.ports.forEach(function (portData) { - global.mysql.query("UPDATE ports SET lastSeen = now(), miners = ? WHERE pool_id = ? AND network_port = ?", [minerCount[portData.port], global.config.pool_id, portData.port]); + let miner_count = 0; + for (let i = 1; i <= numWorkers; ++ i) miner_count += minerCount[i][portData.port]; + global.mysql.query("UPDATE ports SET lastSeen = now(), miners = ? WHERE pool_id = ? AND network_port = ?", [miner_count, global.config.pool_id, portData.port]); }); - }, 10*1000); + }, 30*1000); setInterval(function () { - if (!("" in activeBlockTemplate)) return; + if (!("" in activeBlockTemplates)) return; global.mysql.query("SELECT blockID, port FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { let top_height = 0; - const port = activeBlockTemplate[""].port; - const height = activeBlockTemplate[""].height; + const port = activeBlockTemplates[""].port; + const height = activeBlockTemplates[""].height; rows.forEach(function (row) { if (row.port != port) return; if (row.blockID > top_height) top_height = row.blockID; @@ -1571,7 +2366,7 @@ if (cluster.isMaster) { console.error("!!! Current block height " + height + " is stuck compared to top height (" + top_height + ") amongst other leaf nodes for " + port + " port"); if (!(port in lastBlockFixTime)) lastBlockFixTime[port] = Date.now(); - if (Date.now() - lastBlockFixTime[port] > 5*60*1000) { + if (Date.now() - lastBlockFixTime[port] > 20*60*1000) { if (!(port in lastBlockFixCount)) lastBlockFixCount[port] = 1; else ++ lastBlockFixCount[port]; @@ -1586,22 +2381,7 @@ if (cluster.isMaster) { return; } - global.support.sendEmail(global.config.general.adminEmail, - "Pool server " + global.config.hostname + " has stuck block template", - "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " with current block height " + - height + " is stuck compared to top height (" + top_height + ") amongst other leaf nodes for " + - port + " port\nAttempting to fix..." - ); - if (fs.existsSync(fix_daemon_sh)) { - child_process.exec(fix_daemon_sh + " " + port, function callback(error, stdout, stderr) { - console.log("> " + fix_daemon_sh + " " + port); - console.log(stdout); - console.error(stderr); - if (error) console.error(fix_daemon_sh + " script returned error exit code: " + error.code); - }); - } else { - console.error("No " + fix_daemon_sh + " script was found to fix stuff"); - } + global.coinFuncs.fixDaemonIssue(height, top_height, port); lastBlockFixTime[port] = Date.now(); } } else { @@ -1622,9 +2402,8 @@ if (cluster.isMaster) { console.log('Master cluster setting up ' + numWorkers + ' workers...'); for (let i = 0; i < numWorkers; i++) { - let worker = cluster.fork(); + let worker = cluster.fork({ WORKER_ID: master_cluster_worker_id_map[i + 1] = i + 1 }); worker.on('message', messageHandler); - workerList.push(worker); } cluster.on('online', function (worker) { @@ -1632,45 +2411,91 @@ if (cluster.isMaster) { }); cluster.on('exit', function (worker, code, signal) { - console.log('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal); + console.error('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal); console.log('Starting a new worker'); - worker = cluster.fork(); + const prev_worker_id = master_cluster_worker_id_map[worker.id]; + delete master_cluster_worker_id_map[worker.id]; + worker = cluster.fork({ WORKER_ID: prev_worker_id }); + master_cluster_worker_id_map[worker.id] = prev_worker_id; worker.on('message', messageHandler); - workerList.push(worker); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Started new worker " + prev_worker_id, + "Hello,\r\nMaster thread of " + global.config.hostname + " starts new worker with id " + prev_worker_id); }); - if (!global.config.daemon.activePort) { - console.warn("global.config.daemon.activePort is not defined, using fixed global.config.daemon.port instead"); - global.config.daemon.activePort = global.config.daemon.port; + newCoinHashFactor[""] = lastCoinHashFactor[""] = lastCoinHashFactorMM[""] = 1; + templateUpdate(""); + setTimeout(templateUpdate, DAEMON_POLL_MS, "", true); + + if (global.config.daemon.enableAlgoSwitching) { + if (global.config.daemon.enableAlgoSwitching) COINS.forEach(function(coin) { + newCoinHashFactor[coin] = lastCoinHashFactor[coin] = lastCoinHashFactorMM[coin] = 0; + setInterval(updateCoinHashFactor, 5*1000, coin); + templateUpdate(coin); + setTimeout(templateUpdate, DAEMON_POLL_MS, coin, true); + }); } else { - setInterval(updateActivePort, 3*1000, ""); - ALGOS.forEach(function(algo) { - if ("activePort" + algo in global.config.daemon) { - setInterval(updateActivePort, 5*1000, algo); - templateUpdate(algo); - setTimeout(templateUpdate, 50, algo, true); + console.warn("global.config.daemon.enableAlgoSwitching is not enabled"); + } + + global.support.sendEmail(global.config.general.adminEmail, "Pool server " + global.config.hostname + " online", "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " is online"); + + let block_notify_server = net.createServer(function (socket) { + let timer = setTimeout(function() { + console.error(threadName + "Timeout waiting for block notify input"); + socket.destroy(); + }, 3*1000); + let buff = ""; + socket.on('data', function (buff1) { + buff += buff1; + }); + socket.on('end', function () { + clearTimeout(timer); + timer = null; + const port = parseInt(buff.toString()); + const coin = global.coinFuncs.PORT2COIN(port); + if (typeof(coin) === 'undefined') { + console.error(threadName + "Block notify for unknown coin with " + port + " port"); } else { - console.warn("global.config.daemon." + "activePort" + algo + " is not defined, so ignoring its algo changes"); + //console.log(threadName + "Block notify for coin " + coin + " with " + port + " port"); + templateUpdate(coin, false); } }); - } + socket.on('error', function() { + console.error(threadName + "Socket error on block notify port"); + socket.destroy(); + }); + }); + block_notify_server.listen(BLOCK_NOTIFY_PORT, "127.0.0.1", function() { + console.log(threadName + "Block notify server on " + BLOCK_NOTIFY_PORT + " port started"); + }); + +} else getUniqueWorkerID(function(id, maxId) { + uniqueWorkerId = id; + uniqueWorkerIdBits = 0; + while (maxId) { maxId >>= 1; ++ uniqueWorkerIdBits; } + freeEthExtranonces = [...Array(1 << (16 - uniqueWorkerIdBits)).keys()]; + console.log(threadName + "Starting pool worker with " + uniqueWorkerId + " unique id and " + uniqueWorkerIdBits + " reserved bits"); + + newCoinHashFactor[""] = lastCoinHashFactor[""] = lastCoinHashFactorMM[""] = 1; templateUpdate(""); - setTimeout(templateUpdate, 50, "", true); - global.support.sendEmail(global.config.general.adminEmail, "Pool server " + global.config.hostname + " online", "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " is online"); -} else { - templateUpdate(""); - if (global.config.daemon.activePortHeavy && global.config.daemon.activePortLight) { - ALGOS.forEach(function(algo) { templateUpdate(algo); }); - } + if (global.config.daemon.enableAlgoSwitching) COINS.forEach(function(coin) { + newCoinHashFactor[coin] = lastCoinHashFactor[coin] = lastCoinHashFactorMM[coin] = 0; + templateUpdate(coin); + }); anchorBlockUpdate(); setInterval(anchorBlockUpdate, 3*1000); setInterval(checkAliveMiners, 60*1000); setInterval(retargetMiners, global.config.pool.retargetTime * 1000); setInterval(function () { - bannedIPs = {}; - }, 60*1000); + bannedTmpIPs = {}; + bannedTmpWallets = {}; + }, 10*60*1000); + + setInterval(function () { + bannedBigTmpWallets = {}; + }, 60*60*1000); function add_bans(is_show) { global.mysql.query("SELECT mining_address, reason FROM bans").then(function (rows) { @@ -1709,14 +2534,14 @@ if (cluster.isMaster) { let trust = parseInt(parts[1], 10); let time = parseInt(parts[2], 10); if (Date.now() - time < 24*60*60*1000 && (!(wallet in walletTrust) || trust < walletTrust[wallet])) { - console.log("Adding " + trust.toString() + " trust for " + wallet + " wallet"); + debug("Adding " + trust.toString() + " trust for " + wallet + " wallet"); walletTrust[wallet] = trust; walletLastSeeTime[wallet] = time; } }); } - // dump wallet trust to file + // dump wallet trust and miner agents to file setInterval(function () { let str = ""; for (let wallet in walletTrust) { @@ -1728,10 +2553,51 @@ if (cluster.isMaster) { delete walletLastSeeTime[wallet]; } } - let fn = "wallet_trust_" + cluster.worker.id.toString(); + const fn = "wallet_trust_" + process.env['WORKER_ID'].toString(); fs.writeFile(fn, str, function(err) { if (err) console.error("Error saving " + fn + " file"); }); + + if (process.env['WORKER_ID'] == 1) { + let str2 = ""; + for (let agent in minerAgents) { str2 += agent + "\n"; } + const fn2 = "miner_agents"; + fs.writeFile(fn2, str2, function(err) { if (err) console.error("Error saving " + fn2 + " file"); }); + } + + //cacheTargetHex = {}; + }, 10*60*1000); + // get extra wallets to check + setInterval(function () { + const extra_wallet_verify_fn = "extra_wallet_verify.txt"; + extra_wallet_verify = {}; + fs.access(extra_wallet_verify_fn, fs.F_OK, function(err) { + if (err) return; + let rs = fs.createReadStream(extra_wallet_verify_fn); + rs.on('error', function() { console.error("Can't open " + extra_wallet_verify_fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('line', function (line) { + console.log(threadName + "[EXTRA CHECK] added: '" + line + "'"); + extra_wallet_verify[line] = 1; + }); + const fn = "extra_verify_wallet_hashes_" + process.env['WORKER_ID'].toString(); + fs.writeFile(fn, extra_verify_wallet_hashes.join("\n"), function(err) { if (err) console.error("Error saving " + fn + " file"); }); + extra_verify_wallet_hashes = []; + }); + const wallet_debug_fn = "wallet_debug.txt"; + wallet_debug = {}; + fs.access(wallet_debug_fn, fs.F_OK, function(err) { + if (err) return; + let rs = fs.createReadStream(wallet_debug_fn); + rs.on('error', function() { console.error("Can't open " + wallet_debug_fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('line', function (line) { + console.log(threadName + "[WALLET DEBUG] added: '" + line + "'"); + wallet_debug[line] = 1; + }); + }); + }, 5*60*1000); + let lastGarbageFromIpTime = {}; async.each(global.config.ports, function (portData) { @@ -1742,29 +2608,42 @@ if (cluster.isMaster) { if (!jsonData.id) { console.warn('Miner RPC request missing RPC id'); return; - } - else if (!jsonData.method) { + } else if (!jsonData.method) { console.warn('Miner RPC request missing RPC method'); return; } - else if (!jsonData.params) { - console.warn('Miner RPC request missing RPC params'); - return; - } let sendReply = function (error, result) { - if (!socket.writable) { - return; - } - let sendData = JSON.stringify({ - id: jsonData.id, - jsonrpc: "2.0", - error: error ? {code: -1, message: error} : null, - result: result - }) + "\n"; - socket.write(sendData); + if (!socket.writable) return; + let reply = { + jsonrpc: "2.0", + id: jsonData.id, + error: error ? {code: -1, message: error} : null, + result: result + }; + if (jsonData.id === "Stratum") reply.method = jsonData.method; + debug("[MINER] REPLY TO MINER: " + JSON.stringify(reply)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] reply " + JSON.stringify(reply)); + socket.write(JSON.stringify(reply) + "\n"); }; - handleMinerData(jsonData.method, jsonData.params, socket.remoteAddress, portData, sendReply, pushMessage); + let sendReplyFinal = function (error, timeout) { + setTimeout(function() { + if (!socket.writable) return; + let reply = { + jsonrpc: "2.0", + id: jsonData.id, + error: {code: -1, message: error}, + result: null + }; + if (jsonData.id === "Stratum") reply.method = jsonData.method; + debug("[MINER] FINAL REPLY TO MINER: " + JSON.stringify(reply)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] final reply " + JSON.stringify(reply)); + socket.end(JSON.stringify(reply) + "\n"); + }, (timeout ? timeout : 9) * 1000); + }; + debug("[MINER] GOT FROM MINER: " + JSON.stringify(jsonData)); + handleMinerData(socket, jsonData.id, jsonData.method, jsonData.params, socket.remoteAddress, portData, sendReply, sendReplyFinal, pushMessage); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] recieved " + JSON.stringify(jsonData)); }; function socketConn(socket) { @@ -1773,16 +2652,12 @@ if (cluster.isMaster) { let dataBuffer = ''; - let pushMessage = function (method, params) { - if (!socket.writable) { - return; - } - let sendData = JSON.stringify({ - jsonrpc: "2.0", - method: method, - params: params - }) + "\n"; - socket.write(sendData); + let pushMessage = function (body) { + if (!socket.writable) return; + body.jsonrpc = "2.0"; + debug("[MINER] PUSH TO MINER: " + JSON.stringify(body)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] push " + JSON.stringify(body)); + socket.write(JSON.stringify(body) + "\n"); }; socket.on('data', function (d) { @@ -1831,12 +2706,11 @@ if (cluster.isMaster) { dataBuffer = incomplete; } }).on('error', function (err) { - if (err.code !== 'ECONNRESET') { - console.warn(threadName + "Socket Error from " + socket.remoteAddress + " Error: " + err); - } + //debug(threadName + "Socket Error " + err.code + " from " + socket.remoteAddress + " Error: " + err); }).on('close', function () { - pushMessage = function () { - }; + pushMessage = function () {}; + if (socket.miner_ids) socket.miner_ids.forEach(miner_id => removeMiner(activeMiners.get(miner_id))); + if ("eth_extranonce_id" in socket) freeEthExtranonces.push(socket.eth_extranonce_id); }); } @@ -1869,4 +2743,4 @@ if (cluster.isMaster) { }); } }); -} +}); diff --git a/lib/pool_stats.js b/lib/pool_stats.js new file mode 100644 index 000000000..211c07099 --- /dev/null +++ b/lib/pool_stats.js @@ -0,0 +1,468 @@ +"use strict"; +const debug = require("debug")("pool_stats"); +const async = require("async"); + +const threadName = "Worker Server "; +const max_blocks = 1000; +const max_altblocks = 10000; + +let lastBlockCheckIsFailed = {}; + +let price_btc = 0; +let price_usd = 0; +let price_eur = 0; +let min_block_rewards = {}; +let blockList = []; +let altblockList = []; +let altblockFound = {}; +let altblockFoundDone = 0; + +function get_cmc_price(symbol, callback) { + const slug = global.config.coin.name.toLowerCase(); + global.support.https_get("https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest?slug=" + slug + "&convert=" + symbol + "&CMC_PRO_API_KEY=" + global.config.general.cmcKey, function (res) { + if (res instanceof Object && "data" in res && "quote" in res.data[Object.keys(res.data)[0]] && symbol in res.data[Object.keys(res.data)[0]].quote) { + return callback(parseFloat(res.data[Object.keys(res.data)[0]].quote[symbol].price)); + } else { + console.error("Can't get price data from: " + JSON.stringify(res)); + return callback(0); + } + }); +} + +function get_cmc(callback) { + get_cmc_price("USD", function(usd) { + get_cmc_price("EUR", function(eur) { + get_cmc_price("BTC", function(btc) { + price_btc = btc ? btc : price_btc; + price_usd = usd ? usd : price_usd; + price_eur = eur ? eur : price_eur; + return callback({ btc: price_btc, usd: price_usd, eur: price_eur }); + }); + }); + }); +} + +function updatePoolStats(poolType) { + //console.log("Cleaned " + global.database.env.mdb_reader_check() + " stale readers"); + let cache; + if (typeof(poolType) !== 'undefined') { + cache = global.database.getCache(poolType + "_stats"); + let cache2 = global.database.getCache(poolType + "_stats2"); + cache.totalHashes = cache2.totalHashes; + cache.roundHashes = cache2.roundHashes; + } else { + console.log("Running pool stats"); + cache = global.database.getCache("global_stats"); + let cache2 = global.database.getCache("global_stats2"); + cache.totalHashes = cache2.totalHashes; + cache.roundHashes = cache2.roundHashes; + } + + let port_hash = global.database.getCache('port_hash'); + if (blockList.length > max_blocks) { + const newBlocks = global.database.getBlockList(poolType, 0, max_blocks); + let new_block_count = 0; + let prev_block_index = 0; + for (let block of newBlocks) { + if (block.hash == blockList[prev_block_index].hash) ++ prev_block_index; + else ++ new_block_count; + } + blockList = newBlocks.concat(blockList.slice(max_blocks - new_block_count)); + } else { + blockList = global.database.getBlockList(poolType); + } + if (altblockList.length > max_altblocks) { + const newBlocks = global.database.getAltBlockList(poolType, null, 0, max_altblocks); + let new_block_count = 0; + let prev_block_index = 0; + for (let block of newBlocks) { + if (block.hash == altblockList[prev_block_index].hash) ++ prev_block_index; + else ++ new_block_count; + } + altblockList = newBlocks.concat(altblockList.slice(max_altblocks - new_block_count)); + } else { + altblockList = global.database.getAltBlockList(poolType); + } + let min_block_rewards2 = global.database.getCache('min_block_rewards'); + if (min_block_rewards2) min_block_rewards = min_block_rewards2; + if (!(global.config.daemon.port in min_block_rewards)) min_block_rewards[global.config.daemon.port] = 0; + + async.series([ + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate)"); + return callback(null, cache.hash || 0); + }, + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for miner count for pool stats (miners)"); + return callback(null, cache.minerCount || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for totalHashes"); + return callback(null, cache.totalHashes || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB for lastBlockFoundTime for pool stats"); + let max_time = 0; + if (blockList.length !== 0) { + max_time = Math.floor(blockList[0].ts / 1000); + } + if (altblockList.length !== 0) { + max_time = Math.max(max_time, Math.floor(altblockList[0].ts / 1000)); + } + return callback(null, max_time); + }, + function (callback) { + //debug(threadName + "Checking LMDB for lastBlockFound height for pool stats"); + if (blockList.length === 0) { + return callback(null, 0); + } + return callback(null, blockList[0].height); + }, + function (callback) { + //debug(threadName + "Checking LMDB for totalBlocksFound for pool stats"); + return callback(null, blockList.length); + }, + function (callback) { + //debug(threadName + "Checking MySQL for total miners paid"); + if (typeof(poolType) !== 'undefined') { + global.mysql.query("SELECT payment_address, payment_id FROM payments WHERE pool_type = ? group by payment_address, payment_id", [poolType]).then(function (rows) { + return callback(null, rows.length); + }); + } else { + global.mysql.query("SELECT payment_address, payment_id FROM payments group by payment_address, payment_id").then(function (rows) { + return callback(null, rows.length); + }); + } + }, + function (callback) { + //debug(threadName + "Checking MySQL for total transactions count"); + if (typeof(poolType) !== 'undefined') { + global.mysql.query("SELECT distinct(transaction_id) from payments WHERE pool_type = ?", [poolType]).then(function (rows) { + return callback(null, rows.length); + }); + } else { + global.mysql.query("SELECT count(id) as txn_count FROM transactions").then(function (rows) { + if (typeof(rows[0]) !== 'undefined') { + return callback(null, rows[0].txn_count); + } else { + return callback(null, 0); + } + }); + } + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for roundHashes"); + return callback(null, cache.roundHashes || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB for altblock count for pool stats"); + return callback(null, altblockList.length); + }, + function (callback) { + //debug(threadName + "Checking LMDB for altBlocksFound array for each specific port"); + for (let i in altblockList) { + if (i >= altblockList.length - altblockFoundDone) break; + let block = altblockList[i]; + if (altblockFound.hasOwnProperty(block.port)) ++ altblockFound[block.port]; + else altblockFound[block.port] = 1; + } + altblockFoundDone = altblockList.length; + return callback(null, altblockFound); + }, + function (callback) { + //debug(threadName + "Checking MySQL for activePort value"); + return callback(null, global.config.daemon.port); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for active_ports value"); + let active_ports = global.database.getCache('active_ports'); + return callback(null, active_ports ? active_ports : []); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for xmr_profit value"); + let xmr_profit = global.database.getCache('xmr_profit'); + return callback(null, xmr_profit ? xmr_profit.value : 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for coin_profit value"); + let coin_xmr_profit = global.database.getCache('coin_xmr_profit'); + return callback(null, coin_xmr_profit ? coin_xmr_profit : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for xmr_profit_comment value"); + let coin_comment = global.database.getCache('coin_comment'); + return callback(null, coin_comment ? coin_comment : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for min_block_rewards value to set minBlockRewards"); + return callback(null, min_block_rewards); + }, + function (callback) { + let pending = 0; + for (let i in blockList) { + if (i > max_blocks) break; + const block = blockList[i]; + if (block.valid === true && block.unlocked === false) pending += global.support.coinToDecimal(block.value); + } + for (let i in altblockList) { + if (i > max_altblocks) break; + const altblock = altblockList[i]; + if (altblock.valid === true && altblock.unlocked === false) pending += altblock.port in min_block_rewards ? min_block_rewards[altblock.port] : 0; + } + return callback(null, pending); + }, + function (callback) { + if (typeof(poolType) === 'undefined' && price_btc == 0 && price_usd == 0 && price_eur == 0) { + return get_cmc(function(prices) { return callback(null, prices); }); + } else return callback(null, { btc: price_btc, usd: price_usd, eur: price_eur }); + }, + function (callback) { + let currentEfforts = {}; + for (let port of global.coinFuncs.getPORTS()) { + const value = global.database.getCache(port != global.config.daemon.port ? "global_stats2_" + port : "global_stats2"); + if (value !== false) currentEfforts[port] = value.roundHashes; + } + return callback(null, currentEfforts); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for pplns_port_shares value"); + let pplns_port_shares = global.database.getCache('pplns_port_shares'); + return callback(null, pplns_port_shares ? pplns_port_shares : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for pplns_window_time value"); + let pplns_window_time = global.database.getCache('pplns_window_time'); + return callback(null, pplns_window_time ? pplns_window_time : 0); + }, + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate) per port"); + return callback(null, port_hash || {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for portMinerCount"); + return callback(null, global.database.getCache('portMinerCount') || {}); + }, + function (callback) { + let portCoinAlgo = {}; + for (let port of global.coinFuncs.getPORTS()) portCoinAlgo[port] = global.coinFuncs.algoShortTypeStr(port, 0); + return callback(null, portCoinAlgo); + }, + ], function (err, result) { + global.database.setCache('pool_stats_' + (typeof(poolType) === 'undefined' ? 'global' : poolType), { + hashRate: result[0], + miners: result[1], + totalHashes: result[2], + lastBlockFoundTime: result[3] || 0, + lastBlockFound: result[4] || 0, + totalBlocksFound: result[5] || 0, + totalMinersPaid: result[6] || 0, + totalPayments: result[7] || 0, + roundHashes: result[8] || 0, + totalAltBlocksFound: result[9] || 0, + altBlocksFound: result[10] || {}, + activePort: result[11] || 0, + activePorts: result[12] || [], + activePortProfit: result[13] || 0, + coinProfit: result[14] || {}, + coinComment: result[15] || {}, + minBlockRewards: result[16] || {}, + pending: result[17] || 0, + price: result[18] || {}, + currentEfforts: result[19] || {}, + pplnsPortShares: result[20] || {}, + pplnsWindowTime: result[21] || 0, + portHash: result[22] || {}, + portMinerCount: result[23] || {}, + portCoinAlgo: result[24] || {}, + }); + setTimeout(updatePoolStats, 30*1000, poolType); + }); +} + +function updatePoolPorts(poolServers) { + //debug(threadName + "Updating pool ports"); + let local_cache = {global: []}; + let portCount = 0; + global.mysql.query("select * from ports where hidden = 0 and pool_id < 1000 and lastSeen >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { + rows.forEach(function (row) { + ++ portCount; + if (!local_cache.hasOwnProperty(row.port_type)) { + local_cache[row.port_type] = []; + } + local_cache[row.port_type].push({ + host: poolServers[row.pool_id], + port: row.network_port, + difficulty: row.starting_diff, + description: row.description, + miners: row.miners + }); + if (portCount === rows.length) { + let local_counts = {}; + let port_diff = {}; + let port_miners = {}; + let pool_type_count = 0; + let localPortInfo = {}; + for (let pool_type in local_cache) { // jshint ignore:line + ++ pool_type_count; + local_cache[pool_type].forEach(function (portData) { // jshint ignore:line + if (!local_counts.hasOwnProperty(portData.port)) { + local_counts[portData.port] = 0; + } + if (!port_diff.hasOwnProperty(portData.port)) { + port_diff[portData.port] = portData.difficulty; + } + if (!port_miners.hasOwnProperty(portData.port)) { + port_miners[portData.port] = 0; + } + if (port_diff[portData.port] === portData.difficulty) { + ++ local_counts[portData.port]; + port_miners[portData.port] += portData.miners; + } + localPortInfo[portData.port] = portData.description; + if (local_counts[portData.port] === Object.keys(poolServers).length) { + local_cache.global.push({ + host: { + blockID: typeof(local_cache[pool_type][0].host) === 'undefined' ? 0 : local_cache[pool_type][0].host.blockID, + blockIDTime: typeof(local_cache[pool_type][0].host) === 'undefined' ? 0 : local_cache[pool_type][0].host.blockIDTime, + hostname: global.config.pool.geoDNS, + }, + port: portData.port, + pool_type: pool_type, + difficulty: portData.difficulty, + miners: port_miners[portData.port], + description: localPortInfo[portData.port] + }); + } + }); + if (pool_type_count === Object.keys(local_cache).length) { + //debug(threadName + "Sending the following to the workers: " + JSON.stringify(local_cache)); + global.database.setCache('poolPorts', local_cache); + setTimeout(updatePoolInformation, 30*1000); + } + } + } + }); + }); +} + +function updatePoolInformation() { + let local_cache = {}; + //debug(threadName + "Updating pool information"); + global.mysql.query("select * from pools where id < 1000 and last_checkin >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { + rows.forEach(function (row) { + local_cache[row.id] = { + ip: row.ip, + blockID: row.blockID, + blockIDTime: global.support.formatDateFromSQL(row.blockIDTime), + hostname: row.hostname + }; + if (Object.keys(local_cache).length === rows.length) { + global.database.setCache('poolServers', local_cache); + updatePoolPorts(local_cache); + } + }); + }); +} + +let network_info = {}; + +function updateBlockHeader() { + const ports = global.config.daemon.enableAlgoSwitching ? global.coinFuncs.getPORTS() : [ global.config.daemon.port ]; + async.eachSeries(ports, function(port, next) { + global.coinFuncs.getPortLastBlockHeader(port, function(err, body){ + if (err) return next(); + network_info[port] = { + difficulty: parseInt(body.difficulty), + hash: body.hash ? body.hash : body.hashrate, + height: body.height, + value: body.reward, + ts: parseInt(body.timestamp), + }; + if (port == global.config.daemon.port) { + global.support.rpcPortDaemon(port, 'get_info', [], function (rpcResult) { + network_info.difficulty = rpcResult.result ? rpcResult.result.difficulty : body.difficulty; + network_info.hash = body.hash; + network_info.main_height = body.height; + network_info.height = body.height; + network_info.value = body.reward; + network_info.ts = body.timestamp; + return next(); + }); + } else { + return next(); + } + }, true); + }, function(err, result) { + global.database.setCache('networkBlockInfo', network_info); + setTimeout(updateBlockHeader, 30*1000); + }); +} + +function bad_header_start(port) { + console.error("Issue in getting block header for " + port + " port. Skipping node monitor"); + if (port in lastBlockCheckIsFailed) { + if (++ lastBlockCheckIsFailed[port] >= 5) global.support.sendEmail( + global.config.general.adminEmail, + 'Failed to query daemon for ' + port + ' port for last block header', + `The worker failed to return last block header for ` + port + ` port. Please verify if the daemon is running properly.` + ); + } else { + lastBlockCheckIsFailed[port] = 1; + } + return; +} + +function bad_header_stop(port) { + if (port in lastBlockCheckIsFailed) { + if (lastBlockCheckIsFailed[port] >= 5) global.support.sendEmail( + global.config.general.adminEmail, + 'Quering daemon for ' + port + ' port for last block header is back to normal', + `An warning was sent to you indicating that the the worker failed to return the last block header for ${port} port. + The issue seems to be solved now.` + ); + delete lastBlockCheckIsFailed[port]; + } +} + +function monitorNodes() { + global.mysql.query("SELECT blockID, hostname, ip, port FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { + global.coinFuncs.getPortLastBlockHeader(global.config.daemon.port, function (err, block) { + if (err !== null){ + bad_header_start(global.config.daemon.port); + return; + } + bad_header_stop(); + let top_height = 0; + let is_master_daemon_issue = rows.length > 1 ? true : false; + rows.forEach(function (row) { + if (row.port && row.port != global.config.daemon.port) { + console.error("INTERNAL ERROR: pool node port " + row.port + " do not match master port " + global.config.daemon.port); + is_master_daemon_issue = false; + return; + } + if (top_height < row.blockID) top_height = row.blockID; + if (Math.abs(block.height - row.blockID) > 3) { + global.support.sendEmail(global.config.general.adminEmail, + "Pool server behind in blocks", + "The pool server: " + row.hostname + " with IP: " + row.ip + " is " + (block.height - row.blockID) + " blocks behind for " + row.port + " port" + ); + } else { + is_master_daemon_issue = false; + } + }); + if (is_master_daemon_issue) global.coinFuncs.fixDaemonIssue(block.height, top_height, global.config.daemon.port); + }); + }); +} + +updatePoolStats(); +updatePoolStats('pplns'); +if (global.config.pps.enable === true) updatePoolStats('pps'); +if (global.config.solo.enable === true) updatePoolStats('solo'); +updatePoolInformation(); +updateBlockHeader(); + +monitorNodes(); +setInterval(monitorNodes, 5*60*1000); +setInterval(get_cmc, 15*60*1000, function() {}); + diff --git a/lib/remoteShare.js b/lib/remoteShare.js index e1efc970d..ab9cafb63 100644 --- a/lib/remoteShare.js +++ b/lib/remoteShare.js @@ -24,7 +24,7 @@ app.use(function(req, res, next){ // Master/Slave communication Handling function messageHandler(message) { - if (typeof message.shares === "number"){ + if (typeof message.raw_shares === "number"){ shareData.push(message); } } @@ -82,6 +82,7 @@ app.post('/leafApi', function (req, res) { function storeShares(){ if (Object.keys(shareData).length > 0){ + console.log('Storing ' + Object.keys(shareData).length + ' shares'); global.database.storeBulkShares(shareData); shareData = []; } diff --git a/lib/remote_comms.js b/lib/remote_comms.js index e857034ab..faf018bf0 100644 --- a/lib/remote_comms.js +++ b/lib/remote_comms.js @@ -10,17 +10,14 @@ function Database() { async.doUntil( function (intCallback) { request.post({url: global.config.general.shareHost, body: task.body, forever: true}, function (error, response, body) { - if (!error) { - return intCallback(null, response.statusCode); - } - return intCallback(null, 0); + return intCallback(null, error ? 0 : response.statusCode); }); }, - function (data) { - return data === 200; + function (data, untilCB) { + return untilCB(null, data === 200); }, function () { - callback(); + return callback(); }); }, require('os').cpus().length*32); @@ -66,7 +63,7 @@ function Database() { setInterval(function(queue_obj){ if ((queue_obj.length() > 20 || queue_obj.running() > 20) && global.database.thread_id === '(Master) '){ - console.log(global.database.thread_id + "Queue debug state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); + console.log(global.database.thread_id + "Remote queue state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); } }, 30*1000, this.sendQueue); diff --git a/lib/support.js b/lib/support.js index f21cc108f..e65805479 100644 --- a/lib/support.js +++ b/lib/support.js @@ -2,7 +2,6 @@ const CircularBuffer = require('circular-buffer'); const https = require('https'); const request = require('request'); -const requestJson = require('request-json'); const moment = require('moment'); const debug = require('debug')('support'); const fs = require('fs'); @@ -80,7 +79,7 @@ function sendEmailReal(toAddress, subject, email_body, retry) { } function sendEmail(toAddress, subject, body, wallet){ - if (toAddress === global.config.general.adminEmail && subject.indexOf("FYI") === -1) { + if (toAddress === global.config.general.adminEmail && !subject.includes("FYI")) { sendEmailReal(toAddress, subject, body); } else { let reEmail = /^([a-zA-Z0-9_\.-])+@(([a-zA-Z0-9-])+\.)+([a-zA-Z0-9]{2,4})+$/; @@ -109,50 +108,50 @@ function sendEmail(toAddress, subject, body, wallet){ } } -function jsonRequest(host, port, data, is_wallet, callback, path, timeout) { - let uri; - if (global.config.rpc.https) { - uri = "https://" + host + ":" + port + "/"; - } else { - uri = "http://" + host + ":" + port + "/"; +function sendEmailAdmin(subject, body){ + sendEmail(global.config.general.adminEmail, subject, body); +} + +function jsonRequest(host, port, data, callback, path, timeout) { + let options = { + url: (global.config.rpc.https ? "https://" : "http://") + host + ":" + port + "/" + path, + method: data ? "POST" : "GET", + timeout: timeout, + headers: { + "Content-Type": "application/json", + "Accept": "application/json" + } + }; + if (global.config.daemon.basicAuth) { + options.headers["Authorization"] = global.config.daemon.basicAuth; } - debug("JSON URI: " + uri + path + " Args: " + JSON.stringify(data)); - let client = requestJson.createClient(uri, {timeout: timeout}); - client.headers["Content-Type"] = "application/json"; - client.headers["Content-Length"] = data.length; - client.headers["Accept"] = "application/json"; - if (is_wallet && global.config.payout.rpcPasswordEnabled && global.config.payout.rpcPasswordPath){ - fs.readFile(port === global.config.daemon.port ? global.config.payout.rpcPasswordPath : global.config.payout["rpcPasswordPath" + port.toString()], 'utf8', function(err, data){ - if (err){ - console.error("RPC password enabled, unable to read the file due to: " + JSON.stringify(err)); - return; - } - let passData = data.split(":"); - client.setBasicAuth(passData[0], passData[1]); - request.post(uri, { - auth:{ - user: passData[0], - pass: passData[1], - sendImmediately: false - }, - data: JSON.stringify(data) - }, function (err, res, body) { - if (err) { - return callback(err); - } - debug("JSON result: " + JSON.stringify(body)); - return callback(body); - }); - }); - } else { - client.post(path, data, function (err, res, body) { - if (err) { - return callback(err); - } - debug("JSON result: " + JSON.stringify(body)); - return callback(body); - }); + if (global.config.daemon["X-API-KEY"]) { + options.headers["X-API-KEY"] = global.config.daemon["X-API-KEY"]; + options.headers["api_key"] = global.config.daemon["X-API-KEY"]; } + + if (data) { + const data2 = typeof data === 'string' ? data : JSON.stringify(data); + options.headers["Content-Length"] = data2.length; + options.body = data2; + } + let reply_fn = function (err, res, body) { + if (err) { + if (typeof(err) === "string") console.error("Error doing " + uri + path + " request: " + err); + return callback(err); + } + let json; + try { + json = JSON.parse(body); + } catch (e) { + debug("JSON parse exception: " + body); + return callback("JSON parse exception: " + body); + } + debug("JSON result: " + JSON.stringify(json)); + return callback(json, res.statusCode); + }; + debug("JSON REQUST: " + JSON.stringify(options)); + request(options, reply_fn); } function rpc(host, port, method, params, callback, timeout) { @@ -162,17 +161,11 @@ function rpc(host, port, method, params, callback, timeout) { method: method, params: params }; - return jsonRequest(host, port, data, false, callback, 'json_rpc', timeout); + return jsonRequest(host, port, data, callback, 'json_rpc', timeout); } -function rpc_wallet(host, port, method, params, callback) { - let data = { - id: "0", - jsonrpc: "2.0", - method: method, - params: params - }; - return jsonRequest(host, port, data, true, callback, 'json_rpc', 30*60*1000); +function rpc2(host, port, method, params, callback, timeout) { + return jsonRequest(host, port, params, callback, method, timeout); } function https_get(url, callback) { @@ -237,34 +230,19 @@ function https_get(url, callback) { req.end(); } -function getAlgoHashFactor(algo, callback) { - global.mysql.query("SELECT item_value FROM config WHERE module = 'daemon' and item = 'algoHashFactor" + algo + "'").then(function (rows) { +function getCoinHashFactor(coin, callback) { + global.mysql.query("SELECT item_value FROM config WHERE module = 'daemon' and item = 'coinHashFactor" + coin + "'").then(function (rows) { if (rows.length != 1) { - console.error("Can't get config.daemon.algoHashFactor" + algo + " value"); + console.error("Can't get config.daemon.coinHashFactor" + coin + " value"); return callback(null); } callback(parseFloat(rows[0].item_value)); }); } -function getActivePort(algo, callback) { - global.mysql.query("SELECT item_value FROM config WHERE module = 'daemon' and item = 'activePort" + algo + "'").then(function (rows) { - if (rows.length != 1) { - console.error("Can't get config.daemon.activePort" + algo + " value"); - return callback(null); - } - callback(parseInt(rows[0].item_value)); - }); -} - -function setAlgoHashFactor(algo, algoHashFactor) { - global.mysql.query("UPDATE config SET item_value = ? WHERE module = 'daemon' and item = 'algoHashFactor" + algo + "'", [algoHashFactor]); - global.config.daemon["algoHashFactor" + algo] = algoHashFactor; -} - -function setActivePort(algo, activePort) { - global.mysql.query("UPDATE config SET item_value = ? WHERE module = 'daemon' and item = 'activePort" + algo + "'", [activePort]); - global.config.daemon["activePort" + algo] = activePort; +function setCoinHashFactor(coin, coinHashFactor) { + global.mysql.query("UPDATE config SET item_value = ? WHERE module = 'daemon' and item = 'coinHashFactor" + coin + "'", [coinHashFactor]); + global.config.daemon["coinHashFactor" + coin] = coinHashFactor; } function formatDate(date) { @@ -316,6 +294,12 @@ function tsCompare(a, b) { return 0; } +function port_wallet_ip(port) { + const ip = global.config.wallet["address_" + port.toString()]; + if (ip) return ip; + return global.config.wallet.address; +} + module.exports = function () { return { rpcDaemon: function (method, params, callback) { @@ -324,11 +308,23 @@ module.exports = function () { rpcPortDaemon: function (port, method, params, callback) { rpc(global.config.daemon.address, port, method, params, callback, 30*1000); }, + rpcPortDaemon2: function (port, method, params, callback) { + rpc2(global.config.daemon.address, port, method, params, callback, 30*1000); + }, rpcWallet: function (method, params, callback) { - rpc_wallet(global.config.wallet.address, global.config.wallet.port, method, params, callback); + rpc(port_wallet_ip(global.config.wallet.port), global.config.wallet.port, method, params, callback, 30*60*1000); }, rpcPortWallet: function (port, method, params, callback) { - rpc_wallet(global.config.wallet.address, port, method, params, callback); + rpc(port_wallet_ip(port), port, method, params, callback, 30*60*1000); + }, + rpcPortWallet2: function (port, method, params, callback) { + rpc2(port_wallet_ip(port), port, method, params, callback, 30*60*1000); + }, + rpcPortWalletShort: function (port, method, params, callback) { + rpc(port_wallet_ip(port), port, method, params, callback, 10*1000); + }, + rpcPortWalletShort2: function (port, method, params, callback) { + rpc2(port_wallet_ip(port), port, method, params, callback, 10*1000); }, circularBuffer: circularBuffer, formatDate: formatDate, @@ -339,11 +335,10 @@ module.exports = function () { formatDateFromSQL: formatDateFromSQL, blockCompare: blockCompare, sendEmail: sendEmail, + sendEmailAdmin: sendEmailAdmin, tsCompare: tsCompare, - getAlgoHashFactor: getAlgoHashFactor, - getActivePort: getActivePort, - setAlgoHashFactor: setAlgoHashFactor, - setActivePort: setActivePort, + getCoinHashFactor: getCoinHashFactor, + setCoinHashFactor: setCoinHashFactor, https_get: https_get, }; }; diff --git a/lib/worker.js b/lib/worker.js index 9a013c2b7..f8c03058e 100644 --- a/lib/worker.js +++ b/lib/worker.js @@ -1,290 +1,349 @@ "use strict"; -const debug = require("debug")("worker"); -const async = require("async"); +const debug = require("debug")("worker"); const sprintf = require("sprintf-js").sprintf; -let threadName = "Worker Server "; let cycleCount = 0; let hashrate_avg_min = 10; -let stat_change_alert = 0.3; +let stat_change_alert = 0.6; -let lastBlockCheckIsFailed = {}; let prev_pool_state_time; let prev_pool_hashrate; let prev_pool_workers; let stats_cache = {}; - -function updateShareStats() { +let miner_history_update_time = {}; + +let pool_type_str = {}; +pool_type_str[global.protos.POOLTYPE.PPLNS] = 'pplns'; +pool_type_str[global.protos.POOLTYPE.PPS] = 'pps'; +pool_type_str[global.protos.POOLTYPE.SOLO] = 'solo'; + +let identifiers = {}; +let minerSet = {}; +let minerPortSet = {}; +let localMinerCount = {}; +let localStats = {}; +let localPortHashes = {}; +let localTimes = {}; + +let prevMinerSet = {}; +let cache_updates = {}; +let portMinerCount = {}; + +function updateShareStats2(height, callback) { // This is an omni-worker to deal with all things share-stats related // Time based averages are worked out on ring buffers. // Buffer lengths? You guessed it, configured in SQL. // Stats timeouts are 30 seconds, so everything for buffers should be there. - let currentTime = Date.now(); - async.waterfall([ - function (callback) { - global.coinFuncs.getLastBlockHeader(function (err, body) { - if (err !== null){ - bad_header_start(global.config.daemon.port); - return callback(err, "Invalid block header"); - } - callback(null, body.height + 1); - }); - }, - function (height, callback) { - bad_header_stop(global.config.daemon.port); - console.log("Starting stats collection for " + height + " height"); - - const locTime = currentTime - (hashrate_avg_min*60*1000); - const identifierTime = currentTime - (2*hashrate_avg_min*60*1000); - - let identifiers = {}; - let minerSet = {}; - let minerCount = 0; - let localMinerCount = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 }; - let localStats = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0, miners: {}, miners2: {} }; - let localPortHashes = {}; - let localTimes = { pplns: locTime, pps: locTime, solo: locTime, prop: locTime, global: locTime, miners: {} }; - let loopBreakout = 0; - - async.doUntil(function (callback_until) { - let oldestTime = currentTime; - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - let count = 0; - for (let found = cursor.goToRange(height) === height; found; ++ count, found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, share) { // jshint ignore:line - try { - share = global.protos.Share.decode(share); - } catch (e) { - console.error(share); - return; - } - if (share.timestamp < oldestTime) oldestTime = share.timestamp; - if (share.timestamp <= identifierTime) return; + const currentTime = Date.now(); + // power to ensure we can keep up to global.config.general.statsBufferHours in global.config.general.statsBufferLength array + // here N = log(history_power, global.config.general.statsBufferLength) is number of attemps required on average to remove top left history point (the oldest one) + // we just select history_power so that is till happen on global.config.general.statsBufferHours * 60 attemps on average + const history_power = Math.log(global.config.general.statsBufferLength) / Math.log(global.config.general.statsBufferHours * 60); + + console.log("Starting stats collection for " + height + " height (history power: " + history_power + ")"); + + const locTime = currentTime - (hashrate_avg_min*60*1000); + const identifierTime = currentTime - (2*hashrate_avg_min*60*1000); + + let minerCount = 0; + + identifiers = {}; + minerSet = {}; + minerPortSet = {}; + localMinerCount = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 }; + localStats = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0, miners: {}, miners2: {} }; + localPortHashes = {}; + localTimes = { pplns: locTime, pps: locTime, solo: locTime, prop: locTime, global: locTime, miners: {} }; + + let loopBreakout = 0; + let oldestTime = currentTime; + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + + do { + let count = 0; + for (let found = cursor.goToRange(height) === height; found; ++ count, found = cursor.goToNextDup()) cursor.getCurrentBinary(function (key, share) { // jshint ignore:line + try { + share = global.protos.Share.decode(share); + } catch (e) { + console.error(share); + return; + } + if (share.timestamp < oldestTime) oldestTime = share.timestamp; + if (share.timestamp <= identifierTime) return; - let minerID = typeof(share.paymentID) !== 'undefined' && share.paymentID.length > 10 - ? share.paymentAddress + '.' + share.paymentID : share.paymentAddress; + const minerID = typeof(share.paymentID) !== 'undefined' && share.paymentID.length > 10 + ? share.paymentAddress + '.' + share.paymentID : share.paymentAddress; - const identifier = share.identifier; + const identifier = share.identifier; - if (minerID in identifiers) { - if (identifiers[minerID].indexOf(identifier) < 0) { - identifiers[minerID].push(identifier); - ++ minerCount; - } - } else { - identifiers[minerID] = [identifier]; - ++ minerCount; - } + if (minerID in identifiers) { + if (identifiers[minerID].indexOf(identifier) < 0) { + identifiers[minerID].push(identifier); + ++ minerCount; + } + } else { + identifiers[minerID] = [identifier]; + ++ minerCount; + } - if (share.timestamp <= locTime) return; - - let minerIDWithIdentifier = minerID + "_" + identifier; - localStats.global += share.shares; - if (localTimes.global < share.timestamp) localTimes.global = share.timestamp; - let minerType; - switch (share.poolType) { - case global.protos.POOLTYPE.PPLNS: - minerType = 'pplns'; - localStats.pplns += share.shares; - if (localTimes.pplns < share.timestamp) localTimes.pplns = share.timestamp; - break; - case global.protos.POOLTYPE.PPS: - localStats.pps += share.shares; - minerType = 'pps'; - if (localTimes.pps < share.timestamp) localTimes.pps = share.timestamp; - break; - case global.protos.POOLTYPE.SOLO: - localStats.solo += share.shares; - minerType = 'solo'; - if (localTimes.solo < share.timestamp) localTimes.solo = share.timestamp; - break; - } + if (share.timestamp <= locTime) return; - const port = typeof(share.port) !== 'undefined' && share.port ? share.port : global.config.daemon.port; - if (port in localPortHashes) localPortHashes[port] += share.shares; - else localPortHashes[port] = share.shares; + const minerIDWithIdentifier = minerID + "_" + identifier; + const shares2 = share.shares2; + localStats.global += shares2; + if (localTimes.global < share.timestamp) localTimes.global = share.timestamp; + const minerType = pool_type_str[share.poolType]; + if (!minerType) { + console.error("Wrong share pool type found: " + share.poolType); + return; + } + localStats[minerType] += shares2; + if (localTimes[minerType] < share.timestamp) localTimes[minerType] = share.timestamp; - if (minerID in minerSet) { - localStats.miners[minerID] += share.shares; - localStats.miners2[minerID] += share.shares2 ? share.shares2 : share.shares; - if (localTimes.miners[minerID] < share.timestamp) localTimes.miners[minerID] = share.timestamp; - } else { - ++ localMinerCount[minerType]; - ++ localMinerCount.global; - localStats.miners[minerID] = share.shares; - localStats.miners2[minerID] = share.shares2 ? share.shares2 : share.shares; - localTimes.miners[minerID] = share.timestamp; - minerSet[minerID] = 1; - } - if (minerIDWithIdentifier in minerSet) { - localStats.miners[minerIDWithIdentifier] += share.shares; - localStats.miners2[minerIDWithIdentifier] += share.shares2 ? share.shares2 : share.shares; - if (localTimes.miners[minerIDWithIdentifier] < share.timestamp) localTimes.miners[minerIDWithIdentifier] = share.timestamp; - } else { - localStats.miners[minerIDWithIdentifier] = share.shares; - localStats.miners2[minerIDWithIdentifier] = share.shares2 ? share.shares2 : share.shares; - localTimes.miners[minerIDWithIdentifier] = share.timestamp; - minerSet[minerIDWithIdentifier] = 1; - } - }); + const port = typeof(share.port) !== 'undefined' && share.port ? share.port : global.config.daemon.port; + if (port in localPortHashes) localPortHashes[port] += share.raw_shares; + else localPortHashes[port] = share.raw_shares; + + if (!shares2) return; // use virtual shares from child block mining only for global pool stats + + if (minerID in minerPortSet) { + localStats.miners[minerID] += share.raw_shares; + localStats.miners2[minerID] += shares2; + if (localTimes.miners[minerID] < share.timestamp) localTimes.miners[minerID] = share.timestamp; + } else { + ++ localMinerCount[minerType]; + ++ localMinerCount.global; + localStats.miners[minerID] = share.raw_shares; + localStats.miners2[minerID] = shares2; + localTimes.miners[minerID] = share.timestamp; + minerSet[minerID] = 1; + minerPortSet[minerID] = port; + } + + if (minerIDWithIdentifier in minerSet) { + localStats.miners[minerIDWithIdentifier] += share.raw_shares; + localStats.miners2[minerIDWithIdentifier] += shares2; + if (localTimes.miners[minerIDWithIdentifier] < share.timestamp) localTimes.miners[minerIDWithIdentifier] = share.timestamp; + } else { + localStats.miners[minerIDWithIdentifier] = share.raw_shares; + localStats.miners2[minerIDWithIdentifier] = shares2; + localTimes.miners[minerIDWithIdentifier] = share.timestamp; + minerSet[minerIDWithIdentifier] = 1; + } + }); + debug("On " + height + " height iterated " + count + " elements"); + } while (++loopBreakout <= 60 && --height >= 0 && oldestTime > identifierTime); + cursor.close(); + txn.abort(); + + debug("Share loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + prevMinerSet = global.database.getCache('minerSet'); + if (prevMinerSet === false) prevMinerSet = minerSet; + cache_updates = {}; + // pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 + ['pplns', 'pps', 'solo', 'prop', 'global'].forEach(function (key) { + const hash = localStats[key] / (hashrate_avg_min*60); + const lastHash = localTimes[key]; + const minerCount = localMinerCount[key]; + let cachedData = global.database.getCache(key + "_stats"); + if (cachedData !== false) { + cachedData.hash = hash; + cachedData.lastHash = lastHash; + cachedData.minerCount = minerCount; + if (!cachedData.hasOwnProperty("hashHistory")) { + cachedData.hashHistory = []; + cachedData.minerHistory = []; + } + if (cycleCount === 0) { + cachedData.hashHistory.unshift({ts: currentTime, hs: cachedData.hash}); + if (cachedData.hashHistory.length > global.config.general.statsBufferLength) { + while (cachedData.hashHistory.length > global.config.general.statsBufferLength) { + cachedData.hashHistory.pop(); + } } - cursor.close(); - txn.abort(); - debug("On " + height + " height iterated " + count + " elements"); - return callback_until(null, oldestTime); - - }, function (oldestTime) { - return ++loopBreakout > 60 || --height < 0 || oldestTime <= identifierTime; - - }, function (err) { - debug("Share loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); - let prevMinerSet = global.database.getCache('minerSet'); - if (prevMinerSet === false) prevMinerSet = minerSet; - let cache_updates = {}; - // pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 - ['pplns', 'pps', 'solo', 'prop', 'global'].forEach(function (key) { - let cachedData = global.database.getCache(key + "_stats"); - if (cachedData !== false) { - cachedData.hash = Math.floor(localStats[key] / (hashrate_avg_min*60)) + 1; - cachedData.lastHash = localTimes[key]; - cachedData.minerCount = localMinerCount[key]; - if (!cachedData.hasOwnProperty("hashHistory")) { - cachedData.hashHistory = []; - cachedData.minerHistory = []; - } - if (cycleCount === 0) { - cachedData.hashHistory.unshift({ts: currentTime, hs: cachedData.hash}); - if (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - while (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - cachedData.hashHistory.pop(); - } - } - cachedData.minerHistory.unshift({ts: currentTime, cn: cachedData.minerCount}); - if (cachedData.minerHistory.length > global.config.general.statsBufferLength) { - while (cachedData.minerHistory.length > global.config.general.statsBufferLength) { - cachedData.minerHistory.pop(); - } - } - } - } else { - cachedData = { - hash: Math.floor(localStats[key] / (hashrate_avg_min*60)) + 1, - totalHashes: 0, - lastHash: localTimes[key], - minerCount: localMinerCount[key], - hashHistory: [{ts: currentTime, hs: cachedData.hash}], - minerHistory: [{ts: currentTime, cn: cachedData.hash}] - }; + cachedData.minerHistory.unshift({ts: currentTime, cn: cachedData.minerCount}); + if (cachedData.minerHistory.length > global.config.general.statsBufferLength) { + while (cachedData.minerHistory.length > global.config.general.statsBufferLength) { + cachedData.minerHistory.pop(); } - cache_updates[key + "_stats"] = cachedData; - }); - for (let port in localPortHashes) localPortHashes[port] = Math.floor(localPortHashes[port] / (hashrate_avg_min*60)) + 1; - cache_updates["port_hash"] = localPortHashes; - for (let miner in minerSet) { - let stats; - let keyStats = "stats:" + miner; - let keyHistory = "history:" + miner; - - if (miner in stats_cache) { - stats = stats_cache[miner]; + } + } + } else { + cachedData = { + hash: hash, + totalHashes: 0, + lastHash: lastHash, + minerCount: minerCount, + hashHistory: [{ts: currentTime, hs: hash}], + minerHistory: [{ts: currentTime, cn: minerCount}] + }; + } + cache_updates[key + "_stats"] = cachedData; + }); + for (let port in localPortHashes) localPortHashes[port] = localPortHashes[port] / (hashrate_avg_min*60); + cache_updates["port_hash"] = localPortHashes; + let history_update_count = 0; + + for (let miner in minerSet) { + let stats; + let keyStats = "stats:" + miner; + let keyHistory = "history:" + miner; + + if (miner in stats_cache) { + stats = stats_cache[miner]; + } else { + stats = global.database.getCache(keyStats); + if (!stats) stats = {}; + let history_stats = global.database.getCache(keyHistory); + if (history_stats) { + stats.hashHistory = history_stats.hashHistory; + } else { + stats.hashHistory = []; + } + } + + stats.hash = localStats.miners[miner] / (hashrate_avg_min*60); + stats.hash2 = localStats.miners2[miner] / (hashrate_avg_min*60); + stats.lastHash = localTimes.miners[miner]; + cache_updates[keyStats] = { hash: stats.hash, hash2: stats.hash2, lastHash: stats.lastHash }; + + if (cycleCount === 0) { + stats.hashHistory.unshift({ts: currentTime, hs: stats.hash, hs2: stats.hash2}); + if (stats.hashHistory.length > global.config.general.statsBufferLength) { + const is_worker = miner.indexOf('_') >= 0; + while (stats.hashHistory.length > global.config.general.statsBufferLength) { + if (is_worker) { + stats.hashHistory.pop(); } else { - stats = global.database.getCache(keyStats); - if (!stats) stats = {}; - let history_stats = global.database.getCache(keyHistory); - if (history_stats) { - stats.hashHistory = history_stats.hashHistory; + const last_index = stats.hashHistory.length - 1; + if ((currentTime - stats.hashHistory[last_index].ts) / 1000 / 3600 > global.config.general.statsBufferHours) { + stats.hashHistory.pop(); } else { - stats.hashHistory = []; + // here we remove larger indexes (that are more distant in time) with more probability + const index_to_remove = (last_index * (1 - Math.pow(Math.random(), history_power))).toFixed(); + stats.hashHistory.splice(index_to_remove, 1); } } + } + } + if ( stats.hashHistory.length < global.config.general.statsBufferLength || + !(miner in miner_history_update_time) || + (history_update_count < 5000 && currentTime - miner_history_update_time[miner] > 10*60*1000) + ) { + cache_updates[keyHistory] = { hashHistory: stats.hashHistory }; + miner_history_update_time[miner] = currentTime; + ++ history_update_count; + } + } - stats.hash = Math.floor(localStats.miners[miner] / (hashrate_avg_min*60)) + 1; - stats.hash2 = Math.floor(localStats.miners2[miner] / (hashrate_avg_min*60)) + 1; - stats.lastHash = localTimes.miners[miner]; - cache_updates[keyStats] = { hash: stats.hash, hash2: stats.hash2, lastHash: stats.lastHash }; - - if (cycleCount === 0) { - stats.hashHistory.unshift({ts: currentTime, hs: stats.hash, hs2: stats.hash2}); - if (stats.hashHistory.length > global.config.general.statsBufferLength) { - while (stats.hashHistory.length > global.config.general.statsBufferLength) { - stats.hashHistory.pop(); - } - } - cache_updates[keyHistory] = { hashHistory: stats.hashHistory }; - } + stats_cache[miner] = stats; + } - stats_cache[miner] = stats; - } - debug("History loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); - - // remove old workers - for (let miner in prevMinerSet) { - if (miner in minerSet) continue; // we still have this miner in current set - //debug("Removing: " + miner + " as an active miner from the cache."); - let minerStats = global.database.getCache(miner); - if (!minerStats) continue; - minerStats.hash = 0; - cache_updates[miner] = minerStats; - if (miner.indexOf('_') <= -1) continue; - - // This is a worker case. - let address_parts = miner.split(/_(.+)/); - let address = address_parts[0]; - get_address_email(address, function (email) { - setTimeout(delayed_send_worker_stopped_hashing_email, 5*60*1000, miner, email, currentTime); - }); - } + debug("History loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + + // remove old workers + for (let miner in prevMinerSet) { + if (miner in minerSet) continue; // we still have this miner in current set + //debug("Removing: " + miner + " as an active miner from the cache."); + let minerStats = global.database.getCache(miner); + if (!minerStats) continue; + minerStats.hash = 0; + cache_updates[miner] = minerStats; + if (miner.indexOf('_') <= -1) continue; + + // This is a worker case. + const address_parts = miner.split(/_(.+)/); + const worker = address_parts[1]; + if (typeof(worker) !== 'undefined' && !worker.includes('silent')) { + if (!(miner in workers_stopped_hashing_time)) { + workers_stopped_hashing_time[miner] = currentTime; + setTimeout(delayed_send_worker_stopped_hashing_email, 10*60*1000, miner, currentTime); + } + } + } - debug("Old worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + debug("Old worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + + // find new workers + for (let miner in minerSet) { + if (miner in prevMinerSet) continue; // we still have this miner in previous set + //debug("Adding: " + miner + " as an active miner to the cache."); + if (miner.indexOf('_') <= -1) continue; + + // This is a worker case. + const address_parts = miner.split(/_(.+)/); + const worker = address_parts[1]; + if (typeof(worker) !== 'undefined' && !worker.includes('silent')) { + workers_started_hashing_time[miner] = currentTime; + if (miner in workers_stopped_hashing_email_time) { + delete workers_stopped_hashing_time[miner]; + delete workers_stopped_hashing_email_time[miner]; + const address = address_parts[0]; + get_address_email(address, function (email) { + send_worker_started_hashing_email(miner, email, currentTime); + }); + } + } + } - // find new workers - for (let miner in minerSet) { - if (miner in prevMinerSet) continue; // we still have this miner in previous set - //debug("Adding: " + miner + " as an active miner to the cache."); - if (miner.indexOf('_') <= -1) continue; + debug("New worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); - // This is a worker case. - let address_parts = miner.split(/_(.+)/); - let address = address_parts[0]; - get_address_email(address, function (email) { - send_worker_started_hashing_email(miner, email, currentTime); - }); - } + Object.keys(identifiers).forEach(function (key) { + cache_updates['identifiers:' + key] = identifiers[key]; + }); - debug("New worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + portMinerCount = {}; + for (let miner in minerPortSet) { + const port = minerPortSet[miner]; + if (port in portMinerCount) ++ portMinerCount[port]; + else portMinerCount[port] = 1; + } + cache_updates.portMinerCount = portMinerCount; + cache_updates.minerSet = minerSet; + const db_write_start_time = Date.now(); + try { + global.database.bulkSetCache(cache_updates); + } catch (e) { + console.error("Can't write to pool DB: " + e); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Pool DB is overflowed!", "Can't wite to pool DB: " + e); + } + cache_updates = {}; - Object.keys(identifiers).forEach(function (key) { - cache_updates['identifiers:' + key] = identifiers[key]; - }); - cache_updates.minerSet = minerSet; - global.database.bulkSetCache(cache_updates); - - let pool_hashrate = Math.floor(localStats.global / (hashrate_avg_min*60)) + 1; - let pool_workers = minerCount; - console.log("Processed " + minerCount + " workers for " + ((Date.now() - currentTime) / 1000) + " seconds. Pool hashrate is: " + pool_hashrate); - if (!prev_pool_state_time || currentTime - prev_pool_state_time > hashrate_avg_min*60*1000) { - let pool_hashrate_ratio = prev_pool_hashrate ? pool_hashrate / prev_pool_hashrate : 1; - let pool_workers_ratio = prev_pool_workers ? pool_workers / prev_pool_workers : 1; - if (pool_hashrate_ratio < (1-stat_change_alert) || pool_hashrate_ratio > (1+stat_change_alert) || - pool_workers_ratio < (1-stat_change_alert) || pool_workers_ratio > (1+stat_change_alert)) { - global.support.sendEmail(global.config.general.adminEmail, - "FYI: Pool hashrate/workers changed significantly", - "Pool hashrate changed from " + prev_pool_hashrate + " to " + pool_hashrate + " (" + pool_hashrate_ratio + ")\n" + - "Pool number of workers changed from " + prev_pool_workers + " to " + pool_workers + " (" + pool_workers_ratio + ")\n" - ); - } - prev_pool_hashrate = pool_hashrate; - prev_pool_workers = pool_workers; - prev_pool_state_time = currentTime; - } - callback(null); - }); + let pool_hashrate = localStats.global / (hashrate_avg_min*60); + let pool_workers = minerCount; + console.log("Processed " + minerCount + " workers (" + history_update_count + " history) for " + + ((Date.now() - currentTime) / 1000) + " seconds (" + ((Date.now() - db_write_start_time) / 1000) + " seconds DB write). " + + "Pool hashrate is: " + pool_hashrate + ); + if (!prev_pool_state_time || currentTime - prev_pool_state_time > hashrate_avg_min*60*1000) { + let pool_hashrate_ratio = prev_pool_hashrate ? pool_hashrate / prev_pool_hashrate : 1; + let pool_workers_ratio = prev_pool_workers ? pool_workers / prev_pool_workers : 1; + if (pool_hashrate_ratio < (1-stat_change_alert) || pool_hashrate_ratio > (1+stat_change_alert) || + pool_workers_ratio < (1-stat_change_alert) || pool_workers_ratio > (1+stat_change_alert)) { + global.support.sendEmail(global.config.general.adminEmail, + "FYI: Pool hashrate/workers changed significantly", + "Pool hashrate changed from " + prev_pool_hashrate + " to " + pool_hashrate + " (" + pool_hashrate_ratio + ")\n" + + "Pool number of workers changed from " + prev_pool_workers + " to " + pool_workers + " (" + pool_workers_ratio + ")\n" + ); } - ], function (err, result) { - if (++cycleCount === 3) cycleCount = 0; - setTimeout(updateShareStats, 10*1000); + prev_pool_hashrate = pool_hashrate; + prev_pool_workers = pool_workers; + prev_pool_state_time = currentTime; + } + return callback(); +} + +function updateShareStats() { + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null){ + return setTimeout(updateShareStats, 10*1000); + } + updateShareStats2(body.height + 1, function() { + if (++cycleCount === 3) cycleCount = 0; + setTimeout(updateShareStats, 10*1000); + }); }); } @@ -295,12 +354,14 @@ let minerEmailTime = {}; // worker name -> time let workers_started_hashing_time = {}; +let workers_stopped_hashing_time = {}; let workers_stopped_hashing_email_time = {}; function get_address_email(address, callback) { let currentTime = Date.now(); if (!(address in minerEmailTime) || currentTime - minerEmailTime[address] > 10*60*1000) { minerEmailTime[address] = currentTime; + minerEmail[address] = null; global.mysql.query("SELECT email FROM users WHERE username = ? AND enable_email IS true limit 1", [address]).then(function (rows) { if (rows.length === 0) { delete minerEmail[address]; @@ -308,44 +369,21 @@ function get_address_email(address, callback) { } else { minerEmail[address] = rows[0].email; } - callback(minerEmail[address]); + return callback(minerEmail[address]); + }).catch(function (error) { + console.error("Can't get email address for " + address + ": " + error.message); + return; }); } else if (address in minerEmail) { - callback(minerEmail[address]); + if (minerEmail[address] === null) { // not yet ready (retry again in 10 secs) + if (currentTime - minerEmailTime[address] < 5*1000) return setTimeout(get_address_email, 10*1000, address, callback); + } else { + return callback(minerEmail[address]); + } } } function send_worker_started_hashing_email(miner, email, currentTime) { - workers_started_hashing_time[miner] = currentTime; - if (miner in workers_stopped_hashing_email_time) { - delete workers_stopped_hashing_email_time[miner]; - - let address_parts = miner.split(/_(.+)/); - let address = address_parts[0]; - let worker = address_parts[1]; - // toAddress, subject, body - let emailData = { - worker: worker, - timestamp: global.support.formatDate(currentTime), - poolEmailSig: global.config.general.emailSig - }; - global.support.sendEmail(email, - sprintf(global.config.email.workerStartHashingSubject, emailData), - sprintf(global.config.email.workerStartHashingBody, emailData), - address - ); - } -} - -function delayed_send_worker_stopped_hashing_email(miner, email, currentTime) { - if (miner in workers_started_hashing_time && Date.now() - workers_started_hashing_time[miner] <= 5*60*1000) { - delete workers_started_hashing_time[miner]; - return; - } - - delete workers_started_hashing_time[miner]; - workers_stopped_hashing_email_time[miner] = Date.now(); - let address_parts = miner.split(/_(.+)/); let address = address_parts[0]; let worker = address_parts[1]; @@ -355,488 +393,57 @@ function delayed_send_worker_stopped_hashing_email(miner, email, currentTime) { timestamp: global.support.formatDate(currentTime), poolEmailSig: global.config.general.emailSig }; - global.support.sendEmail(email, - sprintf(global.config.email.workerNotHashingSubject, emailData), - sprintf(global.config.email.workerNotHashingBody, emailData), + sprintf(global.config.email.workerStartHashingSubject, emailData), + sprintf(global.config.email.workerStartHashingBody, emailData), address ); } -function updatePoolStats(poolType) { - if (global.config.daemon.activePort) { - global.support.getActivePort("", function (newActivePort) { - if (newActivePort) global.config.daemon.activePort = newActivePort; - updatePoolStats2(poolType); - }); - } else { - updatePoolStats2(poolType); +function delayed_send_worker_stopped_hashing_email(miner, currentTime) { + if (miner in workers_started_hashing_time && Date.now() - workers_started_hashing_time[miner] <= 10*60*1000) { + delete workers_started_hashing_time[miner]; + return; } -} -let price_btc = 0; -let price_usd = 0; -let price_eur = 0; -let min_block_rewards = {}; - -function updatePoolStats2(poolType) { - let cache; - let port_suffix = global.config.daemon.activePort && global.config.daemon.activePort !== global.config.daemon.port ? "_" + global.config.daemon.activePort.toString() : ""; - if (typeof(poolType) !== 'undefined') { - cache = global.database.getCache(poolType + "_stats"); - if (port_suffix === "") { - let cache2 = global.database.getCache(poolType + "_stats2"); - cache.totalHashes = cache2.totalHashes; - cache.roundHashes = cache2.roundHashes; - } else { - let cache2_total = global.database.getCache(poolType + "_stats2"); - let cache2_round = global.database.getCache(poolType + "_stats2" + port_suffix); - cache.totalHashes = cache2_total.totalHashes; - cache.roundHashes = cache2_round.roundHashes; - } - } else { - cache = global.database.getCache("global_stats"); - if (port_suffix === "") { - let cache2 = global.database.getCache("global_stats2"); - cache.totalHashes = cache2.totalHashes; - cache.roundHashes = cache2.roundHashes; - } else { - let cache2_total = global.database.getCache("global_stats2"); - let cache2_round = global.database.getCache("global_stats2" + port_suffix); - cache.totalHashes = cache2_total.totalHashes; - cache.roundHashes = cache2_round.roundHashes; - } - } + delete workers_started_hashing_time[miner]; - let port_hash = global.database.getCache('port_hash'); - let blockList = global.database.getBlockList(poolType); - let altblockList = global.database.getAltBlockList(poolType); - let min_block_rewards2 = global.database.getCache('min_block_rewards'); - if (min_block_rewards2) min_block_rewards = min_block_rewards2; - if (!(global.config.daemon.port in min_block_rewards)) min_block_rewards[global.config.daemon.port] = 0; - - async.series([ - function (callback) { - //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate)"); - return callback(null, cache.hash || 0); - }, - function (callback) { - //debug(threadName + "Checking Influx for last 5min avg for miner count for pool stats (miners)"); - return callback(null, cache.minerCount || 0); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for totalHashes"); - return callback(null, cache.totalHashes || 0); - }, - function (callback) { - //debug(threadName + "Checking LMDB for lastBlockFoundTime for pool stats"); - let max_time = 0; - if (blockList.length !== 0) { - max_time = Math.floor(blockList[0].ts / 1000); - } - if (altblockList.length !== 0) { - max_time = Math.max(max_time, Math.floor(altblockList[0].ts / 1000)); - } - return callback(null, max_time); - }, - function (callback) { - //debug(threadName + "Checking LMDB for lastBlockFound height for pool stats"); - if (blockList.length === 0) { - return callback(null, 0); - } - return callback(null, blockList[0].height); - }, - function (callback) { - //debug(threadName + "Checking LMDB for totalBlocksFound for pool stats"); - return callback(null, blockList.length); - }, - function (callback) { - //debug(threadName + "Checking MySQL for total miners paid"); - if (typeof(poolType) !== 'undefined') { - global.mysql.query("SELECT payment_address, payment_id FROM payments WHERE pool_type = ? group by payment_address, payment_id", [poolType]).then(function (rows) { - return callback(null, rows.length); - }); - } else { - global.mysql.query("SELECT payment_address, payment_id FROM payments group by payment_address, payment_id").then(function (rows) { - return callback(null, rows.length); - }); - } - }, - function (callback) { - //debug(threadName + "Checking MySQL for total transactions count"); - if (typeof(poolType) !== 'undefined') { - global.mysql.query("SELECT distinct(transaction_id) from payments WHERE pool_type = ?", [poolType]).then(function (rows) { - return callback(null, rows.length); - }); - } else { - global.mysql.query("SELECT count(id) as txn_count FROM transactions").then(function (rows) { - if (typeof(rows[0]) !== 'undefined') { - return callback(null, rows[0].txn_count); - } else { - return callback(null, 0); - } - }); - } - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for roundHashes"); - return callback(null, cache.roundHashes || 0); - }, - function (callback) { - //debug(threadName + "Checking LMDB for altblock count for pool stats"); - return callback(null, altblockList.length); - }, - function (callback) { - //debug(threadName + "Checking LMDB for altBlocksFound array for each specific port"); - let result = {}; - for (let i in altblockList) { - let block = altblockList[i]; - if (result.hasOwnProperty(block.port)) ++ result[block.port]; - else result[block.port] = 1; - } - return callback(null, result); - }, - function (callback) { - //debug(threadName + "Checking MySQL for activePort value"); - return callback(null, global.config.daemon.activePort ? global.config.daemon.activePort : global.config.daemon.port); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for active_ports value"); - let active_ports = global.database.getCache('active_ports'); - return callback(null, active_ports ? active_ports : []); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for xmr_profit value"); - let xmr_profit = global.database.getCache('xmr_profit'); - return callback(null, xmr_profit ? xmr_profit.value : 0); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for coin_profit value"); - let coin_xmr_profit = global.database.getCache('coin_xmr_profit'); - return callback(null, coin_xmr_profit ? coin_xmr_profit : {}); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for xmr_profit_comment value"); - let coin_comment = global.database.getCache('coin_comment'); - return callback(null, coin_comment ? coin_comment : {}); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for min_block_rewards value to set minBlockRewards"); - return callback(null, min_block_rewards); - }, - function (callback) { - let pending = 0; - for (let i in blockList) { - const block = blockList[i]; - if (block.valid === true && block.unlocked === false) pending += global.support.coinToDecimal(block.value); - } - for (let i in altblockList) { - const altblock = altblockList[i]; - if (altblock.valid === true && altblock.unlocked === false) pending += altblock.port in min_block_rewards ? min_block_rewards[altblock.port] : 0; - } - return callback(null, pending); - }, - function (callback) { - if (typeof(poolType) === 'undefined') { - global.support.https_get("https://api.coinmarketcap.com/v1/ticker/" + global.config.coin.name + "/?convert=EUR", function (res) { - if (res != null && res instanceof Array && res.length === 1 && typeof(res[0].price_usd) !== 'undefined' && typeof(res[0].price_eur) !== 'undefined') { - price_btc = parseFloat(res[0].price_btc); - price_usd = parseFloat(res[0].price_usd); - price_eur = parseFloat(res[0].price_eur); - } - return callback(null, { btc: price_btc, usd: price_usd, eur: price_eur }); - }); - } else { - return callback(null, { btc: price_btc, usd: price_usd, eur: price_eur }); - } - }, - function (callback) { - let currentEfforts = {}; - for (let port in min_block_rewards) { - const value = global.database.getCache(port != global.config.daemon.port ? "global_stats2_" + port : "global_stats2"); - if (value !== false) currentEfforts[port] = value.roundHashes; - } - return callback(null, currentEfforts); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for pplns_port_shares value"); - let pplns_port_shares = global.database.getCache('pplns_port_shares'); - return callback(null, pplns_port_shares ? pplns_port_shares : {}); - }, - function (callback) { - //debug(threadName + "Checking LMDB cache for pplns_window_time value"); - let pplns_window_time = global.database.getCache('pplns_window_time'); - return callback(null, pplns_window_time ? pplns_window_time : 0); - }, - function (callback) { - //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate) per port"); - return callback(null, port_hash || {}); - }, - ], function (err, result) { - if (typeof(poolType) === 'undefined') { - poolType = 'global'; - updateBlockHeader(); - } - global.database.setCache('pool_stats_' + poolType, { - hashRate: result[0], - miners: result[1], - totalHashes: result[2], - lastBlockFoundTime: result[3] || 0, - lastBlockFound: result[4] || 0, - totalBlocksFound: result[5] || 0, - totalMinersPaid: result[6] || 0, - totalPayments: result[7] || 0, - roundHashes: result[8] || 0, - totalAltBlocksFound: result[9] || 0, - altBlocksFound: result[10] || {}, - activePort: result[11] || 0, - activePorts: result[12] || [], - activePortProfit: result[13] || 0, - coinProfit: result[14] || {}, - coinComment: result[15] || {}, - minBlockRewards: result[16] || {}, - pending: result[17] || 0, - price: result[18] || {}, - currentEfforts: result[19] || {}, - pplnsPortShares: result[20] || {}, - pplnsWindowTime: result[21] || 0, - portHash: result[22] || {}, - }); - }); -} + const address_parts = miner.split(/_(.+)/); + const address = address_parts[0]; -function updatePoolPorts(poolServers) { - //debug(threadName + "Updating pool ports"); - let local_cache = {global: []}; - let portCount = 0; - global.mysql.query("select * from ports where hidden = 0 and pool_id < 1000 and lastSeen >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { - rows.forEach(function (row) { - ++ portCount; - if (!local_cache.hasOwnProperty(row.port_type)) { - local_cache[row.port_type] = []; - } - local_cache[row.port_type].push({ - host: poolServers[row.pool_id], - port: row.network_port, - difficulty: row.starting_diff, - description: row.description, - miners: row.miners - }); - if (portCount === rows.length) { - let local_counts = {}; - let port_diff = {}; - let port_miners = {}; - let pool_type_count = 0; - let localPortInfo = {}; - for (let pool_type in local_cache) { // jshint ignore:line - ++ pool_type_count; - local_cache[pool_type].forEach(function (portData) { // jshint ignore:line - if (!local_counts.hasOwnProperty(portData.port)) { - local_counts[portData.port] = 0; - } - if (!port_diff.hasOwnProperty(portData.port)) { - port_diff[portData.port] = portData.difficulty; - } - if (!port_miners.hasOwnProperty(portData.port)) { - port_miners[portData.port] = 0; - } - if (port_diff[portData.port] === portData.difficulty) { - ++ local_counts[portData.port]; - port_miners[portData.port] += portData.miners; - } - localPortInfo[portData.port] = portData.description; - if (local_counts[portData.port] === Object.keys(poolServers).length) { - local_cache.global.push({ - host: { - blockID: local_cache[pool_type][0].host.blockID, - blockIDTime: local_cache[pool_type][0].host.blockIDTime, - hostname: global.config.pool.geoDNS, - }, - port: portData.port, - pool_type: pool_type, - difficulty: portData.difficulty, - miners: port_miners[portData.port], - description: localPortInfo[portData.port] - }); - } - }); - if (pool_type_count === Object.keys(local_cache).length) { - //debug(threadName + "Sending the following to the workers: " + JSON.stringify(local_cache)); - global.database.setCache('poolPorts', local_cache); - } - } - } - }); - }); -} + get_address_email(address, function (email) { + workers_stopped_hashing_email_time[miner] = Date.now(); + const worker = address_parts[1]; -function updatePoolInformation() { - let local_cache = {}; - //debug(threadName + "Updating pool information"); - global.mysql.query("select * from pools where id < 1000 and last_checkin >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { - rows.forEach(function (row) { - local_cache[row.id] = { - ip: row.ip, - blockID: row.blockID, - blockIDTime: global.support.formatDateFromSQL(row.blockIDTime), - hostname: row.hostname - }; - if (Object.keys(local_cache).length === rows.length) { - global.database.setCache('poolServers', local_cache); - updatePoolPorts(local_cache); - } - }); + // toAddress, subject, body + const emailData = { + worker: worker, + timestamp: global.support.formatDate(currentTime), + poolEmailSig: global.config.general.emailSig + }; + global.support.sendEmail(email, + sprintf(global.config.email.workerNotHashingSubject, emailData), + sprintf(global.config.email.workerNotHashingBody, emailData), + address + ); }); } -let prev_network_info = {}; -function updateBlockHeader() { - let info = {}; +global.support.sendEmail(global.config.general.adminEmail, "Restarting worker module", "Restarted worker module!"); - let left = 0; - for (let port in min_block_rewards) ++ left; - - for (let port in min_block_rewards) { - global.coinFuncs.getPortLastBlockHeader(port, function(err, body){ - if (err !== null) { - console.error("Last block header request failed for " + port + " port!"); - body.difficulty = prev_network_info[port].difficulty; - body.hash = prev_network_info[port].hash; - body.height = prev_network_info[port].height; - body.reward = prev_network_info[port].value; - body.timestamp = prev_network_info[port].ts; - } - prev_network_info[port] = info[port] = { - difficulty: body.difficulty, - hash: body.hash, - height: body.height, - value: body.reward, - ts: body.timestamp, - }; - if (port == global.config.daemon.activePort) { - info.difficulty = body.difficulty; - info.hash = body.hash; - info.height = body.height; - info.value = body.reward; - info.ts = body.timestamp; - } - if (-- left === 0) { - info.main_height = prev_network_info[global.config.daemon.port].height; - global.database.setCache('networkBlockInfo', info); - } - }); - } -} - -function updateWalletStats() { - async.waterfall([ - function (callback) { - // Todo: Implement within the coins/.js file. - global.support.rpcWallet('getbalance', [], function (body) { - if (body.result) { - return callback(null, { - balance: body.result.balance, - unlocked: body.result.unlocked_balance, - ts: Date.now() - }); - } else { - return callback(true, "Unable to process balance"); - } - }); - }, - function (state, callback) { - // Todo: Implement within the coins/.js file. - global.support.rpcWallet('getheight', [], function (body) { - if (body.result) { - state.height = body.result.height; - return callback(null, state); - } else if (typeof body.error !== 'undefined' && body.error.message === 'Method not found') { - state.height = 0; - return callback(null, state); - } else { - return callback(true, "Unable to get current wallet height"); - } - }); - } - ], function (err, results) { - if (err) { - return console.error("Unable to get wallet stats: " + results); - } - global.database.setCache('walletStateInfo', results); - let history = global.database.getCache('walletHistory'); - if (history === false) { - history = []; - } - history.unshift(results); - history = history.sort(global.support.tsCompare); - if (history.length > global.config.general.statsBufferLength) { - while (history.length > global.config.general.statsBufferLength) { - history.pop(); - } +updateShareStats(); +// clean caches from time to time +setInterval(function() { + console.log("Cleaning caches (" + Object.keys(stats_cache).length + " stats, " + Object.keys(miner_history_update_time).length + " histories)"); + const currentTime = Date.now(); + let stats_cache2 = {}; + for (let miner in stats_cache) { + if (miner in miner_history_update_time && currentTime - miner_history_update_time[miner] < 60*60*1000) { + stats_cache2[miner] = stats_cache[miner]; } - global.database.setCache('walletHistory', history); - }); - -} - -function bad_header_start(port) { - console.error("Issue in getting block header for " + port + " port. Skipping node monitor"); - if (!(port in lastBlockCheckIsFailed)) { - lastBlockCheckIsFailed[port] = 1; - global.support.sendEmail( - global.config.general.adminEmail, - 'Failed to query daemon for ' + port + ' port for last block header', - `The worker failed to return last block header for ` + port + ` port. Please verify if the daemon is running properly.` - ); } - return; -} - -function bad_header_stop(port) { - if (port in lastBlockCheckIsFailed) { - delete lastBlockCheckIsFailed[port]; - global.support.sendEmail( - global.config.general.adminEmail, - 'Quering daemon for ' + port + ' port for last block header is back to normal', - `An warning was sent to you indicating that the the worker failed to return the last block header for ${port} port. - The issue seems to be solved now.` - ); - } -} - -function monitorNodes() { - global.mysql.query("SELECT blockID, hostname, ip, port FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { - rows.forEach(function (row) { - let port = row.port ? row.port : global.config.daemon.port; - global.coinFuncs.getPortLastBlockHeader(port, function (err, block) { - if (err !== null){ - bad_header_start(port); - return; - } - bad_header_stop(); - if (Math.abs(block.height - row.blockID) > 3) { - global.support.sendEmail(global.config.general.adminEmail, - "Pool server behind in blocks", - "The pool server: " + row.hostname + " with IP: " + row.ip + " is " + (block.height - row.blockID) + " blocks behind for " + port + " port" - ); - } - }); - }); - }); -} - -updateShareStats(); -updatePoolStats(); -updatePoolInformation(); -updateWalletStats(); -monitorNodes(); -setInterval(updatePoolStats, 5*1000); -setInterval(updatePoolStats, 5*1000, 'pplns'); -if (global.config.pps.enable === true) setInterval(updatePoolStats, 5*1000, 'pps'); -if (global.config.solo.enable === true) setInterval(updatePoolStats, 5*1000, 'solo'); -setInterval(updatePoolInformation, 5*1000); -setInterval(updateWalletStats, 60*1000); -setInterval(monitorNodes, 5*60*1000); -// clean stats_cache from time to time -setInterval(function() { stats_cache = {}; } , 4*60*60*1000); + stats_cache = stats_cache2; + console.log("After cleaning: " + Object.keys(stats_cache).length + " stats left"); + miner_history_update_time = {}; +}, 2*60*60*1000); diff --git a/manage_scripts/altblock_add.js b/manage_scripts/altblock_add.js index cb24f186b..e19b27fe9 100644 --- a/manage_scripts/altblock_add.js +++ b/manage_scripts/altblock_add.js @@ -24,18 +24,18 @@ require("../init_mini.js").init(function() { const body3 = { "hash": body2.hash, "difficulty": body2.difficulty, - "shares": body2.shares, - "timestamp": body2.timestamp, - "poolType": body2.poolType, - "unlocked": body2.unlocked, - "valid": body2.valid, "port": body2.port, "height": body2.height, - "anchor_height": body2.anchor_height, "value": body2.value, - "pay_value": body2.pay_value, - "pay_stage": body2.pay_stage, - "pay_status": body2.pay_status + "anchor_height": body2.anchor_height, + "timestamp": timestamp * 1000, + "shares": body2.shares || body2.difficulty, + "poolType": body2.poolType || 0, + "unlocked": body2.unlocked || false, + "valid": body2.valid || true, + "pay_value": body2.pay_value || 0, + "pay_stage": body2.pay_stage || "", + "pay_status": body2.pay_status || "" }; if (typeof (body3.hash) === 'undefined' || typeof (body3.difficulty) === 'undefined' || diff --git a/manage_scripts/altblock_add_auto.js b/manage_scripts/altblock_add_auto.js new file mode 100644 index 000000000..fafe2f9a7 --- /dev/null +++ b/manage_scripts/altblock_add_auto.js @@ -0,0 +1,68 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.hash) { + console.error("Please specify hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + global.coinFuncs.getLastBlockHeader(function (err, last_block_body) { + if (err !== null){ + console.error("Can't get last block info"); + process.exit(0); + } + global.coinFuncs.getPortAnyBlockHeaderByHash(port, hash, true, function (err_header, body_header) { + if (err_header) { + console.error("Can't get block info"); + console.error("err:" + JSON.stringify(err_header)); + console.error("body:" + JSON.stringify(body_header)); + process.exit(0); + } + if (!body_header.timestamp) body_header.timestamp = body_header.time; + if (!body_header.timestamp) body_header.timestamp = body_header.mediantime; + if (!body_header.timestamp) { + console.error("Can't get block timestamp: " + JSON.stringify(body_header)); + process.exit(0); + } + if ((Date.now() / 1000) < body_header.timestamp) body_header.timestamp = parseInt(body_header.timestamp / 1000); + if (!body_header.difficulty) { + console.error("Can't get block difficilty: " + JSON.stringify(body_header)); + process.exit(0); + } + if (!body_header.height) { + console.error("Can't get block height: " + JSON.stringify(body_header)); + process.exit(0); + } + + global.database.storeAltBlock(body_header.timestamp, global.protos.AltBlock.encode({ + hash: hash, + difficulty: body_header.difficulty, + shares: 0, + timestamp: body_header.timestamp * 1000, + poolType: global.protos.POOLTYPE.PPLNS, + unlocked: false, + valid: true, + port: port, + height: body_header.height, + anchor_height: last_block_body.height + }), function(data){ + if (!data){ + console.error("Block not stored"); + } else { + console.log("Block with " + port + " port and " + hash + " stored"); + } + process.exit(0); + }); + }); + }); +}); + diff --git a/manage_scripts/altblock_change_stage.js b/manage_scripts/altblock_change_stage.js new file mode 100644 index 000000000..bbd60b1f4 --- /dev/null +++ b/manage_scripts/altblock_change_stage.js @@ -0,0 +1,37 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2), { '--': true }); + +if (!argv.stage) { + console.error("Please specify new stage value"); + process.exit(1); +} +const stage = argv.stage; + +let hashes = {}; +for (const h of argv['--']) { + hashes[h] = 1; +} + +require("../init_mini.js").init(function() { + let changed = 0; + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash in hashes) { + console.log("Found altblock with " + blockData.hash + " hash"); + blockData.pay_stage = stage; + console.log("Put \"" + blockData.pay_stage + "\" stage to block"); + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + console.log("Changed altblock"); + changed = 1; + } + }); + } + cursor.close(); + txn.commit(); + if (!changed) console.log("Not found altblocks with specified hashes"); + process.exit(0); +}); \ No newline at end of file diff --git a/manage_scripts/altblock_del.js b/manage_scripts/altblock_del.js new file mode 100644 index 000000000..d884296ec --- /dev/null +++ b/manage_scripts/altblock_del.js @@ -0,0 +1,17 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.timestamp) { + console.error("Please specify altblock time"); + process.exit(1); +} +const timestamp = argv.timestamp; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + txn.del(global.database.altblockDB, timestamp); + txn.commit(); + console.log("Altblock with " + timestamp + " timestamp removed! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/altblock_fix_raw_reward.js b/manage_scripts/altblock_fix_raw_reward.js new file mode 100644 index 000000000..6be4a892e --- /dev/null +++ b/manage_scripts/altblock_fix_raw_reward.js @@ -0,0 +1,45 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = 0; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (!is_found && blockData.hash === hash) { + console.log("Found altblock with " + blockData.hash + " hash"); + is_found = 1; + global.coinFuncs.getPortAnyBlockHeaderByHash(blockData.port, argv.hash, false, function (err, body) { + if (err) { + cursor.close(); + txn.commit(); + console.error("Can't get block header"); + process.exit(1); + } + console.log("Changing raw block reward from " + blockData.value + " to " + body.reward); + blockData.value = body.reward; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + txn.commit(); + cursor.close(); + console.log("Changed altblock"); + process.exit(0); + }); + } + }); + } + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } +}); diff --git a/manage_scripts/altblock_recalc_distro.js b/manage_scripts/altblock_recalc_distro.js new file mode 100644 index 000000000..73bc18537 --- /dev/null +++ b/manage_scripts/altblock_recalc_distro.js @@ -0,0 +1,44 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = true; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === hash) { + is_found = true; + global.coinFuncs.getPortBlockHeaderByHash(blockData.port, hash, (err, body) => { + if (err !== null) { + console.log("Altblock with " + hash + " hash still has invalid hash for " + blockData.port + " port! Exiting!"); + cursor.close(); + txn.commit(); + process.exit(1); + } + console.log("Changing alt-block pay_ready from " + blockData.pay_ready + " to false"); + blockData.pay_ready = false; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + cursor.close(); + txn.commit(); + console.log("Altblock with " + hash + " hash was validated! Exiting!"); + process.exit(0); + }); + } + }); + } + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } +}); diff --git a/manage_scripts/altblock_revalidate.js b/manage_scripts/altblock_revalidate.js index 42ec2f533..4bcba3638 100644 --- a/manage_scripts/altblock_revalidate.js +++ b/manage_scripts/altblock_revalidate.js @@ -11,19 +11,30 @@ const hash = argv.hash; require("../init_mini.js").init(function() { let txn = global.database.env.beginTxn(); let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = true; for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { cursor.getCurrentBinary(function(key, data){ // jshint ignore:line let blockData = global.protos.AltBlock.decode(data); if (blockData.hash === hash) { + is_found = true; global.coinFuncs.getPortBlockHeaderByHash(blockData.port, hash, (err, body) => { - if (err !== null) { - console.log("Altblock with " + hash + " hash still has invalid hash for " + blockData.port + " port! Exiting!"); + if (err !== null || !body.reward) { + if (blockData.valid) { + blockData.valid = false; + blockData.unlocked = true; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + console.log("Altblock with " + hash + " hash became invalid for " + blockData.port + " port! Exiting!"); + } else { + console.log("Altblock with " + hash + " hash still has invalid hash for " + blockData.port + " port! Exiting!"); + } cursor.close(); txn.commit(); process.exit(1); } blockData.valid = true; blockData.unlocked = false; + //if (blockData.value != body.reward) console.log("Changing alt-block value from " + blockData.value + " to " + body.reward); + //blockData.value = body.reward; txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); cursor.close(); txn.commit(); @@ -33,8 +44,10 @@ require("../init_mini.js").init(function() { } }); } - cursor.close(); - txn.commit(); - console.log("Not found altblock with " + hash + " hash"); - process.exit(1); + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } }); diff --git a/manage_scripts/block_add.js b/manage_scripts/block_add.js index 04e71c2e6..08ae68200 100644 --- a/manage_scripts/block_add.js +++ b/manage_scripts/block_add.js @@ -42,7 +42,7 @@ require("../init_mini.js").init(function() { console.error("Block body is invalid: " + JSON.stringify(body3)); process.exit(1); } - const body4 = global.protos.AltBlock.encode(body3); + const body4 = global.protos.Block.encode(body3); let txn = global.database.env.beginTxn(); txn.putBinary(global.database.blockDB, height, body4); txn.commit(); diff --git a/manage_scripts/dump_shares.js b/manage_scripts/dump_shares.js index 0f0cb4e50..401371d0d 100644 --- a/manage_scripts/dump_shares.js +++ b/manage_scripts/dump_shares.js @@ -9,12 +9,19 @@ if (!argv.user) { } const user = argv.user; +let paymentid; +if (argv.paymentid) paymentid = argv.paymentid; + let worker; if (argv.worker) worker = argv.worker; let depth = 10; if (argv.depth) depth = argv.depth; +console.log("Dumping shares for " + user + " user"); +if (paymentid) console.log("Dumping shares for " + paymentid + " paymentid"); +if (worker) console.log("Dumping shares for " + worker + " worker"); + require("../init_mini.js").init(function() { global.coinFuncs.getLastBlockHeader(function (err, body) { @@ -30,7 +37,7 @@ require("../init_mini.js").init(function() { for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { cursor.getCurrentBinary(function(key, data){ // jshint ignore:line let shareData = global.protos.Share.decode(data); - if (shareData.paymentAddress === user && (!worker || shareData.identifier === worker)) { + if (shareData.paymentAddress === user && (!paymentid || shareData.paymentID === paymentid) && (!worker || shareData.identifier === worker)) { var d = new Date(shareData.timestamp); console.log(d.toString() + ": " + JSON.stringify(shareData)) } diff --git a/manage_scripts/dump_shares_all.js b/manage_scripts/dump_shares_all.js new file mode 100644 index 000000000..7846e21bd --- /dev/null +++ b/manage_scripts/dump_shares_all.js @@ -0,0 +1,35 @@ +"use strict"; + +let range = require('range'); +const argv = require('minimist')(process.argv.slice(2)); + +let depth = 10; +if (argv.depth) depth = argv.depth; + +console.log("Dumping shares"); + +require("../init_mini.js").init(function() { + + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null) { + console.error("Invalid block header"); + process.exit(1); + } + let lastBlock = body.height + 1; + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock, lastBlock - depth, -1).forEach(function (blockID) { + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let shareData = global.protos.Share.decode(data); + var d = new Date(shareData.timestamp); + console.log(d.toString() + ": " + JSON.stringify(shareData)) + }); + } + }); + cursor.close(); + txn.commit(); + process.exit(0); + }); +}); diff --git a/manage_scripts/dump_shares_port.js b/manage_scripts/dump_shares_port.js new file mode 100644 index 000000000..485de4e64 --- /dev/null +++ b/manage_scripts/dump_shares_port.js @@ -0,0 +1,43 @@ +"use strict"; + +let range = require('range'); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port to dump"); + process.exit(1); +} +const port = argv.port; + +let depth = 10; +if (argv.depth) depth = argv.depth; + +console.log("Dumping shares for " + port + " port"); + +require("../init_mini.js").init(function() { + + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null) { + console.error("Invalid block header"); + process.exit(1); + } + let lastBlock = body.height + 1; + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock, lastBlock - depth, -1).forEach(function (blockID) { + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let shareData = global.protos.Share.decode(data); + if (shareData.port === port) { + var d = new Date(shareData.timestamp); + console.log(d.toString() + ": " + JSON.stringify(shareData)) + } + }); + } + }); + cursor.close(); + txn.commit(); + process.exit(0); + }); +}); diff --git a/manage_scripts/fix_negative_ex_xmr_balance.js b/manage_scripts/fix_negative_ex_xmr_balance.js new file mode 100644 index 000000000..072acf42a --- /dev/null +++ b/manage_scripts/fix_negative_ex_xmr_balance.js @@ -0,0 +1,25 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +require("../init_mini.js").init(function() { + const xmr_balance = global.database.getCache("xmr_balance"); + if (xmr_balance !== false) { + if (!xmr_balance.value || xmr_balance.value < 0) { + console.error("Can't fix xmr_balance: " + JSON.stringify(xmr_balance)); + process.exit(1); + return; + } + const xmr_balance2 = { value: -xmr_balance.expected_increase, expected_increase: xmr_balance.expected_increase }; + console.log("In 10 seconds is going to change xmr_balance from " + JSON.stringify(xmr_balance) + " into " + JSON.stringify(xmr_balance2)); + setTimeout(function() { + global.database.setCache("xmr_balance", xmr_balance2); + console.log("Done."); + process.exit(0); + }, 10*1000); + } else { + console.error("Key xmr_balance is not found"); + process.exit(1); + } +}); + diff --git a/manage_scripts/get_block_hash.js b/manage_scripts/get_block_hash.js new file mode 100644 index 000000000..792405493 --- /dev/null +++ b/manage_scripts/get_block_hash.js @@ -0,0 +1,23 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.hash) { + console.error("Please specify hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortAnyBlockHeaderByHash(port, hash, false, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_header.js b/manage_scripts/get_block_header.js new file mode 100644 index 000000000..4f576074f --- /dev/null +++ b/manage_scripts/get_block_header.js @@ -0,0 +1,17 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortLastBlockHeader(port, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_height.js b/manage_scripts/get_block_height.js new file mode 100644 index 000000000..297877d2d --- /dev/null +++ b/manage_scripts/get_block_height.js @@ -0,0 +1,23 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.height) { + console.error("Please specify height"); + process.exit(1); +} +const height = argv.height; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortBlockHeaderByID(port, height, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_template.js b/manage_scripts/get_block_template.js new file mode 100644 index 000000000..9e5e4d8db --- /dev/null +++ b/manage_scripts/get_block_template.js @@ -0,0 +1,16 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortBlockTemplate(port, function (body_header) { + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/mdb_copy.js b/manage_scripts/mdb_copy.js new file mode 100644 index 000000000..64c12d3b1 --- /dev/null +++ b/manage_scripts/mdb_copy.js @@ -0,0 +1,120 @@ +"use strict"; + +const lmdb = require('node-lmdb'); +const fs = require('fs'); + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.dir) { + console.error("Please specify output lmdb dir"); + process.exit(1); +} + +if (fs.existsSync(argv.dir + "/data.mdb")) { + console.error("Please specify empty output lmdb dir"); + process.exit(1); +} + +if (!argv.size) { + console.error("Please specify output lmdb size in GB"); + process.exit(1); +} + +require("../init_mini.js").init(function() { + let env2 = new lmdb.Env(); + env2.open({ + path: argv.dir, + maxDbs: 10, + mapSize: argv.size * 1024 * 1024 * 1024, + useWritemap: true, + maxReaders: 512 + }); + let shareDB2 = env2.openDbi({ + name: 'shares', + create: true, + dupSort: true, + dupFixed: false, + integerDup: true, + integerKey: true, + keyIsUint32: true + }); + let blockDB2 = env2.openDbi({ + name: 'blocks', + create: true, + integerKey: true, + keyIsUint32: true + }); + let altblockDB2 = env2.openDbi({ + name: 'altblocks', + create: true, + integerKey: true, + keyIsUint32: true + }); + let cacheDB2 = env2.openDbi({ + name: 'cache', + create: true + }); + + console.log("Copying blocks"); + { + let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(blockDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying altblocks"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(altblockDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying shares"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(shareDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying cache"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(cacheDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + + + env2.close(); + console.log("DONE"); + process.exit(0); +}); diff --git a/manage_scripts/news.sh b/manage_scripts/news.sh new file mode 100755 index 000000000..c1fe4479b --- /dev/null +++ b/manage_scripts/news.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +subject=$1 +body=$2 + +if [ -z "$subject" ]; then echo "Set subject as first script paaameter"; exit 1; fi +if [ -z "$body" ]; then echo "Set bosy as second script paaameter"; exit 1; fi + +node cache_set.js --key=news --value='{"created": "'$(date +%s)'", "subject": "'"$subject"'", "body": "'"$body"'"}' \ No newline at end of file diff --git a/manage_scripts/user_del.js b/manage_scripts/user_del.js index e72b0f486..19b8b57a0 100644 --- a/manage_scripts/user_del.js +++ b/manage_scripts/user_del.js @@ -19,7 +19,7 @@ require("../init_mini.js").init(function() { console.log("Max payment to remove: " + global.config.payout.walletMin); let rows2remove = 0; - const where_str = payment_id === null ? "payment_address = '" + address + "' AND payment_id IS NULL" + const where_str = payment_id === null ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; async.waterfall([ @@ -46,7 +46,7 @@ require("../init_mini.js").init(function() { } if (rows.length) { console.log("Balance last update time: " + rows[0].last_edited); - if (Date.now()/1000 - global.support.formatDateFromSQL(rows[0].last_edited) < 24*60*60) { + if (Date.now()/1000 - global.support.formatDateFromSQL(rows[0].last_edited) < 12*60*60) { console.error("There was recent amount update. Refusing to continue!"); process.exit(1); } @@ -91,13 +91,13 @@ require("../init_mini.js").init(function() { }); }, function (callback) { - global.mysql.query("DELETE FROM balance WHERE " + where_str, [user]).then(function (rows) { + global.mysql.query("DELETE FROM balance WHERE " + where_str).then(function (rows) { console.log("DELETE FROM balance WHERE " + where_str); callback(); }); }, function (callback) { - global.mysql.query("DELETE FROM payments WHERE " + where_str, [user]).then(function (rows) { + global.mysql.query("DELETE FROM payments WHERE " + where_str).then(function (rows) { console.log("DELETE FROM payments WHERE " + where_str); callback(); }); diff --git a/manage_scripts/user_del_force.js b/manage_scripts/user_del_force.js new file mode 100644 index 000000000..18cfe3a7b --- /dev/null +++ b/manage_scripts/user_del_force.js @@ -0,0 +1,126 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to delete"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + const parts = user.split("."); + const address = parts.length === 1 ? user : parts[0]; + const payment_id = parts.length === 2 ? parts[1] : null; + + console.log("Address: " + address); + console.log("PaymentID: " + payment_id); + console.log("Max payment to remove: " + global.config.payout.walletMin); + let rows2remove = 0; + + const where_str = payment_id === null ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" + : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + console.log("Found rows in users table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + where_str).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + if (rows.length === 1 && rows[0].amount >= global.support.decimalToCoin(global.config.payout.walletMin)) { + console.error("Too big payment left: " + global.support.coinToDecimal(rows[0].amount)); + process.exit(1); + } + console.log("Found rows in balance table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM payments WHERE " + where_str).then(function (rows) { + console.log("Found rows in payments table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM block_balance WHERE " + where_str).then(function (rows) { + console.log("Found rows in block_balance table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + const address = global.database.getCache(user); + const stats = global.database.getCache("stats:" + user); + const history = global.database.getCache("history:" + user); + const identifiers = global.database.getCache("identifiers:" + user); + + if (address != false) console.log("Cache key is not empty: " + user); + if (stats != false) console.log("Cache key is not empty: " + "stats:" + user); + if (history != false) console.log("Cache key is not empty: " + "history:" + user); + if (identifiers != false) console.log("Cache key is not empty: " + "identifiers:" + user); + callback(); + + }, + function (callback) { + if (!rows2remove) { // to check that we accidently do not remove something usefull from LMDB cache + console.error("User was not found in SQL. Refusing to proceed to LMDB cache cleaning"); + process.exit(1); + } + callback(); + + }, + function (callback) { + global.mysql.query("DELETE FROM users WHERE username = ?", [user]).then(function (rows) { + console.log("DELETE FROM users WHERE username = " + user); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM balance WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM balance WHERE " + where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM payments WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM payments WHERE " + where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM block_balance WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM block_balance WHERE " + where_str); + callback(); + }); + }, + function (callback) { + console.log("Deleting LMDB cache keys"); + let txn = global.database.env.beginTxn(); + if (global.database.getCache(user)) txn.del(global.database.cacheDB, user); + if (global.database.getCache("stats:" + user)) txn.del(global.database.cacheDB, "stats:" + user); + if (global.database.getCache("history:" + user)) txn.del(global.database.cacheDB, "history:" + user); + if (global.database.getCache("identifiers:" + user)) txn.del(global.database.cacheDB, "identifiers:" + user); + txn.commit(); + callback(); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/package.json b/package.json index 94d10fb30..05819584f 100644 --- a/package.json +++ b/package.json @@ -1,47 +1,40 @@ { "name": "nodejs-pool", "version": "0.0.1", - "description": "Fairly simple universal cryptonote pool", + "description": "Improved version of Snipa22 nodejs-pool", "main": "init.js", "repository": { "type": "git", - "url": "https://github.com/Snipa22/node-crypto-pool.git" + "url": "https://github.com/MoneroOcean/nodejs-pool.git" }, "author": "Multiple", "license": "MIT", "dependencies": { - "async": "2.1.4", - "bignum": "^0.12.5", - "bluebird": "3.4.7", + "async": "3.2.0", + "bignum": "^0.13.1", "body-parser": "^1.16.0", - "bufferutil": "^1.3.0", "circular-buffer": "1.0.2", "cluster": "0.7.7", "concat-stream": "^1.6.0", "cors": "^2.8.1", - "crypto": "0.0.3", - "debug": "2.5.1", - "express": "4.14.0", + "debug": "2.6.9", + "express": "^4.17.1", "apicache": "1.2.1", - "jsonwebtoken": "^7.2.1", - "minimist": "1.2.0", + "jsonwebtoken": "^8.5.1", + "minimist": ">=1.2.3", "moment": "2.21.0", - "mysql": "2.15.0", - "node-lmdb": "0.4.12", + "mysql": "2.18.1", + "node-lmdb": "git+https://github.com/Venemo/node-lmdb.git#5941c1e553de4ae1d57a67d355b7c2dd87feaea6", "promise-mysql": "3.0.0", "protocol-buffers": "^3.2.1", "range": "0.0.3", - "redis": "^2.6.5", "request": "^2.79.0", - "request-json": "0.6.1", + "request-json": "0.6.5", "shapeshift.io": "1.3.0", - "socketio": "^1.0.0", "sprintf-js": "^1.0.3", - "sticky-cluster": "^0.3.1", - "uuid": "3.0.1", "wallet-address-validator": "0.1.0", - "zmq": "^2.15.3", - "cryptoforknote-util": "git+https://github.com/MoneroOcean/node-cryptoforknote-util.git#v1.0.0", - "cryptonight-hashing": "git+https://github.com/MoneroOcean/node-cryptonight-hashing.git#v2.2.0" + "utf8": "^3.0.0", + "cryptoforknote-util": "git+https://github.com/MoneroOcean/node-cryptoforknote-util.git#v11.0.2", + "cryptonight-hashing": "git+https://github.com/MoneroOcean/node-cryptonight-hashing.git#v26.0.0" } } diff --git a/user_scripts/balance_move_force.js b/user_scripts/balance_move_force.js new file mode 100644 index 000000000..4aefc2437 --- /dev/null +++ b/user_scripts/balance_move_force.js @@ -0,0 +1,92 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.old_user) { + console.error("Please specify old_user address to move balance from"); + process.exit(1); +} +const old_user = argv.old_user; + +if (!argv.new_user) { + console.error("Please specify new_user address to move balance to"); + process.exit(1); +} +const new_user = argv.new_user; + +require("../init_mini.js").init(function() { + const old_parts = old_user.split("."); + const old_address = old_parts.length === 1 ? old_user : old_parts[0]; + const old_payment_id = old_parts.length === 2 ? old_parts[1] : null; + + const new_parts = new_user.split("."); + const new_address = new_parts.length === 1 ? new_user : new_parts[0]; + const new_payment_id = new_parts.length === 2 ? new_parts[1] : null; + + console.log("Old Address: " + old_address); + console.log("Old PaymentID: " + old_payment_id); + console.log("New Address: " + new_address); + console.log("New PaymentID: " + new_payment_id); + + const old_where_str = old_payment_id === null ? "payment_address = '" + old_address + "' AND payment_id IS NULL" + : "payment_address = '" + old_address + "' AND payment_id = '" + old_payment_id + "'"; + + const new_where_str = new_payment_id === null ? "payment_address = '" + new_address + "' AND payment_id IS NULL" + : "payment_address = '" + new_address + "' AND payment_id = '" + new_payment_id + "'"; + + let old_amount; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find old_user!"); + process.exit(1); + } + old_amount = rows[0].amount; + console.log("Old address amount: " + global.support.coinToDecimal(old_amount)); + console.log("Old address last update time: " + rows[0].last_edited); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find new_user!"); + process.exit(1); + } + console.log("New address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = '0' WHERE " + old_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = '0' WHERE " + old_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + console.log("New old address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + console.log("New new address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/email_disable.js b/user_scripts/email_disable.js new file mode 100644 index 000000000..39d7ed270 --- /dev/null +++ b/user_scripts/email_disable.js @@ -0,0 +1,35 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} + +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length != 1) { + console.error("User password and thus email is not yet set"); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE users SET enable_email = '0' WHERE username = ?", [user]).then(function (rows) { + console.log("UPDATE users SET enable_email = '0' WHERE username = " + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/lock_pay.js b/user_scripts/lock_pay.js new file mode 100644 index 000000000..819e7bf89 --- /dev/null +++ b/user_scripts/lock_pay.js @@ -0,0 +1,35 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} + +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length != 1) { + console.error("User password and thus email is not yet set"); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE users SET payout_threshold_lock = '1' WHERE username = ?", [user]).then(function (rows) { + console.log("UPDATE users SET payout_threshold_lock = '1' WHERE username = " + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/pass_set.js b/user_scripts/pass_set.js index b0bae4f60..dbfb7403b 100644 --- a/user_scripts/pass_set.js +++ b/user_scripts/pass_set.js @@ -27,8 +27,8 @@ require("../init_mini.js").init(function() { }); }, function (callback) { - global.mysql.query("INSERT INTO users (username, email) VALUES (?, ?)", [user, pass]).then(function (rows) { - console.log("INSERT INTO users (username, email) VALUES (" + user + ", " + pass + ")"); + global.mysql.query("INSERT INTO users (username, email, enable_email) VALUES (?, ?, 0)", [user, pass]).then(function (rows) { + console.log("INSERT INTO users (username, email, enable_email) VALUES (" + user + ", " + pass + ", 0)"); callback(); }); }, diff --git a/user_scripts/pay_set.js b/user_scripts/pay_set.js new file mode 100644 index 000000000..1c559cc8e --- /dev/null +++ b/user_scripts/pay_set.js @@ -0,0 +1,26 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + const pay = global.support.decimalToCoin(argv.pay ? argv.pay : 0.003); + async.waterfall([ + function (callback) { + global.mysql.query("UPDATE users SET payout_threshold=? WHERE username=?", [pay, user]).then(function (rows) { + console.log("UPDATE users SET payout_threshold=" + pay + " WHERE username=" + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/unban_user.js b/user_scripts/unban_user.js new file mode 100644 index 000000000..59c4fa73a --- /dev/null +++ b/user_scripts/unban_user.js @@ -0,0 +1,33 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to unban"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query('DELETE FROM bans WHERE mining_address = ?', [user]).then(function (rows) { + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM bans").then(function (rows) { + for (let i in rows) { + const row = rows[i]; + console.log(row.mining_address + ": " + row.reason); + } + callback(); + }); + }, + function (callback) { + console.log("Done. User was unbanned."); + process.exit(0); + } + ]); +}); diff --git a/deployment/base.sql b/via06@github.com/create/new/database/moneraocean:master.base.sql similarity index 59% rename from deployment/base.sql rename to via06@github.com/create/new/database/moneraocean:master.base.sql index 11d8cd237..b48f04709 100644 --- a/deployment/base.sql +++ b/via06@github.com/create/new/database/moneraocean:master.base.sql @@ -1,6 +1,8 @@ CREATE DATABASE pool; -GRANT ALL ON pool.* TO pool@`127.0.0.1` IDENTIFIED BY '98erhfiuehw987fh23d'; -GRANT ALL ON pool.* TO pool@localhost IDENTIFIED BY '98erhfiuehw987fh23d'; +CREATE USER pool@`127.0.0.1` IDENTIFIED WITH mysql_native_password BY '98erhfiuehw987fh23d'; +CREATE USER pool@localhost IDENTIFIED WITH mysql_native_password BY '98erhfiuehw987fh23d'; +GRANT ALL ON pool.* TO pool@`127.0.0.1`; +GRANT ALL ON pool.* TO pool@localhost; FLUSH PRIVILEGES; USE pool; ALTER DATABASE pool DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; @@ -17,6 +19,29 @@ CREATE TABLE `balance` ( UNIQUE KEY `balance_payment_address_pool_type_bitcoin_payment_id_uindex` (`payment_address`,`pool_type`,`bitcoin`,`payment_id`), KEY `balance_payment_address_payment_id_index` (`payment_address`,`payment_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `paid_blocks` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `paid_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `found_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `port` int NOT NULL, + `hex` varchar(128) NOT NULL, + `amount` bigint(20) DEFAULT NULL, + PRIMARY KEY (`id`), + KEY `paid_blocks_paid_time` (`paid_time`), + UNIQUE KEY `paid_blocks_hex` (`hex`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `block_balance` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `hex` varchar(128) NOT NULL, + `payment_address` varchar(128) DEFAULT NULL, + `payment_id` varchar(128) DEFAULT NULL, + `amount` float(53) DEFAULT NULL, + PRIMARY KEY (`id`), + UNIQUE KEY `block_balance_id_uindex` (`id`), + UNIQUE KEY `block_balance_hex_payment_address_payment_id_uindex` (`hex`, `payment_address`,`payment_id`), + KEY `block_balance_hex_index` (`hex`), + KEY `block_balance_payment_address_payment_id_index` (`payment_address`,`payment_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `bans` ( `id` int(11) NOT NULL AUTO_INCREMENT, `ip_address` varchar(40) DEFAULT NULL, @@ -34,18 +59,6 @@ CREATE TABLE `notifications` ( PRIMARY KEY (`id`), UNIQUE KEY `notifications_id_uindex` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; -CREATE TABLE `block_log` ( - `id` int(11) NOT NULL COMMENT 'Block Height', - `orphan` tinyint(1) DEFAULT '1', - `hex` varchar(128) NOT NULL, - `find_time` timestamp NULL DEFAULT NULL, - `reward` bigint(20) DEFAULT NULL, - `difficulty` bigint(20) DEFAULT NULL, - `major_version` int(11) DEFAULT NULL, - `minor_version` int(11) DEFAULT NULL, - PRIMARY KEY (`hex`), - UNIQUE KEY `block_log_hex_uindex` (`hex`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `config` ( `id` int(11) NOT NULL AUTO_INCREMENT, `module` varchar(32) DEFAULT NULL, @@ -86,6 +99,13 @@ CREATE TABLE `pools` ( PRIMARY KEY (`id`), UNIQUE KEY `pools_id_uindex` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `pool_workers` ( + `id` tinyint(1) unsigned NOT NULL AUTO_INCREMENT, + `pool_id` int(11) NOT NULL, + `worker_id` int(11) NOT NULL, + PRIMARY KEY (`id`), + UNIQUE KEY `pool_workers_id_uindex` (`pool_id`, `worker_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `port_config` ( `poolPort` int(11) NOT NULL, `difficulty` int(11) DEFAULT '1000', @@ -153,6 +173,7 @@ CREATE TABLE `users` ( `admin` tinyint(1) DEFAULT '0', `payout_threshold` bigint(16) DEFAULT '0', `enable_email` tinyint(1) DEFAULT '1', + `payout_threshold_lock` tinyint(1) DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `users_id_uindex` (`id`), UNIQUE KEY `users_username_uindex` (`username`) @@ -182,26 +203,58 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'banPercent', '25', 'int', 'Percentage of shares that need to be invalid to be banned.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'banThreshold', '30', 'int', 'Number of shares before bans can begin'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustedMiners', 'true', 'bool', 'Enable the miner trust system'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minerThrottleSharePerSec', '5', 'int', 'Number of shares per second (per thread) after pool will throttle shares'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minerThrottleShareWindow', '5', 'int', 'Length of share throttle window in seconds'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'workerMax', '1000', 'int', 'Max number of worker connection before pool starts to issue bans'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'shareAccTime', '60', 'int', 'Length of time shares are accumulated in seconds'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustChange', '1', 'int', 'Change in the miner trust in percent'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustMin', '20', 'int', 'Minimum level of miner trust'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustPenalty', '30', 'int', 'Number of shares that must be successful to be trusted, reset to this value if trust share is broken'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'retargetTime', '60', 'int', 'Time between difficulty retargets'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'address', '127.0.0.1', 'string', 'Monero Daemon RPC IP'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'port', '18081', 'int', 'Monero Daemon RPC Port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'activePort', '0', 'int', 'Currently active daemon RPC port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'activePortHeavy', '0', 'int', 'Currently active heavy algo daemon RPC port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'activePortLight', '0', 'int', 'Currently active light algo daemon RPC port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'activePortFast', '0', 'int', 'Currently active fast algo daemon RPC port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'activePortXTL', '0', 'int', 'Currently active XTL algo daemon RPC port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'algoHashFactorHeavy', '0', 'float', 'Heavy algo hash price factor relative to algoHashFactor'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'algoHashFactorLight', '0', 'float', 'Light algo hash price factor relative to algoHashFactor'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'algoHashFactorFast', '0', 'float', 'Fast algo hash price factor relative to algoHashFactor'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'algoHashFactorXTL', '0', 'float', 'XTL algo hash price factor relative to algoHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'basicAuth', '', 'string', 'Basic auth header if needed by daemon'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'X-API-KEY', '', 'string', 'Turtle wallet API auth header'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'pollInterval', '100', 'int', 'Time in ms between pool daemon checks for new blocks'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRYO', '0', 'float', 'Ryo algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorSUMO', '0', 'float', 'SUMO algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorLOKI', '0', 'float', 'Loki algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXRN', '0', 'float', 'Saronite algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXTNC', '0', 'float', 'XtendCash algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorWOW', '0', 'float', 'Wownero algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorTUBE', '0', 'float', 'BitTube algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXHV', '0', 'float', 'Haven algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorAEON', '0', 'float', 'Aeon algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorMSR', '0', 'float', 'Masari algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXLA', '0', 'float', 'Scala algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorLTHN', '0', 'float', 'Lethean algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorGRFT', '0', 'float', 'Graft algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorTRTL', '0', 'float', 'Turtle algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorIRD', '0', 'float', 'Iridium algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorARQ', '0', 'float', 'ArqMa algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXMV', '0', 'float', 'MoneroV algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXWP', '0', 'float', 'Swap algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXEQ', '0', 'float', 'Equilibria algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorCCX', '0', 'float', 'Conceal algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXTA', '0', 'float', 'Italocoin algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorDERO', '0', 'float', 'Dero algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXMC', '0', 'float', 'XMC algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRVN', '0', 'float', 'RVN algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorETH', '0', 'float', 'ETH algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorBLOC', '0', 'float', 'BLOC algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorERG', '0', 'float', 'ERG algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRTM', '0', 'float', 'RTM algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'enableAlgoSwitching', 'false', 'bool', 'Enable smart miners (need additional altblockManager module)'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'verifyHost', '', 'string', 'Use to extra daemon height verify check'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address', '127.0.0.1', 'string', 'Monero Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_18082', '127.0.0.1', 'string', 'Monero Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_8545', '127.0.0.1', 'string', 'ETH Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_8766', '127.0.0.1', 'string', 'RVN Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_9053', '127.0.0.1', 'string', 'ERG Daemon RPC Wallet IP'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'port', '18082', 'int', 'Monero Daemon RPC Wallet Port'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('rpc', 'https', 'false', 'bool', 'Enable RPC over SSL'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'maxDifficulty', '500000', 'int', 'Maximum difficulty for VarDiff'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minDifficulty', '100', 'int', 'Minimum difficulty for VarDiff'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'maxDifficulty', '10000000000000', 'int', 'Maximum difficulty for VarDiff'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minDifficulty', '10000', 'int', 'Minimum difficulty for VarDiff'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'varDiffVariance', '20', 'int', 'Percentage out of the target time that difficulty changes'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'varDiffMaxChange', '125', 'int', 'Percentage amount that the difficulty may change'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'btcFee', '1.5', 'float', 'Fee charged for auto withdrawl via BTC'); @@ -209,12 +262,14 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'pplnsFee', '.6', 'float', 'Fee charged for the usage of the PPLNS pool'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'propFee', '.7', 'float', 'Fee charged for the usage of the proportial pool'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'soloFee', '.4', 'float', 'Fee charged for usage of the solo mining pool'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeMin', '5', 'float', 'Minimum XMR balance for payout to exchange/payment ID'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'walletMin', '.3', 'float', 'Minimum XMR balance for payout to personal wallet'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeMin', '.1', 'float', 'Minimum XMR balance for payout to exchange/payment ID'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'walletMin', '.01', 'float', 'Minimum XMR balance for payout to personal wallet'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'defaultPay', '.1', 'float', 'Default XMR balance for payout'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'devDonation', '3', 'float', 'Donation to XMR core development'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'poolDevDonation', '3', 'float', 'Donation to pool developer'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'denom', '.000001', 'float', 'Minimum balance that will be paid out to.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'blocksRequired', '60', 'int', 'Blocks required to validate a payout before it''s performed.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'blocksRequired', '30', 'int', 'Blocks required to validate a payout before it''s performed.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'anchorRound', '1', 'int', 'Round anchor height to group payment block pre-calc better. 1 - no round, 2 - round to every even block, 3 - round to every 3-rd block, etc.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'sigDivisor', '1000000000000', 'int', 'Divisor for turning coin into human readable amounts '); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feesForTXN', '10', 'int', 'Amount of XMR that is left from the fees to pay miner fees.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxTxnValue', '250', 'int', 'Maximum amount of XMR to send in a single transaction'); @@ -223,8 +278,10 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'allowBitcoin', 'false', 'bool', 'Allow the pool to auto-payout to BTC via ShapeShift'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeRate', '0', 'float', 'Current exchange rate'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'bestExchange', 'xmrto', 'string', 'Current best exchange'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'mixIn', '6', 'int', 'Mixin count for coins that support such things.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferLength', '480', 'int', 'Number of items to be cached in the stats buffers.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'mixIn', '10', 'int', 'Mixin count for coins that support such things.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'ethWalletPass', '', 'string', 'Ethereum wallet password'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferLength', '1000', 'int', 'Number of items to be cached in the stats buffers.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferHours', '72', 'int', 'Number of hours to be cached in the stats buffers.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pps', 'enable', 'false', 'bool', 'Enable PPS or not'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'shareMulti', '2', 'int', 'Multiply this times difficulty to set the N in PPLNS'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'shareMultiLog', '3', 'int', 'How many times the difficulty of the current block do we keep in shares before clearing them out'); @@ -236,35 +293,56 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_38081', '', 'string', 'Address to mine to for 38081 (MSR) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_48782', '', 'string', 'Address to mine to for 48782 (ITNS) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_34568', '', 'string', 'Address to mine to for 34568 (WOW) port.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19091', '', 'string', 'Address to mine to for 19091 (XMV) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19281', '', 'string', 'Address to mine to for 19281 (XMV) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_12211', '', 'string', 'Address to mine to for 12211 (RYO) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11181', '', 'string', 'Address to mine to for 11181 (Aeon) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_17750', '', 'string', 'Address to mine to for 17750 (Haven) port.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_24182', '', 'string', 'Address to mine to for 24182 (BitTube) port.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_20189', '', 'string', 'Address to mine to for 20189 (Stellite) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_25182', '', 'string', 'Address to mine to for 25182 (BitTube) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11812', '', 'string', 'Address to mine to for 11812 (Scala) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_22023', '', 'string', 'Address to mine to for 22023 (Loki) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_31014', '', 'string', 'Address to mine to for 31014 (Saronite) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_33124', '', 'string', 'Address to mine to for 33124 (XtendCash) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11898', '', 'string', 'Address to mine to for 11898 (Turtle) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_13007', '', 'string', 'Address to mine to for 13007 (Iridium) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19994', '', 'string', 'Address to mine to for 19994 (ArqMa) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19950', '', 'string', 'Address to mine to for 19950 (Swap) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_13102', '', 'string', 'Address to mine to for 13102 (Italocoin) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_20206', '', 'string', 'Address to mine to for 20206 (Dero) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_18181', '', 'string', 'Address to mine to for 18181 (XMC) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_16000', '', 'string', 'Address to mine to for 16000 (CCX) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9231', '', 'string', 'Address to mine to for 9231 (Equilibria) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_8766', '', 'string', 'Address to mine to for 8766 (Ravencoin) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_8545', '', 'string', 'Address to mine to for 8545 (Ethereum) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_2086', '', 'string', 'Address to mine to for 2086 (BLOC) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9053', '', 'string', 'Address to mine to for 9053 (ERG) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9998', '', 'string', 'Address to mine to for 9998 (RTM) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeAddress', '', 'string', 'Address that pool fees are sent to.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'cmcKey', '', 'string', 'CMC API Key for notification'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunKey', '', 'string', 'MailGun API Key for notification'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunURL', '', 'string', 'MailGun URL for notifications'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunNoCert', 'false', 'bool', 'Disable certificate check for MailGun'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'emailFrom', '', 'string', 'From address for the notification emails'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'testnet', 'false', 'bool', 'Does this pool use testnet?'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'blockCleanWarning', '360', 'int', 'Blocks before longRunner cleaner module will start to warn.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'enable', 'true', 'bool', 'Enable PPLNS on the pool.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('solo', 'enable', 'true', 'bool', 'Enable SOLO mining on the pool'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewAmount', '.011', 'float', 'Amount to charge for the txn fee'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewAmount', '.0001', 'float', 'Amount to charge for the txn fee'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewEnd', '4', 'float', 'Value at which txn fee amount drops to 0'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'rpcPasswordEnabled', 'false', 'bool', 'Does the wallet use a RPC password?'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'rpcPasswordPath', '', 'string', 'Path and file for the RPC password file location'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxPaymentTxns', '5', 'int', 'Maximum number of transactions in a single payment'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'shareHost', '', 'string', 'Host that receives share information'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxPaymentTxns', '15', 'int', 'Maximum number of transactions in a single payment'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'shareHost', 'http://localhost/leafApi', 'string', 'Host that receives share information'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingBody', 'Your worker: %(worker)s has stopped submitting hashes at: %(timestamp)s UTC\n', 'string', 'Email sent to the miner when their worker stops hashing'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingSubject', 'Status of your worker(s)', 'string', 'Subject of email sent to miner when worker stops hashing'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerStartHashingBody', 'Your worker: %(worker)s has started submitting hashes at: %(timestamp)s UTC\n', 'string', 'Email sent to the miner when their worker starts hashing'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerStartHashingSubject', 'Status of your worker(s)', 'string', 'Subject of email sent to miner when worker starts hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'adminEmail', '', 'string', 'Email of pool admin for alert notification stuff'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'emailSig', 'NodeJS-Pool Administration Team', 'string', 'Signature line for the emails.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'timer', '120', 'int', 'Number of minutes between main payment daemon cycles'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'timerRetry', '25', 'int', 'Number of minutes between payment daemon retrying due to not enough funds'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'priority', '0', 'int', 'Payout priority setting. 0 = use default (4x fee); 1 = low prio (1x fee)'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'priority', '1', 'int', 'Payout priority setting. 0 = use default (4x fee); 1 = low prio (1x fee)'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'allowStuckPoolKill', 'false', 'bool', 'Allow to kill the pool in case of stuck block template'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'dbSizeGB', '24', 'int', 'LMDB size in GBs'); INSERT INTO pool.users (username, pass, email, admin, payout_threshold) VALUES ('Administrator', null, 'Password123', 1, 0); INSERT INTO pool.port_config (poolPort, difficulty, portDesc, portType, hidden, `ssl`) VALUES (3333, 1000, 'Low-End Hardware (Up to 30-40 h/s)', 'pplns', 0, 0); INSERT INTO pool.port_config (poolPort, difficulty, portDesc, portType, hidden, `ssl`) VALUES (5555, 5000, 'Medium-Range Hardware (Up to 160 h/s)', 'pplns', 0, 0);