Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
b528e18
Bump v0.4.6
nevill Jul 21, 2018
ffb8630
Fix passing Pool to Zongji constructor
normanrz Sep 1, 2018
6386d7e
Bump v0.4.7
nevill Nov 22, 2018
bf6f533
- Require node >= 6.0 to run
nevill May 12, 2019
ecce01c
Running tests on node v6, v8, v10, v12
nevill May 12, 2019
d2d1b1d
Refactor
nevill May 13, 2019
42c7b6c
Refactor on class ZongJi
nevill May 16, 2019
e115659
Update .eslintrc, to support es6 syntax
nevill May 21, 2019
08a027f
Refactor
nevill May 25, 2019
48b6f12
Fix on tests
nevill May 25, 2019
b7b5555
Emit a ready event during ZongJi#_init
nevill May 26, 2019
3b8a171
Rewrite test reconnect_at_pos, to deal with async function calls
nevill May 27, 2019
6ae6681
Update README
nevill May 27, 2019
e66344d
Bump lodash from 4.17.11 to 4.17.14
dependabot[bot] Jul 13, 2019
19ca021
Refactor, rename class BinlogHeader to BinlogPacket
nevill Aug 9, 2019
f2b8c69
Refactor, using generator to split #parse() into two steps, filter th…
nevill Aug 9, 2019
63671f3
Use Buffer.alloc to init a new buffer
nevill Oct 15, 2019
42f43d5
Use more ES6 syntax
nevill Oct 23, 2019
7058a70
Update eslint to 6.5.1
nevill Oct 23, 2019
9653b9a
Lint the code
nevill Oct 23, 2019
a3d2434
- Drop the support of Node.js v6
nevill Oct 24, 2019
417ad5f
Add a new event `stopped`, also update README to describe about all t…
nevill Oct 26, 2019
74c2e3a
Replace nodeunit with tap (#109).
nevill Oct 24, 2019
9502ad8
Refactor, rewrite all the tests by using tap, see #109
nevill Oct 24, 2019
d9701c0
Breaking changes, see #111
nevill Nov 2, 2019
2c9d5db
Add package big-integer as dependency
nevill Nov 5, 2019
fc5ef30
Refactor, exports.parseUInt64 => parseUInt64, exports.parseUInt24 => …
nevill Nov 5, 2019
c3a49c1
Use big-integer to support int64 during parsing
nevill Nov 5, 2019
07b8101
Update README for 0.5.0 release
nevill Nov 5, 2019
9bd02a9
Bump v0.5.0
nevill Nov 5, 2019
a6dea41
Add binlog_row_image support, see #51
nevill Nov 8, 2019
82af342
Bump v0.5.1
nevill Nov 8, 2019
4941612
Use DataView to fix IEEE754 conversion error.
jefbarn Dec 9, 2019
0d2a79e
Merge pull request #117 from jefbarn/master
nevill Dec 11, 2019
94aaac1
Update dependencies to the latest
nevill Mar 22, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .eslintrc
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"env": {
"node": true
"node": true,
"es2020": true
},
"extends": "eslint:recommended",
"rules": {
Expand All @@ -10,6 +11,7 @@
"no-console": "off",
"no-trailing-spaces": ["error", { "skipBlankLines": true }],
"no-unused-vars": "warn",
"no-var": "warn",
"quotes": ["warn", "single", "avoid-escape"],
"semi": ["error", "always"],
"space-before-blocks": "error"
Expand Down
3 changes: 1 addition & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@ language: node_js
sudo: required
dist: trusty
node_js:
- "4"
- "8"
- "12"
services:
- mysql
- docker
before_script:
- npm run lint
Expand Down
40 changes: 26 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@ A MySQL binlog listener running on Node.js.

ZongJi (踪迹) is pronounced as `zōng jì` in Chinese.

This package is a "pure JS" implementation based on [`node-mysql`](https://github.com/felixge/node-mysql). Since v0.2.0, The native part (which was written in C++) has been dropped.
This package is a pure JS implementation based on [`mysql`](https://github.com/mysqljs/mysql). It has been tested to work in MySQL 5.5, 5.6, and 5.7.

This package has been tested to work in MySQL 5.5, 5.6, and 5.7.
# Latest Release

The latest release is v0.5.0, only supports Node.js from v8.

v0.4.7 is the last release which supports Node.js v4.x.

## Quick Start

```javascript
var zongji = new ZongJi({ /* ... MySQL Connection Settings ... */ });
let zongji = new ZongJi({ /* ... MySQL Connection Settings ... */ });

// Each change to the replication log results in an event
zongji.on('binlog', function(evt) {
Expand All @@ -27,7 +31,7 @@ For a complete implementation see [`example.js`](example.js)...

## Installation

* Requires Node.js v4+
* Requires Node.js v8+

```bash
$ npm install zongji
Expand Down Expand Up @@ -59,30 +63,38 @@ For a complete implementation see [`example.js`](example.js)...

The `ZongJi` constructor accepts one argument of either:

* An object containing MySQL connection details in the same format as used by `node-mysql`
* Or, a `node-mysql` `Connection` or `Pool` object that will be used for querying column information.
* An object containing MySQL connection details in the same format as used by [package mysql](https://npm.im/mysql)
* Or, a [mysql](https://npm.im/mysql) `Connection` or `Pool` object that will be used for querying column information.

If a `Connection` or `Pool` object is passed to the constructor, it will not be destroyed/ended by Zongji's `stop()` method.

If there is a `dateStrings` `node-mysql` configuration option in the connection details or connection, `ZongJi` will follow it.
If there is a `dateStrings` `mysql` configuration option in the connection details or connection, `ZongJi` will follow it.

Each instance includes the following methods:

Method Name | Arguments | Description
------------|-----------|------------------------
`start` | `options` | Start receiving replication events, see options listed below
`stop` | *None* | Disconnect from MySQL server, stop receiving events
`set` | `options` | Change options after `start()`
`on` | `eventName`, `handler` | Add a listener to the `binlog` or `error` event. Each handler function accepts one argument.

Some events can be emitted in different phases:

Event Name | Description
-----------|------------------------
`ready` | This event is occurred right after ZongJi successfully established a connection, setup slave status, and set binlog position.
`binlog` | Once a binlog is received and passes the filter, it will bubble up with this event.
`error` | Every error will be caught by this event.
`stopped` | Emitted when ZongJi connection is stopped (ZongJi#stop is called).

**Options available:**

Option Name | Type | Description
------------|------|-------------------------------
`serverId` | `integer` | [Unique number (1 - 2<sup>32</sup>)](http://dev.mysql.com/doc/refman/5.0/en/replication-options.html#option_mysqld_server-id) to identify this replication slave instance. Must be specified if running more than one instance of ZongJi. Must be used in `start()` method for effect.<br>**Default:** `1`
`startAtEnd` | `boolean` | Pass `true` to only emit binlog events that occur after ZongJi's instantiation. Must be used in `start()` method for effect.<br>**Default:** `false`
`binlogName` | `string` | Begin reading events from this binlog file. If specified together with `binlogNextPos`, will take precedence over `startAtEnd`.
`binlogNextPos` | `integer` | Begin reading events from this position. Must be included with `binlogName`.
`filename` | `string` | Begin reading events from this binlog file. If specified together with `position`, will take precedence over `startAtEnd`.
`position` | `integer` | Begin reading events from this position. Must be included with `filename`.
`includeEvents` | `[string]` | Array of event names to include<br>**Example:** `['writerows', 'updaterows', 'deleterows']`
`excludeEvents` | `[string]` | Array of event names to exclude<br>**Example:** `['rotate', 'tablemap']`
`includeSchema` | `object` | Object describing which databases and tables to include (Only for row events). Use database names as the key and pass an array of table names or `true` (for the entire database).<br>**Example:** ```{ 'my_database': ['allow_table', 'another_table'], 'another_db': true }```
Expand All @@ -98,7 +110,7 @@ Event name | Description
`unknown` | Catch any other events
`query` | [Insert/Update/Delete Query](http://dev.mysql.com/doc/internals/en/query-event.html)
`intvar` | [Autoincrement and LAST_INSERT_ID](https://dev.mysql.com/doc/internals/en/intvar-event.html)
`rotate` | [New Binlog file](http://dev.mysql.com/doc/internals/en/rotate-event.html) Not required to be included to rotate to new files, but it is required to be included in order to keep the `binlogName` and `binlogNextPos` properties updated with current values for [graceful restarting on errors](https://gist.github.com/numtel/5b37b2a7f47b380c1a099596c6f3db2f).
`rotate` | [New Binlog file](http://dev.mysql.com/doc/internals/en/rotate-event.html) Not required to be included to rotate to new files, but it is required to be included in order to keep the `filename` and `position` properties updated with current values for [graceful restarting on errors](https://gist.github.com/numtel/5b37b2a7f47b380c1a099596c6f3db2f).
`format` | [Format Description](http://dev.mysql.com/doc/internals/en/format-description-event.html)
`xid` | [Transaction ID](http://dev.mysql.com/doc/internals/en/xid-event.html)
`tablemap` | Before any row event (must be included for any other row events)
Expand All @@ -117,8 +129,8 @@ Name | Description

## Important Notes

* :star2: [All types allowed by `node-mysql`](https://github.com/felixge/node-mysql#type-casting) are supported by this package.
* :speak_no_evil: While 64-bit integers in MySQL (`BIGINT` type) allow values in the range of 2<sup>64</sup> (± ½ × 2<sup>64</sup> for signed values), Javascript's internal storage of numbers limits values to 2<sup>53</sup>, making the allowed range of `BIGINT` fields only `-9007199254740992` to `9007199254740992`. Unsigned 64-bit integers must also not exceed `9007199254740992`.
* :star2: [All types allowed by `mysql`](https://github.com/mysqljs/mysql#type-casting) are supported by this package.
* :speak_no_evil: 64-bit integer is supported via package big-integer(see #108). If an integer is within the safe range of JS number (-2^53, 2^53), a Number object will returned, otherwise, will return as String.
* :point_right: `TRUNCATE` statement does not cause corresponding `DeleteRows` event. Use unqualified `DELETE FROM` for same effect.
* When using fractional seconds with `DATETIME` and `TIMESTAMP` data types in MySQL > 5.6.4, only millisecond precision is available due to the limit of Javascript's `Date` object.

Expand All @@ -131,7 +143,7 @@ Name | Description

I learnt many things from following resources while making ZongJi.

* https://github.com/felixge/node-mysql
* https://github.com/mysqljs/mysql
* https://github.com/felixge/faster-than-c/
* http://intuitive-search.blogspot.co.uk/2011/07/binary-log-api-and-replication-listener.html
* https://github.com/Sannis/node-mysql-libmysqlclient
Expand Down
6 changes: 5 additions & 1 deletion docker-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
MYSQL_HOSTS="mysql55 mysql56 mysql57"

for hostname in ${MYSQL_HOSTS}; do
echo $hostname
echo $hostname + node 8
docker run -it --network=zongji_default -e MYSQL_HOST=$hostname -w /build -v $PWD:/build node:8 npm test
echo $hostname + node 10
docker run -it --network=zongji_default -e MYSQL_HOST=$hostname -w /build -v $PWD:/build node:10 npm test
echo $hostname + node 12
docker run -it --network=zongji_default -e MYSQL_HOST=$hostname -w /build -v $PWD:/build node:12 npm test
done
4 changes: 2 additions & 2 deletions example.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Client code
var ZongJi = require('./');
const ZongJi = require('./');

var zongji = new ZongJi({
const zongji = new ZongJi({
host : 'localhost',
user : 'zongji',
password : 'zongji',
Expand Down
Loading