Data simulation and loading pressing tool MatrixBench

MatrixBench is abbreviated as mxbench. It is a MatrixDB streaming data loading service pressure measurement and data simulation tool. It can quickly generate random data based on the given number of devices, time range, number of indicators, etc., and insert it into the MatrixDB database at high speed through MatrixGate. This tool is located in bin/mxbench in the MatrixDB installation directory.

MatrixBench provides a wealth of parameters to meet different testing needs. The total number of devices is specified by the parameter tag-range, and the generated data is evenly distributed to each device. The timestamp starts with ts-begin. For each ts-rps record inserted, the timestamp is increased by one second. The number of indicators is specified by columns-per-row, and the indicator type is specified by column-data-type. It can be a simple type or a compound type such as JSON or Array. If it is a compound type, the data type of each specific indicator value is specified through point-data-type. The detailed startup parameters and instructions are as follows:

| Parameter name | Parameter value | Parameter meaning | | --- | --- | | --table-name | tableName | A target table name must be specified | | --database | Default postgres | Target database name | | --http-host | Default local address | MatrixGate host name | | --http-port | Default 8086 | MatrixGate host port number | | --parallel | The default is equal to the number of CPU cores | The maximum number of connections connected to MatrixGate | | --batch-size | Default 1000 | Number of rows of data contained per request per connection | | --tag-range | Default 50000 | Number of target devices to load data | | --ts-rps | Default 25000 | The frequency of timestamp change of loaded data, default means that every 25000 pieces of data are inserted, the timestamp is increased by one second | | --ts-begin | Default current time | Start time point corresponding to load data | | --format | Default text | Loading data format: text or csv | | --time-format | Default unix-second | Time stamp unit: unix-second|unix-ms|unix-nano|raw | | --delimiter | Default | | Load the column separator in each row of data | | --columns-per-row | Default 100 | Number of columns per row of loading data | | --column-data-type | Default simple | Indicator column type in the data table, simple single value types currently support, such as float/int, and compound types json and array | | --points-per-column | Default 3 | When --column-data-type is compound types JSON or ARRAY, the number of indicator values contained in each Column | | --point-data-type | Default float8 | When --column-data-type is the compound type JSON and ARRAY, the numerical type of each indicator value in Column supports int, float4, float8 | | --use-gzip | Default FALSE | Whether to enable GZIP compression when sending data |

Use Example

Create a database table in the demo database as follows:

create table sensor_data(
    "time" timestamp with time zone not null,
    tag_id integer,
    tag_value double precision)
Distributed By(tag_id);

Start matrixgate:

mxgate config --db-master-port 5432 --db-database demo --interval 150 > ~/mxgate.conf
mxgate start --config ~/mxgate.conf

Execute mxbench:

mxbench --database demo --table-name sensor_data --columns-per-row 1 --column-data-type simple --point-data-type float8

After running mxbench, you can see the output like this:

Send to http://127.0.0.1:8086
Parallel level is 64
Point format is simple
Tag range is 50000
TIMESTAMP step is 40000 nano second
16036-03-16 16:55:04: sent OK: 4282000, ERR 0, Lat 147.67ms last10s 147.67ms, Row/s: 428200, last10s 428200
16036-03-16 16:55:14: sent OK: 8506000, ERR 0, Lat 148.12ms last10s 148.58ms, Row/s: 425300, last10s 422400
16036-03-16 16:55:24: sent OK: 12794000, ERR 0, Lat 148.26ms last10s 148.52ms, Row/s: 426466, last10s 428800

OK represents the total number of rows of data successfully sent and inserted after mxbench is started, ERR represents the total error data, Lat represents the average delay after startup, last10s after Lat represents the average delay in the last 10 seconds, Row/s represents the average number of rows sent per second after startup, and last10s after Row/s represents the average number of rows sent in the last 10 seconds.

After mxbench is started, the pressure test will be continued until the user presses ctrl-c to exit.