Understanding Progress Information and Statistical Reports

This document provides a detailed explanation of the progress information displayed during MatrixBench execution and the statistical reports printed upon completion.

Note!
mxbench is now open-source. Code review and contributions are welcome. See the README for more details.

1 Progress Information

Example:

● Stdin Writer Report
  ● period start: 2022-04-29 10:08:11, end: 2022-04-29 10:08:16, period: 5.000129734s

  ● count written in total: 637025 rows/ 1500000 rows 42%, 637025 rows in this period

  ● size written in total: 160878718 bytes/ 360000003 bytes 44%, 160878718 bytes in this period

  ● size written to MxGate in total: 350075440 bytes, 350075440 bytes in this period

● Telematics Benchmark Report
  ● stats for query SINGLE_TAG_LATEST_QUERY with parallel 8: progress: 100%

  ● stats for query MULTI_TAG_LATEST_QUERY with parallel 8: progress: 43%

Progress information consists of two parts: Writer and Benchmark.

1.1 Writer

  • period start, period end, period: Start time, end time, and duration of the current reporting interval.
  • count written: Total number of rows written versus total expected rows, along with the completion percentage. "xx in this period" indicates the number of rows written during this interval.
  • size written: Total number of bytes written versus total expected bytes, along with the completion percentage. "xx in this period" indicates the number of bytes written during this interval.
  • size written to MxGate: Total number of bytes sent to MatrixGate after converting data into text format. "xx in this period" indicates the number of bytes written to MatrixGate during this interval.

1.2 Benchmark

Displays the execution progress of a specific query at a given concurrency level (specified by the parallel parameter). When queries run concurrently with data loading, they may execute multiple rounds before data loading finishes. This report shows progress for the most recent round only.

2 Statistical Reports

2.1 Writer

┌───────────────────────────────────────────────────────┐
│            Summary Report for Stdin Writer            │
├─────────────────────────────────┬─────────────────────┤
│ start time:                     │ 2022-04-27 13:29:01 │
├─────────────────────────────────┼─────────────────────┤
│ stop time:                      │ 2022-04-27 13:29:58 │
├─────────────────────────────────┼─────────────────────┤
│ size written to MxGate (bytes): │ 848333400           │
├─────────────────────────────────┼─────────────────────┤
│ lines inserted:                 │ 1500000             │
├─────────────────────────────────┼─────────────────────┤
│ compress ratio:                 │ 1.561276 : 1        │
└─────────────────────────────────┴─────────────────────┘
  • start time: Start time of data loading.
  • stop time: End time of data loading.
  • size written to MxGate (bytes): Total number of bytes written to MatrixGate.
  • lines inserted: Number of data records inserted. Due to UPSERT operations, this number may exceed the actual number of rows in the database.
  • compress ratio: Compression ratio, defined as the ratio between the size of data written to MatrixGate and the actual on-disk size of the table in the database.

2.2 Benchmark

A real-time report is printed for each query at every parallel setting.

┌─────────────────┬───────────────┐
│ Overall Latency │ 29.948370582s │
│ Average Latency │   13.723304ms │
│ P75 Latency     │    14.35972ms │
│ P50 Latency     │   13.654556ms │
│ P25 Latency     │   12.925912ms │
│ TPS             │           582 │
└─────────────────┴───────────────┘
  • Pxx: Latency at the xx-th percentile. For example, P75 = 14.35972ms means 75% of query executions completed within this time, while 25% were slower. P50 represents the median latency.
  • TPS: Transactions per second — the average number of queries executed per second.

Summary Report:

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                Summary Report for Telematics Benchmark                                                │
├─────────────────────┬─────────────────────────────────────┬─────────────────────────────────────┬─────────────────────────────────────┤
│ Parallel\Query Name │ SINGLE_TAG_LATEST_QUERY             │ MULTI_TAG_LATEST_QUERY              │ SINGLE_TAG_DETAIL_QUERY             │
├─────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤
│ 8                   │ ┌─────────────────┬───────────────┐ │ ┌─────────────────┬───────────────┐ │ ┌─────────────────┬───────────────┐ │
│                     │ │ Overall Latency │ 30.008625133s │ │ │ Overall Latency │ 36.407016561s │ │ │ Overall Latency │ 29.948370582s │ │
│                     │ │ Average Latency │    23.81758ms │ │ │ Average Latency │  7.270212105s │ │ │ Average Latency │   13.723304ms │ │
│                     │ │ P75 Latency     │   24.911171ms │ │ │ P75 Latency     │  8.139682559s │ │ │ P75 Latency     │    14.35972ms │ │
│                     │ │ P50 Latency     │   23.411248ms │ │ │ P50 Latency     │  7.161237157s │ │ │ P50 Latency     │   13.654556ms │ │
│                     │ │ P25 Latency     │   20.899622ms │ │ │ P25 Latency     │  6.642459805s │ │ │ P25 Latency     │   12.925912ms │ │
│                     │ │ TPS             │           335 │ │ │ TPS             │             1 │ │ │ TPS             │           582 │ │
│                     │ └─────────────────┴───────────────┘ │ └─────────────────┴───────────────┘ │ └─────────────────┴───────────────┘ │
│                     │ progress: 100%                      │ progress: 100%                      │ progress: 100%                      │
└─────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┘
  • Each row represents results for a specific query concurrency level.
  • Each column represents performance metrics for a specific query across different concurrency levels.
  • If the query set runs multiple rounds (e.g., mixed workloads where queries continue during ongoing data load), only results from the final round are shown.
  • If a query fails or execution is interrupted by the user, the progress indicator shows current completion status, and statistics are based only on completed queries.

Note!
For a quick start with MatrixBench, see MatrixBench Quick Start. For full command-line options, refer to MatrixBench Command-Line Parameters. For an overview of key features, see MatrixBench Key Features.