Skip to content

Performance Considerations

A practical guide to GSP's performance characteristics — what is fast, what is slow, and what to do about it.

Measured numbers

Both editions measured on the same machine (x86_64 Linux), parsing Oracle SQL: Java 4.1.6 on JDK 21, .NET 4.1.0.7 on net10.0. Each figure is the mean of 2,000–3,000 iterations after warm-up.

Workload Java .NET
Simple SELECT (one table, one WHERE) 0.66 ms — 1,509/sec 0.40 ms — 2,490/sec
Complex SELECT (CTE, 2 joins, window function, CASE) 1.06 ms — 943/sec 1.45 ms — 691/sec
Tokenisation only, same complex statement 0.14 ms (7.5× faster than parsing) 0.16 ms (9.4× faster)
First parse in the process (grammar-table init) 1,849 ms 576 ms
Constructing another parser afterwards 0.003 ms 0.218 ms

Treat these as order-of-magnitude guidance. Your numbers will vary with SQL complexity, vendor (Oracle PL/SQL is the slowest grammar), and CPU.

The expensive thing is the first parse, not constructing parsers

"Reuse your parser instances because construction is expensive" is common advice and the reason usually given for it is wrong. Constructing a second parser costs 3 µs on Java and 218 µs on .NET — cheap. What costs 0.6–1.8 seconds is the first parse in the process, which initialises the grammar tables once.

So: expect a one-time startup cost and warm up before timing anything, or before serving the first user request. Pooling parsers is still worth it on .NET, where 218 µs each adds up across a large batch, but on Java construction is effectively free and pooling buys you almost nothing.

What is fast

  • Tokenisation. Skips parsing entirely; roughly 8× faster. Use it when you only need keyword, identifier, and literal information.
  • Re-parsing on a warm process. Once the grammar tables are loaded, individual parse() calls are sub-millisecond for ordinary DML.
  • ANSI DML on most vendors. SELECT/INSERT/UPDATE/DELETE without procedural blocks.

What is slow

  • The first parse. See the warning above.
  • Oracle PL/SQL. The largest single grammar in the library. Procedural blocks are roughly an order of magnitude slower than equivalently sized DML.
  • Pathologically nested expressions. A 1,000-deep nesting of binary operators can exhaust the parser stack, and is slow even when it succeeds. Real SQL does not look like this.
  • Multi-megabyte single statements. Uncommon; the AST scales linearly with statement size.

Tactics

Reuse the parser across a batch

Not for construction cost, but because it keeps the code simple and avoids re-triggering any per-instance setup:

1
2
3
4
5
6
TGSqlParser parser = new TGSqlParser(EDbVendor.dbvoracle);
for (String sql : batch) {
    parser.sqltext = sql;
    if (parser.parse() != 0) continue;
    process(parser.sqlstatements);
}

1
2
3
4
5
6
7
var parser = new TGSqlParser(EDbVendor.dbvoracle);
foreach (string sql in batch)
{
    parser.sqltext = sql;
    if (parser.parse() != 0) continue;
    Process(parser.sqlstatements);
}

Warm up before you measure or serve

Because the first parse pays the grammar-table cost, a cold benchmark or a cold first request is misleading:

1
2
3
4
// Call once at startup, off the request path.
TGSqlParser warm = new TGSqlParser(EDbVendor.dbvoracle);
warm.sqltext = "SELECT 1 FROM dual";
warm.parse();

1
2
3
4
// Call once at startup, off the request path.
var warm = new TGSqlParser(EDbVendor.dbvoracle);
warm.sqltext = "SELECT 1 FROM dual";
warm.parse();

Warm each vendor you intend to use; the grammars initialise independently.

Tokenise when you do not need an AST

1
2
3
4
5
6
parser.sqltext = sql;
parser.tokenizeSqltext();
for (int i = 0; i < parser.sourcetokenlist.size(); i++) {
    TSourceToken token = parser.sourcetokenlist.get(i);
    // ...
}

1
2
3
4
5
6
parser.sqltext = sql;
parser.tokenizeSqltext();
foreach (TSourceToken token in parser.sourcetokenlist)
{
    // ...
}

Parse statement-by-statement for large files

For multi-megabyte scripts, split first and parse one statement at a time. This keeps peak memory bounded.

Threading

TGSqlParser is not thread-safe. Give each thread its own instance.

A ThreadLocal is the cheapest correct answer, and on Java construction is so cheap that a pool is rarely worth the complexity:

1
2
static final ThreadLocal<TGSqlParser> PARSER =
    ThreadLocal.withInitial(() -> new TGSqlParser(EDbVendor.dbvoracle));

A ThreadLocal<T> works the same way; a ConcurrentBag<TGSqlParser> pool is also reasonable here, since .NET construction costs 218 µs:

1
2
static readonly ThreadLocal<TGSqlParser> Parser =
    new(() => new TGSqlParser(EDbVendor.dbvoracle));

See Performance Optimization for fuller examples.

Memory

The AST holds references to the whole token list for line and column tracking, so an AST is considerably larger than the SQL that produced it. Drop the parser reference once you have extracted what you need, and let the collector reclaim the tree.

For long-running services, batch your work and let each batch's ASTs go out of scope rather than accumulating them in a list.

When to suspect the parser

Most performance problems in code that uses GSP are not in the parser. Common culprits, in descending order:

  1. Allocations in your visitor code — building strings inside a hook that runs on every node.
  2. String concatenation in a tight loop — use StringBuilder.
  3. Cold-start cost mistaken for steady-state cost — see the warning above.
  4. Re-parsing the same SQL repeatedly — cache the result.
  5. PL/SQL parsing on workloads that do not need it.

If you have ruled those out and still see slow parsing, file an issue with the offending SQL — it is usually a grammar pessimisation that can be fixed.

Profiling

JFR (java -XX:StartFlightRecording) and async-profiler both work well. For micro-benchmarks use JMH, and make sure the harness warms up — otherwise you will be measuring the one-time grammar-table init.

dotnet-counters monitor --process-id <pid> and dotnet-trace collect work against GSP-using processes. For micro-benchmarks, BenchmarkDotNet is the standard and handles warm-up for you.

See also