Skip to content

Data lineage options, explained

DataFlowAnalyzer is the class that answers one question about a SQL script:

For every column that gets written, which source columns did its value come from — through which joins, functions, CTEs, temp tables, procedures and dynamic SQL?

The answer is a graph. Option is the object that decides how much of that graph you get back and how it is labelled. This page walks through every setting on Option, one at a time, with a small SQL statement and the actual output GSP produces with the setting off and on.

Who this page is for

You are new to data lineage, or new to GSP, and you want to know what each option means before you switch it on. If you already know the option you need and just want its CLI flag or its availability in the SQLFlow web UI, use Configuration options instead — that page is organised by delivery mode (Query / Job / CLI / API); this one is organised by what the option does to the lineage graph.

Verified against
GSP Java version 4.2.8 (TBaseType.versionid), released 2026-08-31
Page last verified 2026-09-08
Options covered 70 mutators on Option

Every SQL sample and every block of output below was produced by running that build — none of it is written by hand. The stamp records what was true for 4.2.8. A newer GSP release does not silently invalidate the page, but the stamp, not the release notes, is what tells you how far behind it might be. See Refreshing this page for how to bring it forward.

1. Before the options: one run, one output

1.1 Your first analysis

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
import gudusoft.gsqlparser.EDbVendor;
import gudusoft.gsqlparser.dlineage.DataFlowAnalyzer;
import gudusoft.gsqlparser.dlineage.dataflow.model.Option;

public class FirstLineage {
    public static void main(String[] args) {
        Option option = new Option();
        option.setVendor(EDbVendor.dbvoracle);

        String sql =
              "INSERT INTO order_report (order_id, customer_name)\n"
            + "SELECT o.order_id, c.name\n"
            + "FROM   orders o\n"
            + "JOIN   customers c ON o.cust_id = c.cust_id;";

        DataFlowAnalyzer analyzer = new DataFlowAnalyzer(sql, option);
        System.out.println(analyzer.generateDataFlow());
    }
}

The same analysis from the command line:

1
2
java -cp gsqlparser-4.2.8.jar:<jaxb-jars> \
     gudusoft.gsqlparser.dlineage.DataFlowAnalyzer /f orders.sql /t oracle

1.2 Reading the output

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<dlineage>
    <process id="7" name="Query Insert-1" procedureName="batchQueries" queryHashId="c59a65ba8edee8b7c7ff46aecc1d6e82" type="sstinsert" coordinate="[1,1,0],[4,45,0]"/>
    <table id="4" name="order_report" type="table" processIds="7" coordinate="[1,13,0],[1,25,0]">
        <column id="5" name="order_id" coordinate="[1,27,0],[1,35,0]"/>
        <column id="6" name="customer_name" coordinate="[1,37,0],[1,50,0]"/>
        <column id="3" name="RelationRows" coordinate="[1,13,0],[1,25,0]" source="system"/>
    </table>
    <table id="11" name="orders" alias="o" type="table" coordinate="[3,8,0],[3,16,0]">
        <column id="12" name="cust_id" coordinate="[4,23,0],[4,32,0]"/>
        <column id="13" name="order_id" coordinate="[2,8,0],[2,18,0]"/>
    </table>
    <table id="17" name="customers" alias="c" type="table" coordinate="[4,8,0],[4,19,0]">
        <column id="18" name="cust_id" coordinate="[4,35,0],[4,44,0]"/>
        <column id="19" name="name" coordinate="[2,20,0],[2,26,0]"/>
    </table>
    <resultset id="21" name="INSERT-SELECT-1" type="insert-select" coordinate="[2,8,0],[2,26,0]">
        <column id="22" name="order_id" coordinate="[2,8,0],[2,18,0]"/>
        <column id="23" name="name" coordinate="[2,20,0],[2,26,0]"/>
        <column id="20" name="RelationRows" coordinate="[2,8,0],[2,26,0]" source="system"/>
    </resultset>
    <relationship id="1" type="fdd" effectType="select">
        <target id="22" column="order_id" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,8,0],[2,18,0]"/>
        <source id="13" column="order_id" parent_id="11" parent_name="orders" parent_alias="o" coordinate="[2,8,0],[2,18,0]"/>
    </relationship>
    <relationship id="2" type="fdd" effectType="select">
        <target id="23" column="name" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,20,0],[2,26,0]"/>
        <source id="19" column="name" parent_id="17" parent_name="customers" parent_alias="c" coordinate="[2,20,0],[2,26,0]"/>
    </relationship>
    <relationship id="5" type="fdd" effectType="insert" processId="7" processType="sstinsert">
        <target id="5" column="order_id" parent_id="4" parent_name="order_report" coordinate="[1,27,0],[1,35,0]"/>
        <source id="22" column="order_id" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,8,0],[2,18,0]"/>
    </relationship>
    <relationship id="6" type="fdd" effectType="insert" processId="7" processType="sstinsert">
        <target id="6" column="customer_name" parent_id="4" parent_name="order_report" coordinate="[1,37,0],[1,50,0]"/>
        <source id="23" column="name" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,20,0],[2,26,0]"/>
    </relationship>
    <relationship id="3" type="fdr" effectType="select" clause="on">
        <target id="20" column="RelationRows" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,8,0],[2,26,0]" source="system"/>
        <source id="12" column="cust_id" parent_id="11" parent_name="orders" parent_alias="o" coordinate="[4,23,0],[4,32,0]" clauseType="joinCondition"/>
        <source id="18" column="cust_id" parent_id="17" parent_name="customers" parent_alias="c" coordinate="[4,35,0],[4,44,0]" clauseType="joinCondition"/>
    </relationship>
    <relationship id="4" type="fdr" effectType="insert">
        <target id="3" column="RelationRows" parent_id="4" parent_name="order_report" coordinate="[1,13,0],[1,25,0]" source="system"/>
        <source id="20" column="RelationRows" parent_id="21" parent_name="INSERT-SELECT-1" coordinate="[2,8,0],[2,26,0]" source="system"/>
    </relationship>
</dlineage>

Six ideas explain that document, and once you have them, every option below is easy:

Element What it is
<table> A real database object the SQL touched — orders, customers, order_report.
<resultset> An intermediate step invented by the query: the SELECT list feeding an INSERT (INSERT-SELECT-1), a CTE, a subquery, a function call. Names like RS-1 are display labels, not object names.
<relationship type="fdd"> Direct lineage. "The value of the target column is built from these source columns." This is what most people mean by column lineage.
<relationship type="fdr"> Indirect lineage. "This source column decided which rows arrived, but its value never became the target value." Join keys, WHERE predicates, GROUP BY keys.
RelationRows A synthetic column standing for "the rows of this relation". fdr edges point at it, which keeps row-influence out of your value-level graph.
coordinate [line,column,offset] start and end in the source text, so you can highlight the exact SQL that produced an edge.

Read the graph by following fdd edges backwards from a target:

1
2
order_report.order_id ← INSERT-SELECT-1.order_id ← orders.order_id
order_report.customer_name ← INSERT-SELECT-1.name ← customers.name

Do not parse this XML in your application

generateDataFlow() returns the serialized XML for humans and for pipes. In Java, call analyzer.getDataFlow() and walk the object model instead — see §12.1.

1.3 Setting options

Every option lives on one Option object that you hand to the constructor:

1
2
3
4
5
6
7
Option option = new Option();
option.setVendor(EDbVendor.dbvmssql);   // required in practice
option.setSimpleOutput(true);           // an option
option.setShowJoin(true);               // another option

DataFlowAnalyzer analyzer = new DataFlowAnalyzer(sqlFileOrText, option);
analyzer.generateDataFlow();

DataFlowAnalyzer mirrors a handful of the same setters (setShowJoin, setIgnoreRecordSet, setSimpleShowFunction, …) and forwards them to the Option it holds, so analyzer.setShowJoin(true) and option.setShowJoin(true) do the same thing. Prefer configuring Option: it holds the full set.

2. Option cheat sheet

Jump straight to the option you need. Default is the value you get if you never touch it.

Dialect, names and qualification

Option Default One line
setVendor (unset) Which SQL dialect to parse. Set it.
setDefaultServer / setDefaultDatabase / setDefaultSchema (unset) Qualify unqualified table names with the session's server/database/schema.
setShowImplicitSchema false Keep the filled-in qualifiers visible in the output.
setNormalizeOutput false Fold every name to the dialect's canonical case.
setFilePathDatabase / setFilePathSchema (unset) Qualify file/stage/URI objects (s3://…) the same way.
setPowerQueryInnerVendor (unset) Dialect of the SQL embedded inside Power Query Value.NativeQuery().
setEnableMssqlColonBindVariables false Let the SQL Server parser accept :name bind variables.

Shape and detail of the output

Option Default One line
setSimpleOutput false Collapse every intermediate step; report only table → table.
setTextFormat false Print x depends on: t.y lines instead of XML (simple output only).
setOutput true false skips XML serialization; consume getDataFlow() instead.
setIgnoreCoordinate false Drop all coordinate attributes.
setTraceTablePosition false Record every place a table is referenced, not just the first.
setTransform / setTransformCoordinate false Attach the expression text (and its position) that transformed each value.
setStartId 0 Offset all generated ids, so several runs can share one graph.
setTraceSQL false Stamp each edge with the hash of the statement that produced it.
setTraceProcedure false Stamp each edge with the id of the procedure that produced it.

What survives simple output

Option Default One line
setSimpleShowFunction false Keep function nodes (COUNT, SUM, UDFs) as visible hops.
setSimpleShowUdfFunctionOnly false Keep only user-defined functions as hops; inline the built-ins.
setSimpleShowVariable false Keep procedure variables as visible hops.
setSimpleShowCursor false Keep cursors as visible hops.
setSimpleShowSynonym false Keep synonyms as visible hops.
setSimpleShowTopSelectResultSet false Keep the final SELECT list — without it a read-only query yields nothing.
setSimpleRetainIntermediate false Keep every intermediate kind at once.
showResultSetTypes (none) Keep intermediates of named kinds only.
setSimpleShowRelationTypes (none) Which edge types simple output emits.
setSqlflowIgnoreFunction false Force-drop function nodes, overriding the switches above.
setIgnoreTemporaryTable false Collapse lineage straight through temp tables.

Extra relationship kinds

Option Default One line
setShowJoin false Emit type="join" records with join type and condition text.
setShowCallRelation false Emit type="call" records for procedure → procedure calls.
setShowERDiagram false Emit type="er" records for declared foreign keys.
setAnalyzeMode dataflow crud adds type="crud" create/read/update/delete records.

Precision and noise

Option Default One line
setIgnoreRecordSet false Remove intermediate result sets from the full output too.
setIgnoreTopSelect false Drop the final SELECT's result set.
setShowConstantTable false Show literals as a synthetic SQL_CONSTANTS-n source.
setIgnoreInsertIntoValues true Whether INSERT … VALUES literals count as constants.
setShowCountTableColumn true Whether COUNT(x) is a direct dependency on x.
setShowCaseWhenAsDirect true Whether a CASE condition column is direct or indirect.
filterRelationTypes (none) Keep only the listed edge types.
setIgnoreUnusedSynonym true Drop synonyms nothing reads from.

Names GSP cannot resolve

Option Default One line
setLinkOrphanColumnToFirstTable false Guess a home table for an unresolvable column.
setShowCandidateTable true List the other tables it might have belonged to.

Procedures, functions and dynamic SQL

Option Default One line
addExcludedProcedureName / addExcludedProcedurePattern (none) Skip named procedures, with * / ? wildcards.
setEnablePipelinedStitching true Carry lineage through Oracle pipelined table functions.
setMaxPipelinedExpansionDepth 8 Nesting bound for that stitching.
setMaxStitchedSourcesPerColumn 64 Fan-in bound for that stitching.
setIdentityFirstVariablePools false Give each routine overload its own variable pool.
setAnalyzeDynamicSql true Analyze SQL built as a string and executed.
setDynamicSqlTrustMode LEGACY SHADOW also reports how complete each dynamic site was.
setReportDynamicSqlSitesAsErrors false Mirror unresolved dynamic sites into the error list.
setAssumeExternalScriptPassthrough true Assume a Python/R external script passes columns through positionally.

Large inputs

Option Default One line
setAutoDetectLargeFile false Delegate oversized inputs to the parallel analyzer instead of risking OOM.
setParallel cores/2 − 1 Worker count for parallel analysis.
thresholds see table What counts as "large".

Integration hooks

Option Default One line
setHandleListener (none) Progress callbacks and cancellation.
setCollectAuthoritativeLineageEvidence false Collect the SLPC evidence sidecar.

3. Dialect, names and qualification

setVendor(EDbVendor)

Default: unset. The dialect GSP parses with. Everything else depends on getting this right — #temp tables, EXEC, PIPE ROW, IDENTIFIER() and quoting rules all differ per vendor. See the list of supported dialects.

1
option.setVendor(EDbVendor.dbvsnowflake);

On the command line this is /t:

1
... DataFlowAnalyzer /f script.sql /t snowflake

setDefaultDatabase(String) · setDefaultSchema(String) · setDefaultServer(String)

Default: unset. Real scripts say FROM orders, not FROM SALESDB.dbo.orders. These three tell GSP what the session defaults were, so unqualified names resolve to fully-qualified objects — which is what makes lineage from two different scripts join up in a catalog.

1
SELECT order_id, amount FROM orders;

1
<table id="4" name="orders" type="table">

1
<table id="4" database="SALESDB" schema="dbo" name="SALESDB.dbo.orders" type="table">

setDefaultSchema is currently ignored (bug #4720)

As of 4.2.8 the value you pass to setDefaultSchema never reaches the output. Qualification is triggered by the default database, and the schema segment is always filled from the dialect's own default:

Options Reported name
setDefaultSchema("myschema") only orders — no qualification at all
setDefaultDatabase("SALESDB") only SALESDB.dbo.orders
both, schema myschema SALESDB.dbo.orders — your value is discarded

The example above uses dbo only because that is what SQL Server would have produced anyway. On Oracle, PostgreSQL, Snowflake and BigQuery the discarded schema shows up as the placeholder DEFAULT.

Until this is fixed, do not rely on setDefaultSchema to attribute lineage to a non-default schema — supply real catalog metadata through setSqlEnv instead, or qualify the names in the SQL. setDefaultServer likewise only matters for dialects that have a server layer above the database.

setShowImplicitSchema(boolean)

Default: false · CLI /showImplicitSchema. Some objects carry an implicit qualifier that the resolver worked out from context rather than from the text — the database a USE statement switched to, the schema that owns a package body, the catalog recorded in supplied metadata. By default that qualifier is not printed; this option adopts it as the object's database/schema.

It is distinct from setDefaultDatabase: that one supplies a session default you know, this one surfaces one GSP inferred. If no implicit qualifier was inferred — which is the usual case when you analyze a bare script with no metadata — the output is unchanged.

setNormalizeOutput(boolean)

Default: false. Rewrites every object and column name to the dialect's canonical case, so orders, Orders and ORDERS in three scripts become one node.

1
SELECT order_id, amount FROM orders;

1
2
3
4
<table id="4" name="orders" type="table">
    <column id="5" name="order_id"/>
    <column id="6" name="amount"/>
</table>

1
2
3
4
<table id="4" name="ORDERS" type="table">
    <column id="5" name="ORDER_ID"/>
    <column id="6" name="AMOUNT"/>
</table>

The canonical case is the dialect's: Oracle and Snowflake fold to upper case, MySQL does not fold at all. Quoted identifiers keep their case, because in these dialects "Store" and Store are genuinely different columns.

setFilePathDatabase(String) · setFilePathSchema(String)

Default: unset. The same idea as setDefaultDatabase, but for objects that are paths rather than tables — external stages, COPY INTO locations, data-lake URIs.

1
2
3
COPY INTO orders_stage
FROM 's3://acme-datalake/orders/2026/'
FILE_FORMAT = (TYPE = CSV);

1
<path id="6" name="'s3://acme-datalake/orders/2026/'" uri="'s3://acme-datalake/orders/2026/'" type="path">

1
<path id="6" database="LAKE" schema="RAW" name="LAKE.RAW.'s3://acme-datalake/orders/2026/'" uri="LAKE.RAW.'s3://acme-datalake/orders/2026/'" type="path">

setPowerQueryInnerVendor(EDbVendor)

Default: unset (inferred). Power Query / M scripts embed native SQL inside Value.NativeQuery(). By default GSP infers that inner dialect from the connector in the M code. Set this when the inference is wrong or the connector is unknown.

setEnableMssqlColonBindVariables(boolean)

Default: false. SQL Server has no :name bind variables, but many JDBC/ORM layers hand GSP SQL that still contains them. Without this switch such a script is a syntax error and you get no lineage at all:

1
2
INSERT INTO order_report (order_id)
SELECT order_id FROM orders WHERE cust_id = :custId;

1
2
3
4
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<dlineage>
    <error errorMessage="syntax error, state:1195(10101) near: :(2,45)" errorType="SyntaxError" file="s27.sql" originCoordinate="[2,45],[2,46]"/>
</dlineage>

1
2
3
4
<relationship id="2" type="fdr" effectType="select" clause="where">
    <target id="13" column="RelationRows" parent_id="14" parent_name="INSERT-SELECT-1" source="system"/>
    <source id="12" column="cust_id" parent_id="10" parent_name="orders" clauseType="where"/>
</relationship>

4. Shape and detail of the output

setSimpleOutput(boolean)

Default: false · CLI /s. The single most useful option, and the first one to try. Full output shows every intermediate hop; simple output collapses them and reports only real objects to real objects.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
CREATE VIEW dept_summary AS
SELECT d.dept_id,
       COUNT(e.emp_id) AS headcount
FROM   employee e
JOIN   department d ON e.dept_id = d.dept_id
WHERE  e.status = 'ACTIVE'
GROUP BY d.dept_id;

INSERT INTO report_dept (dept_id, headcount)
SELECT dept_id, headcount FROM dept_summary;

24 relationships, including RS-1, the COUNT function node, the INSERT-SELECT node and every fdr row-influence edge. Excerpt:

1
2
3
4
5
6
7
8
<relationship id="1" type="fdd" effectType="select">
    <target id="15" column="dept_id" parent_id="14" parent_name="RS-1"/>
    <source id="12" column="dept_id" parent_id="11" parent_name="department" parent_alias="d"/>
</relationship>
<relationship id="13" type="fdd" effectType="create_view" processId="22" processType="sstcreateview">
    <target id="23" column="dept_id" parent_id="21" parent_name="dept_summary"/>
    <source id="15" column="dept_id" parent_id="14" parent_name="RS-1"/>
</relationship>

Four relationships, no intermediates:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
<relationship id="24" type="fdd" effectType="create_view" processId="22" processType="sstcreateview">
    <target id="23" column="dept_id" parent_id="21" parent_name="dept_summary"/>
    <source id="12" column="dept_id" parent_id="11" parent_name="department" parent_alias="d"/>
</relationship>
<relationship id="25" type="fdd" effectType="create_view" processId="22" processType="sstcreateview">
    <target id="24" column="headcount" parent_id="21" parent_name="dept_summary"/>
    <source id="6" column="emp_id" parent_id="4" parent_name="employee" parent_alias="e"/>
</relationship>
<relationship id="26" type="fdd" effectType="insert" processId="31" processType="sstinsert">
    <target id="29" column="dept_id" parent_id="28" parent_name="report_dept"/>
    <source id="23" column="dept_id" parent_id="21" parent_name="dept_summary"/>
</relationship>
<relationship id="27" type="fdd" effectType="insert" processId="31" processType="sstinsert">
    <target id="30" column="headcount" parent_id="28" parent_name="report_dept"/>
    <source id="24" column="headcount" parent_id="21" parent_name="dept_summary"/>
</relationship>

Nothing is lost — the two-hop path employee.emp_id → COUNT → RS-1 → dept_summary.headcount becomes the single edge employee.emp_id → dept_summary.headcount. Use simple output to populate a catalog; use full output to explain why an edge exists.

The rest of §5 is about putting selected intermediates back into simple output.

setTextFormat(boolean)

Default: false · CLI /text. With simple output, print one plain line per edge instead of XML. Perfect for a first look, grep, or a smoke test.

1
2
option.setSimpleOutput(true);
option.setTextFormat(true);

Same script as above:

1
2
3
4
dept_id depends on: department.dept_id
headcount depends on: employee.emp_id
dept_id depends on: dept_summary.dept_id
headcount depends on: dept_summary.headcount

Note

textFormat only applies when simpleOutput is on. The CLI enforces this too: /text is read only after /s.

setOutput(boolean)

Default: true. Set false when your program consumes the object model and never needs the XML string — the serialization step is skipped and generateDataFlow() returns null. See §12.1.

setIgnoreCoordinate(boolean)

Default: false · CLI /ic. Removes every coordinate attribute. Use it when you are diffing two runs, storing output, or reading it by eye — coordinates roughly double the size of the document.

1
2
3
<table id="11" name="orders" alias="o" type="table" coordinate="[3,8,0],[3,16,0]">
    <column id="12" name="cust_id" coordinate="[4,23,0],[4,32,0]"/>
</table>

1
2
3
<table id="11" name="orders" alias="o" type="table">
    <column id="12" name="cust_id"/>
</table>

setTraceTablePosition(boolean)

Default: false. A table referenced five times normally reports one position. Turn this on to record them all — useful for "highlight every mention of this table" features.

1
2
INSERT INTO t_out (a) SELECT x FROM orders;
INSERT INTO t_out (b) SELECT y FROM orders;

1
<table id="10" name="orders" type="table" coordinate="[1,37,0],[1,43,0]">

1
<table id="10" name="orders" type="table" coordinate="[1,37,0],[1,43,0],[2,37,0],[2,43,0]">

setTransform(boolean) · setTransformCoordinate(boolean)

Default: false · CLI /transform, /coor. Lineage tells you that a value flowed; transform tells you what happened to it on the way by attaching the source text of the expression. transformCoordinate adds that expression's position.

1
2
SELECT CASE WHEN t.status = 'A' THEN t.amount ELSE 0 END AS net_amount
FROM   txn t;

1
2
3
4
<relationship id="5" type="fdd" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="6" column="amount" parent_id="4" parent_name="txn" parent_alias="t"/>
</relationship>

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<relationship id="5" type="fdd" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="6" column="amount" parent_id="4" parent_name="txn" parent_alias="t">
        <transforms>
            <transform type="case">
                <code>CASE WHEN t.status = 'A' THEN t.amount ELSE 0 END</code>
            </transform>
        </transforms>
    </source>
</relationship>

1
2
3
4
5
        <transforms>
            <transform type="case" coordinate="[1,8,0],[1,57,0]">
                <code>CASE WHEN t.status = 'A' THEN t.amount ELSE 0 END</code>
            </transform>
        </transforms>

setStartId(long)

Default: 0. Every node and edge gets a small integer id, restarting at 0 on each run. If you analyze 50 files separately and merge the results, those ids collide. Give each run its own range instead.

1
SELECT order_id, amount FROM orders;

1
2
3
4
5
<table id="4" name="orders" type="table">
    <column id="5" name="order_id"/>
    <column id="6" name="amount"/>
</table>
<resultset id="8" name="RS-1" type="select_list">

1
2
3
4
5
<table id="1004" name="orders" type="table">
    <column id="1005" name="order_id"/>
    <column id="1006" name="amount"/>
</table>
<resultset id="1008" name="RS-1001" type="select_list">

setTraceSQL(boolean)

Default: false. Stamps every relationship with sqlHash, the hash of the statement it came from — so you can answer "which statement created this edge?" when a script has hundreds.

1
<relationship id="1" type="fdd" effectType="select">

1
<relationship id="1" type="fdd" effectType="select" sqlHash="a83364d2b279bd24e0604207d5b20c89">

The same hash appears as queryHashId on the <process> element, so you can join edges back to statements.

setTraceProcedure(boolean)

Default: false. The same idea for routines: every relationship carries the procedureId of the procedure whose body produced it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
CREATE PROCEDURE load_staging AS
BEGIN
  INSERT INTO stg_orders (order_id) SELECT order_id FROM orders;
END;
/
CREATE PROCEDURE run_etl AS
BEGIN
  load_staging();
  INSERT INTO dw_orders (order_id) SELECT order_id FROM stg_orders;
END;
/

1
2
<relationship id="2" type="fdd" effectType="insert" processId="7" processType="sstinsert">
<relationship id="5" type="fdd" effectType="insert" processId="22" processType="sstinsert">

1
2
<relationship id="2" type="fdd" effectType="insert" processId="7" processType="sstinsert" procedureId="1">
<relationship id="5" type="fdd" effectType="insert" processId="22" processType="sstinsert" procedureId="16">

procedureId="1" is load_staging, procedureId="16" is run_etl.

5. What survives simple output

Simple output hides all intermediates. Usually that is what you want — but sometimes one kind of intermediate is the whole point of the analysis ("which UDF touched this column?", "which variable carried this value?"). Each option below puts one kind back.

All examples in this section use setSimpleOutput(true) and setTextFormat(true).

setSimpleShowFunction(boolean)

Default: false. Keeps function calls as visible nodes.

1
2
3
4
INSERT INTO report_dept (dept_id, headcount, label)
SELECT d.dept_id, COUNT(e.emp_id), fmt_label(d.dept_name)
FROM   employee e JOIN department d ON e.dept_id = d.dept_id
GROUP BY d.dept_id, d.dept_name;

1
2
3
dept_id depends on: department.dept_id
headcount depends on: employee.emp_id
label depends on: department.dept_name

1
2
3
4
5
COUNT depends on: employee.emp_id
fmt_label depends on: department.dept_name
dept_id depends on: department.dept_id
headcount depends on: COUNT.COUNT
label depends on: fmt_label.fmt_label

setSimpleShowUdfFunctionOnly(boolean)

Default: false. Used together with setSimpleShowFunction(true): keep only user-defined functions as nodes and inline the built-ins. This is usually what you want for governance — nobody needs COUNT in the graph, but a UDF is a piece of business logic worth tracking.

1
2
3
4
5
COUNT depends on: employee.emp_id
fmt_label depends on: department.dept_name
dept_id depends on: department.dept_id
headcount depends on: COUNT.COUNT
label depends on: fmt_label.fmt_label

1
2
3
4
fmt_label depends on: department.dept_name
dept_id depends on: department.dept_id
headcount depends on: employee.emp_id
label depends on: fmt_label.fmt_label

COUNT is gone and headcount now depends directly on employee.emp_id; the UDF fmt_label is still a hop.

setSimpleShowVariable(boolean)

Default: false. Procedure variables are normally invisible plumbing. Turn this on when you need to see which variable carried a value.

1
2
3
4
5
6
7
8
CREATE PROCEDURE load_orders AS
  v_total NUMBER;
BEGIN
  SELECT SUM(o.amount) INTO v_total FROM orders o;

  INSERT INTO order_summary (total_amount) VALUES (v_total);
END;
/

1
total_amount depends on: orders.amount

1
2
v_total depends on: orders.amount
total_amount depends on: v_total.v_total

setSimpleShowCursor(boolean)

Default: false. The same, for cursors.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
CREATE PROCEDURE copy_orders AS
  CURSOR c_orders IS SELECT order_id, amount FROM orders;
  v_id NUMBER;
  v_amt NUMBER;
BEGIN
  OPEN c_orders;
  LOOP
    FETCH c_orders INTO v_id, v_amt;
    EXIT WHEN c_orders%NOTFOUND;
    INSERT INTO order_archive (order_id, amount) VALUES (v_id, v_amt);
  END LOOP;
  CLOSE c_orders;
END;
/

1
2
order_id depends on: orders.order_id
amount depends on: orders.amount

1
2
3
4
order_id depends on: orders.order_id
amount depends on: orders.amount
order_id depends on: c_orders.order_id
amount depends on: c_orders.amount

1
2
3
4
5
6
order_id depends on: orders.order_id
amount depends on: orders.amount
v_id depends on: c_orders.order_id
v_amt depends on: c_orders.amount
order_id depends on: v_id.v_id
amount depends on: v_amt.v_amt

setSimpleShowVariable also reveals cursors, because the fetch target is a variable whose source is the cursor.

setSimpleShowSynonym(boolean)

Default: false. Keeps a synonym as its own node instead of resolving straight through to the base object.

1
2
3
CREATE SYNONYM ord FOR sales.orders;

INSERT INTO order_report (order_id) SELECT order_id FROM ord;

1
order_id depends on: sales.orders.*

1
2
3
ORDER_ID depends on: sales.orders.*
* depends on: sales.orders.*
order_id depends on: ord.order_id

setSimpleShowTopSelectResultSet(boolean)

Default: false · CLI /topselectlist. Simple output reports lineage into targets. A plain SELECT writes nothing, so it has no target — and simple output is empty. This option makes the final SELECT list a target.

1
2
SELECT c.name, o.amount
FROM   customers c JOIN orders o ON c.cust_id = o.cust_id;

1

(empty — the query persists nothing)

1
2
name depends on: customers.name
amount depends on: orders.amount

Tip

If simple output is unexpectedly empty, this is almost always the reason: the script is read-only. Turn this option on, or analyze a script that writes something.

setSimpleRetainIntermediate(boolean)

Default: false. The "keep everything interesting" switch: retains function nodes, variables, cursors and procedure result sets in one go, rather than enabling four options.

1
2
3
4
5
WITH active_orders AS (
  SELECT order_id, cust_id, amount FROM orders WHERE status = 'ACTIVE'
)
INSERT INTO cust_totals (cust_id, total)
SELECT cust_id, SUM(amount) FROM active_orders GROUP BY cust_id;

1
2
cust_id depends on: orders.cust_id
total depends on: orders.amount

1
2
3
SUM depends on: orders.amount
cust_id depends on: orders.cust_id
total depends on: SUM.SUM

showResultSetTypes(String…)

Default: none · CLI /showResultSetTypes. Fine-grained version of the options above: name exactly which kinds of intermediate result set to keep. The value matches the type attribute on <resultset> in the full output (- and _ are interchangeable).

1
option.showResultSetTypes("insert_select", "function");

Accepted names: select_list, array, struct, result_of, cte, insert_select, update_select, merge_update, merge_insert, output, update_set, pivot_table, unpivot_table, alias, function, case_when, cursor, variable.

Using the CTE script above:

1
2
cust_id depends on: orders.cust_id
total depends on: orders.amount

1
2
3
SUM depends on: orders.amount
cust_id depends on: orders.cust_id
total depends on: SUM.SUM

1
2
3
4
5
6
7
order_id depends on: orders.order_id
cust_id depends on: orders.cust_id
amount depends on: orders.amount
cust_id depends on: INSERT-SELECT-1.cust_id
SUM(amount) depends on: INSERT-SELECT-1.amount
cust_id depends on: INSERT-SELECT-2.cust_id
total depends on: INSERT-SELECT-2.SUM(amount)

cte and result_of do not work yet (bug #4719)

A CTE result set is emitted with type="with_cte", which the cte value does not match, so asking for cte silently keeps nothing. result_of has no effect either. To keep CTE nodes today, use setSimpleRetainIntermediate or full output.

setSimpleShowRelationTypes(String…)

Default: none · CLI /simpleShowRelationTypes. Which relationship types simple output emits. Values: fdd (alias direct), fdr (alias indirect), fddi, frd, join, call, er, crud.

1
option.setSimpleShowRelationTypes("fdr");

For the INSERT … SELECT … JOIN script of §1:

1
2
<relationship id="7" type="fdd" effectType="insert" processId="7" processType="sstinsert">
<relationship id="8" type="fdd" effectType="insert" processId="7" processType="sstinsert">

1
<relationship id="7" type="fdr" effectType="insert">

Use fdr to build an impact/filter graph — "which columns decided which rows landed in this table" — separately from the value graph.

setSqlflowIgnoreFunction(boolean)

Default: false. Force-drops function result sets even when setSimpleShowFunction(true) would have kept them. It exists so a host application can veto function nodes globally without rewriting per-request options.

1
2
3
4
5
COUNT depends on: employee.emp_id
fmt_label depends on: department.dept_name
dept_id depends on: department.dept_id
headcount depends on: COUNT.COUNT
label depends on: fmt_label.fmt_label

1
2
3
dept_id depends on: department.dept_id
headcount depends on: employee.emp_id
label depends on: department.dept_name

setIgnoreTemporaryTable(boolean)

Default: false · CLI /withTemporaryTable is the opposite. A staging temp table is real to the engine but noise in a catalog. This option collapses lineage straight through it.

1
2
3
4
SELECT order_id, amount INTO #tmp_orders FROM orders;

INSERT INTO order_report (order_id, amount)
SELECT order_id, amount FROM #tmp_orders;

1
2
3
4
order_id depends on: orders.order_id
amount depends on: orders.amount
order_id depends on: #tmp_orders.order_id
amount depends on: #tmp_orders.amount

1
2
order_id depends on: orders.order_id
amount depends on: orders.amount

…and in XML the temp table is gone but the path is preserved:

1
2
3
4
<relationship id="9" type="fdd" effectType="insert" processId="24" processType="sstinsert">
    <target id="22" column="order_id" parent_id="21" parent_name="order_report"/>
    <source id="5" column="order_id" parent_id="4" parent_name="orders"/>
</relationship>

Lineage is collapsed, not deleted: order_report.order_id still traces to orders.order_id.

6. Extra relationship kinds

The options in this section do not change existing edges; they add new kinds of record to the same document.

setShowJoin(boolean)

Default: false · CLI /j. Emits a dedicated type="join" record per join, carrying the join type and the condition text. Column lineage tells you values flowed; this tells you the tables were joined, and on what.

1
2
3
4
INSERT INTO order_report (order_id, customer_name)
SELECT o.order_id, c.name
FROM   orders o
JOIN   customers c ON o.cust_id = c.cust_id;

1
2
3
4
<relationship id="4" type="join" effectType="select" clause="on" joinType="join" condition="o.cust_id = c.cust_id">
    <target id="18" column="cust_id" parent_id="17" parent_name="customers" parent_alias="c"/>
    <source id="12" column="cust_id" parent_id="11" parent_name="orders" parent_alias="o"/>
</relationship>

For richer join semantics see Join analysis.

setShowCallRelation(boolean)

Default: false. Emits type="call" records — the procedure call graph, alongside the data graph.

Using the two-procedure script from setTraceProcedure:

1
2
3
4
<relationship id="3" type="call" callStmt="load_staging()" callCoordinate="[8,3,0],[8,17,0]">
    <caller id="16" name="run_etl" type="procedure"/>
    <callee id="1" name="load_staging" type="procedure"/>
</relationship>

Note the different endpoint element names: call records use <caller>/<callee>, not <target>/<source>.

setShowERDiagram(boolean)

Default: false · CLI /showER. Emits type="er" records for foreign keys declared in DDL — an entity-relationship view of the same script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
CREATE TABLE customers (
  cust_id INT PRIMARY KEY,
  name    VARCHAR(100)
);

CREATE TABLE orders (
  order_id INT PRIMARY KEY,
  cust_id  INT REFERENCES customers(cust_id),
  amount   DECIMAL(10,2)
);

1
2
3
4
<relationship id="1" type="fdd" effectType="foreign_key" processId="14" processType="sstcreatetable">
    <target id="12" column="cust_id" parent_id="10" parent_name="orders"/>
    <source id="5" column="cust_id" parent_id="4" parent_name="customers"/>
</relationship>

1
2
3
4
<relationship id="2" type="er">
    <target id="12" column="cust_id" parent_id="10" parent_name="orders"/>
    <source id="5" column="cust_id" parent_id="4" parent_name="customers"/>
</relationship>

Column metadata (dataType, primaryKey, foreignKey) is reported from the DDL either way.

setAnalyzeMode(AnalyzeMode)

Default: dataflow. Values: dataflow, crud, dynamic. In crud mode GSP additionally emits type="crud" records describing operations on objects — useful for "what does this script create/read/update/delete?" rather than "where does this value come from?".

1
2
3
4
5
6
7
8
CREATE TABLE staging_orders (order_id INT, amount DECIMAL(10,2));

INSERT INTO staging_orders (order_id, amount)
SELECT order_id, amount FROM orders;

UPDATE staging_orders SET amount = amount * 1.1 WHERE order_id > 100;

DELETE FROM staging_orders WHERE amount IS NULL;

1
2
3
4
5
6
7
8
9
<relationship id="1" type="crud" effectType="create_table">
    <target id="5" column="order_id" parent_id="4" parent_name="staging_orders"/>
</relationship>
<relationship id="2" type="crud" effectType="create_table">
    <target id="6" column="amount" parent_id="4" parent_name="staging_orders"/>
</relationship>
<relationship id="12" type="crud" effectType="delete">
    <target target_id="4" target_name="staging_orders"/>
</relationship>

7. Precision and noise

setIgnoreRecordSet(boolean)

Default: false · CLI /i. Removes intermediate result sets from the full output, keeping everything else (coordinates, fdr edges, functions). Think of it as "simple output for result sets only".

1
2
3
4
INSERT INTO order_report (order_id, customer_name)
SELECT o.order_id, c.name
FROM   orders o
JOIN   customers c ON o.cust_id = c.cust_id;

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
<resultset id="21" name="INSERT-SELECT-1" type="insert-select">
    <column id="22" name="order_id"/>
    <column id="23" name="name"/>
    <column id="20" name="RelationRows" source="system"/>
</resultset>
<relationship id="1" type="fdd" effectType="select">
    <target id="22" column="order_id" parent_id="21" parent_name="INSERT-SELECT-1"/>
    <source id="13" column="order_id" parent_id="11" parent_name="orders" parent_alias="o"/>
</relationship>
<relationship id="5" type="fdd" effectType="insert" processId="7" processType="sstinsert">
    <target id="5" column="order_id" parent_id="4" parent_name="order_report"/>
    <source id="22" column="order_id" parent_id="21" parent_name="INSERT-SELECT-1"/>
</relationship>

1
2
3
4
<relationship id="7" type="fdd" effectType="insert" processId="7" processType="sstinsert">
    <target id="5" column="order_id" parent_id="4" parent_name="order_report"/>
    <source id="13" column="order_id" parent_id="11" parent_name="orders" parent_alias="o"/>
</relationship>

setIgnoreTopSelect(boolean)

Default: false. The mirror image of setSimpleShowTopSelectResultSet: drops the final SELECT's result set from the full output, so only persisted targets remain.

1
2
3
SELECT order_id, amount INTO #tmp_orders FROM orders;

SELECT order_id, amount FROM #tmp_orders;

1
2
3
4
5
6
7
8
9
<resultset id="19" name="RS-2" type="select_list">
    <column id="20" name="order_id"/>
    <column id="21" name="amount"/>
</resultset>
...
<relationship id="5" type="fdd" effectType="select">
    <target id="20" column="order_id" parent_id="19" parent_name="RS-2"/>
    <source id="16" column="order_id" parent_id="10" parent_name="#tmp_orders"/>
</relationship>

RS-2 and both of its edges are gone; the SELECT … INTO #tmp_orders lineage remains.

setShowConstantTable(boolean)

Default: false · CLI /showConstant. A column whose value is a literal has no source column, so by default it gets no lineage at all and simply disappears from the graph. This option gives literals a home: a synthetic SQL_CONSTANTS-n table.

1
2
SELECT 'FIXED' AS src_system, o.order_id
FROM orders o;

1
2
3
4
5
6
7
8
<resultset id="7" name="RS-1" type="select_list">
    <column id="8" name="src_system"/>
    <column id="9" name="order_id"/>
</resultset>
<relationship id="2" type="fdd" effectType="select">
    <target id="9" column="order_id" parent_id="7" parent_name="RS-1"/>
    <source id="5" column="order_id" parent_id="4" parent_name="orders"/>
</relationship>

src_system has no incoming edge.

1
2
3
4
5
6
7
8
<table id="10" name="SQL_CONSTANTS-1" type="constantTable">
    <column id="11" name="'FIXED'"/>
</table>
...
<relationship id="1" type="fdd" effectType="select">
    <target id="8" column="src_system" parent_id="7" parent_name="RS-1"/>
    <source id="11" column="'FIXED'" parent_id="10" parent_name="SQL_CONSTANTS-1"/>
</relationship>

Turn it on when your consumer needs every target column to have a source, even a constant one.

setIgnoreInsertIntoValues(boolean)

Default: true. Decides whether literals in an INSERT … VALUES list count as constants. It only has a visible effect together with setShowConstantTable(true).

1
2
INSERT INTO audit_log (event_id, event_type, created_at)
VALUES (1, 'LOGIN', SYSDATE);

1
2
3
<table id="10" name="SQL_CONSTANTS-1" type="constantTable">
    <column id="11" name="SYSDATE"/>
</table>

Only the function-valued item is kept; the plain literals 1 and 'LOGIN' are ignored.

1
2
3
4
5
6
7
8
9
<table id="10" name="SQL_CONSTANTS-1" type="constantTable">
    <column id="11" name="1"/>
    <column id="12" name="'LOGIN'"/>
    <column id="13" name="SYSDATE"/>
</table>
<relationship id="1" type="fdd" effectType="insert" processId="8" processType="sstinsert">
    <target id="5" column="event_id" parent_id="4" parent_name="audit_log"/>
    <source id="11" column="1" parent_id="10" parent_name="SQL_CONSTANTS-1"/>
</relationship>

setShowCountTableColumn(boolean)

Default: true · CLI /treatArgumentsInCountFunctionAsDirectDataflow. COUNT(x) is philosophically odd: the result does not contain x's value, it counts rows. This option decides whether x → COUNT is reported as a direct (fdd) dependency.

Using the COUNT(e.emp_id) view script from §4:

1
2
3
4
<relationship id="3" type="fdd" effectType="function">
    <target id="19" column="COUNT" parent_id="18" parent_name="COUNT"/>
    <source id="6" column="emp_id" parent_id="4" parent_name="employee" parent_alias="e"/>
</relationship>

That fdd edge is gone. The indirect edges remain:

1
2
3
4
<relationship id="3" type="fdr" function="COUNT" effectType="function">
    <target id="19" column="COUNT" parent_id="18" parent_name="COUNT"/>
    <source id="12" column="dept_id" parent_id="11" parent_name="department" parent_alias="d"/>
</relationship>

Warning

Switching this off removes real edges from your value graph. Only do it if your downstream consumer treats "counted" as row-influence rather than value-flow — and then read the fdr edges, or the dependency disappears entirely.

setShowCaseWhenAsDirect(boolean)

Default: true · CLI /showCaseWhenAsIndirect is the opposite. In CASE WHEN status = 'A' THEN amount ELSE 0 END, amount clearly flows into the result. What about status? It picks which value is returned but its own value never appears. This option decides how status is classified.

1
2
SELECT CASE WHEN t.status = 'A' THEN t.amount ELSE 0 END AS net_amount
FROM   txn t;

1
2
3
4
5
6
7
8
<relationship id="3" type="fdd" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="5" column="status" parent_id="4" parent_name="txn" parent_alias="t"/>
</relationship>
<relationship id="5" type="fdd" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="6" column="amount" parent_id="4" parent_name="txn" parent_alias="t"/>
</relationship>

1
2
3
4
5
6
7
8
<relationship id="3" type="fdd" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="6" column="amount" parent_id="4" parent_name="txn" parent_alias="t"/>
</relationship>
<relationship id="4" type="fdr" effectType="function">
    <target id="12" column="case-when" parent_id="11" parent_name="case-when"/>
    <source id="5" column="status" parent_id="4" parent_name="txn" parent_alias="t" clauseType="selectList"/>
</relationship>

The edge is never dropped — only reclassified from fdd to fdr. Choose false for a strict value-flow graph, true (the default) if a change to status should show up as impacting net_amount.

filterRelationTypes(String…)

Default: none · CLI /filterRelationTypes. Despite the name, this is a whitelist: only the listed relationship types are emitted. Same value set as setSimpleShowRelationTypes, including the direct / indirect aliases.

1
option.filterRelationTypes("fdd");   // value lineage only

For the INSERT … SELECT … JOIN script of §1:

1
2
3
4
5
6
<relationship id="1" type="fdd" effectType="select">
<relationship id="2" type="fdd" effectType="select">
<relationship id="5" type="fdd" effectType="insert" processId="7" processType="sstinsert">
<relationship id="6" type="fdd" effectType="insert" processId="7" processType="sstinsert">
<relationship id="3" type="fdr" effectType="select" clause="on">
<relationship id="4" type="fdr" effectType="insert">

1
2
3
4
<relationship id="1" type="fdd" effectType="select">
<relationship id="2" type="fdd" effectType="select">
<relationship id="5" type="fdd" effectType="insert" processId="7" processType="sstinsert">
<relationship id="6" type="fdd" effectType="insert" processId="7" processType="sstinsert">

1
2
<relationship id="3" type="fdr" effectType="select" clause="on">
<relationship id="4" type="fdr" effectType="insert">

setIgnoreUnusedSynonym(boolean)

Default: true. When the same object appears under several names, a synonym that nothing ever reads from is dropped rather than merged into the graph. Set false to keep every declared synonym node.

8. Names GSP cannot resolve

Sometimes a column cannot be attributed to a table: several tables are in scope, none is qualified, and no catalog metadata was supplied. GSP calls these orphan columns and, by default, refuses to guess.

The real fix is metadata

Supplying a TSQLEnv (CLI /env metadata.json) with the real table/column lists resolves these cases exactly, instead of heuristically. The two options below are for when metadata is not available.

setLinkOrphanColumnToFirstTable(boolean)

Default: false · CLI /lof. Attributes an orphan column to the first table in the FROM clause, and records that it did so.

1
2
SELECT order_no
FROM   orders o, customers c;

1
2
3
4
5
6
7
8
9
<table id="4" name="orders" alias="o" type="table"/>
<table id="8" name="customers" alias="c" type="table"/>
<table id="13" name="pseudo_table_include_orphan_column" type="pseudoTable">
    <column id="14" name="order_no"/>
</table>
<resultset id="10" name="RS-1" type="select_list">
    <column id="11" name="order_no"/>
</resultset>
<error errorMessage="find orphan column(10500) near: order_no(1,8)" errorType="SyntaxHint" file="s8.sql" originCoordinate="[1,8],[1,16]"/>

No lineage edge — order_no sits in a pseudoTable and an error records why.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<relationship id="1" type="fdd" effectType="select">
    <target id="11" column="order_no" parent_id="10" parent_name="RS-1"/>
    <source id="15" column="order_no" parent_id="4" parent_name="orders" parent_alias="o">
        <candidateTables>
            <candidateTable id="4" name="orders"/>
            <candidateTable id="8" name="customers"/>
        </candidateTables>
    </source>
</relationship>
<error errorMessage="Link orphan column [order_no] to the first table [orders o]" errorType="LinkOrphanColumn" file="s8.sql" originCoordinate="[1,8],[1,16]"/>

This is a guess, and it is labelled as one

order_no may well belong to customers. The edge is emitted so the graph is not silently incomplete, but the <candidateTables> list and the LinkOrphanColumn error exist so your consumer can mark it as unproven. Do not present such an edge to users with the same confidence as a resolved one.

setShowCandidateTable(boolean)

Default: true. Whether the <candidateTables> block above is emitted. Set false only if your consumer cannot handle the nested element.

1
2
3
4
<relationship id="1" type="fdd" effectType="select">
    <target id="11" column="order_no" parent_id="10" parent_name="RS-1"/>
    <source id="15" column="order_no" parent_id="4" parent_name="orders" parent_alias="o"/>
</relationship>

The guess is still recorded in the LinkOrphanColumn error, but the alternatives are no longer visible.

9. Procedures, functions and dynamic SQL

addExcludedProcedureName(String) · addExcludedProcedurePattern(String)

Default: none. Skip named routines entirely — logging helpers, audit wrappers, vendor utilities that add nothing but noise. Patterns support * (any run of characters) and ? (one character); . separates name segments. Matching is case-insensitive and quoting (", [], `) is stripped before matching.

1
2
option.addExcludedProcedureName("dbo.sp_write_audit");
option.addExcludedProcedurePatterns("SCHEMA1.*", "*.SP_LOG_*", "*_TEST");

Using the two-procedure script from setTraceProcedure:

1
2
order_id depends on: orders.order_id
order_id depends on: stg_orders.order_id

1
order_id depends on: stg_orders.order_id

load_staging's body is no longer analyzed, so stg_orders ← orders is gone.

setEnablePipelinedStitching(boolean)

Default: true. Oracle pipelined table functions are a lineage cliff: the caller selects from TABLE(get_orders()) and the real source tables are inside the function body. This option stitches the two sides together. Leave it on.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
CREATE TYPE order_row AS OBJECT (order_id NUMBER, amount NUMBER);
/
CREATE TYPE order_tab AS TABLE OF order_row;
/
CREATE FUNCTION get_orders RETURN order_tab PIPELINED AS
BEGIN
  FOR r IN (SELECT o.order_id, o.amount FROM orders o) LOOP
    PIPE ROW(order_row(r.order_id, r.amount));
  END LOOP;
  RETURN;
END;
/
INSERT INTO order_report (order_id, amount)
SELECT t.order_id, t.amount FROM TABLE(get_orders()) t;

1
2
3
4
order_id depends on: orders.order_id
amount depends on: orders.amount
order_id depends on: get_orders.order_id
amount depends on: get_orders.amount

1

(empty — orders and order_report both appear as tables, with no edge between them)

setMaxPipelinedExpansionDepth(int)

Default: 8. How deeply pipelined functions may be expanded into one another before GSP stops. A guard against pathological or recursive nesting.

setMaxStitchedSourcesPerColumn(int)

Default: 64. Upper bound on how many stitched sources one column may collect, so a very wide function body cannot explode the graph.

setIdentityFirstVariablePools(boolean)

Default: false. Advanced. Gives each routine overload its own pool of variables instead of merging same-named formal parameters across overloads. Only relevant to the routine-summary analysis pipelines; leave at the default unless you are working on those.

setAnalyzeDynamicSql(boolean)

Default: true. SQL built as a string and executed (EXEC(@sql), sp_executesql, EXECUTE IMMEDIATE, DBMS_SQL.PARSE) is analyzed like ordinary SQL whenever GSP can materialize its text.

1
2
3
4
CREATE PROCEDURE refresh_report AS
BEGIN
  EXEC('INSERT INTO report_daily (order_id) SELECT order_id FROM orders');
END;

1
2
3
4
5
6
7
8
<relationship id="1" type="fdd" effectType="select">
    <target id="15" column="order_id" parent_id="14" parent_name="INSERT-SELECT-1"/>
    <source id="12" column="order_id" parent_id="11" parent_name="orders"/>
</relationship>
<relationship id="2" type="fdd" effectType="insert" processId="7" processType="sstinsert">
    <target id="6" column="order_id" parent_id="5" parent_name="report_daily"/>
    <source id="15" column="order_id" parent_id="14" parent_name="INSERT-SELECT-1"/>
</relationship>

1
2
3
4
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<dlineage>
    <procedure id="1" name="refresh_report" type="mssqlcreateprocedure"/>
</dlineage>

report_daily, orders and both edges are gone — the procedure appears to touch nothing.

Turn it off only when you must analyze static SQL exclusively; it removes real, provable lineage.

setDynamicSqlTrustMode(DynamicSqlTrustMode)

Default: LEGACY. Values: LEGACY, SHADOW.

Dynamic SQL is frequently only partly known — the table is a literal but the WHERE clause is a runtime parameter. SHADOW publishes exactly the same graph as LEGACY but additionally reports how complete the materialization was, so you can measure the affected surface. Neither mode filters output.

setReportDynamicSqlSitesAsErrors(boolean)

Default: false. Mirrors every non-resolved dynamic-SQL site into the error list, where ordinary error-reading code will see it.

1
2
3
4
5
6
CREATE PROCEDURE refresh_report @where NVARCHAR(200) AS
BEGIN
  DECLARE @sql NVARCHAR(500);
  SET @sql = 'INSERT INTO report_daily (order_id) SELECT order_id FROM orders WHERE ' + @where;
  EXEC(@sql);
END;

Full lineage into report_daily, with the unknown predicate represented by an fdr edge from the @where variable — but nothing tells you the site was incomplete.

1
<error errorMessage="SHADOW retained legacy publication behavior; 1 unresolved fragment(s); 4 unproven candidate relationship(s)" errorType="DynamicSqlUnresolved" file="s23.sql" originCoordinate="[5,3],[5,13]"/>

The structured form of the same information is always available from analyzer.getDynamicSqlSites(), whatever these options are set to.

setAssumeExternalScriptPassthrough(boolean)

Default: true. SQL Server's sp_execute_external_script runs a Python or R script whose body GSP cannot read. By default GSP assumes the input dataset maps positionally to the declared WITH RESULT SETS columns, and marks every such edge effectType="external_script_passthrough" so it stays distinguishable from proven lineage.

1
2
3
4
5
6
INSERT INTO scored_orders (order_id, score)
EXEC sp_execute_external_script
  @language = N'Python',
  @script   = N'OutputDataSet = InputDataSet',
  @input_data_1 = N'SELECT order_id, amount FROM orders'
WITH RESULT SETS ((order_id INT, score FLOAT));

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
<relationship id="3" type="fdd" effectType="external_script_passthrough" processId="20" processType="sstmssqlexec">
    <target id="21" column="order_id" parent_id="19" parent_name="RS-2"/>
    <source id="16" column="order_id" parent_id="15" parent_name="RS-1"/>
</relationship>
<relationship id="4" type="fdd" effectType="external_script_passthrough" processId="20" processType="sstmssqlexec">
    <target id="22" column="score" parent_id="19" parent_name="RS-2"/>
    <source id="17" column="amount" parent_id="15" parent_name="RS-1"/>
</relationship>
<relationship id="5" type="fdd" effectType="insert" processId="7" processType="sstinsert">
    <target id="5" column="order_id" parent_id="4" parent_name="scored_orders"/>
    <source id="21" column="order_id" parent_id="19" parent_name="RS-2"/>
</relationship>

Lineage reaches scored_orders.

The passthrough edges and the RS-2 result set are gone. orders → RS-1 is still reported, but lineage stops at the script boundary — scored_orders has no upstream.

Choose true when a plausible-but-assumed edge is better than a gap, and read effectType to tell the two apart; choose false when your consumer must only ever show proven lineage.

10. Large inputs

setAutoDetectLargeFile(boolean)

Default: false. A very large script or manifest can exhaust the heap in a single-threaded run. With this on, GSP measures the input against the thresholds below and, if any is exceeded, delegates to ParallelDataFlowAnalyzer, which splits the work across a bounded thread pool and merges the results. If delegation fails for any reason, it falls back to single-threaded analysis with a warning.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Option option = new Option();
option.setVendor(EDbVendor.dbvmssql);
option.setAutoDetectLargeFile(true);
option.setLargeSqlInfoCountThreshold(1000);
option.setLargeSqlTotalSizeThreshold(25 * 1024 * 1024L);
option.setLargeQueryCountThreshold(1000);
option.setLargeShardCountThreshold(10);
option.setLargeFileSplitSizeMB(5);
option.setEstimatedMemoryPerTaskMB(2560);
option.setParallel(0);   // 0 = auto-size from CPU cores

DataFlowAnalyzer analyzer = new DataFlowAnalyzer(sqlFiles, option);
analyzer.generateDataFlow();

Thresholds

Option Default Triggers delegation when…
setLargeSqlInfoCountThreshold(int) 1000 the input holds this many SQL entries
setLargeSqlTotalSizeThreshold(long) 26214400 (25 MB) total SQL text reaches this many bytes
setLargeQueryCountThreshold(int) 1000 a sqlflow/grabit manifest holds this many queries
setLargeShardCountThreshold(int) 10 a sharded manifest references this many shard files
setLargeFileSplitSizeMB(int) 5 files above this size are split into chunks
setLargeSqlThresholdMultiplier(double) 1.5 multiplier on the split size that defines a "large" single statement
setEstimatedMemoryPerTaskMB(long) 2560 (2.5 GB) caps the pool size so pool × this fits available memory

setParallel(int)

Default: availableProcessors() / 2 − 1. Worker count for parallel analysis; 0 means auto-size from the CPU count.

Note

setCollectAuthoritativeLineageEvidence is forced off during parallel analysis, because merged ids are remapped and cannot yet be correlated back to worker evidence.

11. Integration hooks

setHandleListener(DataFlowHandleListener)

Default: none. Receives progress callbacks throughout the run, and — through isCanceled() — lets you stop a long analysis. Implement the interface and hand it to the option:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import java.io.File;
import gudusoft.gsqlparser.TCustomSqlStatement;
import gudusoft.gsqlparser.TGSqlParser;
import gudusoft.gsqlparser.dlineage.dataflow.listener.DataFlowHandleListener;

public class ProgressListener implements DataFlowHandleListener {
    private volatile boolean stop = false;

    public void cancel() { this.stop = true; }

    @Override public boolean isCanceled() { return stop; }

    @Override public void startParse(File file, String sql) {
        System.out.println("parsing " + (file == null ? "<text>" : file.getName()));
    }

    @Override public void endAnalyze() { System.out.println("done"); }

    // remaining callbacks: startAnalyze, startParseSQLEnv, endParseSQLEnv, endParse,
    // startAnalyzeDataFlow, startAnalyzeStatment, endAnalyzeStatment,
    // endAnalyzeDataFlow, startOutputDataFlowXML, endOutputDataFlowXML
    @Override public void startAnalyze(File f, long n, boolean isFileCount) { }
    @Override public void startParseSQLEnv() { }
    @Override public void endParseSQLEnv() { }
    @Override public void endParse(boolean isSuccess) { }
    @Override public void startAnalyzeDataFlow(TGSqlParser parser) { }
    @Override public void startAnalyzeStatment(TCustomSqlStatement stmt) { }
    @Override public void endAnalyzeStatment(TCustomSqlStatement stmt) { }
    @Override public void endAnalyzeDataFlow(TGSqlParser parser) { }
    @Override public void startOutputDataFlowXML() { }
    @Override public void endOutputDataFlowXML(long length) { }
}

1
option.setHandleListener(new ProgressListener());

setCollectAuthoritativeLineageEvidence(boolean)

Default: false. Collects an immutable semantic sidecar of same-run AST and catalog facts, keyed by the ids in the returned dataflow, for consumers that project lineage into the SLPC contract. It is opt-in so that ordinary runs pay nothing for the bookkeeping.

1
2
3
4
option.setCollectAuthoritativeLineageEvidence(true);
DataFlowAnalyzer analyzer = new DataFlowAnalyzer(sql, option);
analyzer.generateDataFlow();
AuthoritativeLineageEvidence evidence = analyzer.getAuthoritativeLineageEvidence();

The sidecar never appears in the XML or JSON output.

12. Beyond Option

12.1 Consume the model in Java instead of the XML

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import gudusoft.gsqlparser.EDbVendor;
import gudusoft.gsqlparser.dlineage.DataFlowAnalyzer;
import gudusoft.gsqlparser.dlineage.dataflow.model.Option;
import gudusoft.gsqlparser.dlineage.dataflow.model.xml.dataflow;
import gudusoft.gsqlparser.dlineage.dataflow.model.xml.relationship;
import gudusoft.gsqlparser.dlineage.dataflow.model.xml.sourceColumn;

public class WalkLineage {
    public static void main(String[] args) {
        Option option = new Option();
        option.setVendor(EDbVendor.dbvoracle);
        option.setOutput(false);   // skip XML serialization

        DataFlowAnalyzer analyzer = new DataFlowAnalyzer(
            "INSERT INTO order_report (order_id) SELECT order_id FROM orders;", option);
        analyzer.generateDataFlow();          // returns null when output == false

        dataflow df = analyzer.getDataFlow();
        for (relationship rel : df.getRelationships()) {
            for (sourceColumn src : rel.getSources()) {
                System.out.println(rel.getType() + " " + rel.getEffectType() + ": "
                    + src.getParent_name() + "." + src.getColumn() + " -> "
                    + rel.getTarget().getParent_name() + "." + rel.getTarget().getColumn());
            }
        }
    }
}

1
2
fdd select: orders.order_id -> INSERT-SELECT-1.order_id
fdd insert: INSERT-SELECT-1.order_id -> order_report.order_id

12.2 Other entry points on DataFlowAnalyzer

Call What it gives you
setSqlEnv(TSQLEnv) Real catalog metadata (tables, columns, types). The single biggest accuracy improvement available — it resolves SELECT *, unqualified columns and cross-schema names exactly. CLI /env metadata.json.
getDataFlow() The in-memory model, always populated after generateDataFlow().
getErrorMessages() Parse and analysis errors, including the orphan-column and dynamic-SQL diagnostics shown above.
getDynamicSqlSites() Every dynamic-SQL execution site and whether it was statically resolved.
traceView() View → base-table mapping as CSV, instead of column lineage.
getVersion() / getReleaseDate() The engine version, for provenance stamping.

traceView() on two chained views:

1
2
3
4
5
CREATE VIEW v_active_orders AS
SELECT o.order_id, o.amount FROM orders o WHERE o.status = 'ACTIVE';

CREATE VIEW v_big_orders AS
SELECT order_id, amount FROM v_active_orders WHERE amount > 1000;

1
2
v_active_orders,orders
v_big_orders,orders

Both views trace back to the physical table, not to each other.

12.3 Command-line flags and the options they set

The DataFlowAnalyzer main method accepts a small set of flags:

Flag Sets
/f <file> the SQL file to analyze
/d <dir> a directory of SQL files
/t <vendor> setVendor — default oracle
/s setSimpleOutput(true)
/text setTextFormat(true) (read only after /s)
/i setIgnoreRecordSet(true)
/j setShowJoin(true)
/json print the SQLFlow JSON model instead of XML
/traceView run traceView() (implies /s)
/o <file> redirect output to a file
/log write errors to dataflow.log
/version print engine version and release date 
1
2
3
java -cp gsqlparser-4.2.8.jar:<jaxb-jars> \
     gudusoft.gsqlparser.dlineage.DataFlowAnalyzer \
     /f etl.sql /t mssql /s /text

The distributed CLI has many more flags

The SQLFlow command-line distribution wraps this class and exposes far more options (/env, /showER, /transform, /lof, /filterRelationTypes, CSV output, …). Those are documented in Configuration options.

13. Recipes

Goal Options
Populate a data catalog with table→table and column→column edges setSimpleOutput(true)
Human-readable smoke test setSimpleOutput(true), setTextFormat(true)
Explain why an edge exists, in a UI default (full) output, setTransform(true), setTransformCoordinate(true)
Impact analysis ("what breaks if I drop this column?") full output, read both fdd and fdr edges
Value-flow only, no row influence filterRelationTypes("fdd")
Track business logic in UDFs setSimpleOutput(true), setSimpleShowFunction(true), setSimpleShowUdfFunctionOnly(true)
Hide staging temp tables setSimpleOutput(true), setIgnoreTemporaryTable(true)
Lineage from a read-only reporting query setSimpleOutput(true), setSimpleShowTopSelectResultSet(true)
Merge many runs into one graph setStartId(n) per run, setNormalizeOutput(true), setDefaultDatabase/setDefaultSchema
Audit how much lineage came from dynamic SQL setDynamicSqlTrustMode(SHADOW), setReportDynamicSqlSitesAsErrors(true)
Analyze a multi-gigabyte export setAutoDetectLargeFile(true), setParallel(0)

Refreshing this page

This page is hand-written but machine-verified, so it is refreshed incrementally — you do not rewrite it, you diff the option surface and touch only what moved.

1. Find what changed. From the repository root, list the mutators on Option in the release you are documenting and diff against the previous stamp:

1
2
3
4
grep -oE 'public [^(]*(set|add|show|filter)[A-Za-z0-9_]*\(' \
  gsp_java_core/src/main/java/gudusoft/gsqlparser/dlineage/dataflow/model/Option.java \
  | sed -E 's/.*[^A-Za-z0-9_]([A-Za-z0-9_]+)\($/\1/' \
  | sort -u

At the 4.2.8 stamp this prints 70 names. A different count means an option was added or removed; a same count can still hide a renamed one, so diff the list, not the number.

2. Cover each new option in the section matching what it does, and add its row to the cheat sheet. Keep the entry shape: default, what it is for, minimal SQL, real before/after output in === "..." tabs.

3. Re-run the samples you touched and paste the real output. Nothing on this page may be written from memory — that is the site-wide sample-code rule (site-docs/CLAUDE.md in the repository), and it is the only reason the before/after blocks are trustworthy.

4. Re-check the known limitations. The /// warning and /// note blocks cite open MantisBT issues (currently #4719 and #4720). When one is fixed, replace the caveat with the working behavior rather than leaving a stale warning.

5. Update the stamp at the top: version, its TBaseType.releaseDate, today's date, and the new option count.

6. Verify before publishing:

1
2
3
python3 ../gsp_dotnet/site-docs/tools/check_api_symbols.py \
    --docs site-docs/docs --source gsp_java_core/src/main/java/gudusoft/gsqlparser --lang java
mkdocs build -f site-docs/mkdocs.yml

This version stamp is NOT a release-bump site

The repository's CLAUDE.md lists six sites that must move on every version bump. This page is not one of them, and must not be added. The stamp means "this is the build the samples were produced on", not "this is the current release". Advancing it without re-running the samples turns a true statement into a false one — the page would claim verification that never happened.

See also