gudusoft.gsqlparser.pp2.token

Class Pp2Token

Object

gudusoft.gsqlparser.pp2.token.Pp2Token

public final class Pp2Token
extends Object

Lightweight wrapper around a TSourceToken, owned by pp2.

Carries:

• A reference to the underlying TSourceToken — pp2 never mutates it. This is the cache-safety invariant from plan §5.3. Probe A7 in slice S1 confirmed TGSqlParser reuses the same TSourceTokenList reference across tokenizations, which makes this non-mutation contract mandatory. Callers must build the pp2 token stream and copy any per-token data they need before reusing the parser instance; holding Pp2Token references across a re-tokenization of the same parser will silently observe post-reuse data.
• A mutable EnumSet of TokenRoles — pp2's side-channel annotations. Set during token-stream construction (S7), by designated detector/annotator stages (S9 protected zones, S21 clause scopes, S33 AST overlay), and by layout/island stages (S22 onward).
• precedingBlanks — number of whitespace columns immediately preceding this token in the source (e.g., 4 for " SELECT" at line start after the linebreak).
• precedingLinebreaks — number of newline characters between the previous solid token and this one.

The precedingBlanks / precedingLinebreaks model mirrors the Delphi initTokenArray reconstruction property (plan §7.4 S7): a stream of Pp2Tokens reproduces the original input byte-for-byte by concatenating Σ(linebreaks + blanks + token.text).

Construction does not snapshot the wrapped token's text — it stores the reference and exposes getText() which delegates to TSourceToken.toString(). Pp2 callers must not rely on TSourceToken field stability across tokenization reuse.

Constructor Summary

Constructors

Constructor and Description
Pp2Token(TSourceToken sourceToken) Construct with no roles, zero preceding whitespace.
Pp2Token(TSourceToken sourceToken, int precedingBlanks, int precedingLinebreaks, Set<TokenRole> roles) Construct with the given roles and preceding whitespace counts.

Method Summary

Modifier and Type	Method and Description
void	addRole(TokenRole role)
int	getPrecedingBlanks()
int	getPrecedingLinebreaks()
Set<TokenRole>	getRoles() Read-only view of the role set.
TSourceToken	getSourceToken() Reference to the wrapped TSourceToken.
String	getText() Text of the wrapped token.
boolean	hasRole(TokenRole role)
void	removeRole(TokenRole role)
String	toString()

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Pp2Token

public Pp2Token(TSourceToken sourceToken)

Construct with no roles, zero preceding whitespace.

Parameters:

sourceToken - non-null reference to the underlying source token

Pp2Token

public Pp2Token(TSourceToken sourceToken,
                int precedingBlanks,
                int precedingLinebreaks,
                Set<TokenRole> roles)

Construct with the given roles and preceding whitespace counts.

Parameters:

sourceToken - non-null reference; never mutated

precedingBlanks - count of whitespace columns immediately before this token; must be >= 0

precedingLinebreaks - count of linebreaks immediately before this token; must be >= 0

roles - initial role set; copied into an EnumSet owned by this token; may be null or empty

Method Detail

getSourceToken

public TSourceToken getSourceToken()

Reference to the wrapped TSourceToken. Never mutated by pp2.

getText

public String getText()

Text of the wrapped token. Delegates to TSourceToken.toString() — pp2 never caches or modifies it.

getRoles

public Set<TokenRole> getRoles()

Read-only view of the role set. Use addRole(TokenRole) and removeRole(TokenRole) to mutate; bulk replacement is not supported by design (the invariants live in the mutators).

hasRole

public boolean hasRole(TokenRole role)

addRole

public void addRole(TokenRole role)

removeRole

public void removeRole(TokenRole role)

getPrecedingBlanks

public int getPrecedingBlanks()

getPrecedingLinebreaks

public int getPrecedingLinebreaks()

toString

public String toString()

Overrides:

toString in class Object