Tokens
extends SplFixedArray
in package
Collection of code tokens.
Its role is to provide the ability to manage collection and navigate through it.
As a token prototype you should understand a single element generated by token_get_all.
Tags
Table of Contents
Constants
- BLOCK_TYPE_ARRAY_INDEX_CURLY_BRACE = 7
- BLOCK_TYPE_ARRAY_SQUARE_BRACE = 4
- BLOCK_TYPE_ATTRIBUTE = 11
- BLOCK_TYPE_BRACE_CLASS_INSTANTIATION = 10
- BLOCK_TYPE_COMPLEX_STRING_VARIABLE = 14
- BLOCK_TYPE_CURLY_BRACE = 2
- BLOCK_TYPE_DESTRUCTURING_SQUARE_BRACE = 9
- BLOCK_TYPE_DISJUNCTIVE_NORMAL_FORM_TYPE_PARENTHESIS = 12
- BLOCK_TYPE_DYNAMIC_CLASS_CONSTANT_FETCH_CURLY_BRACE = 13
- BLOCK_TYPE_DYNAMIC_PROP_BRACE = 5
- BLOCK_TYPE_DYNAMIC_VAR_BRACE = 6
- BLOCK_TYPE_GROUP_IMPORT_BRACE = 8
- BLOCK_TYPE_INDEX_SQUARE_BRACE = 3
- BLOCK_TYPE_PARENTHESIS_BRACE = 1
Properties
- $blockEndCache : array<int, int>
- Cache of block ends. Any change in collection will invalidate it.
- $blockStartCache : array<int, int>
- Cache of block starts. Any change in collection will invalidate it.
- $cache : array<non-empty-string, self>
- Static class cache.
- $changed : bool
- Flag is collection was changed.
- $codeHash : non-empty-string|null
- A MD5 hash of the code string.
- $foundTokenKinds : array<int|non-empty-string, int>
- Set of found token kinds.
- $namespaceDeclarations : null|array<int, NamespaceAnalysis>
Methods
- __clone() : mixed
- Clone tokens collection.
- clearAt() : void
- clearCache() : void
- Clear cache - one position or all of them.
- clearChanged() : void
- Clear internal flag if collection was changed and flag for all collection's items.
- clearEmptyTokens() : void
- Clear empty tokens.
- clearRange() : void
- Clear tokens in the given range.
- clearTokenAndMergeSurroundingWhitespace() : void
- countTokenKind() : int
- detectBlockType() : null|array{type: self::BLOCK_TYPE_*, isStart: bool}
- Detect type of block.
- ensureWhitespaceAtIndex() : bool
- Ensure that on given index is a whitespace with given kind.
- findBlockEnd() : int<0, max>
- findBlockStart() : int<0, max>
- findGivenKind() : mixed
- findSequence() : array<string|int, mixed>|null
- Find a sequence of meaningful tokens and returns the array of their locations.
- fromArray() : self
- Create token collection from array.
- fromCode() : self
- Create token collection directly from code.
- generateCode() : string
- generatePartialCode() : string
- Generate code from tokens between given indices.
- getBlockEdgeDefinitions() : array<string|int, mixed>
- getCodeHash() : string
- Get hash of code.
- getMeaningfulTokenSibling() : int|null
- Get index for closest sibling token that is not a whitespace, comment or attribute.
- getNextMeaningfulToken() : int|null
- Get index for closest next token that is not a whitespace or comment.
- getNextNonWhitespace() : int|null
- Get index for closest next token which is non whitespace.
- getNextTokenOfKind() : int|null
- Get index for closest next token of given kind.
- getNonEmptySibling() : int|null
- Get index for closest sibling token which is not empty.
- getNonWhitespaceSibling() : int|null
- Get index for closest sibling token which is non whitespace.
- getPrevMeaningfulToken() : int|null
- Get index for closest previous token that is not a whitespace or comment.
- getPrevNonWhitespace() : int|null
- Get index for closest previous token which is non whitespace.
- getPrevTokenOfKind() : int|null
- Get index for closest previous token of given kind.
- getTokenNotOfKindSibling() : int|null
- Get index for closest sibling token not of given kind.
- getTokenNotOfKindsSibling() : int|null
- Get index for closest sibling token not of given kind.
- getTokenOfKindSibling() : int|null
- Get index for closest sibling token of given kind.
- hasAlternativeSyntax() : bool
- insertAt() : void
- Insert instances of Token inside collection.
- isAllTokenKindsFound() : bool
- Check if all token kinds given as argument are found.
- isAnyTokenKindsFound() : bool
- Check if any token kind given as argument is found.
- isChanged() : bool
- Check if collection was change: collection itself (like insert new tokens) or any of collection's elements.
- isEmptyAt() : bool
- isMonolithicPhp() : bool
- Checks for monolithic PHP code.
- isPartialCodeMultiline() : bool
- isTokenKindFound() : bool
- Check if token kind given as argument is found.
- offsetGet() : Token
- offsetSet() : void
- Set collection item.
- offsetUnset() : void
- Unset collection item.
- overrideRange() : void
- Override tokens at given range.
- removeLeadingWhitespace() : void
- removeTrailingWhitespace() : void
- setCode() : void
- Set code. Clear all current content and replace it by new Token items generated from code directly.
- setSize() : bool
- Set new size of collection.
- toJson() : string
- calculateCodeHash() : non-empty-string
- Calculate hash for code.
- changeCodeHash() : void
- Change code hash.
- extractTokenKind() : int|non-empty-string
- findOppositeBlockEdge() : int<0, max>
- getCache() : self
- Get cache value for given key.
- getTokenNotOfKind() : int|null
- hasCache() : bool
- Check if given key exists in cache.
- isKeyCaseSensitive() : bool
- A helper method used to find out whether a certain input token has to be case-sensitively matched.
- registerFoundToken() : void
- Register token as found.
- removeWhitespaceSafely() : void
- setCache() : void
- unregisterFoundToken() : void
- Unregister token as not found.
Constants
BLOCK_TYPE_ARRAY_INDEX_CURLY_BRACE
public
mixed
BLOCK_TYPE_ARRAY_INDEX_CURLY_BRACE
= 7
BLOCK_TYPE_ARRAY_SQUARE_BRACE
public
mixed
BLOCK_TYPE_ARRAY_SQUARE_BRACE
= 4
BLOCK_TYPE_ATTRIBUTE
public
mixed
BLOCK_TYPE_ATTRIBUTE
= 11
BLOCK_TYPE_BRACE_CLASS_INSTANTIATION
public
mixed
BLOCK_TYPE_BRACE_CLASS_INSTANTIATION
= 10
BLOCK_TYPE_COMPLEX_STRING_VARIABLE
public
mixed
BLOCK_TYPE_COMPLEX_STRING_VARIABLE
= 14
BLOCK_TYPE_CURLY_BRACE
public
mixed
BLOCK_TYPE_CURLY_BRACE
= 2
BLOCK_TYPE_DESTRUCTURING_SQUARE_BRACE
public
mixed
BLOCK_TYPE_DESTRUCTURING_SQUARE_BRACE
= 9
BLOCK_TYPE_DISJUNCTIVE_NORMAL_FORM_TYPE_PARENTHESIS
public
mixed
BLOCK_TYPE_DISJUNCTIVE_NORMAL_FORM_TYPE_PARENTHESIS
= 12
BLOCK_TYPE_DYNAMIC_CLASS_CONSTANT_FETCH_CURLY_BRACE
public
mixed
BLOCK_TYPE_DYNAMIC_CLASS_CONSTANT_FETCH_CURLY_BRACE
= 13
BLOCK_TYPE_DYNAMIC_PROP_BRACE
public
mixed
BLOCK_TYPE_DYNAMIC_PROP_BRACE
= 5
BLOCK_TYPE_DYNAMIC_VAR_BRACE
public
mixed
BLOCK_TYPE_DYNAMIC_VAR_BRACE
= 6
BLOCK_TYPE_GROUP_IMPORT_BRACE
public
mixed
BLOCK_TYPE_GROUP_IMPORT_BRACE
= 8
BLOCK_TYPE_INDEX_SQUARE_BRACE
public
mixed
BLOCK_TYPE_INDEX_SQUARE_BRACE
= 3
BLOCK_TYPE_PARENTHESIS_BRACE
public
mixed
BLOCK_TYPE_PARENTHESIS_BRACE
= 1
Properties
$blockEndCache
Cache of block ends. Any change in collection will invalidate it.
private
array<int, int>
$blockEndCache
= []
$blockStartCache
Cache of block starts. Any change in collection will invalidate it.
private
array<int, int>
$blockStartCache
= []
$cache
Static class cache.
private
static array<non-empty-string, self>
$cache
= []
$changed
Flag is collection was changed.
private
bool
$changed
= false
It doesn't know about change of collection's items. To check it run isChanged
method.
$codeHash
A MD5 hash of the code string.
private
non-empty-string|null
$codeHash
= null
$foundTokenKinds
Set of found token kinds.
private
array<int|non-empty-string, int>
$foundTokenKinds
= []
When the token kind is present in this set it means that given token kind was ever seen inside the collection (but may not be part of it any longer). The key is token kind and the value is always true.
$namespaceDeclarations
private
null|array<int, NamespaceAnalysis>
$namespaceDeclarations
= null
Methods
__clone()
Clone tokens collection.
public
__clone() : mixed
clearAt()
public
clearAt(int $index) : void
Parameters
- $index : int
clearCache()
Clear cache - one position or all of them.
public
static clearCache([null|non-empty-string $key = null ]) : void
Parameters
- $key : null|non-empty-string = null
-
position to clear, when null clear all
clearChanged()
Clear internal flag if collection was changed and flag for all collection's items.
public
clearChanged() : void
clearEmptyTokens()
Clear empty tokens.
public
clearEmptyTokens() : void
Empty tokens can occur e.g. after calling clear on item of collection.
clearRange()
Clear tokens in the given range.
public
clearRange(int $indexStart, int $indexEnd) : void
Parameters
- $indexStart : int
- $indexEnd : int
clearTokenAndMergeSurroundingWhitespace()
public
clearTokenAndMergeSurroundingWhitespace(int $index) : void
Parameters
- $index : int
countTokenKind()
public
countTokenKind(int|string $tokenKind) : int
Parameters
- $tokenKind : int|string
Return values
intdetectBlockType()
Detect type of block.
public
static detectBlockType(Token $token) : null|array{type: self::BLOCK_TYPE_*, isStart: bool}
Parameters
- $token : Token
Return values
null|array{type: self::BLOCK_TYPE_*, isStart: bool}ensureWhitespaceAtIndex()
Ensure that on given index is a whitespace with given kind.
public
ensureWhitespaceAtIndex(int $index, int $indexOffset, string $whitespace) : bool
If there is a whitespace then it's content will be modified. If not - the new Token will be added.
Parameters
- $index : int
-
index
- $indexOffset : int
-
index offset for Token insertion
- $whitespace : string
-
whitespace to set
Return values
bool —if new Token was added
findBlockEnd()
public
findBlockEnd(self::BLOCK_TYPE_* $type, int $searchIndex) : int<0, max>
Parameters
- $type : self::BLOCK_TYPE_*
-
type of block
- $searchIndex : int
-
index of opening brace
Return values
int<0, max> —index of closing brace
findBlockStart()
public
findBlockStart(self::BLOCK_TYPE_* $type, int $searchIndex) : int<0, max>
Parameters
- $type : self::BLOCK_TYPE_*
-
type of block
- $searchIndex : int
-
index of closing brace
Return values
int<0, max> —index of opening brace
findGivenKind()
public
findGivenKind(int|array<int, int> $possibleKind[, int $start = 0 ][, null|int $end = null ]) : mixed
Parameters
- $possibleKind : int|array<int, int>
-
kind or array of kinds
- $start : int = 0
-
optional offset
- $end : null|int = null
-
optional limit
findSequence()
Find a sequence of meaningful tokens and returns the array of their locations.
public
findSequence(array<int, array{0: int, 1?: string}|string|Token> $sequence[, int $start = 0 ][, null|int $end = null ][, array<int, bool>|bool $caseSensitive = true ]) : array<string|int, mixed>|null
Parameters
- $sequence : array<int, array{0: int, 1?: string}|string|Token>
-
an array of token (kinds)
- $start : int = 0
-
start index, defaulting to the start of the file
- $end : null|int = null
-
end index, defaulting to the end of the file
- $caseSensitive : array<int, bool>|bool = true
-
global case sensitiveness or a list of booleans, whose keys should match the ones used in $sequence. If any is missing, the default case-sensitive comparison is used
Return values
array<string|int, mixed>|nullfromArray()
Create token collection from array.
public
static fromArray(array<int, Token> $array[, bool|null $saveIndices = null ]) : self
Parameters
- $array : array<int, Token>
-
the array to import
- $saveIndices : bool|null = null
-
save the numeric indices used in the original array, default is yes
Return values
selffromCode()
Create token collection directly from code.
public
static fromCode(string $code) : self
Parameters
- $code : string
-
PHP code
Return values
selfgenerateCode()
public
generateCode() : string
Return values
stringgeneratePartialCode()
Generate code from tokens between given indices.
public
generatePartialCode(int $start, int $end) : string
Parameters
- $start : int
-
start index
- $end : int
-
end index
Return values
stringgetBlockEdgeDefinitions()
public
static getBlockEdgeDefinitions() : array<string|int, mixed>
Return values
array<string|int, mixed>getCodeHash()
Get hash of code.
public
getCodeHash() : string
Return values
stringgetMeaningfulTokenSibling()
Get index for closest sibling token that is not a whitespace, comment or attribute.
public
getMeaningfulTokenSibling(int $index, -1|1 $direction) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
Return values
int|nullgetNextMeaningfulToken()
Get index for closest next token that is not a whitespace or comment.
public
getNextMeaningfulToken(int $index) : int|null
Parameters
- $index : int
-
token index
Return values
int|nullgetNextNonWhitespace()
Get index for closest next token which is non whitespace.
public
getNextNonWhitespace(int $index[, null|string $whitespaces = null ]) : int|null
This method is shorthand for getNonWhitespaceSibling method.
Parameters
- $index : int
-
token index
- $whitespaces : null|string = null
-
whitespaces characters for Token::isWhitespace
Return values
int|nullgetNextTokenOfKind()
Get index for closest next token of given kind.
public
getNextTokenOfKind(int $index[, array<int, array{: int}|string|Token> $tokens = [] ][, bool $caseSensitive = true ]) : int|null
This method is shorthand for getTokenOfKindSibling method.
Parameters
- $index : int
-
token index
- $tokens : array<int, array{: int}|string|Token> = []
-
possible tokens
- $caseSensitive : bool = true
-
perform a case sensitive comparison
Return values
int|nullgetNonEmptySibling()
Get index for closest sibling token which is not empty.
public
getNonEmptySibling(int $index, -1|1 $direction) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
Return values
int|nullgetNonWhitespaceSibling()
Get index for closest sibling token which is non whitespace.
public
getNonWhitespaceSibling(int $index, -1|1 $direction[, null|string $whitespaces = null ]) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
- $whitespaces : null|string = null
-
whitespaces characters for Token::isWhitespace
Return values
int|nullgetPrevMeaningfulToken()
Get index for closest previous token that is not a whitespace or comment.
public
getPrevMeaningfulToken(int $index) : int|null
Parameters
- $index : int
-
token index
Return values
int|nullgetPrevNonWhitespace()
Get index for closest previous token which is non whitespace.
public
getPrevNonWhitespace(int $index[, null|string $whitespaces = null ]) : int|null
This method is shorthand for getNonWhitespaceSibling method.
Parameters
- $index : int
-
token index
- $whitespaces : null|string = null
-
whitespaces characters for Token::isWhitespace
Return values
int|nullgetPrevTokenOfKind()
Get index for closest previous token of given kind.
public
getPrevTokenOfKind(int $index[, array<int, array{: int}|string|Token> $tokens = [] ][, bool $caseSensitive = true ]) : int|null
This method is shorthand for getTokenOfKindSibling method.
Parameters
- $index : int
-
token index
- $tokens : array<int, array{: int}|string|Token> = []
-
possible tokens
- $caseSensitive : bool = true
-
perform a case sensitive comparison
Return values
int|nullgetTokenNotOfKindSibling()
Get index for closest sibling token not of given kind.
public
getTokenNotOfKindSibling(int $index, -1|1 $direction[, array<int, array{: int}|string|Token> $tokens = [] ]) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
- $tokens : array<int, array{: int}|string|Token> = []
-
possible tokens
Return values
int|nullgetTokenNotOfKindsSibling()
Get index for closest sibling token not of given kind.
public
getTokenNotOfKindsSibling(int $index, -1|1 $direction[, array<int, int> $kinds = [] ]) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
- $kinds : array<int, int> = []
-
possible tokens kinds
Return values
int|nullgetTokenOfKindSibling()
Get index for closest sibling token of given kind.
public
getTokenOfKindSibling(int $index, -1|1 $direction[, array<int, array{: int}|string|Token> $tokens = [] ][, bool $caseSensitive = true ]) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
- $tokens : array<int, array{: int}|string|Token> = []
-
possible tokens
- $caseSensitive : bool = true
-
perform a case sensitive comparison
Return values
int|nullhasAlternativeSyntax()
public
hasAlternativeSyntax() : bool
Return values
boolinsertAt()
Insert instances of Token inside collection.
public
insertAt(int $index, array<int, Token>|Token|Tokens $items) : void
Parameters
isAllTokenKindsFound()
Check if all token kinds given as argument are found.
public
isAllTokenKindsFound(array<int, int|string> $tokenKinds) : bool
Parameters
- $tokenKinds : array<int, int|string>
Return values
boolisAnyTokenKindsFound()
Check if any token kind given as argument is found.
public
isAnyTokenKindsFound(array<int, int|string> $tokenKinds) : bool
Parameters
- $tokenKinds : array<int, int|string>
Return values
boolisChanged()
Check if collection was change: collection itself (like insert new tokens) or any of collection's elements.
public
isChanged() : bool
Return values
boolisEmptyAt()
public
isEmptyAt(int $index) : bool
Parameters
- $index : int
Return values
boolisMonolithicPhp()
Checks for monolithic PHP code.
public
isMonolithicPhp() : bool
Checks that the code is pure PHP code, in a single code block, starting with an open tag.
Return values
boolisPartialCodeMultiline()
public
isPartialCodeMultiline(int $start, int $end) : bool
Parameters
- $start : int
-
start index
- $end : int
-
end index
Return values
boolisTokenKindFound()
Check if token kind given as argument is found.
public
isTokenKindFound(int|string $tokenKind) : bool
Parameters
- $tokenKind : int|string
Return values
booloffsetGet()
public
offsetGet([mixed $offset = ]) : Token
Parameters
- $offset : mixed =
Return values
TokenoffsetSet()
Set collection item.
public
offsetSet(int $index, Token $newval) : void
Warning! $newval
must not be typehinted to be compatible with ArrayAccess::offsetSet
method.
Parameters
- $index : int
- $newval : Token
offsetUnset()
Unset collection item.
public
offsetUnset(int $index) : void
Parameters
- $index : int
overrideRange()
Override tokens at given range.
public
overrideRange(int $indexStart, int $indexEnd, array<int, Token>|Tokens $items) : void
Parameters
removeLeadingWhitespace()
public
removeLeadingWhitespace(int $index[, null|string $whitespaces = null ]) : void
Parameters
- $index : int
- $whitespaces : null|string = null
-
optional whitespaces characters for Token::isWhitespace
removeTrailingWhitespace()
public
removeTrailingWhitespace(int $index[, null|string $whitespaces = null ]) : void
Parameters
- $index : int
- $whitespaces : null|string = null
-
optional whitespaces characters for Token::isWhitespace
setCode()
Set code. Clear all current content and replace it by new Token items generated from code directly.
public
setCode(string $code) : void
Parameters
- $code : string
-
PHP code
setSize()
Set new size of collection.
public
setSize(int $size) : bool
Parameters
- $size : int
Return values
booltoJson()
public
toJson() : string
Return values
stringcalculateCodeHash()
Calculate hash for code.
private
static calculateCodeHash(string $code) : non-empty-string
Parameters
- $code : string
Return values
non-empty-stringchangeCodeHash()
Change code hash.
private
changeCodeHash(non-empty-string $codeHash) : void
Remove old cache and set new one.
Parameters
- $codeHash : non-empty-string
-
new code hash
extractTokenKind()
private
extractTokenKind(array{: int}|string|Token $token) : int|non-empty-string
Parameters
- $token : array{: int}|string|Token
-
token prototype
Return values
int|non-empty-stringfindOppositeBlockEdge()
private
findOppositeBlockEdge(self::BLOCK_TYPE_* $type, int $searchIndex, bool $findEnd) : int<0, max>
Parameters
- $type : self::BLOCK_TYPE_*
-
type of block
- $searchIndex : int
-
index of starting brace
- $findEnd : bool
-
if method should find block's end or start
Return values
int<0, max> —index of opposite brace
getCache()
Get cache value for given key.
private
static getCache(non-empty-string $key) : self
Parameters
- $key : non-empty-string
-
item key
Return values
selfgetTokenNotOfKind()
private
getTokenNotOfKind(int $index, -1|1 $direction, callable(int): bool $filter) : int|null
Parameters
- $index : int
-
token index
- $direction : -1|1
- $filter : callable(int): bool
Return values
int|nullhasCache()
Check if given key exists in cache.
private
static hasCache(non-empty-string $key) : bool
Parameters
- $key : non-empty-string
-
item key
Return values
boolisKeyCaseSensitive()
A helper method used to find out whether a certain input token has to be case-sensitively matched.
private
static isKeyCaseSensitive(array<int, bool>|bool $caseSensitive, int $key) : bool
Parameters
- $caseSensitive : array<int, bool>|bool
-
global case sensitiveness or an array of booleans, whose keys should match the ones used in $sequence. If any is missing, the default case-sensitive comparison is used
- $key : int
-
the key of the token that has to be looked up
Return values
boolregisterFoundToken()
Register token as found.
private
registerFoundToken(array{: int}|string|Token $token) : void
Parameters
- $token : array{: int}|string|Token
-
token prototype
removeWhitespaceSafely()
private
removeWhitespaceSafely(int $index, -1|1 $direction[, string|null $whitespaces = null ]) : void
Parameters
- $index : int
- $direction : -1|1
- $whitespaces : string|null = null
setCache()
private
static setCache(non-empty-string $key, Tokens $value) : void
Parameters
- $key : non-empty-string
-
item key
- $value : Tokens
-
item value
unregisterFoundToken()
Unregister token as not found.
private
unregisterFoundToken(array{: int}|string|Token $token) : void
Parameters
- $token : array{: int}|string|Token
-
token prototype