International PHP Conference Berlin 2025

PhpToken::tokenize

(PHP 8)

PhpToken::tokenizeSplits given source into PHP tokens, represented by PhpToken objects.

Description

public static PhpToken::tokenize(string $code, int $flags = 0): array

Returns an array of PhpToken objects representing given code.

Parameters

code

The PHP source to parse.

flags

Valid flags:

  • TOKEN_PARSE - Recognises the ability to use reserved words in specific contexts.

Return Values

An array of PHP tokens represented by instances of PhpToken or its descendants. This method returns static[] so that PhpToken can be seamlessly extended.

Examples

Example #1 PhpToken::tokenize() example

<?php
$tokens
= PhpToken::tokenize('<?php echo; ?>');

foreach (
$tokens as $token) {
echo
"Line {$token->line}: {$token->getTokenName()} ('{$token->text}')", PHP_EOL;
}

The above examples will output:

Line 1: T_OPEN_TAG ('<?php ')
Line 1: T_ECHO ('echo')
Line 1: ; (';')
Line 1: T_WHITESPACE (' ')
Line 1: T_CLOSE_TAG ('?>')

Example #2 Extending PhpToken

<?php

class MyPhpToken extends PhpToken {
public function
getUpperText() {
return
strtoupper($this->text);
}
}

$tokens = MyPhpToken::tokenize('<?php echo; ?>');
echo
"'{$tokens[0]->getUpperText()}'";

The above examples will output:

'<?PHP '

See Also

add a note

User Contributed Notes

There are no user contributed notes for this page.
To Top