Unreal Engine 4.26 Documentation

Unreal Engine 4.27 Documentation

Unreal Engine 5.0 Documentation

Unreal Engine 5.1 Documentation

Unreal Engine 5.2 Documentation

Unreal Engine 5.3 Documentation

> Unreal Engine API Reference > Runtime > Core > Misc > FTokenStream

Light Theme

Dark Theme

FTokenStream

A token stream wraps up a raw string, providing accessors into it for consuming tokens

Windows

MacOS

Linux

References

Module	Core
Header	/Engine/Source/Runtime/Core/Public/Misc/ExpressionParserTypes.h
Include	#include "Misc/ExpressionParserTypes.h"

Syntax

class FTokenStream

Remarks

A token stream wraps up a raw string, providing accessors into it for consuming tokens

Constructors

	Name	Description
	FTokenStream ( const TCHAR* In )	The stream is only valid for the lifetime of the string provided

Functions

	Name	Description
int32	CharsRemaining()	Get the number of characters remaining in the stream after the current read position
TOptional< F...	GenerateToken ( int32 NumChars, FStringToken* Accumulate )	Generate a token for the specified number of chars, at the current read position (or end of Accumulate)
const TCHAR ...	GetEnd()
FString	GetErrorContext()	Get the error context from the current read position
int32	GetPosition()	Get the current read position from the start of the stream
const TCHAR ...	GetRead()
const TCHAR ...	GetStart()
bool	IsEmpty()	Check if the stream is empty
bool	IsReadPosValid ( const TCHAR* InPos, int32 MinNumChars )	Check if it is valid to read (the optional number of characters) from the specified position
TOptional< F...	ParseSymbol ( FStringToken* Accumulate )	Return a string token for the next character in the stream (or accumulating into the specified existing token)
TOptional< F...	ParseSymbol ( TCHAR Symbol, FStringToken* Accumulate )	Attempt parse out the specified pre-defined string from the current read position (or accumulating into the specified existing token)
TOptional< F...	ParseToken ( TFunctionRef< EParseState)> P..., FStringToken* Accumulate )	Parse out a token using the supplied predicate.
TOptional< F...	ParseToken ( const TCHAR* Symbol, FStringToken* Accumulate )	Attempt parse out the specified pre-defined string from the current read position (or accumulating into the specified existing token)
TOptional< F...	ParseTokenIgnoreCase ( const TCHAR* Symbol, FStringToken* Accumulate )	Parse out a token using the supplied predicate.
TOptional< F...	ParseWhitespace ( FStringToken* Accumulate )	Parse a whitespace token
TCHAR	PeekChar ( int32 Offset )	Peek at the character at the specified offset from the current read position
void	SetReadPos ( const FStringToken& Token )	Set the current read position to the character proceeding the specified token

Help shape the future of Unreal Engine documentation! Tell us how we're doing so we can serve you better.

Take our survey

Dismiss