21.6.5 strtok

strtok is a helper for tokenizing strings. The synopsis of this API are:

type String_Tokenizer =
  struct
  {
    uint<64> i;
    string str;
    computed uint<32> more;

    method get_more = uint<32>: { … }

    method peek = char: { … }
    method pop = char: { … }

    method pop_number = (int<32> base = 10) int<64>: { … }

    method popdelim = (string delimiters) string: { … }
    method poprdelim = (string delimiters) string: { … }
  }

fun strtok = (string a) String_Tokenizer: { … }

Function: String_Tokenizer strtok (string a)

Creates a new tokenizer for the string a, initially on the zero position.

The members of the String_Tokenizer class are:

Instance Variable of String_Tokenizer: uint<64> i

Offset to the next character to be tokenized, i.e. to the first character that has not already been consumed.

Instance Variable of String_Tokenizer: string str

The string being tokenized. This string is never tokenized.

Instance Variable of String_Tokenizer: uint<32> more

A read-only computed property whose value is truthy if there’s more characters and falsey otherwise.

Method on String_Tokenizer: char poke ()

Returns the first unread character of the string, but does not advance the i offset.

Raises E_out_of_bounds if at the end of the string.

Method on String_Tokenizer: char peek ()

Returns the first unread character of the string, and advances the tokenizer.

Raises E_out_of_bounds if at the end of the string.

Method on String_Tokenizer: int<64> pop_number (int<32> base = 10)

Returns the number at the start of the string and advances the tokenizer in the given base. The bases that are supported are the same as for strtoi. See strtoi for a list of supported bases.

Raises E_out_of_bounds if at the end of the string.

Method on String_Tokenizer: string popdelim (string delim)

Returns the substring up to the first character also present in the string delim. Advances the tokenizer to after the delimiter character (i.e. it consumes the delimiter character).

Raises E_out_of_bounds if at the end of the string.

Method on String_Tokenizer: string poprdelim (string delim)

Returns the substring up to the last character also present in the string delim. Advances the tokenizer to after the delimiter character (i.e. it consumes the delimiter character).

Raises E_out_of_bounds if at the end of the string.