Tokens and Delimiters

Often an input string contains groups of characters that have a meaning. Such a group is called a token. Tokens are separated from each other by delimiter characters. For example, say that you need to add up the following numbers:

12  8  5  32

You automatically grouped these characters to form four integers separated by spaces. The tokens in this case are 12 and 8 and 5 and 32. These tokens are delimited by the space character.

What makes a token and what makes a delimiter depends on the application.

643,983,104