Lexing

As per our previous chapters, our language is going to look something like:

new hello = 'world'
print hello

We’ll extend it eventually, but for the moment, while we go over the basic constructs, we’ll keep this as the entirety of the supported syntax.

Before we discuss how to build our lexer, we need to understand the different parts of our source, and how we should interpret them.