strings

What are strings?

In our language, a string is going to be denoted by a ', then any amount of characters, and then another '.

So.. if our current token is ' we're starting a new start, and we want to match everything until the next '.

How?

Utilising the lookahead function we just created, we can now use that to collect everything that is not something we specify. In this case, we want to match evyerthing that is not a '.

Code

if (currentToken === "'") {
  currentPosition++

  const bucket = lookahead(/[^']/)

  out.push({
    type: TokenType.String,
    value: bucket.join('')
  })

  currentPosition += bucket.length + 1
  
  continue
}

Firstly, we only run this if our current token is the start of a string, which is ' in this case.

We could easily implement matching double-quote strings as well, but that can be left as an exercise for the reader.

You may notice that immediately inside our if statement, we do currentPosition++, that’s because we don’t actually care about the ' token, now that we know we’re inside a string, so just skip it.

Next, we create an array of tokens that exist up until the next ' we find! This is what our regex of /[^']/ matches, anything that is not a '.

From here we then add our new string token to our list, then we increment currentPosition by bucket.length + 1.

Why + 1?

Well, if we didn’t do that, the next time the while loop runs, we’d be matching the last ' in the string sequence, but our parser would think it’s a new string, and we’d end up matching the exact inverse of what we wanted to!