Structure

function toAST (tokens: Token[]): ASTNode {
  let currentIndex = 0

  function process (): ASTNode | null {
    const currentToken = tokens[currentIndex]

    // Process our current token, and return an AST node

    return null
  }

  const children: ASTNode[] = []

  while (currentIndex < tokens.length) {
    const next = process()

    if (next) {
      children.push(next)
    }
  }

  return {
    type: ASTNodeType.Program,
    children
  }
}

This may look similar to our lexer code. We're iterating over our tokens and processing them one-by-one.

Let's break this down quickly.

function process (): ASTNode | null {
  const currentToken = tokens[currentIndex]

  // Process our current token, and return an AST node

  return null
}

We define a function called visit. This will be called each time we want to visit - or "consume" - a token.

Each token may either return a new AST Node, or nothing.

return {
  type: ASTNodeType.Program,
  children
}

As mentioned in our type descriptions, we always need a root AST node. As we don't define a program 'start' in our DSL, which will be hardcoded as the first node we return.