We leave the parameter tokens alone, considering it constant, while
the parameter vec_out is used to hold the new stream of tokens. This
allows the caller to have a before and after view on the token stream
and reduces the worry of double frees.
Another great thing for C++: the ability to tell it how to print
structures the way I want. In C it's either:
1) Write a function to print the structure out (preferably to a file
pointer)
2) Write a function to return a string (allocated on the heap) which
represents it
Both are not fun to write, whereas it's much easier to write this.
I've decided to split the module parsing into two modules, one for the
preprocessing stage which only deals with tokens and the parsing stage
which generates bytecode.
