Though practically this would work, as the storage for the half word is
not limited in any way, nevertheless it isn't syntactically right and
it's better to fix now.
I've flip flopped a bit on this but I believe the virtual machine
bytecode format must have a convention on endianness. This is because
of the issue stated in the TODO which may very well happen.
We have distinct functions for the use blocks and the macro blocks,
which each generate wholesale new token streams via `token_copy` so we
don't run into weird errors around ownership of the internal strings
of each token.
Furthermore, process_presults now uses the stream index in each
presult to report errors when stuff goes wrong.
So when a presult_t is constructed it holds an index to where it was
constructed in terms of the token stream. This will be useful when
implementing an error checker in the preprocessing or result parsing
stages.
So %USE <STRING> is the expected call pattern, so there's an error if
there isn't a string after %USE.
The other two errors are file I/O errors i.e. nonexistent files or
errors in parsing the other file. We don't report specifics about the
other file, that should be up to the user to check themselves.
This is different to "%CONST" in that it can take token parameters and
use them. This allows the construction of user code at compile time,
which can be very useful for a variety of use cases.
Essentially importing another file *literally* into the file. This
would happen before parse results are gathered, similar to how
"%CONST" is implemented currently.
Preprocessor handles macros and macro blocks by working at the token
level, not doing any high level parsing or instruction making.
Essentially every macro is recorded in a registry, recording the name
and the tokens assigned to it. Then for every caller it just inserts
the tokens inline, creating a new stream and freeing the old one. It
leaves actual high level parsing to `parse_next` and
`process_presults`.
When an error occurred, because prog->ptr was incremented beforehand
the trace would show the next instruction as the culprit rather than
the actual instruction. This commit fixes that by incrementing the
program if and only if the command was run successfully.
Looks way more high level but parses down to a very simple bytecode.
However, because of lack of inline code processing, it relies on the
call stack quite heavily. With inline labels this would be a much
more compact bytecode.
