Previously, the tokenizer had horrendous errors when lexing raw literal
strings. These have been removed. The remaining issue still persists,
that empty doc comments are serialized in c in a _weird_ way. They
should be merged.
The rust based test were both not sufficient to cover all edge-cases and
so unmaintainable that they nearly always didn't even compile. This new
test framework should alleviate both concerns.
However, one big problem still remains: it does not support test cases
that should fail, so these have just been left in the `./tests`
directory.
Benedikt Peetz