```c
/**
comment one
*/
/**
comment two
*/
/**
comment three
*/
```
is just not as readable as:
```c
/**
* comment one
* comment two
* comment three
*/
```
These would still parse and would generate _stuff_, but the generated
stuff is just not ideal yet. So better not accept them, until their
internal support has matured.
Previously, the tokenizer had horrendous errors when lexing raw literal
strings. These have been removed. The remaining issue still persists,
that empty doc comments are serialized in c in a _weird_ way. They
should be merged.
The rust based test were both not sufficient to cover all edge-cases and
so unmaintainable that they nearly always didn't even compile. This new
test framework should alleviate both concerns.
However, one big problem still remains: it does not support test cases
that should fail, so these have just been left in the `./tests`
directory.
This might be a big diff, but I _hope_ that it does not change much
functionally (hopefully it changes nothing generation specific).
What has changed?
-----------------
- I had to merge the three crates into one, to allow `macros` to impl
functions on Types defined in `parser`.
- As mentioned in the point above, the conversion function are now
inherent to the type they convert (i. e. `r#type.to_rust()` instead of
`type_to_rust(r#type)`).
- The conversion function have been sorted, by what they convert to:
- `to_rust()` converts a type to be used in rust *host* code.
- `to_c()` converts a type to be used in c *host* code.
- `to_auxiliary_c()` converts a type to be used in c *auxiliary*
code.
- The top-most `generate` method of `TrixyConfig` now returns a
`FileTree` instead of writing the files directly. The `FileTree` can
still be materialize with the `materialize` method. But this change
facilitates moving non-generation focused code out of the `generate`
method.
What is the difference between _host_ and _auxiliary_ code?
-----------------------------------------------------------
Auxiliary code is always written in the language it is generated for. So
auxiliary code for c would be written in c (or at least in a subset of c),
as it represents c header files.
Host code is always written in rust. This is the code that is
responsible for implementing the actual ffi stuff. In our case these are
the `extern "C"` functions and the types, defined in trixy.
The rust host code is generating the native rust representations of
these types.
These are implemented right now by simply casting the generic arguments
to void pointers and providing a `type_id` field in every struct
denoting the original type.
This implementation, whilst being extremely unwieldy to work with on
the c side, also fails in a lot of fundamental ways:
1. The `type_id` enum *can* never really support user defined
types because we would already need it to provide the c to rust
value conversion.
2. Even without custom user types the type conversion is extremely
hard to correctly implement in a somewhat performant way: A vector
passed from c code to rust would need to completely reallocated
*one element at a time*. And this only works if the c side has
correctly cast the void pointer to the vectors data before accessing
it, as any other way would have lead to possible unaligned data
(which the rust side had to account for).
3. The c api is just simply bad in this state:
You have to always look at the Trixy file to even be able to deal
with the data the api returns (that is: There is no mention of
a results generics in the c header). Additionally the question
arises if these types should even be leaked into the c code because
than c just becomes a worse version of rust, which undermines the
whole reason of providing a c api in the first place.
One way to fix all these issues would be to change the way generics are
handled by using unions instead of the void pointer and trying to avoid
leaking these rust types in c as far as possible.
This approach would require a lot less binding code (both on the c and
rust side), but also would make the rust based c-header-gen-code harder,
as it would then be required to turn a `Vec<String>` to a `char **` (and
obviously a whole wrapper struct with size and string length), whilst
turning a `Vec<char>` to a `char*` differentiating it from a `string_t`.
Benedikt Peetz