(Re-)Implement C Backend (this time supports callbacks & typed enums)

[yuvatia]

At the company I work for, we use interoptopus to generate C# bindings for a couple of projects and we have been very satisfied with the results, however recently we wanted to generate bindings for a project written in C++ and found the existing (now deprecated) C backend to be extremely lacking. In particular the things that were very apparaent were:

1. Lack of support for callbacks
2. Lack of a nice utility function that resolves all exported functions of the Rust DLL
3. Lack of support for sum types (typed enums)

These gaps motivated me (to motivate Claude) to enhance the backend. Initially this was based on the (now _old) pre-existing C backend, but due to recent structural changes in the repo it was changed to be self-contained.

My intention is to keep this backend relatively maintained (as we plan on using it for production software).

See commit message for detailed overview of the backend.

Some notes

1. The code is mostly AI generated. I tweaked it a bit manually and provided a lot of feedback, but it is still mostly AI-written. From my understanding you're okay with accepting such PRs but lmk if you feel otherwise.
2. This was tested with a real-world project, both on Linux and on Windows).
3. At first I wanted to just reuse the reference project but it does things that are way more complex than what this new backend is capable of at the moment, so I wrote a different reference project that's pretty dumb compared to the one used by the C# backend, but it covers all the gaps mentioned above.
4. Test code is written in C++ and uses GTest since that's the testing framework I'm more comfortable with (my background is in C++ much moreso than in C), but lmk if you prefer a different framework.
5. I'm using CMake. Some people have strong feelings towards CMake, so lmk if you prefer a different build system.

C header generator (crates/backend_c/)

New C header generator built against the 0.16 type system. Organized as three modules: lib.rs (public Generator API with builder pattern), codegen.rs (all emission routines), and topo.rs (deterministic topological sort of types by name before dependency traversal, ensuring stable output across runs).

The generator produces a single .h file containing:

• Type definitions: structs, simple enums, and tagged unions. Rust enums with data payloads emit a C11 tag enum + struct with anonymous union (e.g. SHAPE_TAG enum + SHAPE struct with tag and union fields). This also covers ffi::Option<T> and ffi::Result<T, E> which are internally represented as enums with typed variants.

• Callbacks: each callback! type emits a NAME_fn function pointer typedef (with an implicit trailing const void* context parameter) and a NAME struct containing three fields — callback (the fn pointer), data (context pointer), and destructor (optional cleanup). This matches the Rust #[repr(C)] layout so callbacks round-trip correctly across the FFI boundary.

• Dispatch table: a {name}_api_t struct with one function pointer field per exported function, where {name} is caller-specified (e.g. reference_project_c_api_t). A /* internal helpers */ comment separator is emitted before any interoptopus_* builtin functions (from builtins_string!/builtins_vec!) to visually distinguish user APIs from internal helpers.

• Dynamic loader: a cross-platform {name}_load(path, api) function. The POSIX implementation uses dlopen/dlsym with memcpy to transfer void* into function pointer fields (avoiding the ISO C prohibition on direct void*-to-function-pointer casts, which triggers warnings under -Wpedantic). The Windows implementation converts the UTF-8 path to UTF-16 via MultiByteToWideChar(CP_UTF8, ...) and loads with LoadLibraryW/GetProcAddress. Both validate every symbol and return -1 on failure.

• Static loader: a {name}_load_static(api) function (behind an #ifdef guard) that assigns the forward-declared symbols directly, for use when statically linking the Rust library.

reference_project_c (crates/backend_c/reference_project/)

Comprehensive FFI example exercising all supported patterns: structs, tagged union enums, slices, vecs, options, strings, callbacks (with Shape, Slice, Option, and Vec parameters), and a KitchenSink struct that combines all major FFI types (u64, bool, f64, ffi::String, tagged enum, ffi::Option, ffi::Slice, ffi::Optionffi::String). Exports a public inventory() function consumed by the backend's integration test. Mirrors the role of crates/reference_project/ for the C# backend.

Test infrastructure

• crates/backend_c/tests/: Rust integration test generates the C header from reference_project_c's inventory. C++20 gtest suite (10 tests) under reference_project/ loads the Rust cdylib and validates all FFI types with proper assertions. Uses CMake + FetchContent for gtest. The generated header is gitignored (regenerated by cargo test).

• examples/hello_world/: added a second binding generation test for C (alongside the existing C# one), plus a simple C++20 gtest (2 tests) that validates the Vec2/my_function roundtrip. Renamed bindings/ to bindings_csharp/ for clarity alongside bindings_c/. Also fixed the existing C# Xunit test to only reference types in the inventory and updated the target framework to net10.0. Generated headers are gitignored.

• Justfile: just test-c runs both C++ test suites via a shared _test_c helper (cmake configure/build/ctest with RUST_LIB_DIR). just test-dotnet now also runs the hello_world Xunit test. Both are wired into just ci.

[ralfbiedert]

Hi, thanks for the PR, and thanks a lot for looking after the C backend.

Yes, AI assisted PRs are generally fine if there is sufficient 'human in the loop'. This PR has a few issues though:

Most importantly, the C backend (any backend now) should be 'model + pass' based. Essentially that means each backend should

• First, define a understandable model how the target language works (see the C# backend lang module). Essentially that means defining a taxonomy what items (functions, types, enums, ...) should be observable in its output.
• Next, define one or more model passes (again, see C#) that transform core::Inventory items into its own model. This might mean mapping types, filtering out things that won't work, registering helper types, ...
• Lastly, define one more output passes that (gradually) build up fragments (e.g, type definitions, enum definitions, ...)

The overall structure should be vaguely organized like in the C# backend, in particular the lang items, passes and pipelines (pipelines glue the passes together).

The passes should also be configurable (this might include options about naming conventions, etc.), and setting these config options should again follow how the C# pipeline builders do it.

Most importantly, new backends must use tera templates like the C# backend does it. The whole indented!() we had earlier, and the writeln!() in here, is a total maintenance nightmare.

Testing should also vaguely follow C#. There should be multiple emission / insta snapshot tests for various reference project parts. Neither tests nor codegen should include C++ constructs (for the backend_c), unless its minimal and optional.

[yuvatia]

Thank you for your feedback, will address those comments and update the PR in a few days.

[yuvatia]

I'm debating adding a {{load_fn}}W or something that just takes a PCWSTR to avoid these conversions, wdyt?

[yuvatia]

I updated the PR to address all your comments, and also addressed a few gaps such as missing benchmarks (runnable with just bench-c) + some functionality that was lost from the older backend that I liked (being able to choose between snake_case/camelCase/PascalCase).

[ralfbiedert]

Sorry, but this still has most issues of the C backend flow and structure being vastly different from the C# one.

I recommend you (as a human) look at the C# backend and get a feeling for it, there's too much detail to write everything down item-by-item. While some internals have been agent created (e.g., overload emission and proc macros), the overall backend flow is 'hand-designed' and should be easy to follow (if it isn't that would be an issue).

One bigger thing though I missed the first time, the reliance on cmake, which is often a PITA to set up on platforms that don't ship it out of the box. It's a bit of a weak argument given we rely on dotnet already (and I'm open to discussions how to address that), but I'd be good to explore options (maybe, using cc?) to minimize setup pain.