[WIP] Modular C++20 modules

[sylveon]

This picks back work from #953, bringing modular modules support to C++/WinRT.

This has several advantages over PCH:

• Rebuilds are incremental over a PCH, which rebuilds all or nothing
• Builds are heavily parralelized, as opposed to a PCH that is single-threaded
• Inner loop times are drastically reduced from a PCH in my experience
• The internals are completely unexposed - winrt::impl doesn't even exist when you do import winrt;

Currently, this depends on this fix to build: https://developercommunity.visualstudio.com/t/Error-C3779-using-C20-modules-when-for/10879949

Unfortunately, I have not been able to make it work yet without bringing in the preview MSVC toolchain (which then causes issues with import std;). This issue seems to have finally been fixed, but the above prevents me from fully testing.

Basic usage of winrt::hstring through import winrt; does work though! (test_cpp20 compiles if this is the only test activated)

I've also included #1521 because it has some fixes that help modules (removal of definitions from base_includes, which aren't allowed in the global module fragment).

I'm still posting this as a WIP to let interested people try/look/comment on it.

[DefaultRyan]

This picks back work from #953, bringing modular modules support to C++/WinRT.

This has several advantages over PCH:

• Rebuilds are incremental over a PCH, which rebuilds all or nothing
• Builds are heavily parralelized, as opposed to a PCH that is single-threaded
• Inner loop times are drastically reduced from a PCH in my experience
• The internals are completely unexposed - winrt::impl doesn't even exist when you do import winrt;

Currently, this depends on this fix to build: https://developercommunity.visualstudio.com/t/Error-C3779-using-C20-modules-when-for/10879949

Unfortunately, I have not been able to make it work yet without bringing in the preview MSVC toolchain (which then causes issues with import std;). This issue seems to have finally been fixed, but the above prevents me from fully testing.

Basic usage of winrt::hstring through import winrt; does work though! (test_cpp20 compiles if this is the only test activated)

I've also included #1521 because it has some fixes that help modules (removal of definitions from base_includes, which aren't allowed in the global module fragment).

I'm still posting this as a WIP to let interested people try/look/comment on it.

This is exciting, I'm going to give it a look. (Though I admit, I'm giving myself a crash-course in C++ modules - as in, I've heard about the goodness for years, but haven't actually kicked the tires until very recently)

Just to be sure I understand, when you refer to the need to use the preview toolchain, are you saying that even though the C3779 issue is advertised as fixed in 18.4.0, there is a separate issue that is still blocking us? Is that a separate compiler bug that's being tracked somewhere?

Based on some cursory research, I'm wondering if you are referring to the 14.51 preview toolset? And is it possibly related to this issue https://developercommunity.visualstudio.com/t/ICE-in-VS2026-Insiders-using-c-modules/10977663?sort=recent&viewtype=all?

It might help us to coordinate, track, and test the working state while cutting down on duplicated effort if you could provide some specific toolset version numbers.

[YexuanXiao]

For cppwinrt, the first step toward modularization is to extract all macros into a separate header file. cppwinrt has almost done this, but slim_source_location is placed in the wrong location. The second step is to fix all missing std qualifications and correct anything that is not part of the C++ standard. For example, memcpy_s is not part of the C++ standard and therefore will not be exported by std or std.compat. The third step is to make windows.numeric.impl.h an independent module on its own. This header file is not controlled by cppwinrt and indirectly introduces the standard library and Windows.h, so it must be isolated. The fourth step is to add appropriate macro guard so that the header file can be converted into a module implementation file while also being usable independently as a header file. You can refer to my blog post and the commit history in my repository to see how I achieved this. Additionally, it may be necessary to add inline and extern "C++" to allow both the module and the header file to be used simultaneously. I also recommend implementing at least different modules for different root namespaces, because third-party users, or even the WindowsAppSDK, might establish their own namespaces and should not be confused. I suggest that the module named winrt should only export the declarations in base.h. Finally, modules should not come from macro expansion and modules require that an entity declared in module A must be defined in module A (MSVC currently does not enforce this, but the standard and Clang require it.). Implementing modules in the way described above is actually not difficult.

[sylveon]

Just to be sure I understand, when you refer to the need to use the preview toolchain, are you saying that even though the C3779 issue is advertised as fixed in 18.4.0, there is a separate issue that is still blocking us? Is that a separate compiler bug that's being tracked somewhere?

Based on some cursory research, I'm wondering if you are referring to the 14.51 preview toolset? And is it possibly related to this issue https://developercommunity.visualstudio.com/t/ICE-in-VS2026-Insiders-using-c-modules/10977663?sort=recent&viewtype=all?

The issue is not fixed in the stable toolset on the latest release of VS. The minimum repro code from the developer community post is still broken.

On the preview toolset, the minimum repro code works, however during compilation of the C++/WinRT ixx files, I get the following (which I do not see on stable), which is weird because we don't use stop_token:

Which means I cannot confirm if the C3779 issue has been fixed for us. I've not been able to narrow that issue to a minimum repro either. I've pinged @cdacamar offline to see if he's seen this or can help figuring out the problem.

For cppwinrt, the first step toward modularization is to extract all macros into a separate header file. cppwinrt has almost done this, but slim_source_location is placed in the wrong location.

For this implementation, I moved the macros that are used by the implementation files to a new shared.h header, which are then imported by the implementation files in the global module fragment. Many of the macros used by C++/WinRT are only relevant for base.h, so there is no need to make all of them available to individual implementation files (also means I don't have to deal with slim_source_location 😄).

The second step is to fix all missing std qualifications and correct anything that is not part of the C++ standard. For example, memcpy_s is not part of the C++ standard and therefore will not be exported by std or std.compat

Hilariously, I've not run into that memcpy_s issue, at least not yet. All the ixx files compile just fine. Though it might be exposed by a deeper test of functionality. Swapping it out for memcpy might run afoul of SCI requirements however, I'll need to either include the proper header, or swap it out for e.g. std::copy_n

The third step is to make windows.numeric.impl.h an independent module on its own. This header file is not controlled by cppwinrt and indirectly introduces the standard library and Windows.h, so it must be isolated.

It pulls in a single header, directxmath.h, which is already part of the global module fragment for the base submodule :)

So there is no pollution outside of base thankfully, and it respects the ideal include-then-import order for the STL. Also, it only includes intrinsics and SAL, not the whole of Windows.h.

The fourth step is to add appropriate macro guard so that the header file can be converted into a module implementation file while also being usable independently as a header file.

This has been done in my branch since 2021 :D
The implementation is trivial, we just add WINRT_IMPL_MODULES and guard implementation files to check this before including headers. Though I need to experiment with moving the C++/WinRT version check out of that guard.

Additionally, it may be necessary to add inline and extern "C++" to allow both the module and the header file to be used simultaneously. I also recommend implementing at least different modules for different root namespaces, because third-party users, or even the WindowsAppSDK, might establish their own namespaces and should not be confused. I suggest that the module named winrt should only export the declarations in base.h. Finally, modules should not come from macro expansion and modules require that an entity declared in module A must be defined in module A (MSVC currently does not enforce this, but the standard and Clang require it.)

Some comments here:

• Mixing includes and imports is primarily a problem within the same TU iirc. The heavy nature of C++/WinRT's design makes this less likely to happen than something like the standard library. We can leave it out of an MVP and fix it in subsequent versions if it shows up during adoption.
• I specifically avoid doing different modules because this would require exposing implementation details of C++/WinRT across the module boundary. Keeping everything within the same module (which avoids exposing winrt::impl entirely) is perfectly doable since the source code is entirely generated, and is not plugged into by third party code (undocumented messing around excluded). import winrt; using namespace winrt::Microsoft::WindowsAppSDK; works just fine, as the winrt.ixx file imports the implementation if it was passed to the cppwinrt.exe CLI.
• This implementation does not violate the requirement. An entity declared in submodule A:B can be defined in submodule A:C, as long as the definition ends up visible (export import :C) for consumers of module A. Clang allows this as well: https://godbolt.org/z/Y61W1EcvY

[YexuanXiao]

I guess the reason you don't need to fix a large number of std qualifications and non-standard functions in your implementation is that windows.numeric.impl.h indirectly introduces them, which actually significantly slows down compilation and is also not ideal, see llvm/llvm-project#97239.
After #1521 is merged, I will draft a PR to fix them.

[YexuanXiao]

If you create only one named module winrt, will compiler need to recompile the module after editing their own IDL? I haven’t looked into this, so I suggest verifying it. If different root namespaces use different modules, this issue can be directly avoided. Additionally, I’m not entirely sure how third-party WinRT libraries integrate into the user’s build process, as they may introduce more complex dependencies (they might create general C++ APIs on top of WinRT), which also requires further investigation.

Therefore, I still recommend implementing separate modules for different root namespaces, as this has no drawbacks and avoids the aforementioned issues.

[sylveon]

If you create only one named module winrt, will compiler need to recompile the module after editing their own IDL?

It only rebuilds the affected ixx files.

[YexuanXiao]

It only rebuilds the affected ixx files.

Obviously, modifying the user IDL will recompile the main module winrt, which means that code that only depends on the Windows namespace also needs to be recompiled. Only the unaffected module partitions of winrt do not require recompilation.

[sylveon]

In my implementation, ixx files are split out the same way headers are, so winrt.ixx is just a bunch of export import. The Windows.* module partition are untouched and therefore do not need to be recompiled.

This also means that initial compilation is stupidly parallel, thanks to the compiler's ability to construct a dependency graph of module partitions:

[sylveon]

We might have to do a bit of dancing to get the compiler to agree (e.g. if it only uses timestamp to determine if a rebuild has to be done, we might have to intelligently avoid stomping over unchanged files), but the theory is sound. A full E2E test (which we aren't quite close to yet) will determine if we have to do that.

[YexuanXiao]

I mean, I have code that uses APIs from the Windows namespace, and it does not use APIs from the IDL I wrote. If you only implement a single module winrt, then this code indirectly depends on the IDL I wrote, even if it is not used. This dependency is established through the module winrt, not through module partition. Any modifications to the module partitions of winrt will result in the recompilation of the module winrt, even if it is merely exporting its module partitions. Modifying the IDL will cause this situation. Therefore, when I modify the IDL, any code imports winrt needs to be compiled because the module winrt depends on the modified IDL. The issue I am talking about is not related to recompiling a certain module partition.