Add MSVC and GCC support & add+improve CI Coverage (incl DEBUG runs)

[peterboncz]

Summary

Port FastLanes to compile and pass all tests on GCC Linux and MSVC (Windows x86-64 and ARM64), including shared library (DLL) builds. Also overhaul the CI matrix to cut job count from ~167 to ~50 per PR.

MSVC and GCC Compiler Support

• New compiler.hpp portability header centralizing compiler-specific macros (vectorize pragmas, __builtin_clz/ctz, diagnostic push/pop)
• CMakeLists.txt: accept MSVC alongside Clang, with appropriate flag mappings (/bigobj, /MP, /Zc:__cplusplus)
• Replace ~700 #pragma clang loop vectorize(enable) with FLS_PRAGMA_VECTORIZE
• Replace #pragma GCC/clang diagnostic with portable FLS_DIAG_* macros
• Add Windows implementation for memory_usage.cpp (GetProcessMemoryInfo)
• Add MSVC compat shims to fsst12.h (matching existing fsst.h pattern)
• Guard Clang-only -Wno-macro-redefined flags in 8 subdirectory CMakeLists
• Split interpreter.cpp into encoding/decoding files to reduce MSVC compile time
• Fix C++20 structured binding in range-for (unsupported by MSVC)
• Add msvc_heap_guard test helper to handle SEH guard-page exceptions in GoogleTest

GCC compiler support was an easy add after going through this. We only support gcc on Linux, because it is much harder to use on MacOS and msvc is the preferred Windows compiler. Note that clang works on all three platforms still.

Explicit Symbol Export

we hide all symbols by default in the shared libraries and export the API symbols -- required vor DLLs

• FLS_API macro (__declspec(dllexport/dllimport)) on all public classes and functions
• Explicit = delete of copy ops on classes with unique_ptr members (MSVC eagerly instantiates all special members for dllexport classes)
• Out-of-line destructors where MSVC needs complete types for unique_ptr in dllexport context
• FLS_STATIC propagated as PUBLIC compile definition on the FastLanes target so consumers (tests, examples) link correctly against the static library

Bug Fixes

while it is easy, popular (and justified) to hate on Windows, it does have another independent software stack with strong asan tools and debugger, which unearthed quite some bugs. Also, running the CI tests on debug put the system through more asan testing, exposing some more. In order to get working CI across compilers, platforms, Debug/Release, shared/static quite some issues needed to be fixed to get to green CI on this PR:

1. MSVC portability bug caused by 32-bit unsigned long on Windows (1)
   std::stol/stoul overflow: Values that fit in 64 bits but exceed 32-bit range threw out_of_range. Fixed by using std::stoll/stoull in attribute.cpp and rowgroup.cpp.
2. MSVC portability bug caused by 32-bit unsigned long on Windows (2)
   FSST12 symbol table cast: fsst12.h cast the symbol table to unsigned long*, giving a 4-byte stride on Windows instead of 8. This made symbol lookups read wrong offsets — producing correct-length but
   garbled strings. Fixed by casting to unsigned long long*.
3. FSST12 buffer overrun, as it did a 64-bits wrote from &res (uint32). Only the first 1-3 bytes mattered, but this a buffer overrun and a 4-byte write should be done iso a 8-byte. All FSST12 fixes got ported back to the fsst repo.
4. 2GB reserve in string decoders
   The 8 string decode operators called vector::reserve(1024 * 2MB) = 2GB. Linux overcommits virtual memory so this silently succeeds; Windows does not, throwing bad_alloc. Fixed with an amortized doubling growth strategy.
5. GTest SEH guard-page clash
   When tests process large data (~80MB JPEG base64), the Windows heap allocator touches reserved-but-uncommitted guard pages, triggering a transient STATUS_ACCESS_VIOLATION (0xc0000005). Normally the NT heap manager's own vectored exception handler (VEH) commits the page and resumes transparently. But GoogleTest wraps every TEST body in an SEH __try/__except frame, and SEH frames fire after VEH handlers return EXCEPTION_CONTINUE_SEARCH — so GoogleTest's catch-all __except intercepts the fault first and reports a spurious failure. Fix: a custom first-chance VEH handler (msvc_heap_guard.cpp) that detects read-AVs on committable pages, calls VirtualAlloc(MEM_COMMIT) on the faulting page, and resumes execution — doing exactly what the heap manager would have done.
6. fill_in undefined behavior
   The string column padding loop called push_back() on byte_arr while simultaneously indexing into the same vector to read the last value's bytes. When push_back triggers reallocation, the indices into the old
   buffer become dangling — undefined behavior. On Linux this happened to work (the allocator often didn't relocate), but MSVC's debug CRT caught it. Fix: copy the last value into a separate vector<uint8_t> before the loop, then insert() from that copy.
7. Fix GALP null deref
   make_dec_galp_expr() called column_view.column_descriptor.encoding_rpn()->operand_tokens()->size() to set state.cur_operand. But GALP/ALP columns never emit operand tokens during encoding, so operand_tokens()
   returns null — dereferencing it is a null pointer crash. The fix was simply removing that line, since dec_alp_opr uses hardcoded segment indices and never reads cur_operand.
8. ALP factor_idx out-of-bounds (encoder.cpp)
   In find_top_k_combinations(), the inner loop initialized factor_idx = exponent_idx and iterated down to 0. For float, MAX_EXPONENT is 10 but FACT_ARR has only 10 elements (indices 0-9). When exponent_idx == 10, factor_idx started at 10 — an out-of-bounds access into FACT_ARR[10]. Fix: clamp factor_idx to min(exponent_idx, FACT_ARR.size() - 1). For double this was harmless (MAX_EXPONENT=18, FACT_ARR has 19 elements), but for float it was reading one past the end. MSVC's debug iterator checks caught this.
9. Null map not always allocated (rowgroup.cpp)
   TypedColumnView and NullMapView called .data() on null_map_arr without checking if the vector was empty. Columns without nulls have an empty null_map_arr, so .data() returns nullptr. Downstream code then dereferenced this null pointer. Fix: when null_map_arr is empty, point to a static zero_null_map[65536] array of zeroes (meaning "no nulls") instead.
10. Signed/unsigned conversion warnings in string decoders
    The amortized growth code in all 8 string decode operators called std::max(capacity * 2, size + CFG::String::max_bytes_per_string) where size is size_t (unsigned) but max_bytes_per_string is a signed type. MSVC treats this as a narrowing conversion error. Fix: add static_cast<size_t>(CFG::String::max_bytes_per_string) in all 8 operators.
11. Debug test failure (NextiaJD_Chicago_Crimes_2012_to_2017 Failed: Negative index).
    When the analyzer couldn't find a good compression option (all options had ≥ 20 exceptions), it fell through with the default bw = 64. Then in is_exception(), pow2(64) returned 0 due to the overflow guard (64 >= sizeof(T)*8), making real_upper = base + 0 = 0. This caused val >= 0 to be true for every value, so all 1024 values in the vector were flagged as exceptions. The count of 1024 was written to the file, and on decode FLS_ASSERT_CORRECT_POS correctly rejected it since 1024 > 1023. The fix: Early-return false in is_exception() when bw covers the full type width — by definition, no value can be an exception if you're using all the bits.
12. Debug test failure due to out-of-bounds read (128*8=1024 is clearly out of bounds of the vector size), which gets optimized away in Release (fallback_scalar_aav_1024_uf1_unrsum_src.cpp)
13. ODR violation — two Histogram classes in namespace fastlanes with different layouts
    Renamed the one in analyze_operator.hpp to AnalyzeHistogram4 files: header, impl, .cpp, plus rust vendor copies)

CI Overhaul

• Eliminate push+PR duplication: all workflows now trigger on push: [main, dev] + pull_request: instead of bare push: (was running every job twice)
• Clang-tidy on 1 platform: tidy diagnostics are platform-independent; run on ubuntu-24.04 only, other platforms get lightweight compile-checks
• Reduce shared/static cross-product: static everywhere, shared on one representative per OS family
• Benchmarks: reduced from 2 platforms to ubuntu-latest only
• Dataset generation: reduced from 6 platforms to ubuntu-latest only
• Compiler diversity in tests: GCC on ubuntu-24.04 + ubuntu-24.04-arm, Clang on ubuntu-22.04 + ubuntu-22.04-arm, Clang on macOS, MSVC on Windows. Avoid gcc Debug on ubuntu-arm though, because it is slow and times out while still compiling.

Accelerating Compiles by Splitting

Getting CI to run uncovered another issue: the long compilation times in the interpreter, where hundreds of templates need to be instantiated. This caused in CI timeouts of the Debug compiles. This in turn probably led to Debug CI builds being switched off. And that led to problems getting undetected, because e.g. ASAN and ASSERTs would not get tested. To reduce compilation time, these steps were made:

• split the interpreter.cpp which instantiated hundreds of templates implicitly, in two source files (encoder and decoder).
• split the biggest operators (rsum, slpatch, analyze)
• further split the visit state in a pair of states <enc,dec> and split on that in all visitors for encoding resp. decoding.

this improved compilation time considerably -- but we still have to avoid ubuntu-clang-Debug-static CI as even with all these improvements compilation times out on both arm and x86 (but non-ubuntu, and shared Debug builds do run correctly now).

Motivation

Broader compiler support is not only nice for users with motives not to use clang (which has best auto-vectorization), but also enables vcpkg packaging and is required for the future DuckDB extension for FastLanes, which needs native Windows support. The improvements in compile times and (re-)enabling of Debug testing widens CI coverage and brought to light lingering bugs.