Skip to content

build: Add Cython compilation support for wheel obfuscation #979

@devin-ai-integration

Description

@devin-ai-integration

Summary

Add Cython compilation to the PyAirbyte build pipeline so that PyPI-published wheels contain compiled .so/.pyd extensions instead of readable .py source files, providing basic code obfuscation for distribution from the private repo.

Context

This pattern has been prototyped in the airbyte-ops-mcp and connector-builder-mcp repos:

The approach uses:

  • hatch-cythonize build hook (maintained fork of hatch-cython) to compile .py.c.so during wheel builds
  • Build-time injection of Cython config (not stored in pyproject.toml) to avoid breaking uv sync / editable installs
  • Post-build source stripping to remove .py files from wheels where compiled .so counterparts exist
  • Exclusion of modules that break under Cython compilation (e.g., MCP/Pydantic presentation layers)

Key Learnings from Ops MCP Implementation

  1. Cython config must NOT be in pyproject.toml — it runs during editable installs (uv sync), breaking CI workflows. Solution: inject at build time via a script.
  2. Pydantic/FastMCP modules must be excluded — Cython-compiled functions (cyfunction) don't expose type annotations the same way, breaking Pydantic schema generation.
  3. __init__.py files must be excluded — required for Python package structure.
  4. Platform-specific wheels — must build per OS (Linux, macOS) and Python version (3.10, 3.11, 3.12).

Action Items

  • Evaluate whether this pattern should be applied to PyAirbyte
  • If yes, adapt the enable_cython_build.py and strip_source_from_wheel.py scripts for PyAirbyte's package structure
  • Identify which modules (if any) need to be excluded from compilation
  • Integrate with existing publish/release workflows

Devin session: https://app.devin.ai/sessions/7c8622f9452141d1bbad93f74079278f
Requested by: Aaron ("AJ") Steers (@aaronsteers)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions