feat: Add regex_replace Jinja filter for low-code connector builder

[Lucas Leadbetter (lleadbet)]

Summary

Adds a regex_replace Jinja filter to the low-code connector builder's interpolation system, complementing the existing regex_search filter. This wraps Python's re.sub() and enables regex-based string replacement in declarative manifests:

value: "{{ record['some_field'] | regex_replace('[0-9]+', '') }}"

Closes #902

Review & Testing Checklist for Human

• Verify that no error handling is needed for invalid regex patterns (current behavior matches regex_search — both will raise re.error on bad patterns). Decide if that's acceptable or if a try/except should be added.
• Consider whether this filter should be tested end-to-end within a declarative manifest context (e.g. via ManifestDeclarativeSource) rather than only at the interpolation level.

Notes

• Requested by Lucas Leadbetter (@lleadbet)
• Devin session

Summary by CodeRabbit

Release Notes

• New Features

  • Added regex pattern matching and replacement functionality for advanced text transformation in declarative workflows.

• Tests

  • Added comprehensive test coverage for regex pattern replacement scenarios.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1770835380-add-regex-replace-filter#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1770835380-add-regex-replace-filter

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[coderabbitai]

📝 Walkthrough

Walkthrough

A new regex_replace filter is added to the declarative interpolation system, enabling regex-based string substitution via re.sub(). The filter is exported and available for use in Jinja templates, with comprehensive test coverage validating various replacement scenarios.

Changes

Cohort / File(s)	Summary
Filter Implementation airbyte_cdk/sources/declarative/interpolation/filters.py	Added regex_replace(value: str, regex: str, replacement: str) function wrapping re.sub() for regex-based substitution; registered in _filters_list for export.
Test Coverage unit_tests/sources/declarative/interpolation/test_filters.py	Added parametrized test_regex_replace() covering basic replacement, digit removal, no-match scenarios, and multiple occurrences.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding a regex_replace Jinja filter for the low-code connector builder, which is the primary focus of the PR.
Linked Issues check	✅ Passed	The PR successfully implements all coding objectives from issue #902: adds regex_replace function wrapping re.sub(), registers it in _filters_list, and includes comprehensive test coverage for various replacement scenarios.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to issue #902: the regex_replace function implementation in filters.py and its corresponding test coverage in test_filters.py, with no extraneous modifications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch devin/1770835380-add-regex-replace-filter

---

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments

airbyte_cdk/sources/declarative/interpolation/filters.py (1)

139-143: Implementation looks clean and consistent with regex_search 👍

Two small thoughts:

1. The other filters in this file include usage examples in their docstrings (e.g., showing a YAML snippet). Would it be worth adding a brief example here too for consistency, something like the one from the PR description? Just to help future readers, wdyt?

2. If value happens to be a non-string (e.g., an integer from a record), re.sub will raise a TypeError. The existing regex_search has the same gap, so this is consistent — but if you wanted to be a bit more defensive, you could wrap value with str(value). Totally optional since it matches current behavior.

Optional: add a usage example and str coercion

 def regex_replace(value: str, regex: str, replacement: str) -> str:
     """
     Replace all occurrences of a regular expression pattern in a string.
+
+    For example:
+
+      transformations:
+        - type: AddFields
+          fields:
+            - path: ["cleaned_field"]
+              value: "{{ record['some_field'] | regex_replace('[0-9]+', '') }}"
+
+    :param value: the input string to perform replacements on
+    :param regex: the regular expression pattern to match
+    :param replacement: the replacement string
+    :return: the string with all matches replaced
     """
-    return re.sub(regex, replacement, value)
+    return re.sub(regex, replacement, str(value))

unit_tests/sources/declarative/interpolation/test_filters.py (1)

111-138: Nice parametrized coverage for the happy paths!

Would you consider adding a test case for an invalid regex pattern? Since the PR description mentions that invalid patterns currently raise re.error (matching regex_search behavior), having an explicit test would document that contract and catch any future changes. Something like:

def test_regex_replace_invalid_pattern() -> None:
    expression = "{{ 'hello' | regex_replace('[invalid', 'x') }}"
    with pytest.raises(Exception):
        interpolation.eval(expression, config={})

Also, a test for capture group backreferences (e.g., regex_replace('(\\w+) (\\w+)', '\\2 \\1')) could be a nice addition since that's a common re.sub use case. But totally up to you — wdyt?

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PyTest Results (Fast)

3 860 tests  +3 187   3 848 ✅ +3 187   6m 23s ⏱️ + 2m 39s
    1 suites ±    0      12 💤 +    1 
    1 files   ±    0       0 ❌  -     1 

Results for commit 3540a52. ± Comparison against base commit 796bb34.

[github-actions]

PyTest Results (Full)

3 863 tests  +4   3 851 ✅ +4   10m 57s ⏱️ +6s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 3540a52. ± Comparison against base commit 796bb34.