Skip to content

source-postgres: field type mapping#65983

Merged
Matt Bayley (mwbayley) merged 3 commits intosource-postgres/bulk-cdkfrom
mbayley/source-postgres/field-type-tests
Sep 16, 2025
Merged

source-postgres: field type mapping#65983
Matt Bayley (mwbayley) merged 3 commits intosource-postgres/bulk-cdkfrom
mbayley/source-postgres/field-type-tests

Conversation

@mwbayley
Copy link
Contributor

@mwbayley Matt Bayley (mwbayley) commented Sep 8, 2025

Add field type mapping and associated tests based on the testing framework to the postgres source on the bulk CDK.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
  • /poe source example lock - Alias for /poe connector source-example lock.
  • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
  • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-clickhouse Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-s3 Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-customer-io Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-dev-null Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-mssql Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

source-mysql Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

source-postgres Connector Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@mwbayley Matt Bayley (mwbayley) force-pushed the mbayley/source-postgres/field-type-tests branch from 40cf841 to de8cc06 Compare September 11, 2025 22:47
}
}

class InfJsonEncoder<T>(private val base: JsonEncoder<T>) : JsonEncoder<InfWrapper<T>> {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're using the type of JsonEncoder as a hint for protobuf encoding, this may create a difficulty because the inner type is erased.
Maybe reified can help here.
I'm noting this as a difficulty to myself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inline and reified are new to me. Good idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just played around with this a bit. Maybe we can pair on it in person later this week.

JDBCType.CHAR,
JDBCType.VARCHAR,
// JDBCType.TINYTEXT,
// JDBCType.TEXT,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all these don't exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, forgot to clean this up. Commit incoming.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait, I have this removed locally and the sha on the tip commit matches.... something weird is going on

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, the commented ones are all in red here.... duh. Yes, these JDBC types are not present in column metadata for Postgres.

object PostgresSource {
@JvmStatic
fun main(args: Array<String>) {
// TODO: Remove before merging

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh 🤦

@mwbayley Matt Bayley (mwbayley) merged this pull request into source-postgres/bulk-cdk Sep 16, 2025
30 of 50 checks passed
@mwbayley Matt Bayley (mwbayley) deleted the mbayley/source-postgres/field-type-tests branch September 16, 2025 00:06
Matt Bayley (mwbayley) added a commit that referenced this pull request Sep 16, 2025
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants