Skip to content

Conversation

@freeznet
Copy link
Member

@freeznet freeznet commented Nov 4, 2025

(If this PR fixes a github issue, please add Fixes #<xyz>.)

Fixes #

(or if this PR is one task of a github issue, please add Master Issue: #<xyz> to link to the master issue.)

Master Issue: #

Motivation

Explain here the context, and why you're making that change. What is the problem you're trying to solve.

Modifications

Describe the modifications you've done.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

Check the box below.

Need to update docs?

  • doc-required

    (If you need help on updating docs, create a doc issue)

  • no-need-doc

    (Please explain why)

  • doc

    (If this PR contains doc changes)

@freeznet freeznet self-assigned this Nov 4, 2025
@github-actions
Copy link

github-actions bot commented Nov 4, 2025

@freeznet:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@github-actions github-actions bot added the doc-info-missing This pr needs to mark a document option in description label Nov 4, 2025
@freeznet freeznet marked this pull request as ready for review November 9, 2025 15:24
@freeznet freeznet requested review from a team and nlu90 as code owners November 9, 2025 15:24
@freeznet freeznet requested a review from Copilot November 9, 2025 15:24
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for source and sink connectors in Pulsar functions, enabling functions to use external connectors (e.g., Kafka, Kinesis) as inputs/outputs instead of traditional Pulsar topics. It also adds configurability for Java instance runtime parameters.

Key changes:

  • Added SourceConnectorSpec and SinkConnectorSpec types with fields for connector configuration including className, configs, and type information
  • Extended validation logic to skip input/output topic validation when connectors are configured
  • Added logic to inject connector configuration into function specs and command-line arguments
  • Added instancePath and entryClass fields to JavaRuntime for runtime customization

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated no comments.

Show a summary per file
File Description
api/compute/v1alpha1/function_types.go Defines new SourceConnectorSpec and SinkConnectorSpec types
api/compute/v1alpha1/common.go Adds InstancePath and EntryClass fields to JavaRuntime
controllers/spec/utils.go Implements connector config extraction and application logic
controllers/spec/function.go Updates function command generation to include connector directory
controllers/spec/source.go, sink.go Updates source/sink command generation with instance path/entry class
controllers/spec/common.go Updates MakeJavaFunctionCommand signature with connector directory and instance parameters
pkg/webhook/validate.go Adds skip flags to validateInputOutput for connector scenarios
pkg/webhook/function_webhook.go Sets skip flags based on connector presence
pkg/webhook/source_webhook.go, sink_webhook.go Updates validation calls with skip flags
config/crd/bases/*.yaml Updates CRD schemas with new fields
manifests/*.yaml Regenerated manifests with new CRD schemas
controllers/spec/utils_test.go, function_test.go Adds test coverage for connector functionality

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@freeznet freeznet merged commit 6a4466a into master Nov 10, 2025
16 of 18 checks passed
@freeznet freeznet deleted the freeznet/make-kafka-messaging-support branch November 10, 2025 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-info-missing This pr needs to mark a document option in description

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants