Skip to content

feat(proto): generate generic GRPC API implementations#1950

Open
kkovaacs wants to merge 26 commits intonextfrom
krisztian/grpc-servers
Open

feat(proto): generate generic GRPC API implementations#1950
kkovaacs wants to merge 26 commits intonextfrom
krisztian/grpc-servers

Conversation

@kkovaacs
Copy link
Copy Markdown
Contributor

This PR adds the build-time generation of generic GRPC API implementations for all services discussed in PR #1742.

For each method we're generating a trait like this:

#[tonic::async_trait]
pub trait Status {
    type Input;
    type Output;

    fn decode(request: ()) -> tonic::Result<Self::Input>;

    fn encode(output: Self::Output) -> tonic::Result<crate::generated::validator::ValidatorStatus>;

    async fn handle(&self, input: Self::Input) -> tonic::Result<Self::Output>;

    async fn full(
        &self,
        request: (),
    ) -> tonic::Result<crate::generated::validator::ValidatorStatus> {
        let input = Self::decode(request)?;
        let output = self.handle(input).await?;
        Self::encode(output)
    }
}

The idea is that the actual implementation should consist of implementing all per-method traits for the API server (most of the time just decode / encode / handle), and then the provided generic implementation for the API can be used:

pub trait ApiService: Status + SubmitProvenTransaction + SignBlock {}

impl<T> ApiService for T
where
    T: Status,
    T: SubmitProvenTransaction,
    T: SignBlock,
{
}

#[tonic::async_trait]
impl<T> crate::generated::validator::api_server::Api for T
where
    T: ApiService,
    T: Send,
    T: Sync,
    T: 'static,
{
    async fn status(
        &self,
        request: tonic::Request<()>,
    ) -> tonic::Result<tonic::Response<crate::generated::validator::ValidatorStatus>> {
        #[allow(clippy::unit_arg)]
        <T as Status>::full(self, request.into_inner()).await.map(tonic::Response::new)
    }

    async fn submit_proven_transaction(
        &self,
        request: tonic::Request<crate::generated::transaction::ProvenTransaction>,
    ) -> tonic::Result<tonic::Response<()>> {
        #[allow(clippy::unit_arg)]
        <T as SubmitProvenTransaction>::full(self, request.into_inner())
            .await
            .map(tonic::Response::new)
    }

    async fn sign_block(
        &self,
        request: tonic::Request<crate::generated::blockchain::ProposedBlock>,
    ) -> tonic::Result<tonic::Response<crate::generated::blockchain::BlockSignature>> {
        #[allow(clippy::unit_arg)]
        <T as SignBlock>::full(self, request.into_inner())
            .await
            .map(tonic::Response::new)
    }
}

In this form this PR should be a no-op, because the none of our API implementations have been moved to this new model. That will be done in multiple follow-up PRs.

@kkovaacs kkovaacs added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label Apr 16, 2026
@kkovaacs kkovaacs marked this pull request as ready for review April 16, 2026 11:29
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the crates/proto build-time code generation to also emit generic, per-RPC-method server-side traits and blanket tonic server implementations, aiming to standardize gRPC server implementations across all services referenced in PR #1742.

Changes:

  • Update crates/proto/build.rs to generate additional server facade modules from protobuf descriptors (plus regenerate mod.rs files).
  • Add build-time dependencies needed for code generation and descriptor introspection (codegen, prost-types).
  • Run rustfmt over generated Rust sources at build time.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File Description
crates/proto/build.rs Adds server module generation from descriptors, new mod.rs generation logic, and build-time rustfmt pass.
crates/proto/Cargo.toml Adds build dependencies (codegen, prost-types) needed by build.rs.
Cargo.toml Pins prost-types to align with pinned prost versions in the workspace.
Cargo.lock Locks new dependency resolutions for codegen and prost-types.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/proto/build.rs Outdated
Comment thread crates/proto/build.rs
Comment thread crates/proto/build.rs
Comment thread crates/proto/build.rs
Base automatically changed from krisztian/proto-use-codegen-for-descriptor-generation to next April 16, 2026 19:23
Each FileDescriptorSet includes transitive imports, the same service
(e.g. rpc.Api) will appear in multiple sets and the corresponding
{module_name}.rs will be regenerated/overwritten multiple times.

This commit fixes this.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/proto/build.rs Outdated
Comment thread crates/proto/build.rs
Comment thread crates/proto/build.rs
kkovaacs and others added 5 commits April 17, 2026 11:52
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@kkovaacs
Copy link
Copy Markdown
Contributor Author

I've added code generation for server streaming. The traits we're generating for these look like this (using the mempool subscription as an example):

#[tonic::async_trait]
pub trait MempoolSubscription {
    type Input;
    type Item;
    type ItemStream: tonic::codegen::tokio_stream::Stream<Item = tonic::Result<Self::Item>>
        + Send
        + 'static;

    fn decode(request: ()) -> tonic::Result<Self::Input>;

    fn encode(item: Self::Item) -> tonic::Result<crate::generated::block_producer::MempoolEvent>;

    async fn handle(&self, input: Self::Input) -> tonic::Result<Self::ItemStream>;

    async fn full(
        &self,
        request: (),
    ) -> tonic::Result<
        std::pin::Pin<
            Box<
                dyn tonic::codegen::tokio_stream::Stream<
                        Item = tonic::Result<crate::generated::block_producer::MempoolEvent>,
                    > + Send
                    + 'static,
            >,
        >,
    > {
        use tonic::codegen::tokio_stream::StreamExt as _;
        let input = Self::decode(request)?;
        let stream = self.handle(input).await?;
        Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))
    }
}

And then the implementation can be something like this:

#[tonic::async_trait]
impl proto::server::block_producer_api::MempoolSubscription for BlockProducerRpcServer {
    type Input = ();
    type Item = MempoolEvent;
    type ItemStream = MempoolEventSubscriptionStream;

    fn decode(_request: ()) -> Result<Self::Input, tonic::Status> {
        Ok(())
    }

    fn encode(output: Self::Item) -> tonic::Result<proto::block_producer::MempoolEvent> {
        Ok(proto::block_producer::MempoolEvent::from(output))
    }

    async fn handle(&self, _request: Self::Input) -> tonic::Result<Self::ItemStream> {
        let subscription = self.mempool.lock().await.lock().await.subscribe();
        let subscription = ReceiverStream::new(subscription);

        Ok(MempoolEventSubscriptionStream { inner: subscription })
    }
}

struct MempoolEventSubscriptionStream {
    inner: ReceiverStream<MempoolEvent>,
}

impl tokio_stream::Stream for MempoolEventSubscriptionStream {
    type Item = tonic::Result<MempoolEvent, tonic::Status>;

    fn poll_next(
        mut self: std::pin::Pin<&mut Self>,
        cx: &mut std::task::Context<'_>,
    ) -> std::task::Poll<Option<Self::Item>> {
        self.inner.poll_next_unpin(cx).map(|x| x.map(tonic::Result::Ok))
    }
}

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/proto/build.rs
.line("use tonic::codegen::tokio_stream::StreamExt as _;")
.line("let input = Self::decode(request)?;")
.line("let stream = self.handle(input).await?;")
.line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))");
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ServerStream::as_trait generator emits a full body line with unbalanced parentheses: Ok(Box::pin(stream.map(...))) is missing one closing ). As written, the generated code will not compile for server-streaming RPCs (e.g. MempoolSubscription). Add the missing closing parenthesis in the string passed to .line(...).

Suggested change
.line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))");
.line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i))))))");

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog This PR does not require an entry in the `CHANGELOG.md` file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants