feat(proto): generate generic GRPC API implementations#1950
feat(proto): generate generic GRPC API implementations#1950
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the crates/proto build-time code generation to also emit generic, per-RPC-method server-side traits and blanket tonic server implementations, aiming to standardize gRPC server implementations across all services referenced in PR #1742.
Changes:
- Update
crates/proto/build.rsto generate additional server facade modules from protobuf descriptors (plus regeneratemod.rsfiles). - Add build-time dependencies needed for code generation and descriptor introspection (
codegen,prost-types). - Run
rustfmtover generated Rust sources at build time.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| crates/proto/build.rs | Adds server module generation from descriptors, new mod.rs generation logic, and build-time rustfmt pass. |
| crates/proto/Cargo.toml | Adds build dependencies (codegen, prost-types) needed by build.rs. |
| Cargo.toml | Pins prost-types to align with pinned prost versions in the workspace. |
| Cargo.lock | Locks new dependency resolutions for codegen and prost-types. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Each FileDescriptorSet includes transitive imports, the same service
(e.g. rpc.Api) will appear in multiple sets and the corresponding
{module_name}.rs will be regenerated/overwritten multiple times.
This commit fixes this.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This reverts commit 4cf493e.
|
I've added code generation for server streaming. The traits we're generating for these look like this (using the mempool subscription as an example): #[tonic::async_trait]
pub trait MempoolSubscription {
type Input;
type Item;
type ItemStream: tonic::codegen::tokio_stream::Stream<Item = tonic::Result<Self::Item>>
+ Send
+ 'static;
fn decode(request: ()) -> tonic::Result<Self::Input>;
fn encode(item: Self::Item) -> tonic::Result<crate::generated::block_producer::MempoolEvent>;
async fn handle(&self, input: Self::Input) -> tonic::Result<Self::ItemStream>;
async fn full(
&self,
request: (),
) -> tonic::Result<
std::pin::Pin<
Box<
dyn tonic::codegen::tokio_stream::Stream<
Item = tonic::Result<crate::generated::block_producer::MempoolEvent>,
> + Send
+ 'static,
>,
>,
> {
use tonic::codegen::tokio_stream::StreamExt as _;
let input = Self::decode(request)?;
let stream = self.handle(input).await?;
Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))
}
}And then the implementation can be something like this: #[tonic::async_trait]
impl proto::server::block_producer_api::MempoolSubscription for BlockProducerRpcServer {
type Input = ();
type Item = MempoolEvent;
type ItemStream = MempoolEventSubscriptionStream;
fn decode(_request: ()) -> Result<Self::Input, tonic::Status> {
Ok(())
}
fn encode(output: Self::Item) -> tonic::Result<proto::block_producer::MempoolEvent> {
Ok(proto::block_producer::MempoolEvent::from(output))
}
async fn handle(&self, _request: Self::Input) -> tonic::Result<Self::ItemStream> {
let subscription = self.mempool.lock().await.lock().await.subscribe();
let subscription = ReceiverStream::new(subscription);
Ok(MempoolEventSubscriptionStream { inner: subscription })
}
}
struct MempoolEventSubscriptionStream {
inner: ReceiverStream<MempoolEvent>,
}
impl tokio_stream::Stream for MempoolEventSubscriptionStream {
type Item = tonic::Result<MempoolEvent, tonic::Status>;
fn poll_next(
mut self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
self.inner.poll_next_unpin(cx).map(|x| x.map(tonic::Result::Ok))
}
} |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| .line("use tonic::codegen::tokio_stream::StreamExt as _;") | ||
| .line("let input = Self::decode(request)?;") | ||
| .line("let stream = self.handle(input).await?;") | ||
| .line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))"); |
There was a problem hiding this comment.
The ServerStream::as_trait generator emits a full body line with unbalanced parentheses: Ok(Box::pin(stream.map(...))) is missing one closing ). As written, the generated code will not compile for server-streaming RPCs (e.g. MempoolSubscription). Add the missing closing parenthesis in the string passed to .line(...).
| .line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i)))))"); | |
| .line("Ok(Box::pin(stream.map(|item| item.and_then(|i| Self::encode(i))))))"); |
This PR adds the build-time generation of generic GRPC API implementations for all services discussed in PR #1742.
For each method we're generating a trait like this:
The idea is that the actual implementation should consist of implementing all per-method traits for the API server (most of the time just
decode/encode/handle), and then the provided generic implementation for the API can be used:In this form this PR should be a no-op, because the none of our API implementations have been moved to this new model. That will be done in multiple follow-up PRs.