Generate a gRPC client with Buf

Compile Sift’s Protocol Buffers into a client for your language and protobuf version when an official client doesn’t fit your environment.

After completing this workflow, you can generate a gRPC client for your language and protobuf version, using Buf to compile Sift’s Protocol Buffers.

This workflow can help if, for example:

• Your language isn’t Python, Rust, or Go, the languages Sift’s official clients support.
• Your environment requires a different protobuf version than Sift’s packaged client uses, for example because your firmware or software is pinned to an older version.
• You need raw generated stubs for a custom ingestion pipeline instead of a full SDK.

Before you begin

• Install the Buf CLI.
• Have access to clone the Sift GitHub repository.
• Identify the Buf plugin for your target language. This guide uses Python as an example, but the process applies to any language Buf supports.

How Buf-based client generation works

Buf compiles Sift’s Protocol Buffers into client code using plugins you specify in a configuration file. Each plugin targets a language and can be pinned to a specific version, independent of the protobuf version used inside Sift’s own client libraries.

This is what makes Buf useful when the official client doesn’t fit: you control the exact language, protobuf version, and generated output, rather than taking Sift’s packaged SDK as-is.

Generate a client with Buf

1. Create a new directory for your client, and navigate into it. This directory holds your Buf configuration and the generated code.

   mkdir sift-grpc-python-client
   cd sift-grpc-python-client
   

2. Authenticate to the Buf Schema Registry. The plugins in this workflow run remotely on Buf’s registry, so an unauthenticated CLI will fail with an invalid token error.

   buf registry login
   

3. Clone the Sift GitHub repository to access the Protocol Buffer definitions locally.

   git clone https://github.com/sift-stack/sift.git
   
   Location: The .proto files are located in the sift subdirectory inside protos/ in the repository.

4. In your client directory, create a file named buf.gen.yaml. This file specifies the plugins Buf should use, the versions to pin, and where the generated code should be written.

   nano buf.gen.yaml
   
   version: v1
   managed:
     enabled: true
   plugins:
     - plugin: buf.build/grpc/python:v1.62.1
       out: gen
     - plugin: buf.build/protocolbuffers/python
       out: gen
     - plugin: buf.build/protocolbuffers/pyi:v26.1
       out: gen
   

   This configuration generates Python message classes, gRPC client and server stubs, and type hint files to the gen/ directory.

   Version pinning: To match a protobuf version required by your environment, pin the plugin version, for example buf.build/protocolbuffers/python:v3.19.5.

   Other languages: To generate a client for another language, use a different plugin here. Only the plugin selection and version pinning change; the rest of this workflow stays the same.

5. From inside your client directory, export the .proto files from the Sift repository into a local directory. sift_protos is the output directory in this example, and sift/protos/buf.yaml is the buf.yaml file in the protos subdirectory of the cloned repository.

   buf export sift/protos --output=sift_protos --config sift/protos/buf.yaml
   

6. In the directory containing buf.gen.yaml, generate the client code from the exported .proto files.

   buf generate sift_protos
   
   Dependencies: Generated code often requires additional runtime dependencies to work. Check the documentation for each plugin you use to identify what to install.

Verify the generated client

Check the output directories specified in buf.gen.yaml. For the example configuration above, the gen/ directory should contain:

• Python message classes
• gRPC client and server stubs
• .pyi type hint files

If any of these are missing, confirm the corresponding plugin is listed in buf.gen.yaml and that buf generate completed without errors.