Export data programmatically

Export telemetry data from Sift using the REST API, an official client library, or a client generated with Buf.

After completing this workflow, you can export Channel data from any Run into the tool or pipeline your team uses for analysis.

Sift’s REST API and client libraries let you export telemetry data from Runs programmatically for use in external tools, custom pipelines, and downstream analysis environments such as MATLAB or Python.

Before you export

• You have a Sift API key and your base URLs.
• You have the ID of the Run and the IDs of the Channels you want to export. One option is to find these in the Sift UI or by querying the ListRuns and ListChannels endpoints.

How does programmatic export work

Sift provides two programmatic export methods, data querying and export data to file, each available through the REST API and official client libraries (Python, Rust, and Go). Clients for other languages can be generated using Buf.

Query Channel data

Use the REST API

To query Channel data, call the GetData endpoint.

1. Query Channel data

   curl --request POST \
     --url "https://api.siftstack.com/api/v2/data" \
     --header "Authorization: Bearer $SIFT_API_KEY" \
     --header "Content-Type: application/json" \
     --data '{
       "queries": [
         {
           "channel": {
             "channelId": "YOUR_CHANNEL_ID_1",
             "runId": "YOUR_RUN_ID"
           }
         },
         {
           "channel": {
             "channelId": "YOUR_CHANNEL_ID_2",
             "runId": "YOUR_RUN_ID"
           }
         }
       ],
       "startTime": "2026-03-13T19:43:25.557Z",
       "endTime": "2026-03-13T19:48:25.557Z",
       "sampleMs": 0,
       "pageSize": 100000,
       "pageToken": ""
     }'
   
   endTime: This field is exclusive, so a sample that falls exactly at or after endTime is not included in the response. If a value you expect to see is missing, try extending endTime slightly past the boundary you actually want.

   sampleMs: Set sampleMs to 0, or omit it, to return the full, unsampled dataset, recommended for external data analysis. Any other value downsamples the response using LTTB, a shape-preserving algorithm intended for plotting rather than analysis; it returns approximately one representative point per that many milliseconds rather than sampling at a literal fixed interval.

   Timeseries Panel: The Sift app’s Timeseries Panel offers a choice of sampling methods (LTTB, Min/Max, and Changed Only), but GetData only supports LTTB; the other methods are not currently available through this endpoint.

2. Paginate if needed

   If the response includes a non-empty nextPageToken, resend the same request with pageToken set to that value to retrieve the next batch. Repeat until nextPageToken comes back empty.

   curl --request POST \
     --url "https://api.siftstack.com/api/v2/data" \
     --header "Authorization: Bearer $SIFT_API_KEY" \
     --header "Content-Type: application/json" \
     --data '{
       "queries": [
         {
           "channel": {
             "channelId": "YOUR_CHANNEL_ID_1",
             "runId": "YOUR_RUN_ID"
           }
         }
       ],
       "startTime": "2026-03-13T19:43:25.557Z",
       "endTime": "2026-03-13T19:48:25.557Z",
       "sampleMs": 0,
       "pageSize": 100000,
       "pageToken": "TOKEN_FROM_PREVIOUS_RESPONSE"
     }'
   

Use the Python client

The official Python client provides two methods for querying Channel data:

• client.channels.get_data_as_arrow() returns an Apache Arrow table. Recommended when converting data for use in other tools.
• client.channels.get_data() returns typed Python value objects.

1. Initialize the client

   import os
   from datetime import datetime, timezone
   from dotenv import load_dotenv
   from sift_client import SiftClient
   load_dotenv()
   
   client = SiftClient(
       api_key=os.getenv("SIFT_API_KEY"),
       grpc_url=os.getenv("SIFT_GRPC_URI"),
       rest_url=os.getenv("SIFT_REST_URI"),
   )
   

2. Retrieve the data

   Fetch the Channel object using its ID, then call client.channels.get_data_as_arrow(). The method returns a dict where each key is a Channel name and the value is an Apache Arrow table.

   channel = client.channels.get(channel_id="YOUR_CHANNEL_ID")
   
   result = client.channels.get_data_as_arrow(
       channels=[channel],
       run="YOUR_RUN_ID",
       start_time=datetime(YYYY, MM, DD, HH, MM, SS, microsecond, tzinfo=timezone.utc),  # e.g. datetime(2026, 4, 15, 0, 17, 49, 984000, tzinfo=timezone.utc) = 2026-04-15T00:17:49.984Z
       end_time=datetime(YYYY, MM, DD, HH, MM, SS, microsecond, tzinfo=timezone.utc),    # e.g. datetime(2026, 4, 15, 0, 18, 26, 609000, tzinfo=timezone.utc) = 2026-04-15T00:18:26.609Z
   )
   
   # Access the table by Channel name and convert to a pandas DataFrame
   df = result["YOUR_CHANNEL_NAME"].to_pandas()
   

   Or using client.channels.get_data(), which returns a dict[str, pd.DataFrame] directly with no conversion needed:

   result = client.channels.get_data(
       channels=[channel],
       run="YOUR_RUN_ID",
       start_time=datetime(YYYY, MM, DD, HH, MM, SS, microsecond, tzinfo=timezone.utc),  # e.g. datetime(2026, 4, 15, 0, 17, 49, 984000, tzinfo=timezone.utc) = 2026-04-15T00:17:49.984Z
       end_time=datetime(YYYY, MM, DD, HH, MM, SS, microsecond, tzinfo=timezone.utc),    # e.g. datetime(2026, 4, 15, 0, 18, 26, 609000, tzinfo=timezone.utc) = 2026-04-15T00:18:26.609Z
   )
   
   # Access the DataFrame by Channel name
   df = result["YOUR_CHANNEL_NAME"]
   

Export data to file

Use the REST API

To export data to a file, call the ExportData and GetDownloadUrl endpoints.

1. Submit the export job

   curl --request POST \
     --url "https://api.siftstack.com/api/v1/export" \
     --header "Authorization: Bearer $SIFT_API_KEY" \
     --header "Content-Type: application/json" \
     --data '{
       "runsAndTimeRange": {
         "runIds": ["YOUR_RUN_ID"]
       },
       "channelIds": ["CHANNEL_ID_1", "CHANNEL_ID_2"],
       "outputFormat": "EXPORT_OUTPUT_FORMAT_PARQUET"
     }'
   
   outputFormat: This is a fixed value to select, not a placeholder to replace with your own text.

2. Retrieve the download link

   If the response includes a presignedUrl, the export completed immediately and the file is ready to download. If the response includes a jobId instead, poll this endpoint until presignedUrl is populated:

   curl --request GET \
     --url "https://api.siftstack.com/api/v1/export/YOUR_JOB_ID/download-url" \
     --header "Authorization: Bearer $SIFT_API_KEY"
   

Use the Python client

The official Python client provides two methods for exporting data to file:

• client.data_export.export() submits the export job.
• job.wait_and_download() polls until complete, downloads, and extracts the result automatically. Returns a list of Path objects pointing to the extracted files.

1. Initialize the client

   import os
   from dotenv import load_dotenv
   from sift_client import SiftClient
   from sift_client.sift_types.export import ExportOutputFormat
   
   load_dotenv()
   
   client = SiftClient(
       api_key=os.getenv("SIFT_API_KEY"),
       grpc_url=os.getenv("SIFT_GRPC_URI"),
       rest_url=os.getenv("SIFT_REST_URI"),
   )
   

2. Submit the export job and download the result

   job = client.data_export.export(
       output_format=ExportOutputFormat.PARQUET,
       runs=["YOUR_RUN_ID"],
       channels=["YOUR_CHANNEL_ID_1", "YOUR_CHANNEL_ID_2"],
   )
   
   paths = job.wait_and_download(output_dir=".")  # downloads and extracts to the directory where the script is run