Please follow these steps to ingress data to your pipelines, from GCS buckets!

You can connect to the bucket directly using the service account email address

🚧

Before You Start

These steps require that you have the appropriate permissions to administer sharing on your GCS buckets. Please connect with your IT department or Google Cloud Admin if you do not have those permissions.

These steps also assume that the bucket already exists, and is visible in your "Storage" pane within Google Cloud Console.

Please reach out to Datastreamer Support team for the "Service Account" information that you will need to complete step 4.

Steps to Configure Your GCS Bucket for Datastreamer Support

1. Access the Bucket: Navigate to your desired bucket in the Google Cloud Console's Storage pane.

2. Configure Bucket Details: Click on "Configure" next to the bucket details.

3. Permissions Setup: Proceed to the Permissions tab and click "ADD" next to the permissions label.

4. Add Service Account Permission:

   • Provide the following permissions to the Service Account provided by Datastreamer support. To this member, add the following Cloud Storage roles:
     1. Storage Object Viewer: View objects, their metadata (excluding ACLs), and list bucket contents.
     2. Storage Legacy Bucket Reader: List bucket contents, read bucket metadata (excluding IAM policies), and read object metadata when listing.

That is it! Happy streaming!

📘

Storage Legacy Bucket Reader

Storage Legacy Bucket Reader is the legacy version of Storage Object Viewer.