Google Cloud Storage Ingress

Please follow these steps to ingress data to your pipelines, from GCS buckets!

You can connect to the bucket directly using the service account email address

🚧

Before You Start:

These steps require that you have the appropriate permissions to administer sharing on your GCS buckets. Please connect with your IT department or Google Cloud Admin if you do not have those permissions.

These steps also assume that the bucket already exists, and is visible in your "Storage" pane within Google Cloud Console.

Please reach out to Datastreamer Support team for the "Service Account" information that you will need to complete step 5.

Steps to configure your GCS Bucket

1. Access the desired Bucket within your Storage pane in Google Console.
2. Click configure the Bucket details
3. Click the Permissions tab.
4. Next to the Permissions label, click ADD.
5. Provide the following permissions to the Service Account provided by Datastreamer support. To this member, add the following Cloud Storage roles:
   1. Storage Object Viewer: Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket
   2. Storage Legacy Bucket Reader: Grants permission to list a bucket's contents and read bucket metadata, excluding IAM policies. Also grants permission to read object metadata when listing objects (excluding IAM policies).

That is it! Happy streaming!

📘

Storage Legacy Bucket Reader

Storage Legacy Bucket Reader is the legacy version of Storage Object Viewer.