shollyman
Repos
40
Followers
24
Following
10

Google Cloud Client Libraries for Go.

3112
1018

Sample apps and code written for Google Cloud in the Go programming language.

3665
1510

Events

bigquery: configure write disposition for Inserter

I'm not sure what you mean by automated partition handling. If a DELETE statement has a WHERE clause that removes all data from a given partition, that partition is empty. You can stream afterwards regardless of whether the partition has data or not.

Created at 2 days ago

fix(logging): fix stdout log http request format (#7083)

feat(vmwareengine): start generating apiv1 (#7093)

test(pubsublite): fix flaky TestIntegration_PublishSubscribeSinglePartition (#7098)

Merge branch 'main' into multiplex-refactor-1

Created at 3 days ago
pull request opened
chore(bigquery/storage/managedwriter): internal refactor (flow controller, ids)

This PR adds a new internal mechanism to simplify duplicating flow controllers, and does some preliminary work to wire in a UUID-based ID for managed stream instances. Neither is used elsewhere at this time, but lay the foundation for further refactoring.

Towards: https://github.com/googleapis/google-cloud-go/issues/7103

Created at 3 days ago
shollyman create branch multiplex-refactor-1
Created at 3 days ago
bigquery/storage/managedwriter: multiplexed writes

This issue tracks the PRs related to supporting multiplex connections in managedwriter.

Created at 3 days ago
bigquery: configure write disposition for Inserter

If the partition isn't actively receiving streaming inserts, then you should not have issues.

If however you're actively streaming to a given partition, then wish to drop it and resume streaming then yes you are more likely to encounter issues. It's likely the DML to delete the partition will be rejected due to streaming limitations on DML for example.

Created at 3 days ago
bigquery: configure write disposition for Inserter

The Inserter is for continuous ingestion, and is used for streaming row data into a BigQuery table. It has no affordances for managing the state of the table. We clearly wouldn't want to truncate the table every time you stream a row, for example.

If you want to alter a table prior to streaming, it's likely easiest to do something like execute a DML query (since you mention rewriting a partition rather than a table), and upon successful completion start streaming rows through the inserter.

I've marked this as a question since it's not a feature request we'd consider (assuming I'm understanding your request correctly). Let me know if there's more clarification needed.

Created at 1 week ago
test(bigquery): analyticshub testing
Created at 2 weeks ago