BigQuery Storage API integrations should accept either v1beta1 or v1 client

I notice that the v1beta1 API is still used.

python-bigquery/google/cloud/bigquery/_pandas_helpers.py

Lines 577 to 587 in 952164b

    
           def _download_table_bqstorage_stream( 
        
               download_state, bqstorage_client, session, stream, worker_queue, page_to_item 
        
           ): 
        
               position = bigquery_storage_v1beta1.types.StreamPosition(stream=stream) 
        
               rowstream = bqstorage_client.read_rows(position).rows(session) 
        
               for page in rowstream.pages: 
        
                   if download_state.done: 
        
                       return 
        
                   item = page_to_item(page) 
        
                   worker_queue.put(item)

This is good, as it keeps packages such as pandas-gbq from breaking, but shouldn't we start thinking about migrating folks to the v1 endpoint? To do that without causing too much disruption, the to_dataframe, magics, and DB-API integrations should accept either v1beta1 or v1 clients for an extended period of time.

@tswast

@tswast Now that the BQ Storage client is stable, #55 will start using it as a default, and also migrate to v1 stable from v1beta1.

If this would indeed represent to big a disruption, the PR needs to be adjusted to support both Storage client versions. Possibly with a warning if beta client is used?

Also feel free to bring this point up on the linked PR, if you have time, as this feature request directly affects it, thanks!

Yes, that would be a very big disruption. Definitely a breaking change, as it would break pandas-gbq among others.

Noted. We will add this then, and branch on the BQ Storage client type (and if v1beta1 is passed, the old logic for constructing the session will be used).

Navigation Menu

Repository navigation

BigQuery Storage API integrations should accept either v1beta1 or v1 client #84

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def _download_table_bqstorage_stream(
	download_state, bqstorage_client, session, stream, worker_queue, page_to_item
	):
	position = bigquery_storage_v1beta1.types.StreamPosition(stream=stream)
	rowstream = bqstorage_client.read_rows(position).rows(session)

	for page in rowstream.pages:
	if download_state.done:
	return
	item = page_to_item(page)
	worker_queue.put(item)

BigQuery Storage API integrations should accept either v1beta1 or v1 client #84

Description

Activity

plamut commented on Apr 28, 2020

tswast commented on Apr 28, 2020

plamut commented on Apr 28, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions