Is there any way to get the number of rows actually loaded, but not the total count for a table.

Here is an example, the code snippet is

    # make a call of loadjob and use .result() to wait until it competes

    table = client.get_table(table_id)  # Make an API request.
    print(
        "Loaded {} rows and {} columns to {}".format(
            table.num_rows, len(table.schema), table_id
        )
    )

After the load job completed, it calls get_table to get the num_rows, and prints it. It works if we do a fresh load to an empty table, but when we load a file (or files) to an existing table by using APPEND mode, that number of rows is not the actually loaded count.

My workaround is, get_table() twice and calculate the difference between two counts. I am wondering is there any better approach to get the actually loaded rows (it is like the affected rows in other DBMS).

I am facing the same issue when appending the results of a query to an existing table using the QUERY job. Below code also gives the total count of rows in the destination table after appending:

query_job_config = bigquery.QueryJobConfig(destination_table)
query_job = bq_client.query(sql, job_config=query_job_config)
results = query_job.result()
print(results.total_rows)

Based on staffing and project priorities, this item will not be worked.

Navigation Menu

Repository navigation

Is there any way to get the number of rows actually loaded, but not the total count for a table. #1128

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Is there any way to get the number of rows actually loaded, but not the total count for a table. #1128

Description

Activity

mehul1811 commented on Aug 14, 2022

chalmerlowe commented on Jun 7, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions