Describe the bug
When downloading very large datasources the entire file is held in system memory rather than streamed to disk resulting in excessive memory usage (30GB extract used all available resources on a machine with 64GB RAM).
Versions
- Tableau Server version: 2021.3.8 (20213.22.0213.1115) 64-bit Windows
- Python version: 3.9
- TSC library version: 0.17.0
To Reproduce
server = TSC.Server(tab_server_url)
server.use_server_version()
tab_auth = TSC.TableauAuth(username=user,
password=pw,
site_id=tab_site
)
with server.auth.sign_in(tab_auth):
req_options = TSC.RequestOptions()
req_options.filter.add(TSC.Filter(TSC.RequestOptions.Field.ProjectName,
TSC.RequestOptions.Operator.Equals,
tab_project
)
)
req_options.filter.add(TSC.Filter(TSC.RequestOptions.Field.Name,
TSC.RequestOptions.Operator.Equals,
extract_name
)
)
ds_datasource, ds_pag_item = server.datasources.get(req_options)
server.datasources.download(datasource_id=ds_datasource[0].id,
filepath=str(save_file_path)
)
Results
This download did complete, however system resources were at 100% usage and no file exists within directory until completion.
Describe the bug
When downloading very large datasources the entire file is held in system memory rather than streamed to disk resulting in excessive memory usage (30GB extract used all available resources on a machine with 64GB RAM).
Versions
To Reproduce
server = TSC.Server(tab_server_url)
server.use_server_version()
tab_auth = TSC.TableauAuth(username=user,
password=pw,
site_id=tab_site
)
with server.auth.sign_in(tab_auth):
req_options = TSC.RequestOptions()
req_options.filter.add(TSC.Filter(TSC.RequestOptions.Field.ProjectName,
TSC.RequestOptions.Operator.Equals,
tab_project
)
)
req_options.filter.add(TSC.Filter(TSC.RequestOptions.Field.Name,
TSC.RequestOptions.Operator.Equals,
extract_name
)
)
ds_datasource, ds_pag_item = server.datasources.get(req_options)
server.datasources.download(datasource_id=ds_datasource[0].id,
filepath=str(save_file_path)
)
Results
This download did complete, however system resources were at 100% usage and no file exists within directory until completion.