Repository navigation
Errors when downloading gzipped objects in chunks from GCS #1009
Description
This issue emanates from an issue posted here.
We are trying to get a gzipped GCS file in chunks. The problem comes when we try to read the response content. The problem seems to be related with the fact that response.getContent() automatically creates a GZIPInputStream on top of the raw stream (if the content encoding is gzip). When we attempt to read
the GZIPInputStream given by getContent() we get an error as that stream only holds one chunk of the original file.
Would it be possible to somehow disable the wrapping of the response content into a GzipInputStream when calling getContent()?
Activity
Capstan commentedon Jun 16, 2016
Or have it notice there's a
Content-Rangeheader and disable it itself.mziccard commentedon Sep 30, 2016
Any updates on this?
dalbani commentedon Oct 9, 2017
Hi, I'm very much new to this GCS library and I'm kind of surprised that this simple use case is not currently supported.
Well, a workaround has indeed been implemented AFAIK but I still fail to see how to properly handle reading a gzip-compressed blob from a
ReadChannel, as in:How can I then determine that the data coming out this channel is actually encoded (or not) using GZIP?
Thanks for your help.
ajaaym commentedon Jan 24, 2019
@mziccard do you expect the chunks to be of compressed bytes or non compressed bytes?
sduskis commentedon Mar 8, 2019
I'm going to close this due to staleness. Please open a new issue if this is still a problem.