Skip to content

unpack_from requires a buffer of at least 4 bytes #604

Closed
Closed
unpack_from requires a buffer of at least 4 bytes#604

Description

@analytically
analytically
opened on Dec 2, 2020
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.8/site-packages/prometheus_flask_exporter/__init__.py", line 753, in func
    return f(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/prometheus_flask_exporter/__init__.py", line 273, in prometheus_metrics
    return generate_latest(registry), 200, headers
  File "/usr/lib/python3.8/site-packages/prometheus_client/openmetrics/exposition.py", line 14, in generate_latest
    for metric in registry.collect():
  File "/usr/lib/python3.8/site-packages/prometheus_client/registry.py", line 83, in collect
    for metric in collector.collect():
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 149, in collect
    return self.merge(files, accumulate=True)
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 41, in merge
    metrics = MultiProcessCollector._read_metrics(files)
  File "/usr/lib/python3.8/site-packages/prometheus_client/multiprocess.py", line 61, in _read_metrics
    file_values = MmapedDict.read_all_values_from_file(f)
  File "/usr/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 87, in read_all_values_from_file
    used = _unpack_integer(data, 0)[0]
struct.error: unpack_from requires a buffer of at least 4 bytes for unpacking 4 bytes at offset 0 (actual buffer size is 0)

Activity

  1. brian-brazil commented on Dec 2, 2020

    @brian-brazil
    brian-brazil
    on Dec 2, 2020
    Contributor

    This isn't enough to go on, I have to presume that either you're using the code incorrectly or have data corruption on disk.

  2. analytically commented on Dec 3, 2020

    @analytically
    analytically
    on Dec 3, 2020
    Author

    Possibly corruption.

  3. analytically
    closed this as completedon Dec 3, 2020

  4. felipecwb commented on Aug 13, 2021

    @felipecwb
    felipecwb
    on Aug 13, 2021

    I had this issue too with version 0.11.0.
    I just moved my PROMETHEUS_MULTIPROC_DIR environment from /tmp to /tmp/prometheus and stopped those issues.

  5. Kircheneer commented on Oct 15, 2025

    @Kircheneer
    Kircheneer
    on Oct 15, 2025

    I am having a similar issue. I have identified that the problem comes up because of empty .db files in the multiproc dir:

    $ ls /prom_cache/ -hla | sort -k 5,5 -r |tail -n 20
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:45 gauge_max_456.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:45 gauge_max_401.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:45 counter_456.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:45 counter_401.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:41 histogram_1.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:41 gauge_max_1.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:41 counter_1.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 histogram_80.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 histogram_10.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 gauge_max_80.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 gauge_max_10.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 counter_80.db
-rw-r--r--. 1 1003650000 1003650000  64K Oct 14 11:40 counter_10.db
-rw-r--r--. 1 1003650000 1003650000    0 Oct 14 14:23 gauge_max_25172.db
-rw-r--r--. 1 1003650000 1003650000    0 Oct 14 14:23 gauge_max_25125.db
-rw-r--r--. 1 1003650000 1003650000    0 Oct 14 14:23 gauge_max_25086.db
-rw-r--r--. 1 1003650000 1003650000    0 Oct 14 13:47 gauge_max_19550.db
-rw-r--r--. 1 1003650000 1003650000    0 Oct 14 11:45 gauge_max_358.db
dr-xr-xr-x. 1 root       root         50 Oct 15 08:45 ..
total 45M

    After deleting those files, the generation worked again.

    Is this something I should take to django_prometheus directly?

  6. Kircheneer commented on Oct 15, 2025

    @Kircheneer
    Kircheneer
    on Oct 15, 2025

    Alternatively, what do you think of simply deleting any .db files with length 0 upon read, logging a warning?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions