properly close HDF5 filles and percussions for opening. by sergey-yaroslavtsev · Pull Request #1237 · silx-kit/pymca

sergey-yaroslavtsev · 2026-07-24T15:23:06Z

If user try to open hdf5 fille during fast scan it crashes.

crash should not appear
if any crash happens close file should close it completely (so reopen helps)

I could not test it (no simulation of fast scan exist), so please check it carefully...

Error

CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: source.refresh() failed: Can't synchronously read data (address of object past end of allocation)
CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: <class 'OSError'> Can't synchronously read data (address of object past end of allocation)   File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 587, in rowCount
    return len(self.getProxyFromIndex(index))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 372, in __len__
    return len(self.children)
               ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 228, in children
    self._children = [H5NodeProxy(self.file, i[1], self)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 330, in __init__
    nodeattr = node.attrs[cname]
               ~~~~~~~~~~^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/usr/local/lib/python3.12/dist-packages/h5py/_hl/attrs.py", line 77, in __getitem__
    attr.read(arr, mtype=htype)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 364, in h5py.h5a.AttrID.read
  File "h5py/_proxy.pyx", line 65, in h5py._proxy.attr_rw

CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: <class 'OSError'> Can't synchronously read data (address of object past end of allocation)   File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 587, in rowCount
    return len(self.getProxyFromIndex(index))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 372, in __len__
    return len(self.children)
               ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 228, in children
    self._children = [H5NodeProxy(self.file, i[1], self)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 330, in __init__
    nodeattr = node.attrs[cname]
               ~~~~~~~~~~^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/usr/local/lib/python3.12/dist-packages/h5py/_hl/attrs.py", line 77, in __getitem__
    attr.read(arr, mtype=htype)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 364, in h5py.h5a.AttrID.read
  File "h5py/_proxy.pyx", line 65, in h5py._proxy.attr_rw

woutdenolf · 2026-07-25T13:17:59Z

    """
    sigFileUpdated = qt.pyqtSignal(object)
    sigFileAppended = qt.pyqtSignal(object)
+    sigReadError = qt.pyqtSignal(object)


I learned from @t20100 the signal names are usually somethingHappened or AboutToHappen.

sigReadError -> sigReadFailed?

woutdenolf · 2026-07-25T13:22:23Z

+                "if that does not help, close and open the file again.")
+            msg.exec()
+
+        qt.QTimer.singleShot(0, _show)


First time in PyMca we popup a dialog with qt.QTimer. I'm assuming this is a the UI stays responsive.

Is this the direction we will be taking for all new dialogs? Will be refactor all the existing dialogs to this pattern? Should be use a helper for this?

crash should not appear

But here we pop-up an dialog ...

qt.QTimer here is to make the order:
with: all "drawing" and "build" (the empty three appear - because of return 0) are finished before message
without: message appear after trying to open the file - after message is closed the "drawing" and "build" are finished.

I do not know if it is crucial in this case.
If user somehow manage to change smthg in main window while message is open - then closing error message can lead to a crush. But of course main window is blocked during this message so one can say it is protected...

Let me know what you think about it.

QDialog (which QMessageBox inherits from) has a open non-blocking method, IMO it' worth trying to replace the blocking exec with it and so avoid the QTimer

Sorry, does it mean that message will not block the main window?
Then user instead of closing the message can simply ignore it (e.g. by accidentally clicking oi main window) - which is not a desired behavior.
Do I miss the point?

So we will not block the the main process (good) but it will also not block the main window - providing possibility to ignore the error (not directly bad but questionable) .

With open, the message should block the interaction with the main window, but not the execution of the code, which is what you do here with the QTimer

from that page:

Note: Avoid using this function; instead, use open(). Unlike exec(), open() is asynchronous, and does not spin an additional event loop. This prevents a series of dangerous bugs from happening (e.g. deleting the dialog's parent while the dialog is open via exec()). When using open() you can connect to the finished() signal of QDialog to be notified when the dialog is closed.

does it mean that msg object is not deleted after .open()?

If i understood correctly it require Qt::WA_DeleteOnClose.
Otherwise, can it cause a leak?

P.S.
I've tested - it works as you said 👍

Yes, msg.setAttribute(qt.Qt.WA_DeleteOnClose) is needed.
Adding msg.destroyed.connect(lambda: print("destroyed")) clearly show the difference with and without it.
But wrapper still remain in both cases.

Was not obvious for me.

Indeed, didn't though of that.

sergey-yaroslavtsev · 2026-07-27T15:24:24Z

For me it is not clear if this "closing mechanism" is enough, it defiantly will not harm but is it sufficient?

t20100 · 2026-07-27T15:48:45Z

For me it is not clear if this "closing mechanism" is enough, it defiantly will not harm but is it sufficient?

It should be.
With WA_DeleteOnClose the QMessageBox gets deleted when closed, but since it also has the QNexusWidget as its parent, it also gets deleted when the QNexusWidget is deleted.

sergey-yaroslavtsev · 2026-07-27T16:17:52Z

It should be.
With WA_DeleteOnClose the QMessageBox gets deleted when closed, but since it also has the QNexusWidget as its parent, it also gets deleted when the QNexusWidget is deleted.

Sorry i meant about closing instance for HDF5 source.

woutdenolf · 2026-07-28T17:53:54Z

rowCount fails (seems like a random function which I'm assuming fails first at the moment?) -> absorb error and pop-up message box the first time, nothing related to closing HDF5 here.
_sourceSelectorSlot gets {"event":"SourceClosed"} -> close all HDF5 files.

Those two are not related as far as I can tell.

As for the non-blocking message box: this is the first time we introduce this pattern. Probably good to make a helper?

def non_blocking_user_warning(...)
    msg = qt.QMessageBox(self)
    msg.setIcon(qt.QMessageBox.Warning)
    msg.setWindowTitle("...")
    msg.setText("...")
    msg.setInformativeText("...")
    msg.setAttribute(qt.Qt.WA_DeleteOnClose)
    msg.open()

woutdenolf · 2026-07-29T12:41:38Z

+                # Without this the file stays "open" in the process
+                # and could not be reopened until PyMca restarted.
+                try:
+                    closedSource.close()


If self.sourceList is the only object that holds a reference to the h5py.File object then python's garbage collector will close the file at some point after this function (_sourceSelectorSlot) exists.

I don't mind making sure we close the file here in a finally block. Probably better in case there are weird situations (like a logged error callstack that keeps the h5py.File object alive).

woutdenolf · 2026-07-29T13:08:09Z

+            return len(self.getProxyFromIndex(index))
+        except Exception:
+            if not self._readErrorReported:
+                self._readErrorReported = True


FileModel is per source so you will have the error only once per source. A better solution is needed: perhaps refresh at failure is the solution.

close instance on close file; catch error and notify the user

c66d463

sergey-yaroslavtsev requested a review from woutdenolf July 24, 2026 15:23

sergey-yaroslavtsev changed the title ~~close instance on close file; catch error and notify the user~~ properly close HDF5 filles and percussions for opening. Jul 24, 2026

woutdenolf reviewed Jul 25, 2026

View reviewed changes

sergey-yaroslavtsev added 2 commits July 27, 2026 12:00

error to failed

230c1c3

open+flag instead of QTimer

e81dd76

sergey-yaroslavtsev requested a review from woutdenolf July 27, 2026 15:22

woutdenolf reviewed Jul 29, 2026

View reviewed changes

Uh oh!

Conversation

sergey-yaroslavtsev commented Jul 24, 2026 • edited by woutdenolf Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergey-yaroslavtsev Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergey-yaroslavtsev Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergey-yaroslavtsev Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

t20100 commented Jul 27, 2026

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026

Uh oh!

woutdenolf commented Jul 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sergey-yaroslavtsev commented Jul 24, 2026 •

edited by woutdenolf

Loading

sergey-yaroslavtsev Jul 27, 2026 •

edited

Loading

sergey-yaroslavtsev Jul 27, 2026 •

edited

Loading

sergey-yaroslavtsev Jul 27, 2026 •

edited

Loading

sergey-yaroslavtsev commented Jul 27, 2026 •

edited

Loading