create snapshot of rbd image failed with errno 30

[zhuozh]

When a rbd image is mapped as a LUN, the tcmu-runner will hold the image's exclusive-lock.
If we create a snapshot of this image, it will fail with errno 30 (EROFS: Read-only file system)
Is there any plan or suggestion on how to support rbd snapshot without unbind the LUN, for a reseaon that we may running a VM on this LUN. VM be stopped or crashed is unacceptable when creating snapshot.
#70 is releated to this issue.

[mikechristie]

Hey,

We have sort of discussed in this internally, but any low level details.

I think we would like a gwcli command that calls into ceph-iscsi-config to coordinate the tcmu-runner access and do the ceph/rbd lib calls to do the snapshot.

If you have any ideas, post them!

[dillaman]

@zhuozh There shouldn't be any restriction for creating a snapshot when the image's exclusive lock is owned by another client. The client requesting the snapshot create will send an RPC message to the lock owner to create the snapshot on its behalf. Do you have a reproducer for this behavior? I just tried it in my environment and it worked.

[zhuozh]

@dillaman My expression is a little misleading. Actually the read only error happened when rolling back an image to a snapshot.

1. Create a LUN, mkfs and mount it on initiator node.
2. Write some data into mounted device.
3. Create a snapshot using rbd command
4. iscsiadm logout, unbind clientlun through gwcli.
5. Roll back to a snapshot, it fails with read-only error.

[dillaman]

@zhuozh That was originally a deliberate decision on the part of librbd since we really didn't expect a valid use-case where you should rollback with an alive client. If tcmu-runner knew when the last client disconnected, it could release the lock (but I have a feeling it doesn't know). Instead, as an alternative, we could add a --force optional to the rbd CLI to break the lock before a rollback.

[mikechristie]

Adding @pcuzner

I think Paul wanted to add gwcli commands for snapshots that coordinated with the target/runner to handle this case for the user.

[dillaman]

See new PR #93