Issues with multiple commands sent in quick succession?

[lockyt]

Hi @the1laz - great work with cgateweb. It's a great little script.

I am however having an issue with CBUS which may or may not be the cgateweb script causing the problem which I am hoping you can help with.

To explain the situation. We have a Qnap NAS running docker containers in the following configuration. CBUS(CNI)<->Cgate<->Cgateweb<->mosquito eclipse<->Home Assistant<->Homekit. All works just fine for turning single lights on/off and works super fast. However, when we send more complex commands such as turning off the kitchen lights or downstairs lights, it has problems where it seems some lights fail to report back to HA that they turned on/off. So for example, you tell Siri to turn off all kitchen lights and any lights that are on turn off. However, in HA it shows that some kitchen lights are still on but they have actually turned off. Looking at the cgateweb logs, it shows the command being sent to turn the lights off and they work (lights go out) but the responses don't come back for the lights (which are showing as still on).

Looking at cgate logs, I can see the commands come in to turn on the lights and see the responses go back for all the lights. So it seems that somewhere between cgate and cgateweb the responses to update what happened are getting lost.

I have isolated it down to one of these two (cgate/cgateweb) as everything else seems to be working as expected.

Here are the screenshots of both cgate and cgateweb during the kitchen lights on/off scenario as above, as well as what is happening in Home Assistant:

Turn off all kitchen lights either through HA or SIRI:

all physically turn off as requested but HA doesn't get a response for 28 and 30 so it thinks they are still on

Turn on all kitchen lights through either HA or SIRI:

only turns on those in HA/CBUS that it thinks are off (ie, everything but 28/30)

If you go from the bottom up on the logs you can find where all lights were turned on or off. You can see the device ids being turned on/off and the responses from cgate confirming them. But cgateweb only shows some of the responses coming in, not all and the ones that don't are the lights that stay on/off. Now it could be cgate logging something it isn't sending or perhaps cgateweb not processing and accepting the responses. We have a very large cbus installation and it could be related to that.. a total of ~190 lights and blinds in the place. However this issue happens even on the smallest group - in this instance only 6 lights.

Any help would be greatly appreciated. I will need to confirm what version of cgate we are using and whether there is anything special with the cbus installation (need to speak with the CBUS guy) but thought I'd check with you first.

Thanks for your help in advance.

Locky.

[lockyt]

Hi @the1laz - I spoke with the CBus guy today and he suggested that it could just be the CBus network not being able to process a bunch of commands being sent at once. Although I think we are seeing a response from cgate in the console, it doesn't mean it is actually being sent so I guess it is possible. He is suggesting we send commands with a microsecond or two delay to ensure it doesn't get hit with too many commands all at once. Not sure if that is possible with the script.

He also suggested we use levels instead of on/off commands which I will test. Some of the lights aren't dimmable so it's not a perfect solution and not sure how it would work.

He is also going to setup a scene or two for me to use instead but it kinda defeats the purpose of using Home Assistant/HomeKit. I want to be able to move lights in or out of a room on HomeKit and ask for all lights in that group to be turned on/off. Keep it dynamic. The CBus guy here in Canberra is impossible to get hold of at the best of times, so would prefer not to rely on groups being defined in CBus.

Thanks.

Locky.

[lockyt]

Sorry one other question - does cgateweb use a quality of service parameter with MQTT? If so, is it possible to change it to guaranteed delivery?

Thanks!

[the1laz]

Sorry Locky, I haven't had a chance to have a good look at this but I'm sure it won't be too hard to solve. I didn't really consider sending multiple commands at once when I put this together, so I'm not surprised that it's not working. I'll have a look this weekend.

[lockyt]

Awesome - that'd be great thanks mate. You may want to check my logic is right first as I'd hate you to do a bunch of work to find it is something I am doing stupid.

[lockyt]

Hey Stephen - any luck over the weekend? Get a chance to have a look? Sorry no pressure - just glad you are happy to help.

[the1laz]

No luck last weekend due to my pi crashing at some point then giving me grief when starting up. Got some time today though and stuck a couple of message queues in that'll hopefully help. Seems good at my end, let me know if it fixes it for you.

[the1laz]

Don't forget to grab a copy of your settings before downloading the latest commit. Sorry, the settings file (and all the rest of the code) could be better, but I've not really put much effort into this since I first got it working.

[lockyt]

Ok great thanks Stephen. Unfortunately I won't be able to test it until later in the week but I will let you know how I go!

[lockyt]

Hi Stephen - just tested the new script and definitely a major improvement. I noticed the events have slowed down (your flow control I presume) and whereas it worked properly 10% of the time before, it now works properly 90% of the time. Out of the 10 or so tests I did, we did have a command or two not respond on it's state (which in turn doesn't update HA on the status of the light).

I am hoping it is just a tuning thing now to get the sweet spot of speed vs commands not being responded to? Is that something I can change on the fly or am I over simplifying it? Please let me know.

But definitely a lot better so I think we are going down the right track.

Thanks again mate for all your help.

[lockyt]

This is a screen shot on one of the times it didn't work. The lights actually turned off (so the commands got to CBUS) but the responses from the CBUS through cgateweb don't all get through. This was only one test of 2 out of 10 that didn't work.

Thanks mate.

[the1laz]

Hi Locky, I'm hoping it's just a tuning thing. I've added an option to change the message interval in settings.js and bumped the default up to 200ms between messages. Hopefully that'll work for you, seems to be working for me.

[lockyt]

Great. I'll try again and let you know.

[lockyt]

Hi Stephen - just to let you know that it all now seems to be working a charm. Thanks so much for all your help! Really appreciated.

[the1laz]

No problem Locky, good to hear it's working.

[GledholtHall]

I'm kinda surprised at this - but obviously it works.

There is a TCP socket connection to C-Gate (no dropped info there) and C-Gate handles all the C-Bus communication and includes it's own buffering and flow control. It is true that C-Bus itself has very limited capacity for rapid commands but C-Gate manages this (or should).

The connection to and from MQTT is TCP based so again no dropped messages there so it's a fairly solid path apart from 'within C-Gate' over which you have no control. The fact the buffer solution works implies issues there.

... anyway as it works, this is not too important