I am writing an app to interface with the switches on Hubitat to push any OTA to the switch. However, not sure how long it will work as Hubitat staff constantly patch these vulnerabilities that I am using.
As for Home Assistant, I could create an integration that can push firmware updates to most, if not all, zigbee devices if proper file is supplied. I will start testing with Inovelli switches this week.
Also worth noting - The integration I created for Home Assistant will allow mass parameter editing or based on a custom template. That way no more modifying each switch individually.
Glad to hear you guys are looking into the network issues. It does seem to be the payload’s are too large or too frequent. Could be both. I haven’t dug into the specific traffic enough yet, but it does seem to point to a firmware issue with the switches. It seems the switches keep flooding the network with routing table requests, and large payloads of power information / settings synchronization. I will continue looking into it as well and will provide any information I find to help.
I haven’t been able to replicate, but while having issues pairing a switch I found that specific MAC address was trying to bind to another switch and not the hub. It would then stop blinking blue and just jump to regular blue LED off. It seems if hub isn’t in pairing mode prior to rebooting switch, then switch will try to pair to another switch or random device. Not sure why that is occurring. It has occurred on both v2.08 and v2.10.
For what it’s worth, there is a third party HomeAssistant integration that installs aside ZHA called zha_toolkit that allows you to manually initialize an OTA firmware update on your devices. It does it on a single-device-basis, so you have to either write a script that includes all of your devices, or do it one by one with a service call (eg in the developer tools) to zha_toolkit.ota_notify.
This will pull OTA files from the koenkk git repository, and I have had luck with it grabbing the inovelli OTA updates in whatever manner ZHA handles that. I believe that it will use any OTA file in your configured OTA directory and it will push the firmware meant for the device that you have selected.
I unfortunately have to toggle this option near daily to boot all of my smart bulbs. For whatever reason they have been going offline every 24 hours or so. So I have an automation that toggles smart bulb mode to boot them and it fails most of the time due to the uint_8 exception then resets them all to dimmer. So I now have to basically keep running the automation until it succeeds which is usually the 3rd time.
I’m sitting here dumbfounded…never once had it occurred to me that I could just create an automation as a workaround. That would have saved me so much hassle!
Sorry to hear about the connectivity issues though; I hope you can track it down!
Our engineer has not been able to replicate it on a smaller network, so he is gathering the equipment necessary to test with many devices. He said he could work on this next week so fingers crossed.
Just curious, if you’re willing to be further transparent:
Has anyone on the Inovelli side been able to reproduce this issue or are you relying on the evidence from the community at this point? I’m more than willing to help provide any further data/information if I can be helpful as I’m pretty invested in this one
Also, I recall there was a comment somewhere that suggested that there was a fix coming in this firmware for the issue of button presses not being consistently sent back to the coordinator. I recall it had something to do with the switch sometimes attempting to send the commands to the wrong endpoint (?). Anyhow, that’s another issue I’m still seeing quite commonly and interested if I should hold tight to see if this upcoming firmware fixes that or if I should go ahead and gather evidence and submit a bug for it.
This is great news. I primarily use Hubitat so mostly looking forward to that version. But if the Hubitat staff keeps breaking it, I also have an RPi HA setup to play with and could use that if necessary
Hubitat is a good platform. however, I feel it is limited. Lately, they have stifled innovation by patching the possibility of sending custom OTA commands over the zigbee network. They also are trying their best to prevent OTA files from being uploaded into the local cache / local repo. This is one trick to override pulling from the cloud at all. (It will always check local before reaching out to their server)
You can also downgrade with a trick too.
Another reason I moved to HA this month is that Hubitat’s radio firmware is terrible in the latest version. They knew about the complaints on GitHub but pushed it out anyway.
I have to say, I love Home Assistant and Zigbee2MQTT so much more now. If you do have a problem, you have more tools at your disposal to diagnose and fix it. Always remember: you’ll always be the sysadmin for your smart home - it’s best to not be limited by the software or you’ll never hear the end of a problem from your significant other, lol.
I spent hours inspecting packets. When the hub doesn’t see packets for the action, it’s because the switch isn’t sending them at all. I did not see anything being sent out by the switch during that time. It’s almost as if the switch “gave up”. Within 1 minute, it worked again.
On another note, I’m not sure why everything is multicasted. If a switch setting is updated, or a switch is turned on/off - there is no reason every zigbee device needs to know that information, yet it is sent as a multicast packet. I think when all the network flooding issues are resolved, most of these complaints will disappear with it. The switches seem to have limited RAM and cannot handle much all at once.
Ditto to Terrance’s comment. I’m in the same/similar boat, 47 Blue switches meshed using ZHA on Home assistant, with Juno tunable wafers, and the same network congestion/flooding as others. Happy to to provide any information people need, run tests, or try any beta firmware in an effort to solve this.
This/these issues really are hampering my ability to make the mesh in our remodel stable and or get the whole house up and running.
I’d do whatever it takes to help expedite this process because as of right now I only have about 1/2 the house up and running before inclusions and traffic brings the house of cards crashing down.
Can’t wait to get over these hurdles and get the house set up so programming/automation can commence, but have been in a holding pattern for months now.
Strange. I turned off “Legacy Home Assistant Entity’s”, rebooted HA, and now everything appears stable. Only time will tell, but 3 days in and every switch is responding like normal.
v2.10 on every switch.
Latest HA and latest Z2M.
Intel NUC i7 w/64 GB of RAM (yes, overkill)
SONOFF 3.0 zigbee coordinator (P Texas instruments version) flashed w/ latest stable firmware.
Hooked up via newest USB standard using USB 3.x extension cable.
LQI’s are still low, but I just paired them 3 days ago so it’ll probably get better with time. No problems with network performance though. Now - if it gets worse over time I would presume that the switches are running out of RAM and that may be the core problem. Only time will tell…
Quick update regarding new firmware and updates. While actively testing the various firmwares, we identified an issue that some devices won’t accept an OTA (as mentioned above).
It seems to affect mostly the 1st batch of switches, but still it has been very few. Because of this, we want to roll multiple updates into one to reduce the chances that anything negative might happen.
Since the last update, we’ve gone through a few iterations of firmware and we are currently working on firmware 2.12 with some pretty significant bug fixes regarding routing, traffic reduction, and memory adjustments (as well as some various other bug fixes).
Because of the update issue mentioned we are taking it slow and not releasing it until we know for sure it will solve these issues.
Regarding the issue - we waited to make this public because we wanted to come up with a couple solutions just in case your switch can’t update.
That said we have a tested a successful fix for this issue and are working on an easier solution which would re-write the bootloader via an ota update (this is still being tested).
The current solution requires a USB programmer and we will help anyone out with it if they are having this issue with updating. If you run across this, feel free to reach out to support and we can either let you borrow a programmer, direct you to one on Amazon or you can send you switch(es) in.
Again, this issue has been rare so far, but we just wanted to play it safe.
I was using Hubitat at the time when it was available for download. Now I’m using Z2M. I have the OTA if you want to try it. Flashing via Z2M is possible with any OTA.
Eric, I noticed I killed a switch (temporarily) when it was on v2.00 and I upgraded directly to v2.10. But… v2.00 to 2.08 and then v2.08 to v2.10 worked fine on the same switch. It was an early switch that wasn’t on the bad MAC list. I had to take apart the switch to force a flash.
Just jumping in this thread – I can say that I’m now starting to see similar issues and have a pretty decent sized network:
34x Blue Series 2-1 Switches
4x Blue Series Fan Switches
12x Zigbee Smart Bulbs paired directly to ST
I haven’t read through this entire thread, but in working with the engineers, one of the theories is that the memory for routing is getting filled up and causing problems for people that have large networks where the devices don’t talk directly to the controller. In fact, sometimes if the routing table is full and a device tries to route through it or talk to it, the packet will drop and may even cause it to reboot if it happens too much.
I’ve seen some issues at my house where bindings will drop every now and then. There’s no rhyme or reason as to why, and usually if I turn off and then on the switch again, the bindings work. A minor annoyance, but an annoyance that needs to be fixed. I only have a maximum 3x devices bound together in all my setups, so I can’t imagine the people with a ton of bulbs bound to the switch.
One of the fixes is to increase the routing table size by double. This is what I will be testing over the next week or so once firmware is given to us (should be any day).
I have a Zniffer that is analyzing packets and Eric M’s been sorting through all the logs to help, so I’m hopeful this will solve the binding issues (as well as network drops, etc) on larger networks.
Edit: Forgot to say thanks for volunteering! I’ll leave it up to Eric M if he needs additional help.
Really appreciate the update, and glad to hear others are experiencing the same issues (thought I was going crazy at the beginning). Can’t wait for the fix but completely understand why you guys are taking your time.
