As of about 5:40 AM (US Central time), both my indoor and outdoor monitors dropped offline, and have been showing brief intermittent contact as shown on the dashboard. The network is up and running at about 850MBit speed, and both are up. (Indoor showing purple). Is AG having a server issue, by chance?
1 Like
Ditto here - my two indoor and one outdoor AGs. Purple light on indoor. From the same time (10:40 UK).
As expected, working fine on HA.
Same for all of my I-9PSL (Kit) and O-1PST (Kit) devices.
Same here from about 11:40 Uhr (German time), several O-1PST devices at different places, Wifi stable at all places.
Same here - outdoor monitor noted intermittenly offline around 5:40 AM (US Central time) and indoor monitor had purple light.
Purple light was out at 9:35 AM (US Central time) and outdoor monitor showed as online.
Looks like I have been stable now for a bit as well.
Same boat here. Tried to post a similar topic about an hour ago. Power cycled, reconnected to wifi (all good, and 2.4ghz), and firmware is up to date, but still solid blue/purple in upper left corner.
I am seeing the same, also at about 05:40 PDT. There is no network issue locally, as I can ping the AG and retrieve data in Home Assistant.
There is also no issue locally with internet connectivity
Same here. Two devices at different locations went offline about three hours ago. Wifi and internet at both locations is fine.
Same, likely a server issue since the device itself is reporting readings on the console.
Hi all,
Thanks for the reports!
There was a technical incident on our data centre service provider’s side related to the load balancer. This is what caused monitors to appear offline or send data only intermittently to the AirGradient server, even though the devices themselves were still running and connected locally.
The issue has now been addressed, and monitors have recovered. We are continuing to monitor the situation to make sure the connection remains stable.
In most cases, no action should be needed and monitors should recover automatically.
5 Likes
whats the load like on your servers out of interest. no need to disclose secrets but is it a constant headache ?
How many reliable always on monitors report to the system daily ?
Hi @Pink_House, thanks for the question!
Let me find out with our development team, and I’ll let you know as soon as I get an answer. Thanks!
1 Like
Hi @Pink_House, I was informed that the server load was not that high. The limitation was related to the load balancer, which Hetzner didn’t let us monitor.
1 Like
Great, thank you for the information !
1 Like
My pleasure! We will consistently improve our infrastructure to reduce the chance of situations like this and minimize downtime as much as possible.
Please always feel free to let us know if you experience any issues.
My monitors keep dropping offline for the last two days. Is this issue known? They are online about half the time and have dropped about 40 times each the last two days. I do have a firewall rule, they are only allowed connection to the AirGradient server. Did you change your server configuration and should I whitelist more ip addresses?
Hi @Bart_Netherlands, thanks for reaching out to us. Please let me confirm this with our team, and I’ll give you an update. Could you let me know what IP addresses are currently whitelisted? Thanks!
91.98.9.143 and private networks: 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
@Bart_Netherlands Thanks for the info! Currently, we have two IP addresses for our hardware endpoints.
91.98.9.143
46.225.42.92
Could you please try adding to your whitelist and see if the issue is fixed?