Wow, you really outdid yourself. When I did the AQI calculation a year ago I just settled for that 18h and didnt want to bother with calculating provisional numbers. And just build up data after that initial start. I still had issues with spikes in the input sensor which causes the AQI to bump hours later which should be compensated for I think.
I will try your code both on inside and outside unit soon.
Make sure to pull the latest. I found some issues after my original changes. Things appear to be stable and calculating properly now.
I also have a bit of cleanup for the DIY Pro version (spent way too much time finding better-looking fonts) but will find an appropriate thread to post about those updates.
And in case anyone is curious, here’s the card I’m using in Home Assistant. It uses stack-in-card, card-mod, and mini-graph-card from HACS.
type: custom:stack-in-card
mode: vertical
keep:
background: true
border_radius: false
margin: false
cards:
- type: markdown
content: >
{%- set category =
states('sensor.airgradient_open_air_airgradient_open_air_nowcast_category')
-%}
{% if category == "unknown" %} # NowCast: Calculating
{% set mins =
states('sensor.airgradient_open_air_nowcast_minutes_remaining')
| int %} Available in {% if mins > 60 %}{{ mins // 60 }} hours {% endif
%}{{ mins%60 }} minutes
{% else %}
# NowCast: {{ category }}
{% endif %}
card_mod:
style: |
ha-card {
{% set aqi = states('sensor.airgradient_open_air_airgradient_open_air_nowcast') -%}
{%- set aqi = aqi | float if is_number(aqi) else -1 -%}
{%- if aqi < 0 -%}
background-color: #a0a0a0; color: #393939;
{%- elif aqi < 50 -%}
background-color: #00e400; color: #004000;
{%- elif aqi < 100 -%}
background-color: #ffff00; color: #404000;
{%- elif aqi < 150 -%}
background-color: #ff0000; color: #400000;
{%- elif aqi < 200 -%}
background-color: #8f3f97; color: #300030;
{%- else -%}
background-color: #7e0023; color: #200000;
{% endif %}
text-align: center;
}
h1 {
line-height: 1px;
margin: 0 !important;
padding: 0 !important;
border: 1px solid red !impotant;
}
- type: conditional
conditions:
- entity: sensor.airgradient_open_air_airgradient_open_air_nowcast
state_not: unknown
card:
type: gauge
entity: sensor.airgradient_open_air_airgradient_open_air_nowcast
needle: true
min: 0
max: 500
segments:
- from: 0
color: '#00e400'
- from: 50
color: '#ffff00'
- from: 100
color: '#ff7e00'
- from: 150
color: '#ff0000'
- from: 200
color: '#8f3f97'
- from: 300
color: '#7e0023'
name: ''
- type: custom:mini-graph-card
entities:
- entity: sensor.airgradient_open_air_airgradient_open_air_nowcast
color_thresholds:
- value: 0
color: '#00e400'
- value: 50
color: '#ffff00'
- value: 100
color: '#ff0000'
- value: 200
color: '#8f3f97'
- value: 300
color: '#7e0023'
show:
labels: false
fill: fade
name: false
state: true
icon: false
min_bound_range: 20
line_width: 4
hours_to_show: 24
points_per_hour: 1
card_mod:
style: |
div.states {
display: none !important;
position: absolute;
bottom: 0;
left: 0;
}
ha-card:hover div.states {
display:block !important;
}
- type: markdown
content: |
{%- set category =
states('sensor.airgradient_open_air_airgradient_open_air_aqi_category')
-%} {% if category == "unknown" %} # AQI: Calculating
{% set mins = states('sensor.airgradient_open_air_aqi_minutes_remaining')
| int %} Available in {% if mins > 60 %}{{ mins // 60 }} hours {% endif
%}{{ mins%60 }} minutes
{% else %} # AQI: {{ category }} {% endif %}
card_mod:
style: |
ha-card {
{% set aqi = states('sensor.airgradient_open_air_airgradient_open_air_aqi') -%}
{%- set aqi = aqi | float if is_number(aqi) else -1 -%}
{%- if aqi < 0 -%}
background-color: #a0a0a0; color: #393939;
{%- elif aqi < 50 -%}
background-color: #00e400; color: #004000;
{%- elif aqi < 100 -%}
background-color: #ffff00; color: #404000;
{%- elif aqi < 150 -%}
background-color: #ff0000; color: #400000;
{%- elif aqi < 200 -%}
background-color: #8f3f97; color: #300030;
{%- else -%}
background-color: #7e0023; color: #200000;
{% endif %}
text-align: center;
}
h1 {
line-height: 1px;
margin: 0 !important;
padding: 0 !important;
border: 1px solid red !impotant;
}
- type: conditional
conditions:
- entity: sensor.airgradient_open_air_airgradient_open_air_aqi
state_not: unknown
card:
type: gauge
entity: sensor.airgradient_open_air_airgradient_open_air_aqi
needle: true
min: 0
max: 500
segments:
- from: 0
color: '#00e400'
- from: 50
color: '#ffff00'
- from: 100
color: '#ff7e00'
- from: 150
color: '#ff0000'
- from: 200
color: '#8f3f97'
- from: 300
color: '#7e0023'
name: ''
- type: custom:mini-graph-card
entities:
- entity: sensor.airgradient_open_air_airgradient_open_air_aqi
color_thresholds:
- value: 0
color: '#00e400'
- value: 50
color: '#ffff00'
- value: 100
color: '#ff0000'
- value: 200
color: '#8f3f97'
- value: 300
color: '#7e0023'
show:
labels: false
fill: fade
name: false
state: true
icon: false
min_bound_range: 20
line_width: 4
hours_to_show: 24
points_per_hour: 1
card_mod:
style: |
div.states {
display: none !important;
position: absolute;
bottom: 0;
left: 0;
}
ha-card:hover div.states {
display:block !important;
}
- type: horizontal-stack
cards:
- type: custom:mini-graph-card
entities:
- entity: >-
sensor.airgradient_open_air_airgradient_open_air_particulate_matter_1_0um_concentration
name: PM1
state_adaptive_color: true
color: wheat
line_width: 2
hours_to_show: 24
min_bound_range: 10
show:
labels: false
fill: fade
name: true
icon: false
name_adaptive_color: true
- type: custom:mini-graph-card
entities:
- entity: >-
sensor.airgradient_open_air_airgradient_open_air_particulate_matter_2_5um_concentration
name: PM2.5
state_adaptive_color: true
color: tan
line_width: 2
hours_to_show: 24
min_bound_range: 10
show:
labels: false
fill: fade
name: true
icon: false
name_adaptive_color: true
- type: custom:mini-graph-card
entities:
- entity: >-
sensor.airgradient_open_air_airgradient_open_air_particulate_matter_10_0um_concentration
name: PM10
state_adaptive_color: true
color: burlywood
line_width: 2
hours_to_show: 24
min_bound_range: 10
show:
labels: false
fill: fade
name: true
icon: false
name_adaptive_color: true
columns: 1
For those following along, I submitted a PR for my changes to both the Outdoor/Open Air and DIY Pro models: Quality of life and other (somewhat major) updates for AirGradient Outdoor and DIY Pro by ex-nerd · Pull Request #17 · ajfriesen/ESPHome-AirGradient · GitHub
Is there a way to save/restore 24 hours of data so that on a reboot it doesn’t have the delay for AQI and Now Cast?
It’s possible to store numbers on the flash on the device, but the info I found basically says it has about 10k writes before it starts to degrade. With hourly updates, that’s about a year and a half before the device stops working properly.
Others are doing the calculation in Home Assistant itself, but I haven’t had time to look into how to actually query past/logged states and build that into a value that can be added to a dashboard.
Great to see such an active discussion on the new outdoor monitor and ESPHome integrations.
The flash chip on the C3 is indeed limited with write cycles and I would also recommend to offload the calculation to the server and only transmitting raw data. This gives more flexibility to show also the last current measured data to show trends etc.
I received my outdoor kit last week and have been testing this esphome configuration since then and other than the 18h/12h delays it has been working well.
My problem has been that a few very small power interruptions return the 18h/12h delays on the AQI making it hard to rely on the value.
I also experienced reboots (crashes?) when the device can’t upload to airgradient (internet outage or firewall block) which again end up restarting the counter.
I finally commented out the upload to airgradient and added the calculations to home-assistant. The AQI was pretty straightforward using a 24h statistic sensor and template sensors. But the NowCast is complex to do in Home-Assistant using the official calculations. After testing a few scenarios it seems that using a 3h statistics sensor with the AQI calculations return values that are very close to the NowCast values returned by Esphome.
Regarding the query for statistics from Home-Assistant, this can be done using the websocket API. I don’t think it is documented but the relevant code in Home-Assistant is here
## AQI 3h
sensor:
- platform: statistics
name: PM2.5 Mean 3h
unique_id: pm2_5_mean_3h
entity_id: sensor.airgradient_airgradient_open_air_particulate_matter_2_5um_concentration
state_characteristic: mean
max_age:
hours: 3
- platform: statistics
name: PM10 Mean 3h
unique_id: pm10_mean_3h
entity_id: sensor.airgradient_airgradient_open_air_particulate_matter_10_0um_concentration
state_characteristic: mean
max_age:
hours: 3
- platform: statistics
name: PM2.5 Mean 24h
unique_id: pm2_5_mean_24h
entity_id: sensor.airgradient_airgradient_open_air_particulate_matter_2_5um_concentration
state_characteristic: mean
max_age:
hours: 24
- platform: statistics
name: PM10 Mean 24h
unique_id: pm10_mean_24h
entity_id: sensor.airgradient_airgradient_open_air_particulate_matter_10_0um_concentration
state_characteristic: mean
max_age:
hours: 24
template:
- sensor:
- name: "AQI EPA PM2.5 3h"
unique_id: aqi_epa_pm25_3h
device_class: aqi
state: >
{% set pm25 = states('sensor.pm2_5_mean_3h')|float %}
{% if pm25 < 12.0 %}
{{ ((50.0 - 0.0) / (12.0 - 0.0) * (pm25 - 0.0) + 0.0)|round }}
{% elif pm25 < 35.4 %}
{{ ((100.0 - 51.0) / (35.4 - 12.1) * (pm25 - 12.1) + 51.0)|round }}
{% elif pm25 < 55.4 %}
{{ ((150.0 - 101.0) / (55.4 - 35.5) * (pm25 - 35.5) + 101.0)|round }}
{% elif pm25 < 150.4 %}
{{ ((200.0 - 151.0) / (150.4 - 55.5) * (pm25 - 55.5) + 151.0)|round }}
{% elif pm25 < 250.4 %}
{{ ((300.0 - 201.0) / (250.4 - 150.5) * (pm25 - 150.5) + 201.0)|round }}
{% elif pm25 < 350.4 %}
{{ ((400.0 - 301.0) / (350.4 - 250.5) * (pm25 - 250.5) + 301.0)|round }}
{% elif pm25 < 500.4 %}
{{ ((500.0 - 401.0) / (500.4 - 350.5) * (pm25 - 350.5) + 401.0)|round }}
{% else %}
500
{% endif %}
- name: "AQI EPA PM10 3h"
unique_id: aqi_epa_pm10_3h
device_class: aqi
state: >
{% set pm10 = states('sensor.pm10_mean_3h')|float %}
{% if pm10 < 54.0 %}
{{ ((50.0 - 0.0) / (54.0 - 0.0) * (pm10 - 0.0) + 0.0)|round }}
{% elif pm10 < 154.0 %}
{{ ((100.0 - 51.0) / (154.0 - 55.0) * (pm10 - 55.0) + 51.0)|round }}
{% elif pm10 < 254.0 %}
{{ ((150.0 - 101.0) / (254.0 - 155.0) * (pm10 - 155.0) + 101.0)|round }}
{% elif pm10 < 354.0 %}
{{ ((200.0 - 151.0) / (354.0 - 255.0) * (pm10 - 255.0) + 151.0)|round }}
{% elif pm10 < 424.0 %}
{{ ((300.0 - 201.0) / (424.0 - 355.0) * (pm10 - 355.0) + 201.0)|round }}
{% elif pm10 < 504.0 %}
{{ ((400.0 - 301.0) / (504.0 - 425.0) * (pm10 - 425.0) + 301.0)|round }}
{% elif pm10 < 604.0 %}
{{ ((500.0 - 401.0) / (604.0 - 505.0) * (pm10 - 505.0) + 401.0)|round }}
{% else %}
500
{% endif %}
- name: "AQI EPA 3h"
unique_id: aqi_epa_3h
device_class: aqi
state: >-
{{ [states('sensor.aqi_epa_pm10_3h')|int, states('sensor.aqi_epa_pm2_5_3h')|int]|max }}
- name: "AQI EPA Category 3h"
unique_id: aqi_epa_category_3h
state: >-
{% set aqi = states('sensor.aqi_epa_3h')|int %}
{% if aqi <= 50.0 %}
"Good"
{% elif aqi <= 100.0 %}
"Moderate"
{% elif aqi <= 150.0 %}
"Unhealthy for Sensitive Groups"
{% elif aqi <= 200.0 %}
"Unhealthy"
{% elif aqi <= 300.0 %}
"Very Unhealthy"
{% elif aqi <= 500.0 %}
"Hazardous"
{% else %}
"Extreme Hazardous"
{% endif %}
## AQI EPA 24h
- name: AQI EPA PM2.5
unique_id: aqi_epa_pm25
device_class: aqi
state: >
{% set pm25 = states('sensor.pm2_5_mean_24h')|float %}
{% if pm25 < 12.0 %}
{{ ((50.0 - 0.0) / (12.0 - 0.0) * (pm25 - 0.0) + 0.0)|round }}
{% elif pm25 < 35.4 %}
{{ ((100.0 - 51.0) / (35.4 - 12.1) * (pm25 - 12.1) + 51.0)|round }}
{% elif pm25 < 55.4 %}
{{ ((150.0 - 101.0) / (55.4 - 35.5) * (pm25 - 35.5) + 101.0)|round }}
{% elif pm25 < 150.4 %}
{{ ((200.0 - 151.0) / (150.4 - 55.5) * (pm25 - 55.5) + 151.0)|round }}
{% elif pm25 < 250.4 %}
{{ ((300.0 - 201.0) / (250.4 - 150.5) * (pm25 - 150.5) + 201.0)|round }}
{% elif pm25 < 350.4 %}
{{ ((400.0 - 301.0) / (350.4 - 250.5) * (pm25 - 250.5) + 301.0)|round }}
{% elif pm25 < 500.4 %}
{{ ((500.0 - 401.0) / (500.4 - 350.5) * (pm25 - 350.5) + 401.0)|round }}
{% else %}
500
{% endif %}
- name: "AQI EPA PM10"
unique_id: aqi_epa_pm10
device_class: aqi
state: >
{% set pm10 = states('sensor.pm10_mean_24h')|float %}
{% if pm10 < 54.0 %}
{{ ((50.0 - 0.0) / (54.0 - 0.0) * (pm10 - 0.0) + 0.0)|round }}
{% elif pm10 < 154.0 %}
{{ ((100.0 - 51.0) / (154.0 - 55.0) * (pm10 - 55.0) + 51.0)|round }}
{% elif pm10 < 254.0 %}
{{ ((150.0 - 101.0) / (254.0 - 155.0) * (pm10 - 155.0) + 101.0)|round }}
{% elif pm10 < 354.0 %}
{{ ((200.0 - 151.0) / (354.0 - 255.0) * (pm10 - 255.0) + 151.0)|round }}
{% elif pm10 < 424.0 %}
{{ ((300.0 - 201.0) / (424.0 - 355.0) * (pm10 - 355.0) + 201.0)|round }}
{% elif pm10 < 504.0 %}
{{ ((400.0 - 301.0) / (504.0 - 425.0) * (pm10 - 425.0) + 301.0)|round }}
{% elif pm10 < 604.0 %}
{{ ((500.0 - 401.0) / (604.0 - 505.0) * (pm10 - 505.0) + 401.0)|round }}
{% else %}
500
{% endif %}
- name: "AQI EPA"
unique_id: aqi_epa
device_class: aqi
state: >-
{{ [states('sensor.aqi_epa_pm10')|int, states('sensor.aqi_epa_pm2_5')|int]|max }}
- name: "AQI EPA Category "
unique_id: aqi_epa_category
state: >-
{% set aqi = states('sensor.aqi_epa')|int %}
{% if aqi <= 50.0 %}
"Good"
{% elif aqi <= 100.0 %}
"Moderate"
{% elif aqi <= 150.0 %}
"Unhealthy for Sensitive Groups"
{% elif aqi <= 200.0 %}
"Unhealthy"
{% elif aqi <= 300.0 %}
"Very Unhealthy"
{% elif aqi <= 500.0 %}
"Hazardous"
{% else %}
"Extreme Hazardous"
{% endif %}
Great work, thanks for the info. I’ve been looking into the SQL plugin to get at the raw data to do the AQI and NowCast stuff but this is certainly easier. I’ll probably still do a deeper dive into this because I want to learn the APIs and get an “actual proper calculation” written (just for the sake of doing it … I think the official midnight-to-midnight AQI isn’t helpful within a home assistant context) but this really helps me get a head start on how to get at some of this info.
I started looking into the AQI calculations based on the SQL database itself, and discovered that the Home Assistant database seems to be missing a lot of readings from the AirGradient. I’m not sure if this is something about HA, or if the AirGradient device itself is skipping readings, but it’s definitely another kink to work out in getting everything running.
You’ll obviously need to adjust the sensor ID to match yours, but here’s some sample SQL you can run directly against the sqlite database (I run HA in a docker container, not sure how to access the raw db if you run HA-OS). It returns the average state and number of readings for every hour (should be a reading every 3 minutes, so 20 per hour).
SELECT
datetime(FLOOR(AVG(states.last_updated_ts)/3600) * 3600, 'unixepoch') AS by_hour,
COUNT(*) AS count,
AVG(states.state) AS state
FROM
states
INNER JOIN states_meta ON
states.metadata_id = states_meta.metadata_id
-- LEFT JOIN state_attributes ON
-- states.attributes_id = state_attributes.attributes_id
WHERE
-- states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_2_5um_concentration'
states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_2_5um_concentration_1'
-- OR states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_2_5um_concentration_2'
-- states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_10_0um_concentration'
-- states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_10_0um_concentration_1'
-- OR states_meta.entity_id = 'sensor.airgradient_open_air_airgradient_open_air_particulate_matter_10_0um_concentration_2'
AND states.state != 'unavailable'
AND states.last_updated_ts >= unixepoch(date('now','start of day','-1 day'))
-- AND states.last_updated_ts < unixepoch(date('now','start of day'))
GROUP BY
floor(states.last_updated_ts/3600) -- group by hour
ORDER BY
states.last_updated_ts DESC
Replace 3600 with 60 to group things by minute , which makes it pretty easy to see where there are gaps.
I’ve been watching the on-device logs through its web console and I don’t see any errors or skipped time codes, so I suspect this is a communication issue with esphome and home assistant, rather than something blocking the readings on the device itself.
Anyway, just an annoyance to work through.
After a bit more digging, it seems this is “a feature, not a bug” … meaning that HA only logs sensor states when the value changes. Annoyingly (I say this as a sometimes DBA) they don’t even track the created/updated timestamps so will have to do a bit more db magic to make this all work.
I am using your repo for the unit and noticed I am getting resets according to Home Assistant.
Here is the reset reason:
Timer Group 0 Watch Dog Reset Digital Core
Here is what I am seeing from Home Assistant as to the uptime:
Any thoughts on what to tweak or check out? Thanks!
I don’t. Mine seems to do that when it has trouble reaching the internet. I started digging into an SQL query for home assistant to move these calculations there, but … life got in the way and the on-device calculations are good enough most of the time.
The calculations wouldn’t be particularly difficult but HA only stores a new value when/if it changes so it makes things more complicated to “fill in” values from the hours when nothing changes.
I also just noticed that someone submitted a pull request that should help make some statistics queries easier in home assistant, so I merged that.
See this thread that is likely related, regarding esphome and reboots with the PMS sensor with default settings
I just read your issue with not having all the data in HA. That is because by default it will only store changed values to save on storage. In esphome its possible to force it to update with force_update: true on a sensor.
Because I copy my data to influx I let Grafana ‘fix’ the missing data points so I have best of both worlds.
