I’m guessing that your D1 is unexpectedly restarting. Try adding Uptime Sensor — ESPHome to see how long it stays up. There are other topics on unexpected restarts to diagnose.
Yeah. First things first…
ESPHome uses the platform-espressif8266 library https://github.com/platformio/platform-espressif8266 and the latest version v4.1.0 released 3 weeks ago pulled in the ESP8266 Arduino library v3.1.x which in turns pulled in the latest SoftwareSerial library v7.0.0 which is what is causing a lot of exception 0 crashes.
SoftwareSerial is pushing some new commits to address some of the code-breaking changes of their latest v7.0.0. This is a result of my issue report on the ESP8266 Arduino library repository. We’ll see if this fixes things, but for now, you need to revert to SoftwareSerial v6.17.1. ESP8266 Arduino library has also pushed a commit to revert SoftwareSerial for the time being.
1 Like
Thanks both for the pointers. The uptime sensor indicates it is a reboot issue; the total resets at the same time as the display going unavailable. I tried downgrading the ESPhome platform_version to 4.0.1; it doesn’t seem to have addressed the problem, unfortunately.
Updated ESP2866 YAML
esp8266:
board: d1_mini
framework:
version: recommended
platform_version: 4.0.1
Results
Processing airgradient-pro (board: d1_mini; framework: arduino; platform: platformio/espressif8266 @ 4.0.1)
I guess first thing is to see if your crashes are Exception 0 crashes. Should just plug in to monitor the serial output to see if it spits out an exception 0 with stack dump. If so, it’s highly-likely it’s the same issue and you need to be absolutely sure you have SoftwareSerial version 6.17.1. Download the source files from github and do file comparisons between your installed library and the one you downloaded. And then there’s the possibility: I think in Arduino IDE if I’ve compiled once, replacing one of the library source files and hitting the “re-compiling” didn’t do anything and I had to close the IDE and re-open before re-compiling. Not sure how that works in PlatformIO.
Thanks Ken. I’ve been able to get more details out of the ESPhome setup (along with ruling out a few other things like bus speed increases or adequate power, as far as I can tell). The debug component is giving a bit more detail about restarts even if the logging is unusually awkward to persist (bearing in mind it typically takes 6-8 hours before a reboot happens - I was expecting to be able to e.g. syslog it).
Anyway, the reboot cause is listed as due to the Software Watchdog. I’ll see if I can get more direct log details as you suggest.
Edit: The debug component gave some further info on the fault - sounding like exceptions of some kind:
Reset: Software Watchdog|Fatal exception:4 flag:3 (Software Watchdog) epc1:0x4026d4c2 epc2:0x00000000 epc3:0x0000
For ESPHome specifically, this might be the same phenomenon that @MallocArray saw with theirs. It’s a WDT timeout that was causing it and (s)he reduced the frequency of reads by increasing the update interval and it seems to work. See these posts:
https://forum.airgradient.com/t/airgradient-rebooting-itself/603/36
https://forum.airgradient.com/t/airgradient-rebooting-itself/603/42
This certainly does sound like my issue as well, and while I do believe it is related to the chatty output of the PMS sensor, I don’t know for sure that it is directly related to the espressif8266 release 4.1.0 version. In my testing, if I don’t define a version or platform_version under framework in ESPHome, it defaults to the 3.2.0 version which predates the updated SoftwareSerial library as far as I can tell.
But I have found 2 ways to help with my issue.
The best long term solution I’ve found so far is to use:
logger:
level: INFO
This reduces all of the logging output to only show INFO level which is nearly nothing. No regular updates from the PMS sensor or a TVOC/Temp sensor, although they are still sending to Home Assistant. The issue can be slightly mitigated by changing the update_inverval: value on each sensor. Both the PMS and SPG sensors default to every second, which makes the logs very busy. If I change this to 60s or 120s it drastically reduces the occurances of the reboot, but I was still seeing them happen, just much further between.
So for now, I’m setting the logging to INFO, but now I can’t see the baseline my SPG30 ends up with, which is a pain.
I’m pretty sure the PMS is mostly to blame, as I believe I tested a config with it set to an update_inteval: 120s and left the SGP30 to default of every second and saw reduced reboots, but I may not have let it run log enough to make sure, since the defaults were causing reboots every 0.5 - 2h for me.
To completely silence the serial logging. But still have logging by web. As I do, but still have unexpected reboots.
logger:
baud_rate: 0
Its also possible to silence individual components like
logger:
level: DEBUG
logs:
sensor: INFO
sgp30: ERROR
senseair: INFO
sht3xd: INFO
pmsx003: INFO
Currently I only have psm5003 and sgp30 connected and get reboots. I tried leaving logger at DEBUG but setting those two components to INFO and I still got reboots.
logger level INFO seems to have done it so far for me.
I can try changing the baud_rate with things still at DEBUG and see what that does.
I got lucky and the serial connection captured several stack traces. Ironically, I had configured esphome logs to log to a file, and the file just contains default stack traces, but it looks like doing so meant the only console output (probably stderr) is a decoded set of stack traces, each attached below. Interestingly, they’re not all the same issue, as far as I can tell.
Stack Trace 1
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x40245443: lwip_cyclic_timer at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:261
WARNING Decoded 0x40273a64: etharp_output
WARNING Decoded 0x4024539d: memp_free at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/memp.c:447
WARNING Decoded 0x40266810: pm_keep_active_enable
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x4026c7f1: ets_timer_handler_isr
WARNING Decoded 0x4026c836: ets_timer_handler_isr
WARNING Decoded 0x40234a62: loop_task(ETSEventTag*) at core_esp8266_main.cpp
WARNING Decoded 0x40106015: call_user_start_local
WARNING Decoded 0x4010601b: call_user_start_local
WARNING Decoded 0x4010000d: call_user_start
WARNING Decoded 0x401000ab: app_entry_redefinable
WARNING Decoded 0x4025ca60: cont_ret at cont.S.o
WARNING Decoded 0x4025ca11: cont_continue at cont.S.o
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40100cf1: interrupt_handler
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100260: esphome::ISRInternalGPIOPin::digital_read()
WARNING Decoded 0x40231524: HardwareSerial::write(unsigned char const*, unsigned int) [215/1034]
WARNING Decoded 0x40103807: rcReachRetryLimit
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x401039e4: rcReachRetryLimit
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x401039e4: rcReachRetryLimit
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100c20: interrupt_handler
WARNING Decoded 0x40240930: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:727
WARNING Decoded 0x402403ec: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:484
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40216a13: esphome::ota::OTAComponent::handle_()
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x4023cfa5: __cvt at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_float.c:102
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x4023dae5: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:196 (discriminator 1)
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242054: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:179
WARNING Decoded 0x4023dc14: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:246
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242315: _svfprintf_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:528
WARNING Decoded 0x40242315: _svfprintf_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:528
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4020e927: esphome::api::APINoiseFrameHelper::try_read_frame_(esphome::api::APINoiseFrameHelper::ParsedFrame*)
WARNING Decoded 0x4020f010: esphome::api::APINoiseFrameHelper::state_action_()
WARNING Decoded 0x4026e65c: system_param_load
WARNING Decoded 0x40234b87: yield
WARNING Decoded 0x4020e927: esphome::api::APINoiseFrameHelper::try_read_frame_(esphome::api::APINoiseFrameHelper::ParsedFrame*)
WARNING Decoded 0x4020f010: esphome::api::APINoiseFrameHelper::state_action_()
WARNING Decoded 0x40226be8: ESP8266WiFiSTAClass::SSID() const
WARNING Decoded 0x4020f25e: esphome::api::APINoiseFrameHelper::read_packet(esphome::api::ReadPacketBuffer*)
WARNING Decoded 0x4026f297: wifi_station_get_connect_status
WARNING Decoded 0x4020d3d7: esphome::api::APIConnection::loop()
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x40220518: esphome::wifi::WiFiComponent::wifi_ssid[abi:cxx11]()
WARNING Decoded 0x40212238: esphome::api::APIServer::loop()
WARNING Decoded 0x401068f1: ets_timer_arm_new
WARNING Decoded 0x40234ae1: esp_yield
WARNING Decoded 0x402358fd: delay
WARNING Decoded 0x402213c3: esphome::Application::loop()
Stack Trace 2
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x40245443: lwip_cyclic_timer at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:261
WARNING Decoded 0x40273a5c: etharp_output
WARNING Decoded 0x4024539d: memp_free at /home/earle/src/esp-quick-toolchain/arduino/tools/sdk/lwip2/builder/lwip2-src/src/core/memp.c:447
WARNING Decoded 0x40266810: pm_keep_active_enable
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x4026c7f1: ets_timer_handler_isr
WARNING Decoded 0x4026c836: ets_timer_handler_isr
WARNING Decoded 0x40234a62: loop_task(ETSEventTag*) at core_esp8266_main.cpp
WARNING Decoded 0x40106015: call_user_start_local
WARNING Decoded 0x4010601b: call_user_start_local
WARNING Decoded 0x4010000d: call_user_start
WARNING Decoded 0x401000ab: app_entry_redefinable
WARNING Decoded 0x4025ca60: cont_ret at cont.S.o
WARNING Decoded 0x4025ca11: cont_continue at cont.S.o
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x40105f13: lmacRxDone
WARNING Decoded 0x40103807: rcReachRetryLimit
WARNING Decoded 0x401039e4: rcReachRetryLimit
WARNING Decoded 0x40103e9e: wDev_ProcessFiq
WARNING Decoded 0x40103bc0: wDev_ProcessFiq
WARNING Decoded 0x402090e4: fe25519_mul at ed25519_ref10.c
WARNING Decoded 0x402090cc: fe25519_mul at ed25519_ref10.c
WARNING Decoded 0x40105f13: lmacRxDone
WARNING Decoded 0x40103807: rcReachRetryLimit
WARNING Decoded 0x401039e4: rcReachRetryLimit
WARNING Decoded 0x40103e9e: wDev_ProcessFiq
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40105f13: lmacRxDone
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x40105f13: lmacRxDone
WARNING Decoded 0x40100778: ets_post [115/1034]
WARNING Decoded 0x40216a13: esphome::ota::OTAComponent::handle_()
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40100c20: interrupt_handler
WARNING Decoded 0x40240091: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:352
WARNING Decoded 0x4024008e: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:352
WARNING Decoded 0x40103e9e: wDev_ProcessFiq
WARNING Decoded 0x40103bc0: wDev_ProcessFiq
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40216a13: esphome::ota::OTAComponent::handle_()
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x4023cfa5: __cvt at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_float.c:102
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x4023dae5: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:196 (discriminator 1)
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242054: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:179
WARNING Decoded 0x4023dc14: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:246
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x4023657a: uart_do_write_char at uart.cpp
WARNING Decoded 0x40236580: uart_do_write_char at uart.cpp
WARNING Decoded 0x402365f8: uart_write
WARNING Decoded 0x40231524: HardwareSerial::write(unsigned char const*, unsigned int)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4026e65c: system_param_load
WARNING Decoded 0x40234b87: yield
WARNING Decoded 0x4026e68e: wifi_station_get_config
WARNING Decoded 0x402338f5: String::copy(char const*, unsigned int)
WARNING Decoded 0x40226be8: ESP8266WiFiSTAClass::SSID() const
WARNING Decoded 0x402236fb: void std::__push_heap<__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Sc
heduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, std::unique
_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_val<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Schedule
r::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)> >(__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, s
td::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler
::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp
_val<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::Schedul
erItem> > const&)>&)
WARNING Decoded 0x4021fe38: esphome::wifi::WiFiComponent::wifi_sta_ip()
WARNING Decoded 0x40223acc: void std::__adjust_heap<__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::
Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, std::uniq
ue_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_iter<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Sched
uler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)> >(__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem
, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Schedu
ler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_c
omp_iter<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::Sch
edulerItem> > const&)>)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x4020d5ae: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x40220518: esphome::wifi::WiFiComponent::wifi_ssid[abi:cxx11](
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x4020d5ae: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x40220518: esphome::wifi::WiFiComponent::wifi_ssid[abi:cxx11]()
WARNING Decoded 0x40212238: esphome::api::APIServer::loop()
WARNING Decoded 0x401068f1: ets_timer_arm_new
WARNING Decoded 0x40234ae1: esp_yield
WARNING Decoded 0x402358fd: delay
WARNING Decoded 0x402213c3: esphome::Application::loop()
Stack Trace 3
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x40254a42: sta_input
WARNING Decoded 0x40266810: pm_keep_active_enable
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x4026c7f1: ets_timer_handler_isr
WARNING Decoded 0x4026c836: ets_timer_handler_isr
WARNING Decoded 0x40234a62: loop_task(ETSEventTag*) at core_esp8266_main.cpp
WARNING Decoded 0x40106015: call_user_start_local
WARNING Decoded 0x4010601b: call_user_start_local
WARNING Decoded 0x4010000d: call_user_start
WARNING Decoded 0x401000ab: app_entry_redefinable
WARNING Decoded 0x4025ca60: cont_ret at cont.S.o
WARNING Decoded 0x4025ca11: cont_continue at cont.S.o
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100c20: interrupt_handler
WARNING Decoded 0x401023a8: unsign_to_sign
WARNING Decoded 0x40102410: phy_get_bb_freqoffset
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40100cf1: interrupt_handler
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40100260: esphome::ISRInternalGPIOPin::digital_read()
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40105a46: lmacMSDUAged
WARNING Decoded 0x40102c7c: pp_post
WARNING Decoded 0x40105f13: lmacRxDone
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40103807: rcReachRetryLimit
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100c20: interrupt_handler
WARNING Decoded 0x4021a8f4: esphome::socket::LWIPRawImpl::accept(sockaddr*, unsigned int*)
WARNING Decoded 0x40216a13: esphome::ota::OTAComponent::handle_()
WARNING Decoded 0x4021a8cc: esphome::socket::LWIPRawImpl::accept(sockaddr*, unsigned int*)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::Scheduler
Item, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x4010037e: esphome::uart::ESP8266SoftwareSerial::gpio_intr(esphome::uart::ESP8266SoftwareSerial*)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x40100300: esphome::uart::ESP8266SoftwareSerial::read_bit_(unsigned int*, unsigned int const&)
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40100ce4: interrupt_handler
WARNING Decoded 0x40216a13: esphome::ota::OTAComponent::handle_()
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40241e42: __d2b at /workdir/repo/newlib/newlib/libc/stdlib/mprec.c:779
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40240b9d: _dtoa_r at /workdir/repo/newlib/newlib/libc/stdlib/dtoa.c:854
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x4023dae5: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:196 (discriminator 1)
WARNING Decoded 0x40242118: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:232
WARNING Decoded 0x40242054: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:179
WARNING Decoded 0x4023dc14: _printf_i at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf_i.c:246
WARNING Decoded 0x40242054: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:179
WARNING Decoded 0x40100778: ets_post
WARNING Decoded 0x401067ad: ets_timer_disarm
WARNING Decoded 0x40242054: __ssputs_r at /workdir/repo/newlib/newlib/libc/stdio/nano-vfprintf.c:179
WARNING Decoded 0x402364e3: uart_tx_fifo_available
WARNING Decoded 0x40236580: uart_do_write_char at uart.cpp
WARNING Decoded 0x40232af0: run_scheduled_recurrent_functions()
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x40234b87: yield
WARNING Decoded 0x4023665c: uart_write
WARNING Decoded 0x4023f819: _vsnprintf_r at /workdir/repo/newlib/newlib/libc/stdio/vsnprintf.c:71 (discriminator 4)
WARNING Decoded 0x40231524: HardwareSerial::write(unsigned char const*, unsigned int)
WARNING Decoded 0x4026e65c: system_param_load
WARNING Decoded 0x4026e68e: wifi_station_get_config
WARNING Decoded 0x402338f5: String::copy(char const*, unsigned int)
WARNING Decoded 0x40226be8: ESP8266WiFiSTAClass::SSID() const
WARNING Decoded 0x402236fb: void std::__push_heap<__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_val<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)> >(__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_val<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)>&)
WARNING Decoded 0x4021fe38: esphome::wifi::WiFiComponent::wifi_sta_ip()
WARNING Decoded 0x40223acc: void std::__adjust_heap<__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_iter<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)> >(__gnu_cxx::__normal_iterator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >*, std::vector<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, std::allocator<std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > > > >, int, int, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> >, __gnu_cxx::__ops::_Iter_comp_iter<bool (*)(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)>)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x4020d5ae: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&)
WARNING Decoded 0x402234f0: esphome::Scheduler::SchedulerItem::cmp(std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&, std::unique_ptr<esphome::Scheduler::SchedulerItem, std::default_delete<esphome::Scheduler::SchedulerItem> > const&)
WARNING Decoded 0x40220518: esphome::wifi::WiFiComponent::wifi_ssid[abi:cxx11]()
WARNING Decoded 0x40212238: esphome::api::APIServer::loop()
WARNING Decoded 0x401068f1: ets_timer_arm_new
WARNING Decoded 0x40234ae1: esp_yield
WARNING Decoded 0x402358fd: delay
WARNING Decoded 0x402213c3: esphome::Application::loop()
Trace 1 looks like GPIO issues…
Trace 2 looks related to OTA…
Trace 3 looks like SoftwareSerial issues (this board is running Core 3.0.2)
Potentially related, I am seeing a lot of these in Verbose logs
[01:29:00][V][component:200]: Component ssd1306_base took a long time for an operation (0.32 s).
[01:29:00][V][component:201]: Components should block for at most 20-30ms.
I’m not yet sure what to make of these, other than potentially the entire board has too much to do.
Edit: The other interesting coincidence is that just before each stack trace I see this log - not actually sure what it is but my guess is the particulate sensor?
[01:29:02]pm 1990
It doesn’t have a log level - which feels rather unusual.
I should add I changed my ESPhome config for update intervals based off the prior links to
sht3xd 5s
senseair 30s
pmsx003 120s (also just for spin-down & laser lifetime as I understand)
Hey, sorry to hijack this thread. I’ve been having issues on the AirGradient Pro random restarts too and found this thread on google.
Downgrading SoftwareSerial library to v6.17.1 like you suggested solved my issues completely it seems. This fix didn’t seem to work at first, but I noticed that during compilation after downgrading Software Serial my compilation logs still reported that it was still compiling with a cached compile of v7.0.0.
So just throwing this out there to anyone else who finds this thread on google: if you have an exception 0 and you downgraded SoftwareSerial, make sure v7.0.0 isn’t still cached. If it is, you can deleted the cached compile at C:\Users{user}\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries (path may be different in your case)
And big thank you to @ken830, there’s no way I would have figured that out on my own
1 Like
@Themis3000 You’re welcome! I spent several hours a day for over a week working on the problem and documented my findings in this other thread: https://forum.airgradient.com/t/airgradient-rebooting-itself/603
I’m just glad to hear that the results of my work has been able to help others. Makes it worth my time.
1 Like
@ken830 Wow, that’s a lot of work. I really appreciate your work and you spreading your findings. I’m totally new to Arduino (and c++ for that matter) so I would have probably given up trying to customize the device’s software to my needs and settled for what the 1 click install provided on their website if it weren’t for you. Luckily though, thanks to your answer I was able to program my AirGradient pro to fit my exact needs! This forum comes up near the top of google when searching for this issue, so I’m sure it’ll go on to help others too.
Hope you have a good rest of your night!
1 Like
Much appreciated for the ideas so far in this and the linked thread. Sadly I’m still getting occasional reboots, even with little in the way of logging enabled. I also tried a simple display page and dropping the update interval for the display down; no joy. Between this and the abysmally unreliable OTA code in ESPhome, I’m tempted to pursue a different method of connecting this box to Home Assistant entirely.
I got a second box and can confirm the same behaviour on both, so it’s not dodgy hardware.
When I next have debugging time & energy, I’ll either try a totally different firmware or cut out a bunch of components to try and narrow down what’s going on.
I think to eliminate other factors, you should run default AirGradient code built with SoftwareSerial 6.17.1. And make sure you’re not inadvertently caching SoftwareSerial v7.x.x.
Did you try setting the entire logger to INFO with the code I listed above? I’ve been much more stable since then, where trying to turn down logging for individual components didn’t stabilize it fully.
I’m testing some code from another user where they had multiple fonts defined and some graphs. With 5+ fonts defined, I was stuck in a boot loop and had to connect direct to flash it with only 2 fonts defined and it was back operational. I’ve had 1 reboot in the last 48 hours which could still be related to the extra graphs.
I just got some new sensors in, so planning on putting them in soon, but trying to verify I’m mostly stable at this point
That’s a really good find. I’ll be removing the unused fonts! 
And I almost missed your message about it, oh boy.
I will be curious to hear if you get it to run stably in the end. I did not succeed yet (with logging disabled, baud_rate: 0), I have the same problem as OP. What processing unit do you use, the D1 v4 that was delivered with the kit?