Airgradient rebooting itself?

[ken830]

This is my first time with Arduino or ESP too. I’m a hardware engineer, so my software skills are meager, but I figured I have enough ability to poke around and maybe get other, more experienced, people to take notice.

So the PM sensor does seem to modulate its fan speed at times, but I don’t believe it ever shuts off unless it goes into “sleep” mode either by pulling the SET pin low or sending the 0xE4 + 0x00 sleep command. I also noticed that my custom code made it reboot much more often. And my code putting the PMS sensor to sleep and waking it up periodically was definitely interesting even though I know it wasn’t the sole cause because I noticed the rebooting from day one, before I made any changes to the code.

Early on, I kind of suspected that it could be related to power too because it seemed worse when I powered it via the PCB and better when powered via the D1 Mini directly, but right now I can’t remember if the reboots were improved or just the screen freezing issue. At the time, I wasn’t sure if those were related, but I think I’ve root-caused the screen freezing issue and now I’m just working on the rebooting issue. By the way, you have a v3.3 board: does yours experience the issue where the screen stops updating but still seems to continue to upload to the cloud/dashboard? And does the temperature/humidity reading always work?

[makingmark]

I’m not clear - are y’all running ESP Home? Which is an alternative software for the unit I think?

I’m running what I think of as the stock sofware from the installation guide. The only modifications I’ve made are to set temperature to F and I ran S8_UART examples to turn off CO2 ABC and then do a manual calibration. I do have the SGP41 VOC sensor.

[ken830]

No, not running ESP Home. I ran bone stock and confirmed it reboots. I then did a slight modification to write the uptime to the screen and to the serial port so that I can confirm if there was a reboot by comparing it to a stopwatch I start at the same time I power up the board.

After extensive experiments with the stock code, I’m running my modified version of the stock code which crashes/reboots much more frequently to try to speed up the debug process.

If you want to play along, could you check what version of the ESP8266 library you have installed on your Arduino IDE? If it’s newer than v3.0.0, could you try to roll it back to v3.0.0 and re-compile and see if the reboots go away? I only have one unit and it’s still running the from the very first time I compiled with the older Arduino IDE v1.8… It’s going strong past 7-and-a-half hours now and still no crash and it was crashing every few minutes the last time I loaded the same code, but compiled under Arduino IDE V2.x and PlatformIO. The main difference is that I was forced to use ESP8266 v3.0.0.

[ken830]

@makingmark : Oh, and one more thing, since you mentioned you had to modify the code to change to Fahrenheit – does that mean you have PCB v3.7? Otherwise, there is already stock firmware pre-configured for F for <v3.7 PCBs on the build instructions page.

[philomellia]

@makingmark I am running completely stock software, I flashed the DIY_PRO example code directly to the board and have made no modifications whatsoever.

@ken830 I have not noticed any issues with the OLED freezing. I think I remember skimming through a thread about that while trying to see if anyone else was having the reboot issue, but I don’t remember it being familiar to me. I have had no dropouts in terms of uploading the data to the dashboard, the sensor has not missed a single data point on there besides when I unplugged it for maintenance. The temperature and humidity do seem to update appropriately as well, the only issue I have with those is that the temperature seems to read about half a degree higher than any other thermometer I have been able to test in the same space.

I would be more than willing to try messing around with the code and recompiling, but I’ll have to try and get to that tomorrow or maybe later, as I’m a bit ill at the moment. I’ll get back to you if I have a chance to try those steps, and I might try to implement some debugging measured in my sensor as well so I can at least monitor the uptime.

[ken830]

If you get around to it, running bone-stock FW isn’t feasible because you have to be staring at the screen the entire time waiting for a reboot. Here’s the updateOLED2() function I used in my “near-stock” experiments. This was the only change to the code. I simply squeezed the three lines closer together to make room for a fourth line and on that line, I output the uptime in minutes and also write it to the serial port. Start a stop-watch timer at the same time you plug in the board and as long as they match, you know it hasn’t re-booted.

void updateOLED2(String ln1, String ln2, String ln3) {
      char buf[9];
      u8g2.firstPage();
      u8g2.firstPage();
      do {
          u8g2.setFont(u8g2_font_t0_16_tf);
          u8g2.drawStr(1, 10, String(ln1).c_str());
          u8g2.drawStr(1, 25, String(ln2).c_str());
          u8g2.drawStr(1, 40, String(ln3).c_str());
          u8g2.drawStr(1, 55, String(currentMillis/60000).c_str());
          Serial.println(String(String(currentMillis/60000) + "minutes").c_str());
      } while ( u8g2.nextPage() );
}

[MallocArray]

I am so glad someone else posted about this. I received the AG Pro upgrade kit many months ago. I used my own D1 Mini, SGP30 and PMS sensor, although I didn’t actually migrate the hardware from my original DYI kit. I haven’t bought another S8 CO2 sensor yet

I use ESPHome and noticed the D1 mini was rebooting very regularly. Enough I couldn’t get a baseline for the SGP30. I tried running the D1 and SGP30 on a breadboard and it was perfectly stable so I thought maybe it was the AGPro board, but ultimately it seems to be related to the PMS sensor. If I unplug the PMS cable going to the board, then everything is stable with all of the rest of the components connected.

I’ve been meaning to try switching to the official AirGradient sketch, but I’ve been too lazy to get around to it, but it sounds like the others in this thread may be having a similar issue even with it.

For what it is worth, here is a graph of the uptime of the D1 Mini over the last 24 hours. Every time the graph spikes down, it has rebooted

[Hendrik]

Since running on ESPhome I had the D1 mini rebooting once in a while. It was way worse when I had more screen functions running which relate to memory usage. I also disable the screen update after 5min so the I2C bus is less busy. So I presume that there could be more reasons the ESP crashes/reboots. Also SGP30, SHT30, S8 CO2 and PMS.
My uptime can be more than a day. But eventually it will reboot.

One of the things I want to try but still couldn’t get around it is puttng a wemos C3 in it, which is an ESP32.

[ken830]

My code compiled with ESP8266 v3.0.0 yesterday is approaching 24 hours of uptime now, which is a drastic improvement from just a few minutes with the latest (v3.1.1).

@MallocArray & @Hendrik : I don’t know much about ESPhome, but does the code use the ES8266 board Arduino library?

[ken830]

Quick update:

The board uptime surpassed the 24h mark.

I’m going to call this as “good enough” given the time scale I need to work with to make meaningful progress in debugging this thing.

I moved back to Arduino IDE v2.0.3., opened the exact same sketch, upgraded ESP8266 to the latest (v3.1.1) in Board Manager, re-compiled, and… it crashed with an exception within ~ 1minute. I let it run for another 5 minutes and it crashed again.

I then reverted ESP8266 down to v3.0.0 in Board Manager, re-compiled, and so far it’s running for over 7 minutes with no crashing. I guess if it doesn’t crash, I’m going to let it run for at least 12-24h again. It’s a slow process, but I want to isolate this first before going in too many tangential directions.

[MallocArray]

ESPHome is its own thing. I’m not sure what exactly it does in the background, but what we see is not related to Arduino code at all. They use their own YAML format to configure sensors and displays.

If I unplug the PMS sensor, it seems perfectly stable, but I haven’t tried disabling the OLED and leaving the PMS connected to see if it might be memory related. I might try that soon.

[ken830]

@MallocArray

ESPHome is its own thing. I’m not sure what exactly it does in the background, but what we see is not related to Arduino code at all. They use their own YAML format to configure sensors and displays.

I’m not so sure… From https://esphome.io/components/esp8266.html, it seems like it’s all going back to https://github.com/esp8266/Arduino, which is the same Arduino core used by PlatformIO and Arduino IDE.

[ken830]

I’m no SW expert by any stretch, but looking deeper, I see these releases:

ESP8266 Arduino Core:

• v3.1.1 (2023-01-14) <== CRASHES
• v3.1.0 (2023-01-06)
• v3.0.2 (2021-07-26)
• v3.0.1 (2021-06-26)
• v3.0.0 (2021-05-15) <== WORKS

ESPHome uses PlatformIO. I just compiled on Platform IO a few days ago and it crashed. Platform IO’s last release was v4.1.0:

PlatformIO platform-espressif8266:
*v4.1.0 (2023-01-16), Arduino Core v3.1.0 <== CRASHES
*v4.0.1 (2022-01-01), Arduino Core v3.0.2
*v4.0.0 (2022-05-31), Arduino Core v3.0.2
*v3.2.0 (2021-08-13), Arduino Core v3.0.2

To me, it seems like this Arduino Core was a VERY recent update. Because of this, I became impatient in my testing. My Arduino Code v3.0.0 test ran for 3 hours with no crashes, which is a drastic difference from crashing within 1 minute or 5 minutes of the v3.1.1. I’m fairly confident it will behave the same as yesterday’s 24h test, so I cut it short and decided to narrow down the change between v3.0.2 from 2021 and v3.1.0 from this year.

I’ve installed v3.0.2, recompiled, and the board is running now. 17minutes and counting… My assumption is this will work fine (I’ll give it a few hours), then I will move to v3.1.0 and confirm it’s the recent .1 release that is the significant factor here.

After that, I will move into figuring out how to decode the exception to find the culprit of the illegal operation. Might need some help there, since I’m not quite sure how to approach it at this moment. Wish I had a second board and set of sensors on-hand so I could do some of this in parallel.

[ken830]

EDIT: CAN SOMEONE PLEASE REPLY TO THIS SO THAT I CAN POST AN UPDATE? The forum won’t let me post 3 consecutive replies. And I can’t add my update here because it exceeds the maximum length limit per post.

ESP8266 Arduino Core:

• v3.1.1 (2023-01-14) <== CRASHES
• v3.1.0 (2023-01-06) <== CRASHES
• v3.0.2 (2021-07-26) <== WORKS
• v3.0.1 (2021-06-26) <== WORKS
• v3.0.0 (2021-05-15) <== WORKS

PlatformIO platform-espressif8266:
*v4.1.0 (2023-01-16), Arduino Core v3.1.0 <== CRASHES
*v4.0.1 (2022-01-01), Arduino Core v3.0.2
*v4.0.0 (2022-05-31), Arduino Core v3.0.2
*v3.2.0 (2021-08-13), Arduino Core v3.0.2

It’s very clear that the v3.1.0 release of the ESP8266 Arduino Core is what makes the difference here. So, I set-off trying to figure out a way to decode the exception stack trace/dump. PlatformIO has built-in exception decoder “filter” that I enabled and I captures a slew of exceptions. I guess they are somewhat decoded, but I can’t make enough sense out of it.

Here are a few from PlatformIO:

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (0):
epc1=0x402150d0 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

Illegal instruction
  epc1=0x402150d0 in circular_queue<unsigned int, SoftwareSerial*>::available() const at ??:?

>>>stack>>>

ctx: sys
sp: 3ffffbc0 end: 3fffffb0 offset: 0190
3ffffd50:  401030b3 3ffecfe0 3ffecfb8 056e5a89
3ffffd60:  00007fff 0585c417 3ffed8c4 40103290
3ffffd70:  3ffe9e5c 00000000 00004000 40103290
3ffffd80:  00007fff 0585c417 4010374a c0037035
3ffffd90:  00000000 00000001 0000000e 40100768
3ffffda0:  00000020 00004208 00000000 00000001
3ffffdb0:  3ffe9e5c 00004208 00000000 00000022
3ffffdc0:  3fffc200 401006a4 3fffc258 4000050c
3ffffdd0:  40004748 00000030 00000020 ffffffff
3ffffde0:  40105833 04000002 00000000 00000001
3ffffdf0:  fbf8ffff 04000002 3feffe00 00000100
3ffffe00:  60000200 00eaa101 fffffffe 00000008
3ffffe10:  00000000 00000000 60000600 00000030
3ffffe20:  0000000c 00000000 00000020 00000022
3ffffe30:  4000050c 401006a4 3fffc258 4000050c  
3ffffe40:  40240b26 00000030 00000011 ffffffff
3ffffe50:  00000002 00000002 3ffef446 ffffffff
3ffffe60:  3ffee748 40241018 00000000 bfffffff
3ffffe70:  ffffffff 3fffc6fc 3ffe93b4 00000008
3ffffe80:  3ffee770 058a5a57 60000600 00000030
3ffffe90:  40101ed7 2c9f0300 3fffc228 40106059
3ffffea0:  4000050c 401006a4 3fffc258 4000050c
3ffffeb0:  40000f68 00000030 00000011 ffffffff
3ffffec0:  40000f58 00000000 00000020 00000000
3ffffed0:  40101d76 40241018 3ffe93b4 bfffffff
3ffffee0:  4023dcf1 3ffef430 3ffe93b4 3fffdab0
3ffffef0:  00000000 3fffdcb0 3ffee790 00000030
3fffff00:  000000b0 00000016 3ffe85ec 40239084
3fffff10:  00000008 00000000 f7ffffff 60000600
3fffff20:  00000008 00000000 f7ffffff 40223721
3fffff30:  40240b54 00000008 3ffee770 058a5a57
3fffff40:  40240f66 058a2de7 60000600 4021efd9
3fffff50:  40243294 3ffe93b4 4024102b 3ffe92fc
3fffff60:  3ffee770 01bb1298 3ffee770 60000600
3fffff70:  40248211 3ffe92fc 3ffee770 058a2de7
3fffff80:  40248256 3fffdab0 00000000 3fffdcb0
3fffff90:  3ffee798 40000f65 3fffdab0 00000000
3fffffa0:  40000f49 40000f49 3fffdab0 40000f49
<<<stack<<<

0x401030b3 in rcReachRetryLimit at ??:?
0x40103290 in rcReachRetryLimit at ??:?
0x40103290 in rcReachRetryLimit at ??:?
0x4010374a in wDev_ProcessFiq at ??:?
0x40100768 in interrupt_handler at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40105833 in Cache_Read_Enable_New at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40240b26 in pm_set_sleep_time at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x40101ed7 in ppProcessTxQ at ??:?
0x40106059 in ets_timer_disarm at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40101d76 in fix_cache_bug at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x4023dcf1 in register_chipv6_phy at ??:?
0x40239084 in pm_wakeup_init at ??:?
0x40223721 in sys_timeout_abs at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:189
0x40240b54 in pm_set_sleep_time at ??:?
0x40240f66 in pm_get_sleep_type at ??:?
0x4021efd9 in memp_free at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/memp.c:447
0x40243294 in pp_noise_test at ??:?
0x4024102b in pm_get_sleep_type at ??:?
0x40248211 in ets_timer_handler_isr at ??:?
0x40248256 in ets_timer_handler_isr at ??:?


--------------- CUT HERE FOR EXCEPTION DECODER ---------------

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (0):
epc1=0x402150d0 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

Illegal instruction
  epc1=0x402150d0 in circular_queue<unsigned int, SoftwareSerial*>::available() const at ??:?

>>>stack>>>

ctx: sys
sp: 3ffffbd0 end: 3fffffb0 offset: 0190
3ffffd60:  401030b3 3ffecfb8 3ffffed0 00000000
3ffffd70:  00007fff 03cb1ff9 3ffed8c4 40103290
3ffffd80:  3ffe9e44 00000000 00004000 401001a0
3ffffd90:  00007fff 03cb1ff9 4010374a c0033035
3ffffda0:  00000000 00000001 0000000e 40100768
3ffffdb0:  00000020 00006208 00000000 40102528
3ffffdc0:  3ffe95a0 401057a7 3ffece78 00000022
3ffffdd0:  3fffc200 401006a4 3fffc258 4000050c
3ffffde0:  40004855 00000030 00000020 ffffffff
3ffffdf0:  40101d70 00e8a101 3feffe00 60000200
3ffffe00:  fffdffff 04000001 fffffffe 04000001  
3ffffe10:  04000003 00000001 04000002 00000008
3ffffe20:  00000000 00000000 60000600 00000030
3ffffe30:  4022fed0 3ffffed0 3ffedf38 00000022
3ffffe40:  4000050c 401006a4 3fffc258 4000050c
3ffffe50:  40240f5b 00000030 00000010 ffffffff
3ffffe60:  4024102b 00000008 3ffef446 ffffffff
3ffffe70:  3ffee748 40241018 00000000 bfffffff
3ffffe80:  ffffffff 3fffc6fc 00000000 3ffe92fc
3ffffe90:  3ffee770 03d90c89 60000600 00000030
3ffffea0:  4000050c 401006a4 3fffc258 4000050c
3ffffeb0:  40000f68 00000030 00000010 ffffffff
3ffffec0:  40000f58 00000000 00000020 00000000
3ffffed0:  00000002 40241018 00000000 00000000
3ffffee0:  4023dcf1 3ffe93b4 00000000 3fffdab0
3ffffef0:  00000000 3fffdcb0 3ffee780 00000030
3fffff00:  000000b0 00000016 3ffe85ec 40239084
3fffff10:  00000008 00000000 f7ffffff 60000600
3fffff20:  00000008 00000000 f7ffffff 40223721
3fffff30:  40240b54 00000008 3ffee770 03d90c89
3fffff40:  40240f66 0133925b 60000600 4021efd9
3fffff50:  03d8c006 3ffee770 4024102b 3ffe92fc
3fffff60:  3ffee770 0133aa44 3ffee770 60000600
3fffff70:  40248211 3ffe92fc 3ffee770 03d8c006
3fffff80:  40248256 3fffdab0 00000000 3fffdcb0
3fffff90:  3ffee788 40000f65 3fffdab0 00000000
3fffffa0:  40000f49 3ffe9660 3fffdab0 40000f49  
<<<stack<<<

0x401030b3 in rcReachRetryLimit at ??:?
0x40103290 in rcReachRetryLimit at ??:?
0x401001a0 in ets_post at ??:?
0x4010374a in wDev_ProcessFiq at ??:?
0x40100768 in interrupt_handler at ??:?
0x40102528 in pp_post at ??:?
0x401057a7 in lmacTxFrame at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40101d70 in fix_cache_bug at ??:?
0x4022fed0 in scan_parse_beacon at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40240f5b in pm_get_sleep_type at ??:?
0x4024102b in pm_get_sleep_type at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x4023dcf1 in register_chipv6_phy at ??:?
0x40239084 in pm_wakeup_init at ??:?
0x40223721 in sys_timeout_abs at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:189
0x40240b54 in pm_set_sleep_time at ??:?
0x40240f66 in pm_get_sleep_type at ??:?
0x4021efd9 in memp_free at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/memp.c:447
0x4024102b in pm_get_sleep_type at ??:?
0x40248211 in ets_timer_handler_isr at ??:?
0x40248256 in ets_timer_handler_isr at ??:?


--------------- CUT HERE FOR EXCEPTION DECODER ---------------

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (0):
epc1=0x402150d0 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

Illegal instruction
  epc1=0x402150d0 in circular_queue<unsigned int, SoftwareSerial*>::available() const at ??:?

>>>stack>>>

ctx: sys
sp: 3ffffbc0 end: 3fffffb0 offset: 0190
3ffffd50:  0000ffff ffffffff 00000005 40102528
3ffffd60:  3ffe95a5 401057bf 3ffecfb8 3ffffed0
3ffffd70:  401030b3 3ffecfb8 00004000 00000030  
3ffffd80:  00007fff 0ceca541 3ffed8c4 c0033035
3ffffd90:  00000000 00000001 0000000e 40100768
3ffffda0:  00000020 00000000 00000020 401001a0
3ffffdb0:  3ffe9e2c 7fffffff 00000000 00000022
3ffffdc0:  3fffc200 401006a4 3fffc258 4000050c
3ffffdd0:  400046c9 00000030 00000020 ffffffff  
3ffffde0:  40105833 00000000 00000000 00000001
3ffffdf0:  04000002 00e8a101 3feffe00 00000100
3ffffe00:  60000200 00000001 fffffffe 00000008
3ffffe10:  00000000 00000000 60000600 00000030
3ffffe20:  3ffeda00 3ffffed0 3ffee090 00000022
3ffffe30:  4000050c 401006a4 3fffc258 4000050c
3ffffe40:  40240b51 00000030 00000010 ffffffff
3ffffe50:  40240b4d 00000008 00000000 3feffe00
3ffffe60:  fc7000f7 fffffff7 00000000 bfffffff
3ffffe70:  ffffffff 3fffc6fc 00000000 00000008
3ffffe80:  3ffee770 0cf5ea72 60000600 00000030
3ffffe90:  3ffee770 3fff0f94 3fffc228 40106059
3ffffea0:  4000050c 401006a4 3fffc258 4000050c
3ffffeb0:  40000f68 00000030 00000010 ffffffff
3ffffec0:  40000f58 00000000 00000020 00000000
3ffffed0:  40101d76 40241018 3ffee748 00000000  
3ffffee0:  4023dcf1 3ffe93b4 00000000 3fffdab0
3ffffef0:  00000000 3fffdcb0 3ffee790 00000030
3fffff00:  00000000 0003513c 5b645a1c 40239084
3fffff10:  00000008 00000000 f7ffffff 60000600
3fffff20:  00000008 00000000 f7ffffff 402237c8
3fffff30:  40240b54 00000008 3ffee770 0cf5ea72
3fffff40:  40240f66 040c9eff 3ffefbec 60000600
3fffff50:  0cf5b546 3ffee770 4024102b 3ffe92fc
3fffff60:  3ffee770 040caf9d 3ffee770 60000600
3fffff70:  40248211 3ffe92fc 3ffee770 0cf5b546
3fffff80:  40248256 3fffdab0 00000000 3fffdcb0
3fffff90:  3ffee798 3fffdad0 3ffef458 4020e6e2
3fffffa0:  40000f49 3ffe95f8 3fffdab0 40000f49
<<<stack<<<

0x40102528 in pp_post at ??:?
0x401057bf in lmacRxDone at ??:?
0x401030b3 in rcReachRetryLimit at ??:?
0x40100768 in interrupt_handler at ??:?
0x401001a0 in ets_post at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40105833 in Cache_Read_Enable_New at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40240b51 in pm_set_sleep_time at ??:?
0x40240b4d in pm_set_sleep_time at ??:?
0x40106059 in ets_timer_disarm at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40101d76 in fix_cache_bug at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x4023dcf1 in register_chipv6_phy at ??:?
0x40239084 in pm_wakeup_init at ??:?
0x402237c8 in sys_timeout_LWIP2 at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:306
0x40240b54 in pm_set_sleep_time at ??:?
0x40240f66 in pm_get_sleep_type at ??:?
0x4024102b in pm_get_sleep_type at ??:?
0x40248211 in ets_timer_handler_isr at ??:?
0x40248256 in ets_timer_handler_isr at ??:?
0x4020e6e2 in loop_task(ETSEventTag*) at core_esp8266_main.cpp:?


--------------- CUT HERE FOR EXCEPTION DECODER ---------------

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (0):
epc1=0x402150d0 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

Illegal instruction
  epc1=0x402150d0 in circular_queue<unsigned int, SoftwareSerial*>::available() const at ??:?

>>>stack>>>

ctx: sys
sp: 3ffffbd0 end: 3fffffb0 offset: 0190
3ffffd60:  00007fff 022ef061 3ffed8c4 40103290
3ffffd70:  3ffe9e44 00000000 00000000 ffffff60
3ffffd80:  00007fff 022ef061 00000004 00000100  
3ffffd90:  3ffe9e44 7fffffff 00000000 c0037031
3ffffda0:  00000000 00000001 00000002 40100768
3ffffdb0:  00000020 3ffed030 00000000 022ef061  
3ffffdc0:  000005e0 00000000 401045c5 00000022
3ffffdd0:  3fffc200 401006a4 3fffc258 4000050c
3ffffde0:  40004826 00000030 00000020 ffffffff  
3ffffdf0:  40101d70 00eaa101 3feffe00 60000200
3ffffe00:  fffdffff 04000002 fffffffe 00000008
3ffffe10:  0115006c 00000001 fe7fffff 00000008
3ffffe20:  00000000 00000000 60000600 00000030
3ffffe30:  40104df3 3ffffed0 0000000c 40102528
3ffffe40:  3ffe95ac 40101e72 04000000 00040000
3ffffe50:  0000001b 000000f1 00000004 00000000
3ffffe60:  00000000 00000000 00000000 c0037035
3ffffe70:  00000000 00000001 00000002 40100768  
3ffffe80:  00000000 00000000 0000001f 401001a0
3ffffe90:  402418bd 2c9f0300 3fffc228 40106059
3ffffea0:  4000050c 401006a4 3fffc258 4000050c
3ffffeb0:  40000f68 00000030 00000010 ffffffff
3ffffec0:  40000f58 00000000 00000020 00000000
3ffffed0:  00000002 40241018 3ffee748 00000000
3ffffee0:  4023dcf1 3ffe93b4 00000000 3fffdab0
3ffffef0:  00000000 3fffdcb0 3ffee780 00000030
3fffff00:  000000b0 00000016 3ffe85ec 40239084
3fffff10:  00000008 00000000 f7ffffff 60000600
3fffff20:  00000008 00000000 f7ffffff 40223721
3fffff30:  40240b54 00000008 3ffee770 023387cf
3fffff40:  40240f66 00afdbb7 60000600 60000600
3fffff50:  02335356 3ffee770 4024102b 3ffe92fc  
3fffff60:  3ffee770 00aff0ca 3ffee770 60000600
3fffff70:  40248211 3ffe92fc 3ffee770 02335356
3fffff80:  40248256 3fffdab0 00000000 3fffdcb0
3fffff90:  3ffee788 3fffdad0 3ffef458 4020e6e2
3fffffa0:  40000f49 40000f49 3fffdab0 40000f49
<<<stack<<<

0x40103290 in rcReachRetryLimit at ??:?
0x40100768 in interrupt_handler at ??:?
0x401045c5 in lmacIsIdle at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40101d70 in fix_cache_bug at ??:?
0x40104df3 in lmacRecycleMPDU at ??:?
0x40102528 in pp_post at ??:?
0x40101e72 in pp_soft_wdt_feed_local at ??:?
0x40100768 in interrupt_handler at ??:?
0x401001a0 in ets_post at ??:?
0x402418bd in LightSleepWakedCbSetFunc at ??:?
0x40106059 in ets_timer_disarm at ??:?
0x401006a4 in interrupt_handler at ??:?
0x40241018 in pm_get_sleep_type at ??:?
0x4023dcf1 in register_chipv6_phy at ??:?
0x40239084 in pm_wakeup_init at ??:?
0x40223721 in sys_timeout_abs at /local/users/gauchard/arduino/arduino_esp8266/origin/tools/sdk/lwip2/builder/lwip2-src/src/core/timeouts.c:189
0x40240b54 in pm_set_sleep_time at ??:?
0x40240f66 in pm_get_sleep_type at ??:?
0x4024102b in pm_get_sleep_type at ??:?
0x40248211 in ets_timer_handler_isr at ??:?
0x40248256 in ets_timer_handler_isr at ??:?
0x4020e6e2 in loop_task(ETSEventTag*) at core_esp8266_main.cpp:?


--------------- CUT HERE FOR EXCEPTION DECODER ---------------

I will now try to get Arduino IDE V1.8 working with the new esp8266 arduino core by fixing that stupid bug they introduced. Maybe that decoder works better??

[makingmark]

@ken830

If you want to play along, could you check what version of the ESP8266 library you have installed on your Arduino IDE? If it’s newer than v3.0.0, could you try to roll it back to v3.0.0 and re-compile and see if the reboots go away?

@ken830 I think this is how I got confused about whether others were running stock firmware. I don’t have any library with ESP in the title when I look in the Arduino Library manager. Just the AirGradient library version 2.2.0, the others as directed by the build instuctions, and the S8_UART library I used to modify the CO2 calibration.

However…when I look in the file manager of my Mac, I see a path like: /Users/me/Library/Arduino15/packages/esp8266/hardware/esp8266/3.1.1.

How would I change this version?

[makingmark]

@ken830

@makingmark : Oh, and one more thing, since you mentioned you had to modify the code to change to Fahrenheit – does that mean you have PCB v3.7?

Yes, I have PCB v3.7.

[makingmark]

@ken830

EDIT: CAN SOMEONE PLEASE REPLY TO THIS SO THAT I CAN POST AN UPDATE?

see also my responses below (or above now, I guess)

[ken830]

Okay… Thanks to some posts on the issues and PR sections of the esp8266 Arduino core github page a few days ago , I was able to workaround the bug preventing me from compiling with core v3.1.0 and above by manually replacing the bad mkbuildoptglobals.py file. This allowed me to compile the sketch with the newer Core versions in the older Arduino IDE v1.8.x , which finally allowed me to run the ESP Exception Decoder. And the results look much better:

It crashes so often, I have decoded dozens of exceptions and each one is always the same:

circular_queue ::available() const at c:\users\ken\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec\xtensa-lx106-elf\include\c++\10.3.0\bits/atomic_base.h line 420

atomic_base.h is an “internal header file” and line 420 looks like this:

25  /** @file bits/atomic_base.h
26   *  This is an internal header file, included by other library headers.
27   *  Do not attempt to use it directly. @headername{atomic}
28   */
.
.
.
419      _GLIBCXX_ALWAYS_INLINE __int_type
420      load(memory_order __m = memory_order_seq_cst) const noexcept
421      {
422	memory_order __b = __m & __memory_order_mask;
423	__glibcxx_assert(__b != memory_order_release);
424	__glibcxx_assert(__b != memory_order_acq_rel);

Here are some decoded exceptions:

Exception 0: Illegal instruction
PC: 0x40219d04: circular_queue ::available() const at c:\users\ken\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec\xtensa-lx106-elf\include\c++\10.3.0\bits/atomic_base.h line 420
EXCVADDR: 0x00000000

Decoding stack results
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x4010062b: ets_intr_unlock() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 226
0x40228c85: sys_timeout_abs at core/timeouts.c line 189
0x401000ab: app_entry_redefinable() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 386
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40223559: glue2esp_linkoutput at glue-esp/lwip-esp.c line 301
0x40223787: new_linkoutput at glue-lwip/lwip-git.c line 272
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x4022cb94: ip4_output_if_opt_src at core/ipv4/ip4.c line 1764
0x40223559: glue2esp_linkoutput at glue-esp/lwip-esp.c line 301
0x40223787: new_linkoutput at glue-lwip/lwip-git.c line 272
0x40223be2: ethernet_output at netif/ethernet.c line 312
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x402176d3: Twi::WAIT_CLOCK_STRETCH() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/PolledTimeout.h line 130
0x40217974: Twi::read_bit() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 336
0x40217b48: Twi::writeTo(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 368
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40217b48: Twi::writeTo(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 368
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40212c64: u8x8_byte_arduino_hw_i2c(u8x8_t*, uint8_t, uint8_t, void*) at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\U8x8lib.cpp line 1316
0x40202606: u8x8_cad_ssd13xx_fast_i2c at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_cad.c line 582
0x40212c48: u8x8_byte_arduino_hw_i2c(u8x8_t*, uint8_t, uint8_t, void*) at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\U8x8lib.cpp line 1316
0x402035dc: u8x8_byte_EndTransfer at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_byte.c line 61
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100a4c: micros() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring.cpp line 181
0x40100110: SoftwareSerial::preciseDelay() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\SoftwareSerial\src/SoftwareSerial.h line 246
0x4021790d: Twi::write_bit(bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 325
0x4021787d: Twi::write_stop() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 304
0x40217b01: Twi::readFrom(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 433
0x4021278c: TwoWire::requestFrom(unsigned char, unsigned int, bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 129
0x402127c0: TwoWire::requestFrom(unsigned char, unsigned char, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 137
0x40212966: SensirionI2CCommunication::receiveFrame(unsigned char, unsigned int, SensirionI2CRxFrame&, TwoWire&, CrcPolynomial) at C:\Users\Ken\Documents\Arduino\libraries\Sensirion_Core\src\SensirionI2CCommunication.cpp line 101
0x40216c68: __esp_suspend() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/core_esp8266_features.h line 64
0x40216db9: __esp_delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 161
0x40216e2e: esp_try_delay(unsigned int, unsigned int, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 182
0x40217d90: __delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/coredecls.h line 69
0x402037a7: updateTVOC() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 184
0x40204a84: loop() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 149

Exception 0: Illegal instruction
PC: 0x40219d04: circular_queue ::available() const at c:\users\ken\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec\xtensa-lx106-elf\include\c++\10.3.0\bits/atomic_base.h line 420
EXCVADDR: 0x00000000

Decoding stack results
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40228c85: sys_timeout_abs at core/timeouts.c line 189
0x401000ab: app_entry_redefinable() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 386
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40216e54: esp_yield() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 144
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x40214744: Print::print(__FlashStringHelper const*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\Print.cpp line 106
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40214854: Print::printNumber (unsigned long, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\Print.cpp line 262
0x4021e804: utoa at /workdir/repo/newlib/newlib/libc/stdlib/utoa.c line 74
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40101400: malloc(size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 912
0x40213661: IPAddress::printTo(Print&) const at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/IPAddress.h line 165
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40208a35: WiFiManager::DEBUG_WM__FlashStringHelper const*, IPAddress>(WiFiManager::wm_debuglevel_t, __FlashStringHelper const*, IPAddress) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/IPAddress.h line 88
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x402157a1: String::copy(char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 258
0x40215971: String::operator=(char const*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 303
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x402159f2: String::String(unsigned long, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 82
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40208a86: WiFiManager::DEBUG_WM__FlashStringHelper const*, IPAddress>(__FlashStringHelper const*, IPAddress) at C:\Users\Ken\Documents\Arduino\libraries\WiFiManager\WiFiManager.cpp line 3281
0x402064d4: String::operator!=(char const*) const at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/WString.h line 234
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x402178c8: Twi::write_bit(bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 313
0x402179cb: Twi::write_byte(unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 347
0x40217b48: Twi::writeTo(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 368
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x402127ec: TwoWire::beginTransmission(unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 158
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100110: SoftwareSerial::preciseDelay() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\SoftwareSerial\src/SoftwareSerial.h line 246
0x4021790d: Twi::write_bit(bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 325
0x4021787d: Twi::write_stop() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 304
0x40217b01: Twi::readFrom(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 433
0x4021278c: TwoWire::requestFrom(unsigned char, unsigned int, bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 129
0x402127c0: TwoWire::requestFrom(unsigned char, unsigned char, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 137
0x40212966: SensirionI2CCommunication::receiveFrame(unsigned char, unsigned int, SensirionI2CRxFrame&, TwoWire&, CrcPolynomial) at C:\Users\Ken\Documents\Arduino\libraries\Sensirion_Core\src\SensirionI2CCommunication.cpp line 101
0x40216c68: __esp_suspend() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/core_esp8266_features.h line 64
0x40216db9: __esp_delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 161
0x40216e2e: esp_try_delay(unsigned int, unsigned int, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 182
0x40217d90: __delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/coredecls.h line 69
0x402037a7: updateTVOC() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 184
0x40204a84: loop() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 149

Exception 0: Illegal instruction
PC: 0x40219d04: circular_queue ::available() const at c:\users\ken\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec\xtensa-lx106-elf\include\c++\10.3.0\bits/atomic_base.h line 420
EXCVADDR: 0x00000000

Decoding stack results
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x40228c85: sys_timeout_abs at core/timeouts.c line 189
0x40216bfa: loop_task(ETSEvent*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 273
0x401000ab: app_entry_redefinable() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 386
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x4020f304: ESP8266WiFiSTAClass::status() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\ESP8266WiFi\src\ESP8266WiFiSTA.cpp line 558
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x40214744: Print::print(__FlashStringHelper const*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\Print.cpp line 106
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40214854: Print::printNumber (unsigned long, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\Print.cpp line 262
0x40101400: malloc(size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 912
0x40213661: IPAddress::printTo(Print&) const at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/IPAddress.h line 165
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40208a35: WiFiManager::DEBUG_WM__FlashStringHelper const*, IPAddress>(WiFiManager::wm_debuglevel_t, __FlashStringHelper const*, IPAddress) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/IPAddress.h line 88
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x402157a1: String::copy(char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 258
0x40215971: String::operator=(char const*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 303
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x402159f2: String::String(unsigned long, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\WString.cpp line 82
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40208a86: WiFiManager::DEBUG_WM__FlashStringHelper const*, IPAddress>(__FlashStringHelper const*, IPAddress) at C:\Users\Ken\Documents\Arduino\libraries\WiFiManager\WiFiManager.cpp line 3281
0x402064d4: String::operator!=(char const*) const at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/WString.h line 234
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x4010107e: umm_free_core(umm_heap_context_t*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 642
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40217736: Twi::WAIT_CLOCK_STRETCH() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 167
0x4021790d: Twi::write_bit(bool) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 325
0x40217b48: Twi::writeTo(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 368
0x4021347c: HardwareSerial::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/HardwareSerial.h line 193
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40100c04: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 167
0x40100b40: interrupt_handler(void*, void*) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring_digital.cpp line 138
0x402176e2: Twi::WAIT_CLOCK_STRETCH() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/core_esp8266_features.h line 65
0x40100648: ets_post(uint8, ETSSignal, ETSParam) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 238
0x40101403: malloc(size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 912
0x40101400: malloc(size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\umm_malloc\umm_malloc.cpp line 912
0x4021787d: Twi::write_stop() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 304
0x40217bb1: Twi::writeTo(unsigned char, unsigned char*, unsigned int, unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_si2c.cpp line 393
0x40212662: TwoWire::write(unsigned char const*, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 209
0x40212834: TwoWire::endTransmission(unsigned char) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\Wire\Wire.cpp line 172
0x40212d09: u8x8_byte_arduino_hw_i2c(u8x8_t*, uint8_t, uint8_t, void*) at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\U8x8lib.cpp line 1354
0x40202619: u8x8_cad_ssd13xx_fast_i2c at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_cad.c line 586
0x40202780: u8x8_d_ssd1306_sh1106_generic at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_d_ssd1306_128x64_noname.c line 327
0x402027b4: u8x8_d_sh1106_128x64_noname at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_d_ssd1306_128x64_noname.c line 481
0x4020364c: u8x8_DrawTile at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8x8_display.c line 89
0x40218864: uart_write(uart_t*, char const*, size_t) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\uart.cpp line 547
0x402019b5: u8g2_send_buffer at C:\Users\Ken\Documents\Arduino\libraries\U8g2\src\clib\u8g2_buffer.c line 89
0x40216c68: __esp_suspend() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/core_esp8266_features.h line 64
0x40216db9: __esp_delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 161
0x40216e2e: esp_try_delay(unsigned int, unsigned int, unsigned int) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_main.cpp line 182
0x40217d90: __delay(unsigned long) at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266/coredecls.h line 69
0x40204f20: AirGradient::getCO2_Raw() at C:\Users\Ken\Documents\Arduino\libraries\AirGradient_Air_Quality_Sensor\AirGradient.cpp line 691
0x402038f8: updateCo2() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 206
0x40204a8a: loop() at C:\Users\Ken\Documents\Arduino\sketch_jan24a/sketch_jan24a.ino line 151

[ken830]

@makingmark

@ken830 I think this is how I got confused about whether others were running stock firmware. I don’t have any library with ESP in the title when I look in the Arduino Library manager. Just the AirGradient library version 2.2.0, the others as directed by the build instuctions, and the S8_UART library I used to modify the CO2 calibration.

However…when I look in the file manager of my Mac, I see a path like: /Users/me/Library/Arduino15/packages/esp8266/hardware/esp8266/3.1.1.

How would I change this version?

This is the esp8266 Arduino Core. It’s installed under Board Manager, not Library manager. If you look there, you should see the version v3.1.1 installed and you should be able to revert down to v3.0.2, recompile, and flash the board.

@makingmark

see also my responses below (or above now, I guess)

Thanks! I have a lot more to report, but it’s past 6am here and I need to get some sleep. Typing this on my phone in bed.

[ken830]

NOTE: This is my 3rd post in a row, so someone reply to this so I can continue to post.

I opened an issue in the Arduino Core repository and one of the collaborators there has been responding and has been very helpful and patient with me. Their first response was that it’s likely something to do with SoftwareSerial. I think because circular_queue is part of SoftwareSerial. And they pointed out that EXCVADDR is pointing at nil (0x00000000) instead of an object.

He/she quoted this from my exception:

 Exception 0: Illegal instruction
PC: 0x40219d04: circular_queue ::available() const at c:\users\ken\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.1.0-gcc10.3-e5f9fec\xtensa-lx106-elf\include\c++\10.3.0\bits/atomic_base.h line 420
EXCVADDR: 0x00000000
...
0x40100110: SoftwareSerial::preciseDelay() at C:\Users\Ken\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\SoftwareSerial\src/SoftwareSerial.h line 246

So, that gave me a couple of leads to pursue. The first was the nil/NULL comment stuck me and I remember always see the warning about 2 NULL conversions whenever I compiled the code, but I noticed I didn’t get that warning when I built using <v3.0.2. Here are the build outputs from each.

Core v3.1.1

C:\Users\Ken\Documents\Arduino\libraries\AirGradient_Air_Quality_Sensor\AirGradient.cpp: In member function 'TMP_RH AirGradient::returnError(TMP_RH_ErrorCode)':
C:\Users\Ken\Documents\Arduino\libraries\AirGradient_Air_Quality_Sensor\AirGradient.cpp:548:14: warning: converting to non-pointer type 'float' from NULL [-Wconversion-null]
  548 |   result.t = NULL;
      |              ^~~~
C:\Users\Ken\Documents\Arduino\libraries\AirGradient_Air_Quality_Sensor\AirGradient.cpp:549:15: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
  549 |   result.rh = NULL;
      |               ^~~~
. Variables and constants in RAM (global, static), used 33372 / 80192 bytes (41%)
║   SEGMENT  BYTES    DESCRIPTION
╠══ DATA     1516     initialized variables
╠══ RODATA   3832     constants       
╚══ BSS      28024    zeroed variables
. Instruction RAM (IRAM_ATTR, ICACHE_RAM_ATTR), used 62695 / 65536 bytes (95%)
║   SEGMENT  BYTES    DESCRIPTION
╠══ ICACHE   32768    reserved space for flash instruction cache
╚══ IRAM     29927    code in IRAM    
. Code in flash (default, ICACHE_FLASH_ATTR), used 355008 / 1048576 bytes (33%)
║   SEGMENT  BYTES    DESCRIPTION
╚══ IROM     355008   code in flash   
esptool.py v3.0
Serial port COM4
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 84:f3:eb:24:4b:d0
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 394432 bytes to 280039...
Writing at 0x00000000... (5 %)
Writing at 0x00004000... (11 %)
Writing at 0x00008000... (16 %)
Writing at 0x0000c000... (22 %)
Writing at 0x00010000... (27 %)
Writing at 0x00014000... (33 %)
Writing at 0x00018000... (38 %)
Writing at 0x0001c000... (44 %)
Writing at 0x00020000... (50 %)
Writing at 0x00024000... (55 %)
Writing at 0x00028000... (61 %)
Writing at 0x0002c000... (66 %)
Writing at 0x00030000... (72 %)
Writing at 0x00034000... (77 %)
Writing at 0x00038000... (83 %)
Writing at 0x0003c000... (88 %)
Writing at 0x00040000... (94 %)
Writing at 0x00044000... (100 %)
Wrote 394432 bytes (280039 compressed) at 0x00000000 in 6.4 seconds (effective 495.1 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Core v3.0.2

Executable segment sizes:
ICACHE : 32768           - flash instruction cache 
IROM   : 353468          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 29585   / 32768 - code in IRAM          (IRAM_ATTR, ISRs...) 
DATA   : 1516  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 3840  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 27904 )         - zeroed variables      (global, static) in RAM/HEAP 
Sketch uses 388409 bytes (37%) of program storage space. Maximum is 1044464 bytes.
Global variables use 33260 bytes (40%) of dynamic memory, leaving 48660 bytes for local variables. Maximum is 81920 bytes.
esptool.py v3.0
Serial port COM4
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 84:f3:eb:24:4b:d0
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 392560 bytes to 277859...
Writing at 0x00000000... (5 %)
Writing at 0x00004000... (11 %)
Writing at 0x00008000... (17 %)
Writing at 0x0000c000... (23 %)
Writing at 0x00010000... (29 %)
Writing at 0x00014000... (35 %)
Writing at 0x00018000... (41 %)
Writing at 0x0001c000... (47 %)
Writing at 0x00020000... (52 %)
Writing at 0x00024000... (58 %)
Writing at 0x00028000... (64 %)
Writing at 0x0002c000... (70 %)
Writing at 0x00030000... (76 %)
Writing at 0x00034000... (82 %)
Writing at 0x00038000... (88 %)
Writing at 0x0003c000... (94 %)
Writing at 0x00040000... (100 %)
Wrote 392560 bytes (277859 compressed) at 0x00000000 in 6.3 seconds (effective 497.5 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Here is the code snippet from AirGradient.cpp. This is part of the overall set of code that reads the temperature (TMP) and relative humidity (RH) from the sensor. I’m guessing this piece defines the values that are returned if the sensor reading results in an error.

546   TMP_RH AirGradient::returnError(TMP_RH_ErrorCode error) {
547     TMP_RH result;
548     result.t = NULL;
549     result.rh = NULL;
550   
551     result.t_char[0] = 'N';
552     result.t_char[1] = 'U';
553     result.t_char[2] = 'L';
554     result.t_char[3] = 'L';
555   
556     result.rh_char[0] = 'N';
557     result.rh_char[1] = 'U';
558     result.rh_char[2] = 'L';
559     result.rh_char[3] = 'L';
560   
561     result.error = error;
562     return result;
}

But this turns out to not likely be related, Tmp/RH is not on SoftwareSerial, the warning isn’t indicative of the exception error, and there was a change in compiler warnings since the #8495 PR that’s probably suppressing the still-existent warnings.

The second lead was that I remembered that in the PlatformIO decoder filter output (the ones that I said weren’t very useful), there was SoftwareSerial mentioned:

Exception (0):
epc1=0x402150d0 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

Illegal instruction
  epc1=0x402150d0 in circular_queue<unsigned int, SoftwareSerial*>::available() const at ??:?

>>>stack>>>

I posted this and mentioned that the output wasn’t very helpful and looked incomplete to which the response was that I needed -g in build_flags or build_type = debug (which does that for us) for the decoder to work. So, in an attempt to see if the PlatformIO decoder can give us more details/clues, I set the build_type = debug flag , re-compiled and ran it waiting for an exception… after waiting 15 minutes, I grew suspicious. So, I removed the flag, re-compiled, and ran it again. This time, it crashed within 5 minutes. I let it run for 15 minutes and it crashed a couple more times as expected. I then re-added the build_type = debug flag, re-compiled, and ran it. To my surprise, it didn’t crash for over an hour. I let it run and went to bed. I got up 6 hours later to find it was still running with no crashes/reboots! As I type this, it’s been over 7 hours and still going.

My assumption is building with the debug flag should only minimally affect code behavior, otherwise, it wouldn’t be very useful in debugging purposes. And so with that premise, I’m hopeful this clue can really narrow-in on possible root causes. I’m doing some reading or maybe someone with a better-suited skillset will be able to help solve this or point me in the right direction.