Serial UART stuff

serialuart/README.md

@@ -0,0 +1,137 @@
+# Serial UART communication
+
+![breadboard](breadboard.png)
+
+This project shows using the ESP32-S3 serial port for communication via UART.
+
+## Lessons Learned
+
+- The Arduino HAL is surprisingly thick
+- You can't use the ESP32-S3 UART for communication via USB when you're debugging
+- libc is libc anywhere you go
+- How to use my digital logic analyzer to snoop on, record and inspect UART protocol traffic
+- How to use my oscilloscope to look at UART traffic over the wire, and how that compares to using the logic analyzer
+
+## libc is libc anywhere you go
+
+The original tutorial for this project just had us setting up the Serial port at 115.2k baud, printing a startup message, and printing another message every second or so. YAAAAAWN.
+
+![oog is bored](bored.jpg)
+
+Instead of that, I decided to implement a very simple command language over the serial port for setting the mode of a given pin, and then setting that pin high or low. So you could, for example, send this over serial:
+
+```
+setpin 2 output
+writepin 2 high
+```
+
+... to turn an LED on the breadboard on. Then send this
+
+```
+writepin 2 low
+```
+
+... to turn it off. Simple, but it gave me a chance to poke around at the C compiler in the arduino IDE. To do this, I needed a few C features and functions:
+
+* Structures
+* `strtok` for tokenizing a string
+* `strncmp` for comparing N characters of two strings
+* `strncpy` for copying N characters of two strings
+* Pointers
+
+The Arduino IDE *says* that it is using C++. I'll be honest, I'm not sure how true that statement is. I've written a decent amount of C++, and it doesn't look exactly like what I'm seeing in the Arduino IDE. But it looks a lot like C, so I decided to treat it like C, and write my parser using standard libc functions, since that seemed like a simpler ramp in this environment. And, as usual, bare C failed to disappoint me.
+
+C and libc just work, basically everywhere you go, and there's very little magic behind it.
+
+Feels good man.
+
+## The Arduino HAL is thick
+
+The language and library setup in the Arduino IDE feels really thick to me. If you're looking for a bare metal embedded C experience, this ain't it. It's fun, and it's waaaaay closer than what you will get in 99.9999% of environments these days, but there's a remarkably thick layer between you and the hardware.
+
+First, there's the fact that the language (in the `.ino` files anyway) is supposedly C++ but doesn't really behave like C++. Take how we initialize the serial port, for example.
+
+```
+Serial.begin(SERIALSPEED)
+```
+
+Notice that:
+
+* I didn't have to include any headers
+* I didn't have to link any libraries
+* I didn't have to construct a Serial object and call an initializer on it
+
+But beyond that, there's lots of things about establishing a UART link over Serial that I didn't have to bother with at all:
+
+* I didn't have to enable or disable the TX/RX pins
+* I didn't have to manually write anything to TX or read anything from RX
+* I didn't have to go to the trouble of establishing UART over those pins
+* I didn't have to parse any response from the pin using UART protocol
+* I didn't have to manage any error correction, parity, etc
+
+For the Arduino audience, that's probably fine. I doubt most Arduino users are really that interested in any of that. They probably just want to open the serial port and use the damn thing already. And that's all fine and dandy. But there is a TON of work going on under the hood. But you can actually crack open the hardware abstraction library and see what that code is doing.
+
+For example, to see what `Serial.begin()` is doing, you can right-click on the method name, and `Go to definition`. The hardware abstraction library C++ code will pop open and there you are. We can dig into this code to find out things like:
+
+* We can set up the serial on different pins than the default (neat)
+* We can have more than one UART going at the same time (very neat)
+* We can control how many bytes can be ready before the FIFO (First In First Out) buffer on that UART is full, which triggers an interrupt
+
+All this is pretty neat. It also tells me that, theoretically, you could replace the HAL with one of your own, if you really wanted to. Which is pretty cool, because you can see how it's *supposed* to work in the default HAL from the manufacturer, but if you want to dig down deeper, you can implement a different HAL. I haven't tried doing that yet, but it seems possible, and that's pretty cool for educational growth.
+
+## Snooping UART with the Logic Analyzer
+
+One of the most useful tools in enterprise IT is something called `tcpdump`, which allows us to dump all the packets coming over a network connection and inspect them. There are lots and lots of errors that jump right out at you when you're looking at a tcpdump, that would be very difficult to figure out if you were just looking at application log files.
+
+When I decided to get into embedded development, I wanted to be able to do the same thing with the digital inputs and outputs on my dev boards. We can use a voltmeter to check electrical connections, and we can use an oscilloscope to check waveforms, but how do we (without great effort) figure out exactly what is going over that wire at any given time? How do we capture information going over that wire, examine it, and maybe even play it back later? How do we put a man in the middle between two pins and put something there that lets us not only capture but also inject signals?
+
+I did a little research, and picked up a [Digilent Digital Discovery logic analyzer](https://digilent.com/reference/test-and-measurement/digital-discovery/start). It's a pretty slick little device that can do all kinds of things
+
+## ESP32-S3 Serial monitor when using the USB JTAG
+
+It looks like you can't use the default UART when using the USB JTAG for debugging. I tried firing up the debugger to figure out why command processing was failing, and while i was able to connect to the debugger, the serial monitor no longer did what I expected.
+
+Firing it up and watching the serial monitor on the Ardruino IDE, I saw a lot of useful info, but not what I wanted:
+
+```
+ESP-ROM:esp32s3-20210327
+Build:Mar 27 2021
+rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
+Saved PC:0x40388542
+pro cpu reset by JTAG
+SPIWP:0xee
+mode:DIO, clock div:1
+load:0x3fce2820,len:0x10cc
+load:0x403c8700,len:0xc2c
+load:0x403cb700,len:0x30c0
+```
+
+... And if I fire up my logic analyzer to watch the serial UART traffic, I get some binary traffic, instead of the plain text protocol I'm expecting to see
+
+![JTAG traffic on the UART pins, not UART serial](jtagnotuart.png)
+
+Apparently when the USB JTAG is used, pins 43 and 44 no longer receive UART output. There are some details behind the exact mechanism, and I'm not really clear on how it all works - the datasheets don't make it stunningly clear - but the Espressif IDF I/O documentation says that depending on the value of a config flag (that I couldnt find in the HAL), the default `Serial` object's output may be routed away from the UART pins, making them appear inactive. 
+
+One suggestion for getting around this was to use a second Serial object, explicitly using those pins. This will guarantee that UART remains configured on those pins.
+
+```
+Serial0.begin(SERIALSPEED, SERIAL_8N1, 44, 43);
+```
+
+... Unfortunately this didn't work as expected, because apparently something inside the Freenove ESP32-S3 WROOM devkit holds onto port 44. I was able to send data out from the ESP over pin 43 and see it on my logic analyzer, but anything I sent back from my logic analyzer never got processed by the ESP32. I did some experimentation and wound up moving the second Serial object to pins 21 and 14. At that point....
+
+![Debugger running and analyzer connected to UART](debugandanalyze.png)
+
+Voila! The debugger can run through the USB JTAG, while the logic analyzer can manage serial communication over the `Serial0` object that was manually configured.
+
+I tried to figure out what might have been using pin 44, but the ESP32 datasheets didn't make it immediately clear. The Freenove board documentation didn't either. Maybe I'll revisit this more later, but for now, I think I'm happy with it just working.
+
+## Watching UART on the Oscilloscope vs the Logic Analyzer
+
+I experimented with examining UART traffic with the oscilloscope, and the logic analyzer. The clear winner was the logic analyzer.
+
+The oscilloscope could show the waveforms of the data coming off the serial TX/RX pins, but it didn't really give me much of an opportunity to understand what was actually being transmitted. It also didn't really help me understand the timing of the transmissions. That could be an issue of my oscilloscope - the Hanmatek DOS1102 is not the most advanced thing in the world, and I'm not an expert at using it. Anyway, while it worked for showing lines moving on the pins, it didn't really help me do any analysis.
+
+The Digilent logic analyzer, however, was a whole other beast. Not only could it capture the transmission on the pins as square waves, it would map them up to a millisecond scale timeline, and it would overlay the ASCII on top of it so I could see how the waveform was actually sending the data! That's pretty damn slick.
+
+![Logic analyzer showing UART traffic at the line level](logicuart.png)

serialuart/serialuart.ino

@@ -0,0 +1,113 @@
+#define BUFLEN      512
+#define SERIALSPEED 115200
+#define ERROR_SYNTAX  1
+#define ERROR_COMMAND 2
+
+typedef struct CmdParser {
+  char buffer[BUFLEN] = {0};
+  char atoibuf[32] = {0};
+  char *tok = NULL;
+  char *atoiptr = NULL;
+  int pin = 0;
+} CmdParser;
+
+void setup() {
+  Serial0.begin(SERIALSPEED, SERIAL_8N1, 21, 14);
+}
+
+int parse_digit_from_buffer(CmdParser *parser) 
+{
+  parser->atoiptr = parser->tok;
+  while ( *parser->atoiptr != ' ' && *parser->atoiptr != 0x00 ) {
+    if ( !isDigit((char)*parser->atoiptr) ) { 
+      Serial0.printf("Expected digit, found %s\n", parser->tok);
+      return ERROR_SYNTAX;
+    } else if (  parser->atoiptr == ((char *)&parser->buffer + 512 ) ) {
+      Serial0.printf("Reached end of string looking for next token : %s\n", parser->tok);
+      return ERROR_SYNTAX;
+    }
+    parser->atoiptr++;
+  }
+  strncpy((char *)&parser->atoibuf, parser->tok, (int)(parser->atoiptr - parser->tok));
+  parser->tok = parser->atoiptr;
+  return 0;
+}
+
+int command_writepin(CmdParser *parser)
+{
+  int rc = 0;
+  parser->tok = strtok(NULL, " ");
+  rc = parse_digit_from_buffer(parser);
+  if ( rc != 0 ) {
+    return rc;
+  }
+  parser->pin = atoi((char *)&parser->atoibuf);
+  parser->tok = strtok(NULL, " ");
+  if ( strncmp(parser->tok, "high", 4) == 0 ) {
+    digitalWrite(parser->pin, HIGH);
+    Serial0.printf("digitalWrite(%d, HIGH)", parser->pin);
+  } else if ( strncmp(parser->tok, "low", 3) == 0 ) {
+    digitalWrite(parser->pin, LOW);
+    Serial0.printf("digitalWrite(%d, LOW)", parser->pin);
+  } else {
+    Serial0.printf("Expected HIGH or LOW, found %s", parser->tok);
+    return ERROR_SYNTAX;
+  }
+  return 0;
+}
+
+int command_setpin(CmdParser *parser)
+{
+  int rc = 0;
+  parser->tok = strtok(NULL, " ");
+  rc = parse_digit_from_buffer(parser);
+  if ( rc != 0 ) {
+    return rc;
+  }
+  parser->pin = atoi((char *)&parser->atoibuf);
+  parser->tok = strtok(NULL, " ");
+  if ( strncmp(parser->tok, "input", 5) == 0 ) {
+    pinMode(parser->pin, INPUT);
+    Serial0.printf("pinMode(%d, INPUT)", parser->pin);
+  } else if ( strncmp(parser->tok, "output", 6) == 0 ) {
+    pinMode(parser->pin, OUTPUT);
+    Serial0.printf("pinMode(%d, OUTPUT)", parser->pin);
+  } else {
+    Serial0.printf("Expected input or output, found %s", parser->tok);
+    return ERROR_SYNTAX;
+  }
+  return 0;
+}
+
+int parse_command(CmdParser *parser)
+{
+  parser->tok = strtok((char *)&parser->buffer, " ");
+  if ( strcmp(parser->tok, "setpin") == 0 ) {
+    return command_setpin(parser);
+  } else if ( strcmp(parser->tok, "writepin") == 0 ) {
+    return command_writepin(parser);
+  }
+  return ERROR_COMMAND;
+}
+
+void loop() {
+  CmdParser parser;
+  memset((void *)&parser, 0x00, sizeof(CmdParser));
+  Serial0.println("");
+  Serial0.println("READY");
+  while ( Serial0.available() == 0 ) {
+    delay(10);
+  }
+  if ( Serial0.readBytesUntil('\0', (char *)&parser.buffer, BUFLEN) != 0 ) {
+    switch ( parse_command(&parser) ) {
+      case ERROR_SYNTAX:
+        Serial0.println("Syntax error");
+        break;
+      case ERROR_COMMAND:
+        Serial0.println("Unknown command");
+        break;
+      case 0:
+        return;
+    }
+  }
+}