Handle an M5Stack Core2 for AWS – C/C++
15 min read
The primary article mentioned M5Stack administration based mostly on the Micropython language. Now, we have to dive a lot deeper right into a rabbit gap. Let’s attempt to use C and C++ solely. A very powerful benefit of utilizing C is the potential of full, low-lever management of all controller points. A very powerful drawback of utilizing C is the need of full, low-lever management of all controller points. Effectively… with nice energy comes nice duty.
FreeRTOS
AWS FreeRTOS is a real-time working system devoted to AWS cloud and resource-constrained gadgets.
There may be a whole lot of code to jot down this time, so we’ll use an instance immediately from AWS. There isn’t a have to burn any firmware with the burning instrument; nonetheless, we nonetheless have to cross the USB port to the WSL surroundings utilizing usbip, as we’ve completed within the „Micropython” part of the primary chapter.
You may obtain the code we’re going to make use of from https://github.com/m5stack/Core2-for-AWS-IoT-EduKit.git . The one subdirectory we’d like is Blinky-Good day-World, however the repository is admittedly small, so utilizing a sparse checkout is pointless, and you’ll merely clone the whole repo.
Open VSCode and set up a plugin known as PlatformIO. There’s a bug in PlatformIO, so you possibly can’t see any information out of your WSL surroundings utilizing the PlatformIO browser (Windows WSL: I can’t open any files(projects) in the PIO open browser. · Issue #2316 · platformio/platformio-home · GitHub). To repair it, shut VSCode, edit ~/.platformio/packages/contrib-piohome/essential.*.min.js file in Home windows, change „”: „/” with „/”: „/”, and open VSCode once more.
To confirm the connection between PlatformIO and your controller, open PlatformIO from the very left menu after which choose „Units” from the primary left menu. You need to see /dev/ttyUSB0 within the heart a part of the display. Please bear in mind to cross the USB gadget to WSL utilizing usbip and to permit all customers to make use of the port with chmod.
If every part seems to be good up to now, you possibly can open the Blinky-Good day-World listing (not the whole cloned repository) as a venture from the PlatformIO dwelling display. Now you possibly can observe the important parts of the official instruction supplied under.
You could have AWS CLI v2 put in in your machine. When you don’t, you possibly can set up it utilizing the official handbook: Installing or updating the latest version of the AWS CLI – AWS Command Line Interface (amazon.com)
Now guarantee you’ve a sound token, and you’ll work together along with your AWS account utilizing CLI (I suggest itemizing some assets because the verification, e.g., aws s3 ls).
We’ll use the built-in script to create a Factor in AWS IoT. Simply open a terminal utilizing PlatformIO (normal bash terminal gained’t work, so it’s essential to open it from Miscellaneous -> New Terminal from the primary PlatformIO menu in VSC), be sure you’re in Blinky-Good day-World listing, and run pio run -e core2foraws-device_reg -t register factor. The script will create the Factor and obtain the required certificates/key information. You are able to do it manually if you happen to don’t belief such scripts; nonetheless, this one is created by the AWS group, so I consider it’s reliable.
Within the AWS IoT console, go to Handle -> All gadgets -> Issues and see the brand new Factor created by the script. The Factor title is autogenerated. In my case, it’s 0123FAA32AD40D8501.
OK, the subsequent step is to permit the gadget to connect with the Web. There may be one other script that can assist you with this job. Name pio run ‐‐surroundings core2foraws ‐‐goal menuconfig. You’ll see a easy menu. Navigate to AWS IoT EduKit Configuration and arrange WiFi SSID abd WiFi Password. Bear in mind that your community’s SSID and password shall be saved as plaintext in a number of information in your code now.
Let’s construct the appliance. Simply name pio run ‐‐surroundings core2foraws from the PlatformIO terminal after which pio run ‐‐surroundings core2foraws ‐‐goal add ‐‐goal monitor to run it in your gadget and monitor logs.
Now you should use the MQTT check shopper from the AWS IoT console to ship something to <<factor title>>/blink matter. In my case, it’s 0123FAA32AD40D8501/blink. The message payload doesn’t matter for this instance. Simply ship one thing to start out blinking and ship something once more to cease it.
As you possibly can see, we have now completed lots simply to speak between AWS Cloud and the controller. It was a lot easier with Micropython and much more with UiFlow. Nevertheless, C is far more highly effective, and what’s most necessary right here, we are able to prolong it with libraries.
TensorFlow Lite for Microcontrollers
TensorFlow is an end-to-end open-source platform for machine studying. TensorFlow Lite is a library for deploying fashions on cell, microcontrollers, and different edge gadgets.
TensorFlow Lite for Microcontrollers is only a light-weight model of TensorFlow Lite designed to run machine studying fashions on microcontrollers and different gadgets with just a few kilobytes of reminiscence. The core runtime suits in 16 KB on an Arm Cortex M3 and may run many primary fashions. It doesn’t require working system assist, any normal C or C++ libraries, or dynamic reminiscence allocation.
TensorFlow Lite isn’t designed to work on ESP32 processors, so the one one out there for M5Stack is TensorFlow Lite for Microcontrollers. It has some limitations – it helps only a restricted subset of TensorFlow operations and gadgets, it requires handbook reminiscence administration in Low-level C++ API, and it doesn’t assist on-device coaching. Due to this fact, to construct a „studying on the edge” answer, you want a extra highly effective IoT Edge gadget, e.g. Raspberry Pi.
However you possibly can nonetheless run ML fashions on the M5Stack controller.
Now, let’s attempt to modify our Blinky-Good day-World so as to add the TensorFlow Lite for the Microcontrollers library.
TensorFlow Lite for Microcontrollers in FreeRTOS
The primary difficulty to unravel is the place to get the TensorFlow supply code from. In the primary TensorFlow repository, you’ll find data that it’s moved to a standalone one (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro) even when a lot of the documentation and examples nonetheless level there. The standalone repository used to comprise a makefile for ESP32, nevertheless it appears to be deleted when transferring examples to one more repository (https://github.com/tensorflow/tflite-micro/commit/66cfa623cbe1c1ae3fcc8a4903e9fed1a345548a). At this time, one of the best supply appears to be this repository: https://github.com/espressif/tflite-micro-esp-examples/tree/master/components.
We’ll want tfline-lib, nevertheless it doesn’t work with out esp-nn, so it’s best to copy each to the parts listing in your Blinky-Good day-World venture.
Let’s modify our code, ranging from together with tensorflow headers at the start of the primary.c file.
#embody "tensorflow/lite/micro/all_ops_resolver.h" #embody "tensorflow/lite/micro/micro_error_reporter.h" #embody "tensorflow/lite/micro/micro_interpreter.h" #embody "tensorflow/lite/schema/schema_generated.h"
Now we are able to attempt to use it. For instance, simply earlier than the void app_main() operate, let’s declare TF error reporter and use it within the operate.
tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main() Core2ForAWS_Init(); Core2ForAWS_Display_SetBrightness(80); ui_init(); TF_LITE_REPORT_ERROR(error_reporter, "Good day TensorFlow" "That is only a check message/n" ); initialise_wifi(); xTaskCreatePinnedToCore(&aws_iot_task, "aws_iot_task", 4096 * 2, NULL, 5, NULL, 1); xTaskCreatePinnedToCore(&blink_task, "blink_task", 4096 * 1, NULL, 2, &xBlink, 1); }
Clearly, it’s not an precise utilization of TensorFlow, nevertheless it proves the library is linked and can be utilized no matter you want.
In the primary listing, you have to additionally add new libraries tflite-lib and esp-nn to the required parts in CMakeLists.txt
set(COMPONENT_REQUIRES "nvs_flash" "esp-aws-iot" "esp-cryptoauthlib" "core2forAWS" "tflite-lib" "esp-nn")
It seems to be good, nevertheless it gained’t work but. Throughout compilation utilizing pio run –surroundings core2foraws, you’ll discover out that the whole Blinky-Good day-World is made in pure C, and TensorFlow Lite for Microcontrollers library requires C++. The best approach to convert it’s as follows:
- Rename essential.c to essential.cc
- Change essential.c to essential.cc within the first line of essential/CMakeList.txt
- Create extern „C” part for the whole essential file code aside from tensorflow imports.
It ought to look by some means like that:
#embody "tensorflow/lite/micro/all_ops_resolver.h" #embody "tensorflow/lite/micro/micro_error_reporter.h" #embody "tensorflow/lite/micro/micro_interpreter.h" #embody "tensorflow/lite/schema/schema_generated.h" extern "C" ######authentic essential.c content material goes right here###### tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main() #the primary operate code from the itemizing above
- In essential.cc, delete TaskHandle_t xBlink; declaration as a result of it’s already declared in one other file
- In platform.ini, in [env:core2foraws] part add build_flags = -fpermissive to alter permissive compilation errors into warnings
Now you possibly can construct the venture once more. When working it with the goal –monitor, you’ll see the „Good day TensorFlow” message in logs, which suggests the TensorFlow library is included and dealing appropriately.
Now, you are able to do no matter you need with an out-of-the-box machine studying library and AWS integration.
Arduino
As you possibly can see, C is far more highly effective however requires far more work. Let’s attempt to join the identical blocks (tensorflow, AWS IoT, and M5Stack library) however utilizing a extra user-friendly surroundings.
Arduino is an open-source digital prototyping platform enabling customers to create interactive digital objects. Let’s attempt to mix the official M5Stack Core 2 for AWS with the Arduino IDE handbook (https://docs.m5stack.com/en/quick_start/core2_for_aws/arduino) with TensorFlow Lite for Microcontrollers (https://github.com/tanakamasayuki/Arduino_TensorFlowLite_ESP32).
Good day world!
Firstly, set up Arduino IDE from the official web page https://www.arduino.cc/en/software. I assume you have already got the CP210x driver put in, and the USB mode chosen in your gadget.
Open the IDE, go to File -> Preferences, and add the boards’ administration URL: https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/arduino/package_m5stack_index.json.
Then open the Boards supervisor from the left menu and set up M5Stack-Core2. Now join the controller to the pc and select it from the highest drop-down menu.
To make use of the M5Stack-specific library within the code, it’s essential to open Sketch -> Embody Libraries -> Library catalog and set up M2Core2.
Now you possibly can write the straightforward „Good day World!” code and run it with the inexperienced arrow within the IDE high menu.
#embody <M5Core2.h>
void setup()
M5.start();
M5.Liquid crystal display.print("Good day World");
void loop()
Typically, Arduino can’t reset the controller through an RTS pin, so it’s essential to reboot it manually after writing a brand new code to it.
Thus far, so good.
TensorFlow and AWS integration
The TensorFlow-official, Arduino_TensorFlowLite_ESP32 library isn’t designed for use with M5Stack. Let’s adapt it. Clone the library and replica the Good day World instance to a different listing. You may open it from Arduino IDE now. It’s a totally working instance of the utilization of the TensorFlow mannequin. Let’s adapt it to make use of the M5Core2 library. To hello_world.ino it’s essential to add #embody <M5Core2.h> at the start of the file and likewise M5.start(); at the start of void setup() operate. You may also add M5.Axp.SetLed(true); after this line to activate the small inexperienced led and make sure the gadget is working.
Now, begin the appliance. You may see TensorFlow output within the Serial Monitor tab. Simply change the baud price to 115200 to make it human-readable.
Can we combine it with AWS IoT integration? Sure, we are able to.
We’ll use the PubSubClient library by Nick O’Leary, so open the library catalog in Arduino IDE and set up it, after which let’s connect with AWS IoT and MQTT.
Utilizing Arduino IDE, create a brand new file secrets and techniques.h. We want a number of declarations there:
#outline AWS_IOT_PUBLISH_TOPIC " m5stack/pub" #outline AWS_IOT_SUBSCRIBE_TOPIC " m5stack/sub" #outline WIFI_SSID "ThisIsMyWiFiSSID" #outline WIFI_PASSWORD "Do not use really easy passwords!" int8_t TIME_ZONE = 2; #outline MQTT_HOST "xxxx.iot.eu-west-1.amazonaws.com" #outline THINGNAME "UiFlow_test" static const char* ca_cert = R"KEY( -----BEGIN CERTIFICATE----- … -----END CERTIFICATE----- )KEY"; static const char* client_cert = R"KEY( -----BEGIN CERTIFICATE----- … -----END CERTIFICATE----- )KEY"; static const char* privkey = R"KEY( -----BEGIN RSA PRIVATE KEY----- … -----END RSA PRIVATE KEY----- )KEY";
AWS_IOT_PUBLISH_TOPIC and AWS_IOT_SUBSCRIBE_TOPIC are our check subjects we’re going to make use of on this instance. WIFI_SSID and WIFI_PASSWORD are our WiFi credentials. TIME_ZONE is the time zone offset. MQTT_HOST is the general public AWS IoT endpoint (the identical as within the first UiFlow instance). THINGNAME is the title of Factor in AWS (I’ve used the identical as within the UiFlow instance). client_cert and privkey, it’s essential to copy from the secrets and techniques generated when creating Factor for the UiFlow instance. ca_cert is the general public key of AWS certificates authority, so you possibly can get hold of it from the Factor creation wizard (certificates step) or from https://good.sca1a.amazontrust.com/).
Now it’s time to adapt the primary hello_world.ino file.
We must always add new imports (together with our secret.h file).
#embody <WiFiClientSecure.h> #embody <PubSubClient.h> #embody "secrets and techniques.h" #embody <time.h>
Then we’d like a number of new fields.
WiFiClientSecure internet; PubSubClient shopper(internet); time_t now; time_t nowish = 1510592825;
The sphere nowish is just a few timestamp up to now.
Within the setup() operate, we have to open a WiFi reference to our native community and the Web, arrange the time to examine certificates, set up the certificates, arrange the MQTT shopper, and open the AWS IoT connection.
delay(3000);
WiFi.mode(WIFI_STA);
WiFi.start(WIFI_SSID, WIFI_PASSWORD);
WiFi.waitForConnectResult();
whereas (WiFi.standing() != WL_CONNECTED)
Serial.print(".");
delay(1000);
M5.Liquid crystal display.println(String("Making an attempt to connect with SSID: ") + String(WIFI_SSID));
M5.Liquid crystal display.println(WiFi.localIP());
M5.Liquid crystal display.print("Setting time utilizing SNTP");
configTime(TIME_ZONE * 3600, 0 * 3600, "pool.ntp.org", "time.nist.gov");
now = time(nullptr);
whereas (now < nowish)
delay(500);
Serial.print(".");
now = time(nullptr);
M5.Liquid crystal display.println("completed!");
struct tm timeinfo;
gmtime_r(&now, &timeinfo);
M5.Liquid crystal display.print("Present time: ");
M5.Liquid crystal display.print(asctime(&timeinfo));
internet.setCACert(ca_cert);
internet.setCertificate(client_cert);
internet.setPrivateKey(privkey);
shopper.setServer(MQTT_HOST, 8883);
shopper.setCallback(messageReceived);
M5.Liquid crystal display.println("Connecting to AWS IOT");
whereas (!shopper.join(THINGNAME))
Serial.print(".");
delay(1000);
if (!shopper.linked())
M5.Liquid crystal display.println("AWS IoT Timeout!");
return;
shopper.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);
M5.Liquid crystal display.println("AWS IoT Related!");
That is a complete code wanted to arrange the appliance, however I suggest splitting it into a number of smaller and extra readable capabilities. As you possibly can see, I take advantage of Serial output for debugging.
To obtain messages, we’d like a brand new operate (the title matches the declaration in shopper.setCallback(messageReceived);)
void messageReceived(char *matter, byte *payload, unsigned int size)
M5.Liquid crystal display.print("Obtained [");
M5.Lcd.print(topic);
M5.Lcd.print("]: ");
for (int i = 0; i < size; i++)
M5.Liquid crystal display.print((char)payload[i]);
M5.Liquid crystal display.println();
The very last thing to do is to loop the shopper with the whole utility. To do this, simply add a one-liner to the loop() operate:
shopper.loop();
You want one other one-liner to ship one thing to AWS, however I’ve added two extra to make it seen on the controller’s show.
M5.Liquid crystal display.println("Sending message");
shopper.publish(AWS_IOT_PUBLISH_TOPIC, ""message": "Good day from M5Stack"");
M5.Liquid crystal display.println("Despatched");
The communication works each methods. You may subscribe to m5stack/pub utilizing the MQTT Take a look at shopper within the AWS console to learn messages from the controller, and you’ll publish to m5stack/sub to ship messages to the controller.
As you possibly can see, utilizing Arduino is less complicated than utilizing FreeRTOS, however sadly, it’s a bit bit babyish. Now we’ll attempt to keep away from all IDE’s and use pure console solely.
Espressif IoT Improvement Framework
Principally, there are 3 ways to burn software program to the controller from a Linux console – Arduino, esptool.py, and ESP-IDF. While you create a brand new venture utilizing PlatformIO, you possibly can choose Arduino or ESP-IDF. Now, let’s attempt to take away the IDE from the equation and use a pure bash.
To start with, it’s essential to set up a number of stipulations after which obtain and set up the library.
sudo apt set up git wget flex bison gperf python3 python3-venv cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0 mkdir -p ~/esp cd ~/esp git clone --recursive <a href="https://github.com/espressif/esp-idf.git">https://github.com/espressif/esp-idf.git</a> cd ~/esp/esp-idf ./set up.sh esp32 ./export.sh
Please be aware it’s essential to run set up and export (final two instructions) everytime you open a brand new WSL console. With the library, you even have some examples downloaded. Run one among them to examine, does every part work.
cd examples/get-started/hello_world/ idf.py set-target esp32 set ESPPORT=/dev/ttyUSB0 idf.py construct flash monitor
You need to see the output like this one.
Good day world! That is an esp32 chip with 2 CPU core(s), WiFi/BT/BLE, silicon revision 300, 2MB exterior flash Minimal free heap dimension: 295868 bytes Restarting in 10 seconds... Restarting in 9 seconds... Restarting in 8 seconds... Restarting in 7 seconds... Restarting in 6 seconds... Restarting in 5 seconds... Restarting in 4 seconds... Restarting in 3 seconds... Restarting in 2 seconds... Restarting in 1 seconds... Restarting in 0 seconds... Restarting now.
To cease the serial port monitor, press CRTL + ]. Bear in mind that the appliance continues to be working on the controller. You could energy off the gadget by the {hardware} button on the facet to cease it.
If you wish to use TensorFlow Lite for Microcontrollers with ESP-IDF, it’s essential to create a brand new venture and add a correct library. You need to use the command idf.py create-project <<project_name>> to create a venture. My venture title is hello_tf. The script creates a pure C venture; we have to rename hello_tf.c file to hello_tf.cc. Then, we are able to copy tflite-micro and esp-nn libraries from FreeRTOS instance and place them within the parts listing. The essential/CMakeList.txt content material needs to be like that.
set(COMPONENT_SRCS "hello_tf.cc") set(COMPONENT_REQUIRES "tflite-lib" "esp-nn") register_component()
As you possibly can see, the default parts sources definition is modified, and new libraries are added.
Now, let’s see the primary hello_tf.cc file content material.
#embody "tensorflow/lite/micro/all_ops_resolver.h" #embody "tensorflow/lite/micro/micro_error_reporter.h" #embody "tensorflow/lite/micro/micro_interpreter.h" #embody "tensorflow/lite/schema/schema_generated.h" extern "C" tflite::MicroErrorReporter micro_error_reporter; tflite::ErrorReporter* error_reporter = µ_error_reporter; void app_main(void) TF_LITE_REPORT_ERROR(error_reporter, "Good day from TensorFlown");
As you possibly can see, we had to make use of extern „C” block once more as a result of, by default, ESP-IDF runs the void app_main() operate from C, not C++ context.
To run the appliance run idf.py construct flash monitor.
In the identical means, you possibly can add different libraries wanted, however with out PlatformIO, dependency administration is hard, particularly for the core2forAWS library with a number of dependencies. Alternatively, you should use https://github.com/m5stack/M5Stack-IDF as a library with M5Stack dependencies to manage the I/O gadgets of the controller.
Abstract
As I wrote at the start of this text, with C++, you are able to do far more; nonetheless, you’re pressured to handle the whole gadget by your self. Sure, you should use AWS integration, M5Stack I/O interfaces, and TensorFlow (TensorFlow Lite for Microcontrollers model solely) library collectively, nevertheless it requires a whole lot of code. Can we do something to affix some great benefits of utilizing Micropython and C collectively? Let’s attempt to do it within the final chapter.
