First, the big news. The latest updates to the Micropython code base now allow it to be built with Espressif’s ESP-IDF version 5.2. At the same time these changes were pushed out into the Micropython repository, Espressif dropped ESP-IDF 5.2.1, which I installed on my development system, then used to build my updated Micropython with the latest changes noted above. Micropython successfully built and I was then able to flash one of my ESP32-S3 development boards, and away I went with my stand-alone WiFi test board.

I then used the same ESP-IDF tool chain to build one of my C++ embedded applications, in which it joins my home WiFi access point and allows me to then bring up its simple web page. Again, everything built and flashed. This is with an ESP32-S3 development board with 32 MiB flash and 8 MiB external ram.

CircuitPython Update

One other update comes from CircuitPython: they’ve just dropped version 9.0.0 release candidate 0. I’ve downloaded and installed it on two of my boards that can handle it, and so far it’s doing just fine. If you like CircuitPython and use it for ESP32-S3-based boards, then by all means consider downloading and installing this version. It’s clean and stable enough for you to work with. Quoting from the release note, the key features out of many that caught my eye are:

• Merge updates from MicroPython v1.19.1, v1.20.0, and v1.21.0.
• Espressif: update to ESP-IDF v5.1.3.

Micropython and CircuitPython are moving rapidly, adding support for the latest boards as well as adding new functionality for existing devices, especially on the Espressif side of things.

I purchased the recently release Arduino UNO R4 WiFi from Amazon for US$27.50, the same price that Arduino charges on its website. I was drawn to it because if its intriguing mix of a Renesas RA4M1 (Arm® Cortex®-M4) running at 48MHz and an Espressif ESP32-S3 microcontroller peripheral for WiFi and Bluetooth support. I’ve barely gotten started with working with it, so I don’t have much to say about what its fit for purpose for, but so far it’s been extremely easy to program when paired with Arduino’s latest Arduino IDE, version 2.3.2.

Pairing this latest Arduino board to the Arduino IDE is incredibly easy. After first plugging in the R4 and then starting the IDE, the IDE immediately identified the board and was ready for me to build a sketch and then flash that sketch to the R4. A little bit of searching uncovered the C/C++ source to the initial application the board is shipped with (see links below), as well as a lot of other programming examples that exercise the WiFi and Bluetooth features of the ESP32-S3 peripheral.

One big feature I like is that the new Arduino IDE 2 is based on Eclipse Theia, an alternative framework for building IDEs. That is completely unlike Visual Studio Code, which is built on top of Electron. That makes the Arduino IDE 2 a much more agile and leaner tool than Visual Studio Code. I prefer Arduino IDE over Visual Studio Code.

One little feature about this board, and all other Arduino boards like it; it ships with a clear plastic cover snapped onto the bottom of the board. That means you can start to work with the board as soon as you remove it from its box, because sitting it down won’t cause any part of the board to short, especially if you sit it on a conductive surface. I find that original Arduino boards have a touch of engineering class, if you will.

I also find that the low cost of the UNO 4 compared to recent Raspberry Pi boards (both the 4 and the 5) is a lot cheaper, yet it’s powerful enough to run embedded applications that can do quite a bit. Unless you’ve got a strong use case for running embedded Linux, the UNO 4 is more than capable of performing quite a bit without the much higher cost and overhead of running an embedded operating system to support your embedded application.

Bottom line, I’m quite happy to have purchased this board, and I’m looking forward to seeing what I can do with it. I’m sure I’ll be able to do quite a bit.

Links

• Arduino® UNO R4 WiFi — https://store-usa.arduino.cc/products/uno-r4-wifi
• Getting Started with UNO R4 WiFi (initial Tetris animation sketch) — https://docs.arduino.cc/tutorials/uno-r4-wifi/r4-wifi-getting-started/
• Arduino® UNO R4 WiFi Tutorials — https://docs.arduino.cc/hardware/uno-r4-wifi/#tutorials
• The Eclipse Theia Platform — https://theia-ide.org/

I’ve been busy these past few weeks with my part-time job. The week of February 12th I spent out in Louisiana and billed 65 hours for the week. This week I’ve been doing other tasks, both to catch up from being on travel the week before as well as doing much less work in my new part time job. I still managed to read a bit and discovered, while out in Louisiana, that updates had been made to the open source project Micropython (version 1.22.2 was released) as well as Espressif’s commercial but open ESP-IDF C/C++ embedded tool chain.

When I got back this week I updated my personal fork of Micropython on Github, then updated my home Micropython development area via git, then set about testing all the updates in Micropython by using ESP-IDF 5.0.4 to build it, then began to experiment to see if Micropython would build with ESP-IDF 5.1.2. I mention ESP-IDF version 5.1.2 because Adafruit wrote that the latest CircuitPython beta, 9.0.0 beta 2, was built with ESP-IDF 5.1.2 after rolling in the latest changes from Micropython. When Micropython 1.22.0 was released in January the developers had locked in ESP-IDF 5.0.4 as the version that would build Micropython. I tried to use 5.1.2 and Micropython failed to build, so I stayed with ESP-IDF 5.0.4. Now the latest Micropython updates build successfully with 5.1.2, which the bottom of the little test web page provided by one of my ESP32-S3 N8R8 boards shows (see above). I’m quite happy that main Micropython builds with ESP-IDF 5.1.2 because I currently have three versions of ESP-IDF; 5.0.4, 5.1.2, and 5.2. I would like to drop the oldest (5.0.4) and even move Micropython compilation up to 5.2 so I can drop 5.1.2 as well. I do all my development with 5.2, only using an older version to build Micropython.

This next example is another Espressif board, the ESP32-S3 N32R8 running code written in C++ and compiled with ESP-IDF 5.2. As you should note at the bottom it’s been built with ESP-IDF 5.2, which is the current stable release. So far everything is working well, especially WiFi. I’m in the process of finishing up a MAX7219 C++ class using SPI for communication with the MAX7219. Once finished I’ll write about it here as well as push the source up to my modest Github space.

I added the ability to define what all my devices were inside a JSON file, then modified my Python version of this utility to read in that file and use those descriptions to better remind me devices what I had plugged in. Along the way I also created the same type of utility in C++ and Rust, which were adventures in and of themselves.

This entry documents additional evolution of the C++ version of the utility. The changes from the first version of the C++ utility are;

1. Read in and parse the devices.json file that contains the human understandable annotations.
2. The ability to create a “raw” version of the devices.json file if you don’t have one.

The devices.json file must reside in the same folder as the devices utility itself. Furthermore, if you decide to generate a new devices.json file then you need to have write privileges to that folder; that’s why mine resides in my home folder’s bin directory. Finally, so that you don’t shoot yourself in the foot and try to generate a new annotations file on top of your existing annotations file, devices will abort and not write over an existing devices.json file.

Listing

#include <string>#include <array>#include <map>#include <algorithm>#include <iostream>#include <fstream>#include <filesystem>#include <regex>#include <exception>#include <boost/program_options.hpp>#include <boost/json.hpp>using std::cout;using std::cerr;using std::exception;using std::string;using std::array;using std::map;using std::pair;using std::regex;using std::regex_search;using std::replace;using std::smatch;using boost::program_options::options_description;using boost::program_options::value;using boost::program_options::variables_map;using boost::program_options::store;using boost::program_options::parse_command_line;using boost::program_options::notify;using std::filesystem::directory_iterator;using std::filesystem::path;using std::filesystem::read_symlink;using std::filesystem::exists;void split(const string &subj, const regex &rgx, array<string, 2> &vars) {smatch match;regex_search(subj, match, rgx);vars[0] = subj.substr(0, match.position(0));replace(vars[0].begin(), vars[0].end(), '_', ' ');vars[1] = match.str(0).erase(0,1);vars[1].erase(vars[1].end()-1);}map<string, array<string,2>> devices;map<string, string> device_annotations;void find_devices() {const regex hregex{"_[0-9A-Ia-f]+-"};for (const auto &entry : directory_iterator("/dev/serial/by-id/")) {string slink = read_symlink(entry).stem().generic_string();string sname = entry.path().stem().generic_string().erase(0,4);array<string, 2> results;split(sname, hregex, results);devices[slink] = results;}}void print_devices() {if (not device_annotations.empty()) {for (auto [device, values] : devices) {cout << device << ", " << (device_annotations.contains(values[1]) ?device_annotations[values[1]] : values[1]) << "\n";}}else {for (auto [device, values] : devices) {cout << device << ", "  << values[0] << ", " << values[1] << "\n";}}}namespace json = boost::json;using std::filesystem::file_size;void parse_devices_json_annotations(int argc, char **argv) {if (argc <= 0 or strlen(argv[0]) == 0) {cout << "JSON no application name in environment to work with\n";return;}auto json_name = path(argv[0]).stem().string().append(".json");auto json_path = read_symlink("/proc/self/exe").replace_filename(json_name);if (not exists(json_path)) {cout << "JSON annotation file missing - no annotations" << "\n";return;}auto json_file_size = file_size(json_path);if (json_file_size == 0) {cout << "JSON file size is 0 - aborting\n";return;}std::ifstream json_file(json_path);if (not json_file.is_open()) {cout << "JSON file did not open - aborting\n";return;}json::stream_parser parser;json::error_code err_code;auto memblock = new char[json_file_size];std::memset(memblock, 0, json_file_size);json_file.read(memblock, json_file_size);json_file.close();parser.write(memblock, json_file_size, err_code);delete [] memblock;if (err_code) {cout << "JSON failed to parse - aborting\n";return;}parser.finish(err_code);if (err_code) {cout << "JSON failed to finish parsing - aborting\n";return;}json::value json_data = parser.release();if (json_data.is_null()) {cout << "JSON data is empty - aborting\n";}if (auto obj = json_data.if_object()) {auto annotations = obj->at("devices");if (annotations.is_array()) {for( auto element : annotations.get_array() ) {auto my_key = element.at("hexid").as_string().c_str();auto my_val = element.at("description").as_string().c_str();device_annotations[my_key] = my_val;}}}else {cout << "JSON top level data is not an array - aborting\n";}}void generate_default_json_annotations(int argc, char **argv) {if (argc <= 0 or strlen(argv[0]) == 0) {cout << "JSON no application name in environment to work with\n";return;}find_devices();if(devices.empty()) {cout << "No devices found. No annotations file generated.\n";return;}auto json_name = path(argv[0]).stem().string().append(".json");auto json_path = read_symlink("/proc/self/exe").replace_filename(json_name);if (exists(json_path)) {cout << "JSON annotation file already exists, will not overwrite." << "\n";return;}std::ofstream json_file(json_path);cout << "Generating default JSON annotations file "<< json_path << "\n";if (json_file.is_open()) {auto device_count = devices.size();cout << "Device count: " << device_count << "\n";json_file << "{ \"devices\" : [\n";for (auto [device, values] : devices) {json_file << "{\n\"hexid\" : \"";json_file << values[1] << "\",\n";json_file << "\"description\" : \"" << values[0];json_file << (--device_count ? "\"\n},\n" : "\"\n}\n");}json_file << "] }\n";json_file.close();return;}else {cout << "JSON output file did not open - aborting.\n";return;}}int main(int argc, char **argv) {options_description options("devices options");options.add_options()("help", "Help using application.")("json", "Generate starting JSON annotations file.");try {variables_map varmap;store(parse_command_line(argc, argv, options), varmap);notify(varmap);if (varmap.count("help")) {cout << options << "\n";return 0;}if (varmap.count("json")) {generate_default_json_annotations(argc, argv);return 0;}}catch (exception &ex) {cerr << "Error: " << ex.what() << "\n";cerr << "\n" << options << "\n";return -1;}parse_devices_json_annotations(argc, argv);find_devices();print_devices();return 0;}

Example Runs

A run of the utility without a JSON annotations file. Note the comment on the first line, and a sorted list (by device) of all the connected development board devices. Without the annotations file all that’s listed are the name of each USB adapter on each board followed by each boards unique hexadecimal ID.

JSON annotation file missing - no annotationsttyACM0, Adafruit Adafruit Feather ESP32S3 4MB Flash 2MB PSRAM, 4F21AF95CC4DttyUSB0, Silicon Labs CP2102N USB to UART Bridge Controller, 7eab1642dbfaeb1198773ca4c6d924ecttyUSB1, Silicon Labs CP2102N USB to UART Bridge Controller, e08d862b0867ec118f12a17089640db2ttyUSB2, Silicon Labs CP2102N USB to UART Bridge Controller, 964756d6d823ed119c228ee8f9a97352ttyUSB3, Silicon Labs CP2102N USB to UART Bridge Controller, 80e3350df417ec11baa043103803ea95

Here’s what happens when I create a raw devices.json file.

$ ./build/devices --jsonGenerating default JSON annotations file "/home/mint/Develop/CPPwork/devices/build/devices.json"Device count: 5

Now when I re-run the utility, the annotations file is read in and parsed. Note that the hexadecimal IDs are gone. That’s because they’ve been used to read out the default annotations, which are the USB adapter names. This can be confusing at first. This is where you, as the developer, open up devices.json and edit those USB names and replace them with something that has meaning to you.

ttyACM0, Adafruit Adafruit Feather ESP32S3 4MB Flash 2MB PSRAMttyUSB0, Silicon Labs CP2102N USB to UART Bridge ControllerttyUSB1, Silicon Labs CP2102N USB to UART Bridge ControllerttyUSB2, Silicon Labs CP2102N USB to UART Bridge ControllerttyUSB3, Silicon Labs CP2102N USB to UART Bridge Controller

Here’s what happens when I run the utility with my personal devices.json file.

ttyACM0, CircuitPython 8.2 - WiFi/UDP/JSONttyUSB0, ESP32-C3-DevKitC-1-N4 - ESP-IDF 5.2-dev DisplaysttyUSB1, ESP32-S3-DevKitC-1.1-N32R8 - ESP-IDF 5.2-dev WiFittyUSB2, ESP32-C6-DevKitC-1-N8 - ESP-IDF 5.2-dev NeoPixelttyUSB3, ESP32-S3-DevKitC-1.1-N8R8 - MicroPython 1.20.0 WiFi

Building

I use cmake to build the application.

cmake_minimum_required (VERSION 3.27.0)cmake_policy(SET CMP0144 NEW)project (devices)set(Boost_NO_WARN_NEW_VERSIONS 1)find_package(Boost 1.83.0 REQUIREDCOMPONENTS json)# The switch -Weffc++ (using Effective C++) generates tremendous# warnings, including source you have no control over, such as in Boost.# Makes you wonder.#### add_compile_options(-Wall -Wextra -Wpedantic -Weffc++)add_compile_options(-Wall -Wextra -Wpedantic -std=c++20)include_directories(/home/mint/Develop/boost/)link_directories(/home/mint/Develop/boost/stage/lib)add_executable (devices main.cpp)### set_property(TARGET devices PROPERTY CXX_STANDARD 20)target_link_libraries(devices ${Boost_LIBRARIES} boost_program_options)

I use cmake to keep everything straight. For just 18 lines for cmake I get a 181 GNU makefile, which is correct. I could probably do better with my cmake file, and I probably will in the future. Today I’m happy everything builds and works as planned.

Links

additions to listing usb-connected development boards

python vs c++ vs rust — a personal adventure

Note that I did not install support for Android (Android SDK and Android Studio) nor the Web (via Chrome). All I’m doing is writing Flutter desktop applications.

The Flutter plugin within VSCode automates quite a lot. Using the VSCode Command Palette, you can create an initial Flutter application (Flutter: New Project) that will create skeleton code and build files that compiles and runs the graphical equivalent of hello, world.

As you can see, the Flutter tools for both the framework and the VSCode plugin provide an effortless on-ramp to developing applications. Creating the start of an application is ridiculously easy, and it’s capable of building and executing from the get-go. One feature this has and I use quite a lot is hot reload. When I make changes to the code, when I save the changes they’re rebuilt and the running application restarted immediately showing those changes. That’s why I use this setup.

And here is the initial application’s extremely simple desktop window. When this works well, it’s a little like magic.

Unfortunately I ran into a problem where Clang, one of Flutter’s dependencies, could not link a C++ application that is part of the Flutter application. The error message reported by the Flutter tooling was (with much superfluous reporting snipped away):

.../usr/bin/ld: cannot find -lstdc++: No such file or directory...

This really annoyed me because I have this same setup on my other Linux Mint system, the UM250. With a bit of googling I discovered what was wrong, and how I’d accidentally side-stepped the issue on the UM250. I solved the problem on the Series 7 with sudo apt install libstdc++-12-dev. After that the Flutter tool chain just worked.

Why didn’t I see this issue on the UM250? Because I’d installed gcc/g++ version 12 to support another small project I was working on. On the UM250 I have both versions 11 and 12. Then when I later updated an older installation of Flutter and ran the same simple tests, everything worked there as well. I don’t intend to install gcc/g++ version 12 on the Series 7 as there is no need.

What I don’t understand is how Clang and GCC got out of sync on Linux Mint 22.1. The version of Clang installed is version 14, and requires libstdc++-12. Unfortunately Linux Mint 22.1 installs GCC version 11, which means that if you wanted to build with Clang you’re out of luck unless you know of Clang 14’s dependencies.

I managed to order three of the ESP32-C6-DevKitC-1 developer boards from the Espressif store on Amazon. They’ve been advertised since mid-2022, and where-ever I went to try to buy them, they were advertised as out of stock. Finally I stumbled onto them on Amazon, and grabbed three examples. They arrived at my home earlier this week. They had to wait until today before I could sit down and begin to work with them.

These are not supported in any existing release of Espressif’s ESP-IDF. I had to clone from the master, or trunk, of their GitHub repo. This is what I used to perform that clone:

git clone --recursive https://github.com/espressif/esp-idf.git esp-idf-main

Once cloned I installed the ESP-IDF software and enabled the development environment I then moved into my wifi-scan folder that I’d migrated to ESP-IDF V5, and ran the following:

idf.py --preview set-target esp32c6

You have to run --preview for everything to be properly set up for the build. After that you can run a regular idf.py build, followed by the flash and monitor commands. You will need to do this before the boards will work. Out-of-the-box the boards, when powered up via USB-C, will immediately go into an endless boot loop because they’re flashed with the wrong code.

Here’s a small portion of what I saw on successful boot after a successful flash.

...ESP-ROM:esp32c6-20220919Build:Sep 19 2022rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)SPIWP:0xeemode:DIO, clock div:2load:0x4086c510,len:0xd3cload:0x4086e610,len:0x2a64load:0x40875720,len:0x1790entry 0x4086c510I (23) boot: ESP-IDF v5.1-dev-3462-g045163a2ec 2nd stage bootloaderI (24) boot: compile time Feb 17 2023 13:48:04I (24) boot: chip revision: v0.0I (28) boot.esp32c6: SPI Speed  : 40MHzI (33) boot.esp32c6: SPI Mode   : DIOI (38) boot.esp32c6: SPI Flash Size : 2MBI (42) boot: Enabling RNG early entropy source...W (48) bootloader_random: bootloader_random_enable() has not been implemented yet...

Here’s some interpretation of those output lines. Line 2 shows that it’s an ESP32-C6 due to the boot ROM on the chip. Line 11 says it was built with ESP-IDF v5.1-dev. I cloned that specific tag, but it won’t support ESP32-C6. You have to use master. I found that a little concerning as the README says that ESP-IDF v5.1, when it’s formally released, is supposed to support the ESP32-C6 device. Oh well.

Everything else seems to be behaving normally, especially support for C++20 in the tool chain. This chip is also supposed to support Zigbee, but I haven’t researched that bit yet. Another goody to check out later.

I’ve reached a point where the central part of the USB Device Monitor, which contains the device table at top and the console at the bottom, has evolved into its own class. It started out as a class function of the primary class, MainWindow. But it quickly became evident I needed to better organize specific functionality into its own compilation unit. The best way to do that was with a unique class that encapsulated the data and structures unique to both the table and console. That’s the reason I like C++ and other object-oriented languages; encapsulation. Not inheritance, encapsulation.

Here is the unique header for CentralWidget:

#pragma once#include <QSplitter>#include <QWidget>#include <QString>#include <QStringList>#include <QHeaderView>#include <QTableWidget>#include <QTableWidgetItem>#include <QTextEdit>#include <QPlainTextEdit>#include <QSettings>#include <QPalette>#include <QSerialPort>#include <QByteArray>#include <QScrollBar>#include <QFont>class CentralWidget : public QSplitter {public:const QString CWIDGET_STATE{"cwidget/state"};explicit CentralWidget(QWidget *parent);void saveCWState(QSettings &settings);void restoreCWState(QSettings &settings);private:QTextEdit *console;QSerialPort *serialPort;QTableWidget *table;void readSerialData();void openUSBPort(QTableWidgetItem *);void closeUSBPort();};

And the implementation, with some beginning comments:

#include "CentralWidget.hpp"#include "devices.hpp"CentralWidget::CentralWidget(QWidget *parent) :QSplitter(parent),console(new QTextEdit),serialPort(new QSerialPort),table(new QTableWidget) {setOrientation(Qt::Vertical);// Configure the console onto which we will communicate with a given// USB device.//QPalette colors = palette();colors.setColor(QPalette::Base, Qt::black);colors.setColor(QPalette::Text, Qt::white);QFont font("Monospace", 12);font.setStyleStrategy(QFont::PreferAntialias);console->setPalette(colors);console->setFont(font);console->setCursorWidth(8);// Connect the console with the serial port so that any data that is// recieved by the serial port will be displayed on the console.//connect(serialPort, &QSerialPort::readyRead, this,&CentralWidget::readSerialData);// Find all the USB devices connected to our system. Put their data in a// spreadsheet-like table.//auto devices = getDevices();table->setRowCount(devices.size());QStringList headers = {"Device", "Adapter Name", "Hex Identification"};table->setColumnCount(headers.size());table->setHorizontalHeaderLabels(headers);table->horizontalHeader()->setStretchLastSection(true);int row{0};// Populate the table with eacj USB device, one row/devices.//for (auto [device, values] : devices) {QString dev(device.c_str());QString v1(values[0].c_str());QString v2(values[1].c_str());table->setItem(row, 0, new QTableWidgetItem(dev));table->setItem(row, 1, new QTableWidgetItem(v1));table->setItem(row, 2, new QTableWidgetItem(v2));++row;}table->resizeColumnsToContents();// Connect the table with the class function so that when a user// clicks on a port in the table, that port will be opened and// communicating with the console.//connect(table, &QTableWidget::itemClicked, this,&CentralWidget::openUSBPort);// Finish construction by putting the table at the top and the// console beneath it.//addWidget(table);addWidget(console);}void CentralWidget::readSerialData() {QByteArray data = serialPort->readAll();console->insertPlainText(data);QScrollBar *scrollBar = console->verticalScrollBar();scrollBar->setValue(scrollBar->maximum());}void CentralWidget::openUSBPort(QTableWidgetItem *cell) {if (cell->column() == 0) {closeUSBPort();QString portName = "/dev/" + cell->text();console->insertPlainText(QString("\nPort open: %1\n").arg(portName));QScrollBar *scrollBar = console->verticalScrollBar();scrollBar->setValue(scrollBar->maximum());serialPort->setPortName(portName);serialPort->setBaudRate(QSerialPort::Baud115200);serialPort->setDataBits(QSerialPort::Data8);serialPort->setParity(QSerialPort::NoParity);serialPort->open(QIODevice::ReadOnly);serialPort->clear(QSerialPort::AllDirections);}}void CentralWidget::closeUSBPort() {if (serialPort->isOpen()) {serialPort->clear();serialPort->close();}}void CentralWidget::saveCWState(QSettings &settings) {settings.setValue(CWIDGET_STATE, saveState());}void CentralWidget::restoreCWState(QSettings &settings) {if(settings.contains(CWIDGET_STATE)) {restoreState(settings.value(CWIDGET_STATE).toByteArray());}}

The key features of the new class are:

• The table is built based on the number of entries found by getDevices(), and the data now fits into the table accordingly. The code was reorganized so that the devices are found before the table is filled with the resultant data.
• The table cell click event is now connected to openUSBPort(), so that clicking on a cell that contains a port will close any port that may be open, then open the new port clicked on.
• The class handles saving and restoring its own internal state, instead of having MainWindow do it, via saveCWState() and restoreCWState().

Now when you start the application, it just sits there not displaying any information. When you click on a port, such as ttyACM0, then that port is opened and any data being sent from the device connected to that port is displayed on the console. The screen capture leading this post shows how this works now. The ttyACM0 was clicked, and then the port was opened.

The biggest problem right now is that if an text is displayed on the console that contains ANSI escape sequences, they are not interpreted and instead are just printed as raw data. I’m going to have to either find a Qt class that can interpret those sequences and display the text accordingly (usually colors), or else write my own. What I may wind up doing is stripping that ANSI data before writing to a log file.

Another chunk of future work is to put functional controls on the dock panel. I have some ideas what I want.

Once I get those tasks out of the way, I will probably release this as an open tool for others to use. As I’ve written before, more to come…

Links

building a qt 6.2 application — beginning usb comms

My Qt 6 application isn’t simple any more. It’s grown more complicated as I continue to add features and evolve it towards what I finally want. In this post I’m adding very basic serial communications via USB to my attached development boards. For this post I’m only communicating with the Raspberry Pi Pico W. I’ll be using QSerialPort to develop that functionality. I’ll show what additional source and libraries I had to install, the core modified code, and the application I had running on the Pico W to produce test output.

First things first. I had to install the headers and library files to support developing with Qt’s QSerialPort. Those extra packages are libqt6serialport6 and libqt6serialport6-dev. This will put the necessary header files down /usr/include/x86_64-linux-gnu/qt6/QtSerialPort/. You’ll also need to link against Qt6SerialPort.

Here’s the modified code module. I’ll highlight the pertinent sections after the code block.

#include <QStringList>#include <QHeaderView>#include <QTableWidgetItem>#include <QSplitter>#include <QPalette>#include <QSerialPort>#include <QByteArray>#include <QScrollBar>#include <QFont>#include "MainWindow.hpp"#include "devices.hpp"void MainWindow::addCentralWidget() {console = new QPlainTextEdit(this);QPalette colors = palette();colors.setColor(QPalette::Base, Qt::black);colors.setColor(QPalette::Text, Qt::white);QFont font("Monospace", 12);font.setStyleStrategy(QFont::PreferAntialias);console->setPalette(colors);console->setFont(font);serialPort = new QSerialPort(this);connect(serialPort, &QSerialPort::readyRead, this, &MainWindow::readSerialData);serialPort->setPortName("/dev/ttyACM0");serialPort->setBaudRate(QSerialPort::Baud115200);serialPort->setDataBits(QSerialPort::Data8);serialPort->setParity(QSerialPort::NoParity);serialPort->open(QIODevice::ReadOnly);table = new QTableWidget(this);table->setColumnCount(3);table->setRowCount(6);QStringList headers = {"Device", "Adapter Name", "Hex Identification"};table->setHorizontalHeaderLabels(headers);table->horizontalHeader()->setStretchLastSection(true);int row{0};for (auto [device, values] : getDevices()) {QString dev(device.c_str());QString v1(values[0].c_str());QString v2(values[1].c_str());table->setItem(row, 0, new QTableWidgetItem(dev));table->setItem(row, 1, new QTableWidgetItem(v1));table->setItem(row, 2, new QTableWidgetItem(v2));++row;}table->resizeColumnsToContents();QSplitter *splitter = new QSplitter(Qt::Vertical, this);splitter->addWidget(table);splitter->addWidget(console);setCentralWidget(splitter);}void MainWindow::readSerialData() {QByteArray data = serialPort->readAll();console->insertPlainText(data);QScrollBar *scrollBar = console->verticalScrollBar();scrollBar->setValue(scrollBar->maximum());}

The text box is being used again as a console. Starting at line 15, we set up the foreground and background colors and the font we want to see any text in. This gives us a console with white text on a black background, with a 12pt Monospaced font using anti-aliasing. The default font is horrible.

Lines 24 through 27 are critical to this working. While Qt calls the ability to handle callbacks as signals (the call) and slots (the callback), this is their nomenclature for a feature as old as C, which is the ability to call pointers to functions. This was a feature in the Unix widget toolkit, Motif, in which controls had the ability to be programatically configured with pointers to functions that were called when a given control was activated. Line 26 sets up the callback (MainWindow::readSerialData) to be called by the serialPort when any data is present. The callback reads that data and places it to be displayed on the console. This is all done asynchronously such that the GUI event queue isn’t blocked. This is very similar to what Java tries to do with its GUI toolkits. The callback is in lines 61-66. When the application is running, it will open the port to the Pico W and then begin to read the data that the Pico W sends out.

These are early days for this. I only wrote enough to get the callback going and to read from the port. You’ll note that the port is hard coded for the Pico W. In the future I’ll be able to pick any port just by clicking on the entry in the upper table and the console will be fully bi-directional. I’m also going to add the ability to log everything in the console. As a stretch goal I’d like to be able to edit Micropython source in the attached microcontroller as well. But that’s a stretch goal which is going to require almost as much work as all the work leading up to finishing the first three goals.

Now for the interesting part. The Pico W is running Circuit Python 8 beta 6.

# SPDX-FileCopyrightText: 2022 Liz Clark for Adafruit Industries## SPDX-License-Identifier: MITimport osimport timeimport sslimport wifiimport socketpoolimport microcontrollerimport adafruit_requestsimport jsonimport gc# Adafruit quotes URL#quotes_url = "https://www.adafruit.com/api/quotes.php"# Connect to your local hostspot.# WIFI_SSID and WIFI_PASSWORD should be defined in your settings.toml# file.#wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'))pool = socketpool.SocketPool(wifi.radio)session = adafruit_requests.Session(pool, ssl.create_default_context())print("Fetching quotes from ", quotes_url)print()while True:try:time.sleep(5)gc.collect()# Get a quote.#response = session.get(quotes_url)dic = dict(json.loads(response.text)[0])qstr = dic['text'] + "."print("   " + qstr)print(".. " + dic['author'])print("")response.close()gc.collect()# pylint: disable=broad-exceptexcept ValueError as e:print("Error: bad JSON")response.close()except Exception as e:print("Error:\n", str(e))print("Resetting microcontroller in 10 seconds")time.sleep(10)microcontroller.reset()

The original code came from Adafruit. I cleaned up the while loop to make the output more readable. The original code just spat out the JSON strings that came back from the URL. I’ve done some formatting and cleanup instead.

That’s it for the time being. I’ll continue to clean up the C++ code. Hopefully I’ll have more progress to post by the weekend.

My Qt 6.2 application continues to evolve. I’ve now named it USB Device Monitor, or udm. It has a dock that can be switched to either left or right side and a table in the center with specific Linux information replacing the text editing widget I had before. All of this is building on from the basic plumbing entry and my earlier utility to read Linux USB devices (see links below).

Comments will be shorter than usual, with code more than commentary.

cmake_minimum_required (VERSION 3.25.0)## Build requirements#include_directories( /usr/include/x86_64-linux-gnu/qt6/ /usr/include/x86_64-linux-gnu/qt6/QtCore/ /usr/include/x86_64-linux-gnu/qt6/QtGui/ /usr/include/x86_64-linux-gnu/qt6/QtWidgets/ )link_libraries( Qt6Core Qt6Widgets Qt6Gui )add_compile_options( -fPIC -std=c++20 )## Project specific#project (usb-device-monitor)add_executable (udmmain.cppMainWindow.cppAddCentralWidget.cppAddWindowFileMenu.cppAddDockingPanel.cppdevices.cpp)

The last third of the CMake file shows that MainWindow is a rename of AppWindow, and three new files have been added, AddCentralWidget, AddDockingPanel and devices.

#pragma once#include <QApplication>#include <QMainWindow>#include <QMenuBar>#include <QPlainTextEdit>#include <QSettings>#include <QString>#include <QDockWidget>#include <QCheckBox>#include <QComboBox>#include <QTableWidget>class AppWindow : public QMainWindow {public:const QString SCOPE_NAME{"Qt6BasedSoftware"};const QString APP_NAME{"USBDeviceMonitor"};const QString APP_GEOMETRY{"app/geometry"};const QString APP_STATE{"app/state"};const int INITIAL_WIDTH{900};const int INITIAL_HEIGHT{600};AppWindow();private:QTableWidget *table;QPlainTextEdit *textEdit;void addCentralWidget();void createFileMenu();QMenu *fileMenu;QAction *newAction;QAction *openAction;QAction *saveAction;QAction *saveAsAction;QAction *aboutAction;QAction *exitAction;void addDockingPanel();QDockWidget *dock;QCheckBox*cb1,*cb1x,*cb2,*cb2x,*cb3,*cb3x,*cb4,*cb4x;QComboBox *comboBox;void closeEvent(QCloseEvent*);};

#include <QScreen>#include <QSize>#include <QStatusBar>#include "MainWindow.hpp"AppWindow::AppWindow() {QCoreApplication::setOrganizationName(SCOPE_NAME);QCoreApplication::setApplicationName(APP_NAME);setWindowTitle(APP_NAME);addCentralWidget();createFileMenu();addDockingPanel();statusBar()->showMessage(tr("Statusbar Placeholder"));QSettings settings(QCoreApplication::organizationName(),QCoreApplication::applicationName());if (settings.contains(APP_GEOMETRY) and settings.contains(APP_STATE)) {restoreGeometry(settings.value(APP_GEOMETRY).toByteArray());restoreState(settings.value(APP_STATE).toByteArray());cb1->setChecked(settings.value("dock/cb1").toBool());cb2->setChecked(settings.value("dock/cb2").toBool());cb3->setChecked(settings.value("dock/cb3").toBool());cb4->setChecked(settings.value("dock/cb4").toBool());comboBox->setCurrentIndex(settings.value("dock/combo1").toInt());return;}QScreen *primaryScreen = QGuiApplication::primaryScreen();QSize screenSize = primaryScreen->availableSize();auto screenWidth  = screenSize.width();auto screenHeight = screenSize.height();auto upperLeftX  = (screenWidth - INITIAL_WIDTH)/2;auto upperLeftY = (screenHeight - INITIAL_HEIGHT)/2;setGeometry(upperLeftX, upperLeftY, INITIAL_WIDTH, INITIAL_HEIGHT);}void AppWindow::closeEvent(QCloseEvent *event) {QSettings settings(QCoreApplication::organizationName(),QCoreApplication::applicationName());settings.setValue(APP_GEOMETRY, saveGeometry());settings.setValue(APP_STATE, saveState());settings.setValue("dock/cb1",cb1->isChecked());settings.setValue("dock/cb2",cb2->isChecked());settings.setValue("dock/cb3",cb3->isChecked());settings.setValue("dock/cb4",cb4->isChecked());settings.setValue("dock/combo1", comboBox->currentIndex());QMainWindow::closeEvent(event);}

The biggest changes in MainWindow besides the renaming is in the constructor and closeEvent. The constructor now calls external class functions to build the subsystems used in the application, and then recalls saved state in order to put the application back the way it was when it was closed the time before. I made a change to the conditional if block by removing the else and adding a return at the end of the if block. I’ve developed an aversion to cascading conditional statements and the “pyramid of doom” that they can create. I now organize my tests such that if I need to check for the health of the application then I’ll check that first and bail if there’s a problem, or as in this case, put the most common action that should occur nearly all the time, perform that task and then leave. The use of if/else blocks might look proper to purists, but I’m no longer a purist.

The closeEvent now does a bit of state saving. I didn’t mention it in the last post, but for Linux the save file is located in $HOME/.config/Qt6BasedSoftware/ USBDeviceMonitor.conf. That folder and file are defined in SCOPE_NAME and APP_NAME, respectively, in MainWindow.hpp, lines 16 and 17.

#include <QStringList>#include <QHeaderView>#include <QTableWidgetItem>#include "MainWindow.hpp"#include "devices.hpp"void AppWindow::addCentralWidget() {table = new QTableWidget(this);table->setColumnCount(3);table->setRowCount(6);QStringList headers = {"Device", "Adapter Name", "Hex Identification"};table->setHorizontalHeaderLabels(headers);table->horizontalHeader()->setStretchLastSection(true);int row{0};for (auto [device, values] : getDevices()) {QString dev(device.c_str());QString v1(values[0].c_str());QString v2(values[1].c_str());table->setItem(row, 0, new QTableWidgetItem(dev));table->setItem(row, 1, new QTableWidgetItem(v1));table->setItem(row, 2, new QTableWidgetItem(v2));++row;}table->resizeColumnsToContents();setCentralWidget(table);}

The class function addCentralWidget creates a three column by six row table, looking pretty much like a spreadsheet, to contain the data that the class function getDevices finds in the Linux operating system. The first half of this function sets up the table while the second half formats the data returned by the call to getDevices. This is the minimum code necessary to just display the data. No other functionality is enabled.

#pragma once#include <array>#include <map>#include <string>#include <iostream>typedef std::map<std::string, std::array<std::string,2>> device_collection;device_collection getDevices();

#include <algorithm>#include <filesystem>#include <regex>#include "devices.hpp"using std::array;using std::string;using std::pair;using std::regex;using std::regex_search;using std::replace;using std::smatch;using std::filesystem::directory_iterator;using std::filesystem::path;using std::filesystem::read_symlink;void split(const string &subj, const regex &rgx, array<string, 2> &vars) {smatch match;regex_search(subj, match, rgx);vars[0] = subj.substr(0, match.position(0));replace(vars[0].begin(), vars[0].end(), '_', ' ');vars[1] = match.str(0).erase(0,1);vars[1].erase(vars[1].end()-1);}device_collection getDevices() {const string DEVICE_PATH{"/dev/serial/by-id/"};const regex hregex{"_[0-9A-Fa-f]+-"};device_collection devices;for (const auto &entry : directory_iterator(DEVICE_PATH)) {if (entry.is_symlink()) {string slink = read_symlink(entry).stem().generic_string();string sname = entry.path().stem().generic_string().erase(0,4);array<string, 2> results;split(sname, hregex, results);devices[slink] = results;}}return devices;}

This is the utility I wrote about a week back where I was trying to write the same utility in Python, C++, and Rust. The original C++ utility was stand-alone. I removed the main and the printout, and just converted it into a function that returned a collection of the processed data to be displayed in the QTableWidget.

#include <QGroupBox>#include <QVBoxLayout>#include <QDockWidget>#include <QGridLayout>#include "MainWindow.hpp"void AppWindow::addDockingPanel() {dock = new QDockWidget(tr("Docking Panel"), this, Qt::Widget);dock->setObjectName(tr("Docking_Panel"));dock->setFeatures(QDockWidget::DockWidgetMovable);dock->setAllowedAreas(Qt::LeftDockWidgetArea | Qt::RightDockWidgetArea);addDockWidget(Qt::LeftDockWidgetArea, dock);QWidget *mainWidget = new QWidget;QGridLayout *grid = new QGridLayout;mainWidget->setLayout(grid);QGroupBox *checkboxSelection = new QGroupBox(tr("Non-exclusive Checkboxes"));QVBoxLayout *cslayout = new QVBoxLayout;cb1 = new QCheckBox(tr("One"), this);cslayout->addWidget(cb1, 0, Qt::AlignLeft);cb2 = new QCheckBox(tr("Two"), this);cslayout->addWidget(cb2, 1, Qt::AlignLeft);cb3 = new QCheckBox(tr("Three"), this);cslayout->addWidget(cb3, 2, Qt::AlignLeft);cb4 = new QCheckBox(tr("Four"), this);cslayout->addWidget(cb4, 3, Qt::AlignLeft);checkboxSelection->setLayout(cslayout);grid->addWidget(checkboxSelection, 0, 0);QGroupBox *groupBox = new QGroupBox(tr("Dropdown"), dock);QVBoxLayout *layout = new QVBoxLayout();comboBox = new QComboBox();comboBox->addItem(tr("One"));comboBox->addItem(tr("Two"));comboBox->addItem(tr("Three"));comboBox->addItem(tr("Four"));layout->addWidget(comboBox);groupBox->setLayout(layout);grid->addWidget(groupBox, 1, 0);QWidget *filler = new QWidget;QVBoxLayout *flayout = new QVBoxLayout;flayout->addSpacerItem(new QSpacerItem(20, 800, QSizePolicy::Minimum,QSizePolicy::Expanding));filler->setLayout(flayout);grid->addWidget(filler);dock->setWidget(mainWidget);}

This creates a dockable panel that can be moved either to the right or the left of the main application. It does nothing but display a few controls. QDockWidget has changed a bit since Qt 4. In particular, I had to create a single widget to set within the QDockWidget instance; that’s what mainWidget is for. I also had to create a special widget to encapsulate the QSpacerItem in order for all the other widgets to line up properly and not waste space on the dock.

The class function createFileMenu hasn’t changed.

Not much to say at this point except that now I need to decide what real functionality I want to enable in this monitor, and then add controls that actually do something. A bit more state is going to have to be preserved. The utility is going to have to monitor the USB devices to indicate when devices are added or removed. And I’m going to want to add a column to each row to hold additional user added information (text). So a bit more work to do.

Links

building a simple qt 6 application — basic plumbing

python vs c++ vs rust — a personal adventure

The first change has been to the CMake file, CMakeLists.txt.

cmake_minimum_required (VERSION 3.25.0)## Build requirements#include_directories( /usr/include/x86_64-linux-gnu/qt6/ /usr/include/x86_64-linux-gnu/qt6/QtCore/ /usr/include/x86_64-linux-gnu/qt6/QtGui/ /usr/include/x86_64-linux-gnu/qt6/QtWidgets/ )link_libraries( Qt6Core Qt6Widgets Qt6Gui )add_compile_options( -fPIC -std=c++20 )## Project specific#project (control-app)add_executable (control-appmain.cppAppWindow.cppAppWindowFileMenu.cpp)

The first change is the addition of two more include directories on lines 6 and 7. The add_executable CMake call now has three source files instead of the original one.

The next big change was to break up the original single source file into three. The original source file was renamed main.cpp, and the source that defined the AppWindow functionality was put into two files, one for the main part of AppWindow and the other for just creating the File menu structure. Let’s go in the file order of CMakeLists.txt.

The first file is main.cpp.

#include "AppWindow.hpp"int main(int argc, char **argv) {QApplication app (argc, argv);AppWindow window;window.show();return app.exec();}

This file is what is left of the original simple-app.cpp, stripped of everything not needed and renamed. The only functionality required of main is to instantiate AppWindow and then kick everything off with window.show().

#pragma once#include <QApplication>#include <QMainWindow>#include <QMenuBar>#include <QPlainTextEdit>#include <QSettings>#include <QString>class AppWindow : public QMainWindow {public:const QString SCOPE_NAME{"Qt6BasedSoftware"};const QString APP_NAME{"ControlApp"};const QString APPWIN_GEOMETRY{"AppWindow/geometry"};const QString APPWIN_STATE{"AppWindow/state"};const int INITIAL_WIDTH{800};const int INITIAL_HEIGHT{600};AppWindow();private:QSettings *settings;QPlainTextEdit *textEdit;void createFileMenu();QMenu *fileMenu;QAction *newAction;QAction *openAction;QAction *saveAction;QAction *saveAsAction;QAction *aboutAction;QAction *exitAction;void closeEvent(QCloseEvent*);};

If you compare this header file with the AppWindow class definition in simple-app.cpp, you’ll notice that the header file alone is longer than the original simple-app.cpp. At this point there aren’t all that many class functions; there’s the public constructor and then the private function createFileMenu which can only be called internally to the class, specifically from the constructor. The majority of the header file so far is primarily data structures, although I expect that will change as this application continues to evolve.

#include <QScreen>#include <QSize>#include <QStatusBar>#include <iostream>#include "AppWindow.hpp"AppWindow::AppWindow() {QCoreApplication::setOrganizationName(SCOPE_NAME);QCoreApplication::setApplicationName(APP_NAME);settings = new QSettings(SCOPE_NAME, APP_NAME);textEdit = new QPlainTextEdit;setCentralWidget(textEdit);createFileMenu();setWindowTitle(APP_NAME);statusBar()->showMessage(tr("Statusbar Placeholder"));if (settings->contains(APPWIN_GEOMETRY) and settings->contains(APPWIN_STATE)) {std::cout << APP_NAME.toStdString() << ": found full state\n";restoreGeometry(settings->value(APPWIN_GEOMETRY).toByteArray());restoreState(settings->value(APPWIN_STATE).toByteArray());}else {std::cout << APP_NAME.toStdString() << ": did not find full state\n";QScreen *primaryScreen = QGuiApplication::primaryScreen();QSize screenSize = primaryScreen->availableSize();auto screenWidth  = screenSize.width();auto screenHeight = screenSize.height();auto upperLeftX  = (screenWidth - INITIAL_WIDTH)/2;auto upperLeftY = (screenHeight - INITIAL_HEIGHT)/2;setGeometry(upperLeftX, upperLeftY, INITIAL_WIDTH, INITIAL_HEIGHT);}}void AppWindow::closeEvent(QCloseEvent *event) {std::cout << APP_NAME.toStdString() << ": close event\n";settings->setValue(APPWIN_GEOMETRY, saveGeometry());settings->setValue(APPWIN_STATE, saveState());QMainWindow::closeEvent(event);}

This is the meat of the application so far. There are two key sections, the constructor and the definition of function closeEvent.

The constructor sets up the overall environment for QSettings so that the application can save, and then recall, its state when it starts up and shuts down respectively. I followed the directions on the QSettings documentation page ( https://doc.qt.io/qt-6.2/qsettings.html ). Inside the constructor, starting on line 19, I check to see if the application geometry and state are available from a prior run. If they are not, then I default to a width and a height and a location that places the application squarely in the middle of the screen. If the geometry and state are loaded, however, the application goes back to the state that was last saved. Note that, on 13, that a QPlainTextEdit widget is set as the center widget for AppWindow. This is because Qt states that simply showing an empty QMainWindow isn’t supported (well, they won’t guarantee it works), so I added this widget just to fill the slot. In fact lines 13 through 16 are basically enabling the application (center widget, menu, window title and status line).

In order to make sure that the geometry and state or saved, I’ve redefined QMainWindow::closeEvent() such that everything is saved. After saving the call to QMainWindow::closeEvent() is chained to make sure that the Qt application closes properly.

#include "AppWindow.hpp"void AppWindow::createFileMenu() {fileMenu = menuBar()->addMenu(tr("&File"));newAction = new QAction(tr("&New"), this);fileMenu->addAction(newAction);openAction = new QAction(tr("&Open"), this);fileMenu->addAction(openAction);saveAction = new QAction(tr("&Save"), this);fileMenu->addAction(saveAction);saveAsAction = new QAction(tr("Save as"), this);fileMenu->addAction(saveAsAction);fileMenu->addSeparator();aboutAction = new QAction(tr("&About Qt"), this);connect(aboutAction, &QAction::triggered, qApp, &QApplication::aboutQt);fileMenu->addAction(aboutAction);fileMenu->addSeparator();exitAction = new QAction(tr("E&xit"), this);exitAction->setShortcuts(QKeySequence::Quit);connect(exitAction, &QAction::triggered, this, &QWidget::close);fileMenu->addAction(exitAction);}

The class function that builds the File menu is in its own compilation unit. I did this to keep any one source file from growing too large. I believe in fine granularity. As more menus are needed, and defined, they’ll go into their own files. Additional functionality will also go into their own unique files.

More to come…

Links

building a simple qt 6 application with cmake and make

As I wrote earlier I’ve switched back to Visual Studio Code on my Linux Mint 21.1 system for C++ development. In order to support Qt 6 C++ development, I needed to modify the C++ plugin’s include path to point to specific Qt 6 include files. The easiest way to do this is (and this assumes you have the Microsoft C/C++ extension installed to begin with):

1. Select Extensions in Visual Studio Code on the far left.
2. On the C/C++ Extension left-click the gear.
3. On the menu that appears, select Extension Settings.
4. In the Settings screen that opens, scroll down to C_Cpp > Default: Include Path.
5. Click the blue Add Item button.
6. In the input text box that appears, type in the full path to the Qt include files. Note in my example I have two paths to the Qt 6 include files.

Once you’ve updated the include paths just close the Settings tab. If you don’t have those additional paths in the Microsoft C/C++ plugin, then any and all Qt include files will have the dreaded Red Squiggly Lines underlining all those include files it can’t find.

I’ve been working on C++ applications, especially those that use C++17 and C++20 standards features. My approach is to start small and simple and then build up from there. For this example I decided to bring my ancient Qt skills up to date with Qt 6. The last time I wrote any serious Qt applications in native C++ was back around 2006/2007, and that was using Qt 4.

I also wanted to incorporate other tools I’ve been learning how to use, specifically CMake. CMake ( https://cmake.org ) is a powerful build tool that can help create a cross-platform configuration and build tool. I’ve been using with it with Espressif’s ESP-IDF framework. Because of this, CMake’s use has started to “leak” over into other C++ applications I’ve been working on. Then very recently I decided to pick up developing Qt applications for Linux, and I decided to use CMake to help build them from the get-go.

This application, which I’ve called simple-app, is a bare-bones skeleton that will create a bare window on the desktop and a single drop-down menu with just two entries. You can see a screen capture of it leading this post.

In order to develop with Qt 6, I installed Linux Mint’s Qt 6 package with sudo apt install qt6-base-dev. This installs Qt 6.2.4, the long-term support or LTS version that the KDE desktop uses. I could have gone with a more recent release, such as 6.4, but that would have caused too much friction trying to distribute it to others to use. Going with the version in the repos makes building and distributing it so much easier. And I don’t have to go hunt down all the odd build dependencies that Qt needs to build.

Once you install Qt, you need to install cmake if you don’t have it on your system already. You can either install it via apt, or you can do like I did and download the latest pre-built binary from the CMake website and use it locally in your home directory.

With cmake installed, it’s time to create a build folder and put the necessary files into it. The first file we’ll look at is CMakeLists.txt.

cmake_minimum_required (VERSION 3.25.0)## Build requirements#include_directories( /usr/include/x86_64-linux-gnu/qt6/ /usr/include/x86_64-linux-gnu/qt6/QtWidgets/ )link_libraries( Qt6Core Qt6Widgets Qt6Gui )add_compile_options( -fPIC -std=c++20 )## Project specific#project (simple-app)add_executable (simple-app simple-app.cpp)

These are the bare CMake essentials necessary to create a make file that will build the application.

• Line 1 is the cmake minimum version this will work with. If you have an older version of cmake then change this to match it. It should work with older versions, but I can’t guarantee that.
• Lines 5 and 6 set up the include paths necessary to find the Qt C++ include files. These are the paths created from the installation of qt6-base-dev.
• Line 7 lists the specific link libraries necessary for the application.
• Line 8 are specific C++ compiler flags. The -fPIC flag is required by the Qt libraries. The -std=c++20 is required to make sure the code is being compiled to the C++20 standard.
• Line 12 is the project name. It is actually compiled into the title of the window, which is rather nice.
• Line 13 associates the application with the source file.

Again, these are the absolute minimum CMake statements.

Here’s the C++ source file.

#include <QApplication>#include <QMainWindow>#include <QMenuBar>class AppWindow : public QMainWindow {public:AppWindow() {createFileMenu();resize(800, 600);}private:QMenu *fileMenu;QAction *aboutAction;QAction *exitAction;void createFileMenu() {fileMenu = menuBar()->addMenu(tr("&File"));aboutAction = new QAction(tr("&About"), this);connect(aboutAction, &QAction::triggered, qApp, &QApplication::aboutQt);fileMenu->addAction(aboutAction);fileMenu->addSeparator();exitAction = new QAction(tr("E&xit"), this);exitAction->setShortcuts(QKeySequence::Quit);connect(exitAction, &QAction::triggered, this, &QWidget::close);fileMenu->addAction(exitAction);}};int main(int argc, char **argv) {QApplication app (argc, argv);AppWindow window;window.show();return app.exec();}

This application is about as minimal as possible, at least for me. I wanted a bare window and a single simple drop-down menu, and this produces that. Here’s a basic source breakdown.

• main() has as little as possible. It instantiates QApplication and AppWindow. It then shows the window (AppWindow), meaning the window created through QMainWindow, and then it starts the entire application execution. That’s it, and frankly, that’s really all that should be there.
• The public AppWindow is derived from QMainWindow. This appears to be the recommended way to get a window up on a screen. All the other window features are created and then added to QMainWindow.
• The public constructor, AppWindow::AppWindow, does very little itself. It calls a private function to create the file menu, and then sets the size in pixels (width = 800, height = 600).
• The other function of note is of course createFileMenu. It shows, in order, how all the elements in the File menu are created. In the past I would create private functions for each menu on the menu bar. It makes it a lot easier to add, modify, or disable menus.

Here are the instructions to build the application. I’m assuming at this point you’ve got Qt 6 installed along with cmake.

1. Create a folder somewhere in your home directory. For the purposes of this post I’ll give it the same name as the application, or simple-app.
2. Step into the folder and copy CMakeLists.txt and simple-app.cpp into it.
3. Run cmake: cmake -B build. This will create a folder beneath your current location, into which the makefile and all other support files are created.
4. Run make: make -C build. This will build the application, leaving the executable binary in the build folder.
5. Run the application: ./build/simple-app

You don’t have to create the build subdirectory, but if you don’t then cmake and make litter the work area with a lot of individual build byproduct files. I use the build subdirectory method so that if I check this into Git then my .gitignore file is very simple; all I have to do to have git ignore the build subdirectories is put in an entry for build.

Right now the only “real” working function is the About menu selection. It produces the following dialog when selected:

I want to see Qt’s About because it also includes the licensing terms, which I always check.

I wanted to share this because while it’s minimal, it’s still a useful starting point. I hope you find it useful as well.

Embedded development goes back as far as regular computing, at least to when the first mini-computers were released, because mini-computers were small enough and inexpensive enough (relative to mainframes) to be purposed towards specific tasks and then left alone to run unattended. As the computers continued to shrink in size and grow in capability, embedded system design leveraged those advances, including when Ethernet first showed up on systems. Why networked embedded systems suddenly became the Internet of Things makes no sense, considering that networked embedded systems began to appear when DEC (Digital Equipment Corporation) began to ship VAXen with DECnet Phase IV, going back to the early 1980s. While VAXen were rather large, there were microVAXes and smaller PDP-11/44s in particular that wound up in unattended control systems.

The real advances began with VME (1979) and other “big board” systems. As for me, the really interesting and affordable evolution began with PC/104 in the early 1990s. Boards got a lot smaller and more capable, while older designs kept getting cheaper. That evolution continued relentlessly until today, when I can go onto Amazon or Adafruit and purchase single board computers about the size of a pack of chewing gum with embedded 32-bit SoCs for anywhere from US$4 up to around US$25. Today I work with Raspberry Pi Picos as well as a number of Espressif ESP32-S3 and ESP32-C3 developer boards.

What’s key about these little boards is how they communicate with personal computers. They’ve standardized on using USB as the method for communicating with a host computer. All of my boards show up on my systems (macOS, Linux, and Windows) as a terminal device. For example, on Linux one of the devices might show up as /dev/ttyACM0, another as /dev/ttyUSB0, and so forth. I need to know the device ID so I can open up those devices and work with them, whether I’m using something like screen, or programming tools such as Thonny or the Arduino IDE.

Life is simple when you only have one device plugged in. It’s real obvious what the device is. Life gets more complicated when you have more than one. At one time I had eight plugged in and running, and I can assure you that trying to keep them all straight came to be a challenge. To start I wrote a very small bit of shell script to search for any and all connected boards:

alias devices='ll /dev/tty[A,U]*'

When I typed devices at a shell prompt, I’d get the following:

crw-rw---- 1 root dialout 166, 0 Dec 31 17:27 /dev/ttyACM0crw-rw---- 1 root dialout 188, 0 Dec 21 21:00 /dev/ttyUSB0crw-rw---- 1 root dialout 188, 1 Dec 23 11:37 /dev/ttyUSB1crw-rw---- 1 root dialout 188, 2 Dec 21 21:00 /dev/ttyUSB2crw-rw---- 1 root dialout 188, 3 Dec 25 20:06 /dev/ttyUSB3

Looks good, except for one small problem. What is actually attached to each port? I know through experience that the ACM0 port is connected to a Raspberry Pi Pico W. The other four are Espressif ESP32 developer boards. The Pico is running CircuitPython 8 beta 6. The ESP32 developer boards are running diverse software loads, only one of which is Micropython, while the other three are C++ applications I’ve compiled with the ESP-IDF and flashed on the various boards. So I decided to document which boards were on which port in a small spreadsheet. Worked good until one day, when I’d unplugged one of the boards to add some components to it, Duke Energy decided to drop power to the house. When power returned and I plugged the developer board back in, all the Espressive ports had been scrambled. What used to be on USB0 was on USB2, USB1 had dropped down to USB0, etc, etc. So I went looking deeper into the Linux kernel and how it manages serial devices, and learned that all those USB devices were listed down /dev/serial/by-id. So here’s what a full directory listing looks like:

lrwxrwxrwx 1 root root 13 Dec 31 17:27 usb-Raspberry_Pi_Pico_W_E6614C311B633831-if00 -> ../../ttyACM0lrwxrwxrwx 1 root root 13 Dec 21 21:00 usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_7eab1642dbfaeb1198773ca4c6d924ec-if00-port0 -> ../../ttyUSB2lrwxrwxrwx 1 root root 13 Dec 21 21:00 usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_8698c1b85519ec11982039bff95a09b1-if00-port0 -> ../../ttyUSB3lrwxrwxrwx 1 root root 13 Dec 23 11:37 usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_d0ed30374c18ec1199b543103803ea95-if00-port0 -> ../../ttyUSB1lrwxrwxrwx 1 root root 13 Dec 21 21:00 usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_e08d862b0867ec118f12a17089640db2-if00-port0 -> ../../ttyUSB0

Note that those are symlinks to the devices. So now I had an idea for a utility to write.

I’ve spent the last few weeks tinkering with three different languages to create a small utility to list the information out of /dev/serial/by-id, and process the output as strings to list in a cleaner way. I also wanted to get that information to create some simple data structures, especially the long hexadecimal ID which does not change. The terminal devices may change, but the hex IDs remain the same.

Here’s the first utility I wrote in Python. Took me about an hour of tinkering (where I was interrupted by Life a few times; remember I’m retired and this is supposed to be a hobby).

#!/bin/env python3import os, reDEVICE_PATH = '/dev/serial/by-id/'devices = {}for device in os.listdir(DEVICE_PATH):link_name = os.path.basename(os.readlink(DEVICE_PATH + device))re_result = re.search(r'_[0-9A-Fa-f]+-', device)interface_name = device.split(re_result[0])[0].replace('usb-','').replace('_',' ')hex_str = re_result[0].lstrip('_').rstrip('-')item = [interface_name, hex_str]devices[link_name] = itemfor device in sorted(devices.keys()):values = devices[device]print("{}, {}, {}".format(device, values[0], values[1]))

And here’s the output it produces when it’s run on my system.

ttyACM0, Raspberry Pi Pico W, E6614C311B633831ttyUSB0, Silicon Labs CP2102N USB to UART Bridge Controller, e08d862b0867ec118f12a17089640db2ttyUSB1, Silicon Labs CP2102N USB to UART Bridge Controller, d0ed30374c18ec1199b543103803ea95ttyUSB2, Silicon Labs CP2102N USB to UART Bridge Controller, 7eab1642dbfaeb1198773ca4c6d924ecttyUSB3, Silicon Labs CP2102N USB to UART Bridge Controller, 8698c1b85519ec11982039bff95a09b1

You can see how I reordered the data as well as cleaned up the device names by replacing all the underscores with white space. I’m not about to present this little Python utility as perfect idiomatic Python. I’m not interested in being called out over how this or that isn’t truly Pythonic. Python is, for me, a very easy prototyping language, a tool to make tools.

Having created a tool to give me the gist of what I wanted, I could have gone further in Python and added more features. Instead, I decided to implement the same functionality in C++ and Rust. Let it be known that I was going to implement it first in Rust and then C++, but I ran into so may problems trying to research what was Rust’s “simple” way to manipulate strings that I gave up in frustration and turned to C++. Here’s the C++ equivalent.

#include <string>#include <array>#include <map>#include <algorithm>#include <iostream>#include <filesystem>#include <regex>using std::string;using std::replace;using std::cout;using std::filesystem::directory_iterator;using std::filesystem::path;using std::filesystem::read_symlink;using std::regex;using std::regex_search;using std::smatch;using std::map;using std::array;using std::pair;void split(const string &subj, const regex &rgx, array<string, 2> &vars) {  smatch match;  regex_search(subj, match, rgx);  vars[0] = subj.substr(0, match.position(0));  replace(vars[0].begin(), vars[0].end(), '_', ' ');  vars[1] = match.str(0).erase(0,1);  vars[1].erase(vars[1].end()-1);}int main() {  const string DEVICE_PATH{"/dev/serial/by-id/"};  map<string, array<string,2>> devices;  regex hregex{"_[0-9A-Fa-f]+-"};  for (const auto &entry : directory_iterator(DEVICE_PATH)) {if (entry.is_symlink()) {  string slink = read_symlink(entry).stem().generic_string();  string sname = entry.path().stem().generic_string().erase(0,4);  array<string, 2> results;  split(sname, hregex, results);  devices[slink] = results;}  }  for (auto [device, values] : devices) {cout << device << ", " << values[0] << ", " << values[1] << "\n";  }  return 0;}

Lots of C++ code. Considerably many more lines of C++ than Python. If you’re going to get shocked about how much longer it is, don’t. To start with, there’s 11 lines of book keeping code (lines 9 to 20) that I prefer to use instead of a blanket using std that I’ve seen at the beginning of too much C++ source code. So I create a list of explicit usings in order to explicitly list everything individually. I have never cared for C++’s namespace notation, so explicit listing is a compromise for me. When run it produces the same results as the Python version.

One appreciated C++ map container feature is that the entries are stored pre-sorted. When I read them out I don’t have to explicitly pull out the keys, sort them, and then read the values from the map using the sorted keys.

Now we come to the Rust version. Let’s list it out for all to see.

use std::io;use std::fs;use regex::Regex;use std::collections::HashMap;fn main() -> io::Result<()> {const DEVICE_PATH : &str = "/dev/serial/by-id/";let hregex = Regex::new("_[0-9A-Fa-f]+-").unwrap();let trimchars: &[_] = &['_', '-', '.', '/'];let mut devices: HashMap<String, [String; 2]> = HashMap::new();for entry in fs::read_dir(DEVICE_PATH)? {// Split the extraction of file_name from path.// Combine the two, as you might naturally think to do in// another language, and you'll generate an error saying that// entry.path() creates a temporary that is freed while still// in use, meaning down the chain after the call to path().// This is a bug, not a feature.//let path = entry?.path();let device = path.read_link().unwrap();let device = device.to_str().unwrap().trim_matches(trimchars);// To give credit where credit is due:// Question: How do I print an OsStr without the quotes?// https://stackoverflow.com/questions/70266860/how-do-i-print-an-osstr-without-the-quotes// Who would have thought this was the answer to how// to extract the string from the file name?//let filename = path.file_name().unwrap().to_str().unwrap();let filename = filename.trim_start_matches("usb-");let caps = hregex.captures(filename).unwrap();let mut hexval = caps.get(0).unwrap().as_str();hexval = hexval.trim_matches(trimchars);let mat = hregex.find(filename).unwrap();let filename = filename.split_at(mat.start()).0;let filename = filename.replace("_", " ");devices.insert(device.to_owned(),   [filename.to_owned(), hexval.to_owned()]);}let mut sorted: Vec<_> = devices.iter().collect();sorted.sort_by_key(|a| a.0);for (device, values) in sorted.iter() {println!("{}, {}, {}", device, values[0], values[1]);}Ok(())}

The shine has gone off Rust (no pun intended), at least for me. I got caught up in the hype, but this little exercise (and it is a little exercise) has pretty much sand blasted that away. It took me about an hour to write the Python version of this utility, and then a day for the C++ version. It took me nearly a week to learn everything I needed to write it in rust, even though the number of source lines is about half-way between the Python version and the C++ version. Why?

1. Rust is constantly evolving, which isn’t necessarily a bad thing. It gets bad, if not worse, because the documentation and examples aren’t keeping up with the evolution. For example, see the declaration of the HashMap on line 10. The documentation says that all I have to do is declare just the HashMap, and that the compiler will determine the necessary types for the HashMap when you perform an insert into the HashMap. Nope. I had to explicitly declare it with the key and values to get it to compile. This is just one of many little documentation gotchas I kept running into.
2. I don’t know who came up with the design, but having to get the string value for a directory path by using “.unwrap().to_string().unwrap()” is all sorts of wrong. I left a comment right above that section of code, along to where I found a solution for it. Other languages manage to offer a simple convenience function, usually called to_string() or toString(), that will return the string representation of the class or structure. But not in Rust, it would appear.
3. Function chaining is a powerful feature, supported by all languages I know of. Unfortunately it doesn’t work all the time for Rust. The Rust designer(s) seem obsessed with “ownership” above all else, which seems to break the expected feature of function chaining. Start reading my comment from line 13 on down to the code. This fits in with (2) above, it would seem.
4. The odd difference between tuple indexing (.#) vs regular indexing ([#]). I used both, and I still don’t know why I couldn’t use just one indexing form throughout.

I will say that I appreciate some of the string manipulation functions, such as trim_matches(…) … once you get a string out to work with. Sorting a map’s keys and then reading out the (key, value) pairs is a bit odd (line 42).

At this point I think I’ll stick with Python and C++ (using the C++17 and C++20 standards), and let Rust continue to evolve for a while. Right now, as far as I’m concerned, Rust looks to be little more than C++ poorly implemented.

The first is that I’ve moved all my Espressif project work to my GitHub account here: https://github.com/wbeebe/EspressifProjects . The projects are broken into ESP-IDF V4.x projects, and ESP-IDF V5.x projects. The older version is for those who want to see what I’ve done and who are still working with some version of V4.x. I know those tool chains are still being developed because ESP-IDF v4.1.4 was released two days ago, on 12 December. But all of my work going forward will be with version 5.

The second is a test application I wrote using ESP-IDF v5.0 with the ESP32-S3-DevKitC-1-N32R8V, which is a development board with 32 MB of FLASH and 8 MB of PSRAM. I’ve already successfully enabled the PSRAM and it appears, from my very limited testing, to work. However, I tried to enable the entire 32 MB of FLASH and the ESP-IDF framework will only allow up to 16MB of FLASH to be enabled. What follows is the code followed by a sample run. Note that this code is also checked into my GitHub repo.

#include <array>#include <cinttypes>#include <string>#include "freertos/FreeRTOS.h"#include "freertos/task.h"#include "driver/gpio.h"#include "esp_log.h"#include "led_strip.h"#include "sdkconfig.h"#include "esp_chip_info.h"#include "esp_system.h"static const char *TAG = "ESP32-S3-DevKitC-1.1-N32R8";#define stringify( _name ) #_namestatic const char* translate_esp_chip_model(esp_chip_model_t model){switch(model) {case CHIP_ESP32:   return stringify(CHIP_ESP32);case CHIP_ESP32S2: return stringify(CHIP_ESP32S2);case CHIP_ESP32S3: return stringify(CHIP_ESP32S3);case CHIP_ESP32C3: return stringify(CHIP_ESP32C3);case CHIP_ESP32H2: return stringify(CHIP_ESP32H2);case CHIP_ESP32C2: return stringify(CHIP_ESP32C2);//case CHIP_ESP32C6: return stringify(CHIP_ESP32C6);}return "CHIP_UNKNOWN";}using std::string;static void decode_features(uint32_t features, string &results) {if (features == 0) {results = "NO_FEATURES";return;}if (features & CHIP_FEATURE_EMB_FLASH) results += "FLASH ";if (features & CHIP_FEATURE_WIFI_BGN) results += "WIFI ";if (features & CHIP_FEATURE_BLE) results += "BLE ";if (features & CHIP_FEATURE_BT) results += "BT ";if (features & CHIP_FEATURE_IEEE802154) results += "IEEE802154 ";if (features & CHIP_FEATURE_EMB_PSRAM) results += "PSRAM";}// Create an array of color arrays to cycle through continuously.//using std::array;const array<array<int, 3>, 7> colors {{{32,0,0},  // red{0,32,0},  // green{0,0,32},  // blue{0,32,32}, // cyan{32,0,32}, // magenta{32,16,0}, // yellow{0,0,0}// black}};static led_strip_handle_t led_strip;static void initialize_neo_pixel() {led_strip_config_t strip_config = {.strip_gpio_num = CONFIG_BLINK_GPIO,.max_leds = 1, // at least one LED on board};led_strip_rmt_config_t rmt_config = {.resolution_hz = 10000000, // 10 MHz};ESP_ERROR_CHECK(led_strip_new_rmt_device(&strip_config, &rmt_config, &led_strip));// Darken NeoPixel by setting individual LED values to 0//led_strip_clear(led_strip);}static void blink_neo_pixel() {for(auto color : colors) {led_strip_set_pixel(led_strip, 0, color[0], color[1], color[2]);led_strip_refresh(led_strip);vTaskDelay(800 / portTICK_PERIOD_MS);led_strip_clear(led_strip);vTaskDelay(200 / portTICK_PERIOD_MS);}}extern "C" void app_main(void) {esp_chip_info_t chip_info;esp_chip_info(&chip_info);string features;decode_features(chip_info.features, features);ESP_LOGI(TAG, "BEGIN");ESP_LOGI(TAG, "ESP-IDF VERSION %s", esp_get_idf_version());ESP_LOGI(TAG, "CHIP MODEL %s", translate_esp_chip_model(chip_info.model));ESP_LOGI(TAG, "CHIP FEATURES %s", features.c_str());ESP_LOGI(TAG, "REVISION %X", chip_info.revision);ESP_LOGI(TAG, "FREE HEAP %" PRIu32, esp_get_free_heap_size());ESP_LOGI(TAG, "INITIALIZE NEOPIXEL");initialize_neo_pixel();ESP_LOGI(TAG, "ENTER MAIN LOOP");while (true) {blink_neo_pixel();}}

And here is a sample run.

ESP-ROM:esp32s3-20210327Build:Mar 27 2021rst:0x1 (POWERON),boot:0x18 (SPI_FAST_FLASH_BOOT)SPIWP:0xeeOctal Flash Mode EnabledFor OPI Flash, Use Default Flash Boot Modemode:SLOW_RD, clock div:1load:0x3fce3810,len:0x167cload:0x403c9700,len:0xbe8load:0x403cc700,len:0x2e98entry 0x403c9904I (32) boot: ESP-IDF v5.0 2nd stage bootloaderI (32) boot: compile time 08:19:25I (32) boot: chip revision: v0.1I (34) boot_comm: chip revision: 1, min. bootloader chip revision: 0I (41) boot.esp32s3: Boot SPI Speed : 80MHzI (46) boot.esp32s3: SPI Mode   : SLOW READI (51) boot.esp32s3: SPI Flash Size : 16MBI (56) boot: Enabling RNG early entropy source...I (61) boot: Partition Table:I (65) boot: ## LabelUsage  Type ST Offset   LengthI (72) boot:  0 nvs  WiFi data01 02 00009000 00006000I (80) boot:  1 phy_init RF data  01 01 0000f000 00001000I (87) boot:  2 factory  factory app  00 00 00010000 00100000I (95) boot: End of partition tableI (99) boot_comm: chip revision: 1, min. application chip revision: 0I (106) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=0d30ch ( 54028) mapI (127) esp_image: segment 1: paddr=0001d334 vaddr=3fc92f00 size=02ce4h ( 11492) loadI (130) esp_image: segment 2: paddr=00020020 vaddr=42000020 size=1e860h (125024) mapI (162) esp_image: segment 3: paddr=0003e888 vaddr=3fc95be4 size=004f0h (  1264) loadI (163) esp_image: segment 4: paddr=0003ed80 vaddr=40374000 size=0ee10h ( 60944) loadI (185) esp_image: segment 5: paddr=0004db98 vaddr=50000000 size=00010h (16) loadI (191) boot: Loaded app from partition at offset 0x10000I (191) boot: Disabling RNG early entropy source...I (205) octal_psram: vendor id: 0x0d (AP)I (205) octal_psram: dev id   : 0x02 (generation 3)I (205) octal_psram: density  : 0x03 (64 Mbit)I (210) octal_psram: good-die : 0x01 (Pass)I (215) octal_psram: Latency  : 0x01 (Fixed)I (221) octal_psram: VCC  : 0x00 (1.8V)I (226) octal_psram: SRF  : 0x01 (Fast Refresh)I (232) octal_psram: BurstType: 0x01 (Hybrid Wrap)I (238) octal_psram: BurstLen : 0x01 (32 Byte)I (243) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)I (249) octal_psram: DriveStrength: 0x00 (1/1)I (254) esp_psram: Found 8MB PSRAM deviceI (259) esp_psram: Speed: 40MHzI (263) cpu_start: Pro cpu up.I (267) cpu_start: Starting app cpu, entry point is 0x403754480x40375448: call_start_cpu1 at /home/mint/Develop/esp/esp-idf-v5.0/components/esp_system/port/cpu_start.c:142I (0) cpu_start: App cpu up.I (1004) esp_psram: SPI SRAM memory test OKI (1014) cpu_start: Pro cpu start user codeI (1014) cpu_start: cpu freq: 240000000 HzI (1014) cpu_start: Application information:I (1018) cpu_start: Project name: esp32-s3-r8n32I (1023) cpu_start: App version:  1I (1028) cpu_start: Compile time: Dec 14 2022 08:19:15I (1034) cpu_start: ELF file SHA256:  52102a3bd7bc4632...I (1040) cpu_start: ESP-IDF:  v5.0I (1045) heap_init: Initializing. RAM available for dynamic allocation:I (1052) heap_init: At 3FC96B90 len 00052B80 (330 KiB): D/IRAMI (1059) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAMI (1065) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAMI (1072) heap_init: At 600FE010 len 00001FF0 (7 KiB): RTCRAMI (1078) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocatorI (1087) spi_flash: detected chip: mxicW (1090) spi_flash: Detected flash size > 16 MB, but access beyond 16 MB is not supported for this flash model yet.I (1101) spi_flash: flash io: dioW (1105) spi_flash: Detected size(32768k) larger than the size in the binary image header(16384k). Using the size in the binary image header.I (1120) cpu_start: Starting scheduler on PRO CPU.I (0) cpu_start: Starting scheduler on APP CPU.I (1139) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocationsI (1139) ESP32-S3-DevKitC-1.1-N32R8: BEGINI (1149) ESP32-S3-DevKitC-1.1-N32R8: ESP-IDF VERSION v5.0I (1149) ESP32-S3-DevKitC-1.1-N32R8: CHIP MODEL CHIP_ESP32S3I (1159) ESP32-S3-DevKitC-1.1-N32R8: CHIP FEATURES WIFI BLE I (1169) ESP32-S3-DevKitC-1.1-N32R8: REVISION 1I (1169) ESP32-S3-DevKitC-1.1-N32R8: FREE HEAP 8726608I (1179) ESP32-S3-DevKitC-1.1-N32R8: INITIALIZE NEOPIXELI (1189) gpio: GPIO[38]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (1189) ESP32-S3-DevKitC-1.1-N32R8: ENTER MAIN LOOP

The biggest line to point out in this run is line 69. Even though the ESP32-S3 devkit has 32 MB of FLASH, the latest ESP-IDF tool chain, v5.0, will not enable all 32 MB of FLASH, only 16 MB, or half of it. I would swear that I thought I read that version 5 would allow the full 32 MB of FLASH, but I guess I misread that somewhere. At least I have 16 MB and I have all 8 MB of the PSRAM. What I’m working on now is an application that can take full advantage of both FLASH and PSRAM maximum amounts.

It’s heading into Christmas and I have a lot of family things to do, so this may be one of my last posts for 2022.

We need to configure certain MCU attributes with idf.py menuconfig, the most important the enabling of the external SPIRAM.

Configuration

The ESP32-S3-DevKitC-1-N8R8 comes with an ESP32-S3-WROOM-1 SOC, 8 GB (N8) of flash and 8 GB (R8) of external RAM. We’re going to configure some of the properties of the devkit to tell the software tool chain how much flash is present, and to enable the use of the external RAM. To do this we’re going to use idf.py menuconfig.

• Set the amount of flash from the default of 2 GB to 8 GB. At the top-level of menuconfig select Serial flasher config; select Flash size (2 MB); move down to 8 MB and select it, then return to the top of menuconfig.
• Enable the external RAM. At top-level of menuconfig select Component config; scroll down and select ESP PSRAM; enable Support for external, SPI-connected RAM; move down to and select SPI RAM config; select Mode (QUAD/OCT) of SPI RAM chip in use (Quad Mode PSRAM); select Octal Mode PSRAM; step back one level (left arrow).
  • move down to Initialize SPI RAM during startup and enable it;
  • move down to Run memory test on SPI RAM initialization and enable it.
• Press ‘Q’ key and save these changes.

With the external RAM enable, the system will find it and allow it to become part of the overall memory pool, which means you now have 8.5 MB of memory to work with.

Updated Code

The code comes from the initial ESP32-S3 code I wrote a while back that flashes an external LED and the on-board NeoPixel. I’ll speak to changes after the listing.

#include <stdio.h>#include <array>#include <cinttypes>#include "freertos/FreeRTOS.h"#include "freertos/task.h"#include "driver/gpio.h"#include "esp_log.h"#include "led_strip.h"#include "sdkconfig.h"#include "esp_system.h"#include "esp_psram.h"// Create an array of color arrays to cycle through continuously.//using std::array;const array<array<int, 3>, 7> colors {{{32,0,0},  // red{0,32,0},  // green{0,0,32},  // blue{0,32,32}, // cyan{32,0,32}, // magenta{32,16,0}, // yellow{0,0,0}// black}};static led_strip_handle_t led_strip;static void initialize_neo_pixel() {led_strip_config_t strip_config = {.strip_gpio_num = CONFIG_BLINK_GPIO,.max_leds = 1, // at least one LED on board};led_strip_rmt_config_t rmt_config = {.resolution_hz = 10000000, // 10 MHz};ESP_ERROR_CHECK(led_strip_new_rmt_device(&strip_config, &rmt_config, &led_strip));// Darken NeoPixel by setting individual LED values to 0//led_strip_clear(led_strip);}// Task 1.//static void task_blink_neo_pixel(void * pvParameters) {while(true) {for(auto color : colors) {led_strip_set_pixel(led_strip, 0, color[0], color[1], color[2]);led_strip_refresh(led_strip);vTaskDelay(500 / portTICK_PERIOD_MS);led_strip_clear(led_strip);vTaskDelay(500 / portTICK_PERIOD_MS);}}}// Task 2.//static void task_blink_led(void * pvParameters) {gpio_reset_pin(GPIO_NUM_46);// Set the GPIO as a push/pull output//gpio_set_direction(GPIO_NUM_46, GPIO_MODE_OUTPUT);// Task 2 endless loop.//while (true) {gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(2700 / portTICK_PERIOD_MS);}}// Main entry.//extern "C" void app_main(void) {static const char *TAG = "DUAL_BLINK_V5";int core = xPortGetCoreID();ESP_LOGI(TAG, "BEGIN");ESP_LOGI(TAG, "ESP-IDF VERSION %s", esp_get_idf_version());ESP_LOGI(TAG, "APP_MAIN CORE %i", core);ESP_LOGI(TAG, "FREE HEAP %" PRIu32, esp_get_free_heap_size());ESP_LOGI(TAG, "INITIALIZE NEOPIXEL");initialize_neo_pixel();// Create task 1.//ESP_LOGI(TAG, "CREATE TASK 1");TaskHandle_t xHandle1 = NULL;static uint8_t ucParameterToPass1 = 1;xTaskCreate(task_blink_neo_pixel,"BlinkNeoPixel",// human readable task name.4096,   // stack size in bytes.&ucParameterToPass1,tskIDLE_PRIORITY,&xHandle1);configASSERT(xHandle1);// Create task 2.//ESP_LOGI(TAG, "CREATE TASK 2");TaskHandle_t xHandle2 = NULL;static uint8_t ucParameterToPass2 = 1;xTaskCreate(task_blink_led,"BlinkLED", // human-readable task name.4096,   // stack size in bytes.&ucParameterToPass2,tskIDLE_PRIORITY,&xHandle2);configASSERT(xHandle2);// Stay in an endless loop. Don't return from this task.//ESP_LOGI(TAG, "ENTER MAIN LOOP");while (true) {vTaskDelay(5000 / portTICK_PERIOD_MS); // 5 second period}}

The first issue I came across after enabling external RAM was with the NeoPixel blinking task. Before the external RAM was enabled, both tasks ran just fine. After, BlinkNeoPixel crashed its stack and caused the application to enter a continuous boot loop. I fixed that problem by first removing the NeoPixel initialization code and putting it into its own subroutine, which is called once to set up the NeoPixel (see lines 36 through 53). I should note that that is the way the original Espressif example code handled it. It was my decision to pull it into the NeoPixel blink task, even if it was only executed once before running in the while loop. This was a bad decision because it forces the task to consume more stack space than necessary. Once it was split apart again, there were no more stack crashes on that task. I also decided to increase the stack size on both tasks from 2048 bytes to 4096 bytes.

The second issue is with the external RAM itself. The way it’s configured is that it’s “lumped in” with the SoC’s 512K RAM, allowing regular applications to access it via malloc and free. I tried to access the memory more directly via esp_psram, but while I could write code with functions found in esp_psram, they failed to link because the psram subsystem would not build. And I couldn’t figure out how to configure my project to build that library. So I’m leaving the code as is and will move on with “normal” memory access to see what happens.

Here’s one run of the code on the idf.py monitor.

Hard resetting via RTS pin...Executing action: monitorRunning idf_monitor in directory /home/mint/Develop/esp/dualblink_v5Executing "/home/mint/.espressif/python_env/idf5.0_py3.10_env/bin/python /home/mint/Develop/esp/esp-idf-v5.0/tools/idf_monitor.py -p /dev/ttyUSB0 -b 115200 --toolchain-prefix xtensa-esp32s3-elf- --target esp32s3 /home/mint/Develop/esp/dualblink_v5/build/dualblink_v5.elf -m '/home/mint/.espressif/python_env/idf5.0_py3.10_env/bin/python' '/home/mint/Develop/esp/esp-idf-v5.0/tools/idf.py' '-p' '/dev/ttyUSB0'"...--- idf_monitor on /dev/ttyUSB0 115200 ------ Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---p_image: segment 2: paddr=0001dbb8 vaddr=40374000 size=02460h (  9312)�ESP-ROM:esp32s3-20210327Build:Mar 27 2021rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)SPIWP:0xeemode:DIO, clock div:1load:0x3fce3810,len:0x167cload:0x403c9700,len:0xbe8load:0x403cc700,len:0x2e98entry 0x403c9904I (25) boot: ESP-IDF v5.0 2nd stage bootloaderI (25) boot: compile time 21:49:42I (25) boot: chip revision: v0.1I (26) boot_comm: chip revision: 1, min. bootloader chip revision: 0I (33) boot.esp32s3: Boot SPI Speed : 80MHzI (38) boot.esp32s3: SPI Mode   : DIOI (43) boot.esp32s3: SPI Flash Size : 8MBI (48) boot: Enabling RNG early entropy source...I (53) boot: Partition Table:I (57) boot: ## LabelUsage  Type ST Offset   LengthI (64) boot:  0 nvs  WiFi data01 02 00009000 00006000I (71) boot:  1 phy_init RF data  01 01 0000f000 00001000I (79) boot:  2 factory  factory app  00 00 00010000 00100000I (86) boot: End of partition tableI (91) boot_comm: chip revision: 1, min. application chip revision: 0I (98) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=0a9b4h ( 43444) mapI (114) esp_image: segment 1: paddr=0001a9dc vaddr=3fc92d00 size=031d4h ( 12756) loadI (117) esp_image: segment 2: paddr=0001dbb8 vaddr=40374000 size=02460h (  9312) loadI (125) esp_image: segment 3: paddr=00020020 vaddr=42000020 size=1d974h (121204) mapI (153) esp_image: segment 4: paddr=0003d99c vaddr=40376460 size=0c880h ( 51328) loadI (165) esp_image: segment 5: paddr=0004a224 vaddr=50000000 size=00010h (16) loadI (171) boot: Loaded app from partition at offset 0x10000I (172) boot: Disabling RNG early entropy source...I (185) octal_psram: vendor id: 0x0d (AP)I (185) octal_psram: dev id   : 0x02 (generation 3)I (185) octal_psram: density  : 0x03 (64 Mbit)I (190) octal_psram: good-die : 0x01 (Pass)I (195) octal_psram: Latency  : 0x01 (Fixed)I (201) octal_psram: VCC  : 0x01 (3V)I (206) octal_psram: SRF  : 0x01 (Fast Refresh)I (212) octal_psram: BurstType: 0x01 (Hybrid Wrap)I (217) octal_psram: BurstLen : 0x01 (32 Byte)I (223) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)I (229) octal_psram: DriveStrength: 0x00 (1/1)I (234) esp_psram: Found 8MB PSRAM deviceI (239) esp_psram: Speed: 40MHzI (243) cpu_start: Pro cpu up.I (247) cpu_start: Starting app cpu, entry point is 0x4037535c0x4037535c: call_start_cpu1 at /home/mint/Develop/esp/esp-idf-v5.0/components/esp_system/port/cpu_start.c:142I (0) cpu_start: App cpu up.I (984) esp_psram: SPI SRAM memory test OKI (993) cpu_start: Pro cpu start user codeI (993) cpu_start: cpu freq: 160000000 HzI (994) cpu_start: Application information:I (996) cpu_start: Project name: dualblink_v5I (1002) cpu_start: App version:  1I (1006) cpu_start: Compile time: Dec  6 2022 21:49:36I (1013) cpu_start: ELF file SHA256:  5c83520c39687425...I (1019) cpu_start: ESP-IDF:  v5.0I (1023) heap_init: Initializing. RAM available for dynamic allocation:I (1031) heap_init: At 3FC96960 len 00052DB0 (331 KiB): D/IRAMI (1037) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAMI (1044) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAMI (1050) heap_init: At 600FE010 len 00001FF0 (7 KiB): RTCRAMI (1057) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocatorI (1065) spi_flash: detected chip: genericI (1069) spi_flash: flash io: dioI (1074) cpu_start: Starting scheduler on PRO CPU.I (0) cpu_start: Starting scheduler on APP CPU.I (1094) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocationsI (1094) DUAL_BLINK_V5: BEGINI (1104) DUAL_BLINK_V5: ESP-IDF VERSION v5.0I (1104) DUAL_BLINK_V5: APP_MAIN CORE 0I (1114) DUAL_BLINK_V5: FREE HEAP 8727288I (1114) DUAL_BLINK_V5: INITIALIZE NEOPIXELI (1124) gpio: GPIO[48]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (1134) DUAL_BLINK_V5: CREATE TASK 1I (1134) DUAL_BLINK_V5: CREATE TASK 2I (1134) DUAL_BLINK_V5: ENTER MAIN LOOPI (1154) gpio: GPIO[46]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 

So what’s happening during a boot?

1. Lines 20-22 — The external RAM chip is first detected, and the size is displayed.
2. Lines 39-51 — The external RAM chip is probed and detail information is printed.
3. Line 71 — PSRAM is added to the full heap space.
4. Line 80 — The initial amount of free heap is printed in bytes. This is printed by line 100 in dualblink_v5.cpp.

There’s more to come…

• First thing I do is create a new work folder in which to migrate to version 5.
• Within that folder I recursively copy the contents of a version 5 example that most closely matches what I want to port. In my case it was underneath the examples folder, examples/get-started/blink.
• Once copied I step into the main folder and copy the main folder source contents only of the application to be ported. Adjust the CMake file in the main folder to pick up the new source files you just copied.
• I step out of the main folder and back into the top level migration folder
• I perform an idf.py set-target esp32s3 or esp32c3 for the hardware I need to compile for.
• I them start to compile the code. You’re going to get errors, so you need to keep an editor open in another console tab or within a graphical editor, and fix the source that’s causing compiler errors.
• I managed to complete the migration in no more than three steps.
• Once compiled, then flash the new binary to the test board and see if it works. In both of my cases so far, they have.

Here’s one of the easiest of my apps to port so far.

#include <stdio.h>#include <array>#include "freertos/FreeRTOS.h"#include "freertos/task.h"#include "driver/gpio.h"#include "esp_log.h"#include "led_strip.h"#include "sdkconfig.h"// Create an array of color arrays to cycle through continuously.//using std::array;const array<array<int, 3>, 7> colors {{{32,0,0},  // red{0,32,0},  // green{0,0,32},  // blue{0,32,32}, // cyan{32,0,32}, // magenta{32,16,0}, // yellow{0,0,0}// black}};// Task 1.//static void task_blink_neo_pixel(void * pvParameters) {// Initialized the NeoPixel.//static led_strip_handle_t led_strip;led_strip_config_t strip_config = {.strip_gpio_num = CONFIG_BLINK_GPIO,.max_leds = 1, // at least one LED on board};led_strip_rmt_config_t rmt_config = {.resolution_hz = 10000000, // 10 MHz};ESP_ERROR_CHECK(led_strip_new_rmt_device(&strip_config, &rmt_config, &led_strip));// Darken NeoPixel by setting individual LED values to 0//led_strip_clear(led_strip);// Task 1 endless loop.//while(true) {for(auto color : colors) {led_strip_set_pixel(led_strip, 0, color[0], color[1], color[2]);led_strip_refresh(led_strip);vTaskDelay(500 / portTICK_PERIOD_MS);led_strip_clear(led_strip);vTaskDelay(500 / portTICK_PERIOD_MS);}}}// Task 2.//static void task_blink_led(void * pvParameters) {gpio_reset_pin(GPIO_NUM_46);// Set the GPIO as a push/pull output//gpio_set_direction(GPIO_NUM_46, GPIO_MODE_OUTPUT);// Task 2 endless loop.//while (true) {gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(2700 / portTICK_PERIOD_MS);}}// Main entry.//extern "C" void app_main(void) {static const char *TAG = "DUAL_BLINK";int core = xPortGetCoreID();ESP_LOGI(TAG, "BEGIN");ESP_LOGI(TAG, IDF_VER);ESP_LOGI(TAG, "APP_MAIN CORE %i", core);// Create task 1.//TaskHandle_t xHandle1 = NULL;static uint8_t ucParameterToPass1 = 1;xTaskCreate(task_blink_neo_pixel,"BlinkNeoPixel",// human readable task name.2048,   // stack size in bytes.&ucParameterToPass1,tskIDLE_PRIORITY,&xHandle1);configASSERT(xHandle1);// Create task 2.//TaskHandle_t xHandle2 = NULL;static uint8_t ucParameterToPass2 = 1;xTaskCreate(task_blink_led,"BlinkLED", // human-readable task name.2048,   // stack size in bytes.&ucParameterToPass2,tskIDLE_PRIORITY,&xHandle2);configASSERT(xHandle2);// Stay in an endless loop. Don't return from this task.//while (true) {vTaskDelay(5000 / portTICK_PERIOD_MS); // 5 second period}}

The solidly highlighted section above shows what had to be changed. It looks like a lot, but it’s not. The functionality is still the same as it was in older versions, it’s how that functionality is invoked that’s changed. What I found more interesting is that none of the multitasking functionality changed. I wasn’t sure what to expect, but the changes to using the NeoPixel are rather minor, all things considered. What I’m looking forward to is if I can use the external PSRAM on all the boards. Currently I can’t with ESP_IDF 4.x.

There’s also a problem compiling Micropython with version 5. I’ve already checked and there is no support in Micropython 1.19.1 for ESP_IDF 5. I found an closed ticket in the Micropython forums talking about this and asking if anyone wanted to step forward and work on this.

Micropython Issue 8607: https://github.com/micropython/micropython/issues/8607

Update: std::array is (as of C++11) part of the C++ standard. The differences between boost::array and std::array are minimal. If you are using C++11, you should consider using std::array instead of boost::array.

The following code shows some of the ways this container is used. If you change the #include <array> to <boost/array>, as well as the corresponding using statement, then it will compile and execute the same.

#include <array>#include <algorithm>#include <iostream>using std::array;using std::cout;using std::sort;using std::string;int main() {array<int, 10> iarray = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};cout << "Array of integers:" << "\n";for (const auto value : iarray) {cout << "Integer array value: " << value << "\n";}cout << "\n" << "Array of standard strings:" << "\n";array<string, 10> sarray ={"one","two","three","four","five","six","seven","eight","nine","ten"};for (const auto value : sarray) {cout << "String array value: " << value << "\n";}cout << "\n" << "Array of sorted strings:" << "\n";sort(sarray.begin(), sarray.end());for (const auto value : sarray) {cout << "String array value: " << value << "\n";}return 0;}

Compiling It

Compilation is a simple one liner.

g++ -std=c++20 array-test.cpp -o array-test

Sample Run

Here’s what happens when we run it.

~ ./array-test Array of integers:Integer array value: 1Integer array value: 2Integer array value: 3Integer array value: 4Integer array value: 5Integer array value: 6Integer array value: 7Integer array value: 8Integer array value: 9Integer array value: 10Array of standard strings:String array value: oneString array value: twoString array value: threeString array value: fourString array value: fiveString array value: sixString array value: sevenString array value: eightString array value: nineString array value: tenArray of sorted strings:String array value: eightString array value: fiveString array value: fourString array value: nineString array value: oneString array value: sevenString array value: sixString array value: tenString array value: threeString array value: two

Comments

What makes std::array better than std::vector is that there is no need for extra code to support a dynamic container like vector. The container array is like the simpler C fixed array, in that it’s fixed and static, but has enough container traits that it can be used in a C++ ranged for loop (lines 15,23, and 29), as show in the example above.

Using std::array in an Embedded Application

I have been writing ESP32-S3 and ESP32-C3 C++ code. I’ve published the following code more than once, but this time, I just want to highlight how I replaced the C++ standard library vector and tuple type with array. Here is where std::array helps in dropping the size of the binary. I checked the size of the generated binary (everything else the same) and discovered that I save a bit of 10K of generated binary, meaning that I saved 10K by switching to array. What’s more the ability to access the elements of the inner array in line 44 makes a lot more sense than the older way of declaring the inner colors a tuple and the convoluted way I accessed tuple members the way I did.

#include <stdio.h>#include <array>#include "freertos/FreeRTOS.h"#include "freertos/task.h"#include "driver/gpio.h"#include "esp_log.h"#include "led_strip.h"#include "sdkconfig.h"using std::array;const array<array<int, 3>, 7> colors {{{32,0,0},  // red{0,32,0},  // green{0,0,32},  // blue{0,32,32}, // cyan{32,0,32}, // magenta{32,16,0}, // yellow{0,0,0}// black}};// Task 1.//static void task_blink_neo_pixel(void * pvParameters) {static led_strip_t *pStrip_a;pStrip_a = led_strip_init(CONFIG_BLINK_LED_RMT_CHANNEL, CONFIG_BLINK_GPIO, 1);pStrip_a->clear(pStrip_a, 50);// Stay in an endless loop. Don't return from this task.//while(true) {for(auto color : colors) {pStrip_a->set_pixel(pStrip_a, 0, color[0], color[1], color[2]);pStrip_a->refresh(pStrip_a, 100);vTaskDelay(500 / portTICK_PERIOD_MS);// Set NeoPixel LED dark by clearing all its individual LEDs.pStrip_a->clear(pStrip_a, 50);vTaskDelay(500 / portTICK_PERIOD_MS);}}}// Task 2.//static void task_blink_led(void * pvParameters) {gpio_reset_pin(GPIO_NUM_46);// Set the GPIO as a push/pull output//gpio_set_direction(GPIO_NUM_46, GPIO_MODE_OUTPUT);// Stay in an endless loop. Don't return from this task.//while (true) {gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, true);   // LED onvTaskDelay(100 / portTICK_PERIOD_MS);gpio_set_level(GPIO_NUM_46, false);  // LED offvTaskDelay(2700 / portTICK_PERIOD_MS);}}extern "C" void app_main(void) {static const char *TAG = "DUAL_BLINK";int core = xPortGetCoreID();ESP_LOGI(TAG, "app_main running on core %i", core);ESP_LOGI(TAG, "CONFIG_BLINK_GPIO %i", CONFIG_BLINK_GPIO);// Create task 1.//TaskHandle_t xHandle1 = NULL;static uint8_t ucParameterToPass1 = 1;xTaskCreate(task_blink_neo_pixel,"BlinkNeoPixel",// human readable task name.2048,   // stack size in bytes.&ucParameterToPass1,tskIDLE_PRIORITY,&xHandle1);configASSERT(xHandle1);// Create task 2.//TaskHandle_t xHandle2 = NULL;static uint8_t ucParameterToPass2 = 1;xTaskCreate(task_blink_led,"BlinkLED", // human-readable task name.2048,   // stack size in bytes.&ucParameterToPass2,tskIDLE_PRIORITY,&xHandle2);configASSERT(xHandle2);// Stay in an endless loop. Don't return from this task.//while (true) {vTaskDelay(1000 / portTICK_PERIOD_MS);}}

#include <algorithm>#include <iostream>#include <iomanip>#include <stdexcept>#include <filesystem>#include <string>#include <boost/multiprecision/cpp_int.hpp>using std::cout;using std::cerr;using std::left;using std::right;using std::setw;using std::fixed;using std::stoi;using std::string;using std::filesystem::path;using std::invalid_argument;using boost::multiprecision::cpp_int;void factorial(cpp_int n) {cpp_int a = 1;cout << right << setw(4) << fixed << n;while ( n > 0 ) {a = a * n;n--;}cout << left << "! = " << a << "\n";}int main(int argc, char *argv[]) {string appname = path(argv[0]).stem();if (argc == 1) {cout << appname << " can be invoked with one or more integer arguments.\n";cout << "Example: " << appname << " 10 20 30 40 50 60 70 80 90 100\n";}int index{1};try {for (; index < argc; factorial(stoi(argv[index++])));}catch(invalid_argument ia) {index--;cerr << apname << " invalid argument: " << argv[index] << "\n";return -1;}return 0;}

General Description

You’ve seen this code before, but in a far simpler code format than this. This version of the application has a try/catch block at lines 43 and 46 to catch an exception that stoi will throw if it’s given a non-numeric argument to parse. The catch statement explicitly catches an invalid argument and will print out what caused the application to stop. This is a lot better than the core dump it performed before this. Furthermore, if you call this application without any arguments it will give you several lines of help (at least, I hope they’re helpful). I’ve pulled a little bit of C++ slight-of-hand here by using std::filesystem::path to pull out the name of the application, or at least the name it was invoked by. This is an old trick in bash scripts and Python, but I’m using functionality from the C++ 17 standard.

How to Build

Building this is a one-liner:

g++ -std=c++20 SimpleFactorial.cpp -o SimpleFactorial

You’ll note that I didn’t use the -static switch. Since there were no specific Boost libraries to link, I decided to just build a regular Linux application that will dynamically link against standard Linux C and C++ libraries. Those are common to every modern Linux distribution.

Sample Runs

The first example shows what happens when you invoke SimpleFactorial without any arguments.

~ ./SimpleFactorial SimpleFactorial can be invoked with one or more integer arguments.Example: SimpleFactorial 10 20 30 40 50 60 70 80 90 100

So what happens if we invoke that example?

~ ./SimpleFactorial 10 20 30 40 50 60 70 80 90 100  10! = 3628800  20! = 2432902008176640000  30! = 265252859812191058636308480000000  40! = 815915283247897734345611269596115894272000000000  50! = 30414093201713378043612608166064768844377641568960512000000000000  60! = 8320987112741390144276341183223364380754172606361245952449277696409600000000000000  70! = 11978571669969891796072783721689098736458938142546425857555362864628009582789845319680000000000000000  80! = 71569457046263802294811533723186532165584657342365752577109445058227039255480148842668944867280814080000000000000000000  90! = 1485715964481761497309522733620825737885569961284688766942216863704985393094065876545992131370884059645617234469978112000000000000000000000 100! = 93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

What happens if we invoke SimpleFactorial with bad input data?

~ ./SimpleFactorial 10 20 foo  10! = 3628800  20! = 2432902008176640000SimpleFactorial invalid argument: foo

There’s another “hidden” feature of the application. What happens if you rename SimpleFactorial to something else, invoking it differently than it’s original name? What if you rename your source and build the application as something else? That’s where the path function comes into play. Let’s see what happens if we change the name of the application by way of a soft link.

~ ln -s SimpleFactorial foo~ ./foofoo can be invoked with one or more integer arguments.Example: foo 10 20 30 40 50 60 70 80 90 100

What happens when we have an error with this new name?

~ ./foo barfoo invalid argument: bar

This capability doesn’t seem all that important, but for more complex applications it comes in quite handy when you’re chasing down errors from within a long log file.

If you’re wondering why I didn’t use Boost’s Program Options, it’s because I didn’t understand the documentation well enough to learn how to pass an arbitrary number of arguments to the application. Maybe later…

Link: https://www.boost.org/doc/libs/1_80_0/libs/multiprecision/doc/html/index.html

#include <exception>#include <iostream>#include <boost/program_options.hpp>using std::cout;using std::cerr;using std::exception;using boost::program_options::options_description;using boost::program_options::value;using boost::program_options::variables_map;using boost::program_options::store;using boost::program_options::parse_command_line;using boost::program_options::notify;int main(int argc, char *argv[]) {options_description options("Options");options.add_options()("help", "Help using application.")("compression", value<int>(), "Set compression level to integer value");try {variables_map varmap;store(parse_command_line(argc, argv, options), varmap);notify(varmap);if (varmap.empty() or varmap.count("help")) {cout << options << "\n";}if (varmap.count("compression")) {cout << "Compression level set to "<< varmap["compression"].as<int>;() << "\n";}}catch (exception &amp;ex) {cerr << "Error: " << ex.what() << "\n";cerr << "\n" << options << "\n";return -1;}return 0;}

The using Conundrum

When C++ namespaces were introduced, they also provided a way to avoid having to preface every namespace element with the namespace name, such as std::cout. That feature was the using namespace declaration, such as using namespace std. My problem with that is it completely obfuscates those function that are part of the std namespace, which can be a real pain when you’re trying to read code you’re not familiar with, or worse, your code that you haven’t touched in a while under stressful circumstances. Of course if you don’t use using namespace std then you’re going to put the namespace declaration in front of every standard library element, wherever they’re being used. I was pleasantly surprised with the using variant, where the namespace can be combined with the element, allowing you to just use the element in the body of you code. That’s what I did in lines 5-13 highlighted above.

Two good things have occurred. First, I have a list of every element I’m using in all the namespaced libraries at the top of the code in a list I can quickly read. Second, I can use the bare elements within the code body, producing cleaner code, more quickly. It might not be perfect, but it’s better that the first two original options, and it’s a coding standard I think I’m going to stick with from now on.

The End of Line Conundrum

I have seen quite a bit of C++ code these days that uses "\n" at the end of a line, instead of std::endl. For example:

std::cout << options << "\n";

versus

std::cout << options << std::endl;

I haven’t read anywhere that might explain why it appears std::endl has fallen out of favor as it were. I personally would rather use the simpler form, and from this point forward, I will just as I have in the listing at the top of this post.

Wrap Up

I removed that last else statement because it made no sense to have it. I then broke up the remain if statements into two smaller conditional blocks, which makes it easier to read. The entire program is organized in a logical fashion from declarations to initializations to work sections, and the try/catch block handles all the exceptions you can throw at it. It might not be perfect, but it’s good enough for me. Bitter experience has taught me over the decades that perfect is always the enemy good.

#include <exception>#include <iostream>#include <boost/program_options.hpp>namespace po = boost::program_options;int main(int argc, char *argv[]) {po::options_description options("Options");options.add_options()("help", "Help using application.")("compression", po::value<int>(), "Set compression level to integer value");try {po::variables_map varmap;po::store(po::parse_command_line(argc, argv, options), varmap);po::notify(varmap);if (varmap.empty() or varmap.count("help")) {std::cout << options << std::endl;}else if (varmap.count("compression")) {std::cout << "Compression level set to "<< varmap["compression"].as<int>() << std::endl;}else {std::cout << "Compression level was not set.\n";return -1;}}catch (std::exception &ex) {std::cerr << "Error: " << ex.what() << std::endl;std::cout << std::endl << options << std::endl;return -1;}return 0;}

The example above started with an example provided on the Boost Program Options Getting Started page ( https://www.boost.org/doc/libs/1_80_0/doc/html/program_options/tutorial.html#id-1.3.30.4.3 ). The first code example in the tutorial won’t compile. In order to compile that code you need to include three libraries as shown in the first three lines of the listing above. I will make an editorial comment about the code examples in the Boost tutorial, and say that they’re all bad. Not only will they not compile, they won’t run very well either. I’ll get to that running poorly in a moment.

The bulk of the original tutorial logic is between lines 5 to 30. Everything else I had to add in order to produce an application that would behave reasonably, especially when erroneous input is given to the application.

The try/catch block was added in order to handle exceptions that the Boost library throws when it has an error. That isn’t documented anywhere that I can find for this library. While handling exceptions is really no big deal, and is considered the Python Way (using try/except) within Python, at least you should tell someone what will happen if your library reaches a point where an error occurs. I discovered this behavior when I successfully compiled the initial version without any sort of try/catch logic. Every time I gave it bad switches, or nothing at all, it would throw an exception and core dump. Yeah, that’s gonna work just fine. Not.

The rest of the code is the try/catch block, with some logic in the catch section to tell you what the error is and then prints out help. That’s what should happen.

If you just type the name of the application, line 20 will check to see if any switches were passed to the application. If none were, or if you entered --help, then you get help. The original code would not check for lack of any options and throw an exception and core dump. By the way, the library will only accept an option if it is preceded by --. Anything else, such as one dash, is treated as an invalid input.

The only problem with the listing is the last else statement, lines 27 to 30. With the logic as it currently exists, this last statement is no longer executed and should be removed. I left it in place because it’s part of the original Boost tutorial.

Setting up the Boost Build Environment

There are two ways to set up the Boost build environment. The first is to download the source tarball, unpack that into a work area, build it, then install it. The directions on the Boost website ( https://www.boost.org ) are clear and complete on this. I’ve done it myself just to see what it’s like and if it provides any advantage over installing from the Ubuntu repos. That depends. The current Boost release is version 1.80, released this November. The version installed from the repo is 1.74, released in August 2020, over two years ago. If you want the latest release with all its bug fixes and newer features, then yes, build Boost from the source and then build your application(s) on top of that. For the most part start with the repo and then build Boost from latest if you find a problem or you want a feature in version 1.80 that’s not in version 1.74.

The second way is to install from the repo. Specifically, sudo apt install libboost-all-dev will install all header and library files. The header files are installed in /usr/include/boost/, and the library files are installed in /usr/lib/x86_64-linux-gnu/.

Building the Boost Example

I used to be a make file nerd back in the day because make was all you had, and so you learned to use it. One of the primary reasons for make back in the day was because build systems were so slow compared to today’s systems. Make checked dependencies and only rebuilt those portions of a project that had changed, then rebuilt those changed source files and then relinked the full application binary. That’s not such a problem today, especially when my $300 2 GHz 8 core AMD Ryzen 5 blows the doors off those SUN Microsystems build servers I used to use. And Linux Mint is so much better than Solaris.

I’ve since fallen away from make, especially when all I want to do is build some simple applications. These days I’ll use Bash or Python, or if it gets complicated, CMake. If I write in Go or Rust, those two languages have a complete toolset that will build my application for me. For building this application I wrote a simple Bash script.

#!/bin/env bashg++ -std=c++20 -c cli.cppg++ -static -o cli cli.o -l boost_program_options

Line 1 is the standard shebang. Line two invokes the GNU C++ compiler to compile the application using the C++20 standard rules. I set the standard to C++20 because that’s what I’m brushing up on at the moment. If you’re running on a Linux distribution that has GNU C++ version 11 or later, then you’re all set. Linux Mint 21 has version 11.3.0. Line 3 is the linker line. Note the -static command line switch, and also note that I’ve had to select the specific library to link against with the -l boost_program_options command line switch. I’m building a static application because that is how Go and Rust build binaries. I heartily approve of static linking so that you can give your customers a single binary that will run just about anywhere. Having your application depend upon whether they have every supporting library installed is a royal pain. That may be the way things were done twenty or more years ago, but not today.

Running the Application

Here are a few runs of the application.

~ ./cli Options:  --helpHelp using application.  --compression arg Set compression level to integer value

Simply running the application without any input produces help. Just as you would want it to do.

~ ./cli --helpOptions:  --helpHelp using application.  --compression arg Set compression level to integer value

Running the application for explicit help.

~ ./cli --compressionError: the required argument for option '--compression' is missingOptions:  --helpHelp using application.  --compression arg Set compression level to integer value

We passed a proper option, but the argument is missing, thus the error. And once again we print out the help.

~ ./cli --compression 10Compression level set to 10

Now we have a successful invocation. But what happens if we try to pass a float argument? Remember, help says to pass an integer argument.

~ ./cli --compression 10.1Error: the argument ('10.1') for option '--compression' is invalidOptions:  --helpHelp using application.  --compression arg Set compression level to integer value

We get an error and another dose of help. What about a totally different option?

~ ./cli --fooError: unrecognised option '--foo'Options:  --helpHelp using application.  --compression arg Set compression level to integer value

Which is what we would expect. Before you ask, it will bale on on the first unrecognized option. And I know that “unrecognised” is misspelled in the error message. But that error message is part of the package, nothing that I can easily fix. There’s also one other feature. If I pass --comp or even --c, the code will recognize it because it’s still a partial match and it’s unique.

Some Final Thoughts

I choose to dig deeper into Boost because of the general good chatter around the library set. And the first few forays into Boost were very easy. Those first few forays used header-only libraries that didn’t require linking. This is the first instance where I had to link. To be honest, this experience has been a bit rough. If I’d had experience with this first, I might not have gone any farther. But it does show that quality in Boost is uneven. Not just in the tutorials, but in the implementation. For example consider the penchant to throw exceptions all over the place. Boost has two exception libraries, a regular Exception and another called LEAF. If you go to the Exception documentation they tell you that if you’re past a certain Boost version then you should use LEAF. I looked at LEAF and left in confusion. I tried Boost’s Exception and discovered that Boost’s Exception printed out that it was using regular C++ exception and that maybe the application should use the more advance Boost Exception. Really?

I’m going to continue to survey Boost as well as move on my own into C++ 20 style C++.