LED Clock, Part III: Alexa?

Whilst I await delivery of more WS2812 LEDs, I wanted to start investigating how I can leverage the Alexa Gadget Toolkit integration, so that when I set a timer using Alexa, my LED clock can show the countdown.

Amazon make an Echo Wall Clock which does just that.

I found an open source project called nixie-timer, which had an Alexa Integration and was written for the ESP32

After a few hours of digging around, I had a very basic idea of what the code was doing. I started by trying to replicate the flow using the OOB ESP-IDF.

After many, many, many hours, I realised that I really didn’t know what I was doing, so I went back to the nixie-timer code and added the btstack ESP32 port into my code. This meant I could at least follow the samples provided.

I began trying to run some code myself, but the Bluetooth radio wouldn’t even start. I then took one of the BTStack examples and used that as a starting point. At least they worked.

After more hours, I got the code running, with the occasional kernel panic as I figured stuff out..

Eventually, I got it responding to the Alexa queries and receiving the messages for wakeword, timeinfo and timer.

Exactly what I wanted to achieve. I’m not 100% sure I know what’s going on, but I’ll get more understanding over time.

For starters, I want to use timeinfo to set the internal clock and then I want to use the timer command to display the countdown of an alexa timer.

Now that I’ve moved back to ESP-IDF, I’m going to have to bin my existing Arduino code which powers the LED strip and look at how to make that work.

Happy with progress.

My other WS2812 strip arrived the other day, so I’ve got to look at cutting and connecting the two strips to form one long 180 LED strip. Then I get get a feel of the overall diameter and cut some MDF to house the damn thing.

LED Clock, Update!

I managed to rework the time calculation and hour hand movement!

The clock is ticking!

I removed the RTC component and just let the loop run, with simple counters to simulate the passage of second, minutes and hours.

I think the transition of the “hands” needs to be smoother, more analogue. I’ll have to investigate if FastLED can do this.

LED Clock, Part II: Tick, Tock

In Part I, I covered the basics of controlling the LEDs. This covered the hands of the clock.

The second part of my investigation covers time and how to make the clock tick. The ESP 8266 I am using for this prototyping work doesn’t include a Real Time Clock, so I was required to add an external one.

Connecting the RTC module to my Wemos D1

With the module connected, I turned to the code. I’m using Arduino for this as it’s easy to prototype and there are lots of libraries!

#include <Wire.h>      //I2C library
#include <RtcDS3231.h> //RTC library

Setup is achieved in a couple of steps. To get me started, I just set the time.

RtcDS3231<TwoWire> rtcObject(Wire);

void setup()
{
  Serial.begin(9600);

  rtcObject.Begin();

  RtcDateTime currentTime = RtcDateTime(20, 04, 22, 0, 0, 0);

  rtcObject.SetDateTime(currentTime);
}

This starts the clock. In reality, as the module I’m using is battery backed, this step should only be performed once, but I found that having a fixed “time” would make debugging the lights a little easier.

I plan to revisit this code to ensure that I grab the current time from the internet when required.

Using the time is achieved by reading it.

void loop()
{
  Serial.println("Loop()");

 //get the time from the RTC
  RtcDateTime currentTime = rtcObject.GetDateTime();

  char str[20]; //declare a string as an array of chars

  //print the time for debugging.
  sprintf(str, "%d/%d/%d %d:%d:%d", 
          currentTime.Year(),       //get year method
          currentTime.Month(),      //get month method
          currentTime.Day(),        //get day method
          currentTime.Hour(),       //get hour method
          currentTime.Minute(),     //get minute method
          currentTime.Second()      //get second method
  );

Each part of the time is available in a named method, so to get an idea of where my LEDs should be, I grabbed the hour, minute and second.

// Fetch the current time
//
hour = currentTime.Hour();
minute = currentTime.Minute() * 2;
second = currentTime.Second() * 2;

I multiply them as I’m using a strip with 120 LEDs on it.

Building on my LED control code, I make the clock tick by turning off LEDs in the right place.

fill_solid(leds, NUM_LEDS, CRGB::DarkGreen);

// Fetch the current time
//
hour = currentTime.Hour();
minute = currentTime.Minute() * 2;
second = currentTime.Second() * 2;

// Hour
leds[hour - 1] = CRGB::Black;
leds[hour] = CRGB::Black;
leds[hour + 1] = CRGB::Black; 
leds[hour + 2] = CRGB::Black;
  
// Minute
leds[minute] = CRGB::Black;
leds[minute + 1] = CRGB::Black;

// Seconds
//
leds[second] = CRGB::Black;
leds[second + 1] = CRGB::Black;

FastLED.show();

I use multiple lights to make it more visible. With it now ticking, I hastily assembled the LED strip into a circle using cardboard and sellotape. I ran a length of alarm cable from the strip to the breadboard. To see it in action, I sellotaped it to the wall!

After letting it run for quite a while, I was surprised the hour “hand” hadn’t moved. It took my wife to point out that there aren’t 60 hours on a clock face 🤣

I’m pleased with these initial results.

As for my next steps, I’m not 100% sure. I’ve got to refine the “hands” code. I also need to think about how to mount it on the wall. I suspect it won’t be self contained and I’ll probably have a control box somewhere.

I’ve ordered another strip from the excellent Pimoroni and this will, I hope, let me add another 36 LEDs to the strip, taking it up to 180, which I think will give me a nice resolution.

I’ve got to think about power too. My estimate is that I’ll need around 3.5A to power the whole affair.

Expect another blog post soon!

Lockdown LED Clock

A year or two ago, I picked up a WS2812B LED strip, but, for the life of me, could never think of anything useful to do with it.

Recently, however, I came across these LED powered clocks.

This feels like a project that will combine a few things:

  • ESP MCU
  • Real Time Clock
  • WS2812B strip
  • Maybe some Alexa integration?

Since I’m working from home for the duration, due to COVID-19, I get to recoup my usual commute time and put it towards something.

My starting point will be hooking up the strip to an ESP32 and trying to light one of the LEDs. I plan on a series of posts as I work towards my own clock!

Part I – https://tomasmcguinness.com/2020/04/14/led-clock-part-i-the-leds/

Part II – https://tomasmcguinness.com/2020/04/25/led-clock-part-ii-tick-tock/

Do we have any hot water?

A common occurrence in our house is bath time.

Another common occurrence is not having enough water for afore mentioned bath.

Our hot water is provided by a system boiler/unvented cylinder arrangement and I use a Nest to control the hot water (more on that later). The current schedule has it running the boiler for thirty minutes each day.

What is very annoying is that on some bath days we have ample hot water and on others, we don’t. We end up boiling the kettle to make up the difference. I don’t like the idea of just turning on the hot water as we might just be heating water that is already hot enough.

How much hot water do we have?

The first thing to figure out is how much how water we actually have. As I’ve mentioned, one some days we have ample and on other days we run out. Why does this happen?

Here are a few of the main factors that spring to mind:

  • The duration of the showers in the morning.
  • The ambient temperature in the loft.
  • How much washing up we’ve done.

The image above shows the basic operation of an unvented cylinder. Cold water enters the tank at the bottom and hot water exits at the top. The coil inside the tank is connected to the boiler and is responsible for actually heating the water in the tank. The whole thing works off the principal that the hot water will sit on top of the cold water and be pushed out the top.

My plan, therefore, was to attach temperature probes to the cold water entry and hot water exit and see what I could measure.

I was also interested in trying to measure the effect of the loft temperature on heat loss.

The Sensor

With my requirement of three sensors, I set about building a circuit that included an ESP-8266 and three DS18B20 temperature probes.

The DS18B20 uses something called the OneWire protocol, which means you can connect multiple sensors in parallel and use a single GPIO to read all the values. I found an excellent tutorial on the subject here – https://lastminuteengineers.com/multiple-ds18b20-arduino-tutorial/

With the long DS18B20 probes attached
The three temperature probes connected to the D1

For power, I wanted to use an off the shelf power supply. As these are mostly 5v, I added an LV1117V33 LDO. This little device accepts up to 7v input and outputs 3.3v, which was perfect to power the D1 and the sensors. To this, I connected a barrel input.

Powering the device with a 5v power supply

Software

I had originally planning on writing my own software to read the values from the sensors and send via MQTT to my HA instance. I actually did write my own software and even got the auto-discovery working, but as I wanted a web UI running on the device, so I could change easily change WiFi and MQTT settings, going down the custom road made little sense.

I was vagly aware that the excellent Tasmota Firmware (which I use in lots of smart sockets) did have some form of sensor support, so I did some reading. It turns out that not only did it support the DS18B20 sensor, it supported multiple ones 🙂

I flashed it onto the D1 mini and, after setup, navigated to the Configuration page.

You can select Module type as Generic (18) to gain access to the various GPIO pins on the device. I had connected the data wire of my sensors to D4, which is GPIO2. I choose teh DS18x20 option.

Setting up GPIO2 to handle the temp sensors

Once I hit save, the device rebooted and, lo and behold, the temperature started flowing into the UI!

Housing

Knowing how dusty my attic can be, I wanted to house the electronics in something, just to keep it clean and tidy. I had some Hammond enclosures (https://www.hammfg.com/electronics/small-case) and one of them seemed perfect.

I drilled four holes. One for each of the temperature probes and one for the power connector. The probes are held by three IP68 glands. Not necessary at all, but I just had them and thought they’d suit.

All the holes drilled and the sensor glands added
Circuit board installed
Lid on.

Once I had it all assembled, I plugged it in. It was at this point I wished I had some sort of indicator light to tell me it was powered or connected to the internet. I had to wait a minute, but eventually the Tasmota appeared on my list of connected clients.

Installation

I have a pretty standard unvented cylinder. There are two pipes entering at the bottom, the cold water inlet and the hot water return to the boiler. The hot water outlet is at the top.

My hot water tank in all its glory!
I stuffed one of the probes under the lagging at the top
The cold water inlet at the back – It isn’t even fully lagged!
I stuff another probe up against the cold water.

The third probe I just left hanging in the air, supported by a piece of furniture. My loft really is full of junk!

The data

The reading from the probes

From the three probes, I could have a guess as to which probe was were!

The 15.8 was the ambient temperature in the loft, with the 19 and 48 degree measures pointing to the inline and outlets.

Measures taken first thing in the morning

My hot water turns on for 30 minutes each morning (30 mins because the Nest Thermostat won’t let me do anything shorter). I looked at the temperatures when I got up around 7am and I could see the top temp was at almost 60 degrees.

Next Steps

I’m going to let this setup run for a few weeks and note the days we run low/out of hot water.

Combined with better control of my hot water (I’m thinking of replacing my Nest’s control), I think there is an opportunity to reduce the time my hot water needs to actually be turned on.

Other things might be possible, such as extra hot water when we have guests over (using their connection to the house WiFi as an indicator of presence). Or ensuring these is hot water for washing the dishes at 6pm each night.

I’ll post an update, once I’ve got some findings!

Update 24th Feb

With the sensor installed for over a week, I pulled a graph from Home Assistant.

A week’s worth of readings

The green line, which reaches the highest values is the outlet temperature. You can see the rather pronounced peaks and troughs. The orange line represents the inlet temp and that does spike a little, each morning, when the heating is turned on. The red line is the ambient temp of the loft. It does vary by a few degrees.

From the graph, it’s very obvious when we’re very low on hot water!

I also find the temperature loss over the course of the day to be interesting. This is obviously a function of the ambient temperature in the loft, but also, I suppose, of the temperature of the incoming water.

I’m in the process of upgrading my Home Assistant instance and I hope to be able to retain more data than I can do right now. I’d like to be able to compare summer months to winter months, so I need a few months of stored data to perform that comparison.

More to come!

Shelly 1 Relay

With Den Automation facing immanent demise, I’ve been looking at alternatives.

I came across the Shelly 1 relay, quite by accident, whilst looking at some Sonoff stuff. It immediately piqued my interest, mostly because it taught me that I didn’t know anything about lighting circuits in the UK.

I’ve installed my fair share of lighting fixtures during the rennovation of my home and whilst I was astonished at the number of connections in a ceiling rose, I never really took any time to investigate why.

Image result for ceiling rose wiring
Typical ceiling rose wiring arrangement in UK home

I had made my own attempts at automating lighting when I first moved in. This was a crude affair, which involved putting a Sonoff Basic relay in between the rose and bulb.

Whilst functional, the downside was that flipping the physical switch turned the relay off. Bye, bye, remote switching.

This is where the Shelly 1 comes into play.

Shelly 1 Wi-Fi WLAN switching actuator 16 A SHELLY SHELLY 1
The Shelly 1 relay. Small, but powerful 🙂

I came across an excellent post, which explained how to connect the little unit to a UK ceiling rose, which is well worth checking out – https://gist.github.com/lordneon/aecf24035a4dc5e6b950977e37aeb930

Essentially, the Shelly 1 is connected to the permanent supply, so it’s always on, regardless of the physical switch. The Shelly is also connected to the physical switch, so it knows when it’s been flipped.

This means that the light can be turned on and off via an API *and* by the existing light switch! I immediately ordered some.

A Christmas tree of relays.

My order included two Sonoff Minis, but I choose the Shelly1 to begin with as it supports MQTT control out of the box (which bypasses the Shelly Cloud – local control all the way). I know the Minis have to be flashed with Tasmota firmware to support MQTT and I didn’t want to get distracted!

Installation

WARNING! Please don’t attempt anything related to mains electricity unless your confident in your ability. Always remember to isolate circuits from the fuse board before doing anything!! I am not responsible if anything happens.

Installation was very straight forward with the help of the excellent WAGO connectors. Following the diagram, I installed the Shelly 1.

Wiring Diagram
https://gist.github.com/lordneon/aecf24035a4dc5e6b950977e37aeb930
The Shelly 1 wired in!

Once I’d restored power from the fuse board, I went through the configuration process. This followed the standard pattern of connecting to the Shelly’s WiFi network, connecting to my own WiFi network and then entering the MQTT server address and credentials.

It worked first time!

I set it’s Switched Live mode to edge, so that means that any change in the physical switch will turn it on or off, perfect for my needs.

Adding the Shelly to my Home Assistant required the addition of an MQTT entity with the appropriate topics.

  - platform: mqtt
    name: "Shelly-1"
    state_topic: "shellies/shelly1-B8B728/relay/0"
    command_topic: "shellies/shelly1-B8B728/relay/0/command"
    payload_on: "on"
    payload_off: "off"   

I repeated the process on another light in my house.

This was installed into a standard plastic ceiling rose, but unfortunately the cover wouldn’t close due to the thickness of the Shelly.

As this is pretty unsightly, I’ve decided to find a nice ceiling rose with the space to hold the Shelly before I install any more.

I’m so happy with these, they are a perfect replacement for my defunct Den switches. I’ll be replacing all of Den’s switches with these Shelly 1s once I find a good rose.

Den Automation and the Cloud

A few months ago, Den Automation Ltd appeared to drop off the face of the earth. My hub disconnected from the internet and the Den app would only work whilst I was at home.

I should point out that I invested in Den via Seedrs.

Continued Operation is possible

I took time to try and understand the protocol and eventually managed to write some code that allowed me to continuing using them. I created a Flow in my Home Automation installation using NodeRed. This essentially bridged messages from my Hass over to my Den Hub.

This has been reasonable successful, but there are some issues.

#1 The Den Hub needs to be restarted every week or so. I don’t know what’s happening, but I suspect the MQTT server they are using has a memory leak. After a few days of successful operation, it disconnects from NodeRed and refuses new connections. A restart clears this.

#2 Speed is inconsistent. Sometimes operating a light or socket from within HA will result in speedy action from the Den unit. Sometimes it takes several seconds and sometimes nothing happens.

#3 Disconnections from the switches and sockets. This has happened my bathroom switch. It just no longer operates or sends its state. I guess it has lost contact with the Hub. I might try and reset it, but I think that’s probably a crap shoot since I have no App to guide the process.

Liquidation

Den Automation Ltd. are officially in liquidation.

As somebody who invested in both shares and product, I’ve been hit twice.

I’m also in the process of trying to recover some of the money I spend on pre-orders, as some of the items were never delivered. I’ve submitted the paperwork, but I have yet to get a response.

End of the line

With Den officially in Liquidation, there is no longer any hope for the company.

I’ve also decided to start the process of removing all their products from my home. First to go was the double socket connected to my Kitchen’s TV. I’ve replaced this with a Sonoff switch (flashed with Tasmota).

I’ve had a great experience with the Shelly 1 relay product, so my plan is to remove all the Den light switches upstairs and restore the original switches, with Shelly relays going into the lights.

Cloud Control

The loss of Den’s cloud crippled the product for most people who had purchased it. I was lucky that I was had the knowledge and tools to determine the local protocol and leverage it from my own ends. Sadly, I wasn’t able to help anyone else.

Den’s local control ability was the primary thing that attracted me to their product line. The relied on the Cloud for user account’s and when that went, their app became useless, actually logging the users out. Not as “local” as I had hoped.

Sonos’s recent announcement about their hardware has made me come to the conclusion that any hardware that needs a Cloud is bad. It’s hardware you own, but don’t own. I’ve got £250 of Den products that I can’t really use because a company was poorly managed, despite the product itself being fully operational.

Google did the same thing a few years back killing off a smart home company called Revolv. Pulled the plug and turned off all the hardware.

I’ve got some Nest products in my house and all this has me wondering how long they will last!

Den Automation – API

With the very sad news that Den Automation is having financial troubles, I decided it was time to revisit my investigation into the protocol that Den uses.

I had read on their website that they used both a cloud based and local network approach. In fact, this was one of the things that appealed to me most. Offering local control is an important aspect in any smart home as it insulates you from internet outages.

I had looked into their protocol a few weeks after setting up my Den hub, but it turned up a dead end for me.

Based on what I’d learned my own adventures in IoT, I started working under the assumption that the local protocol would be HTTP based and use probably use mDNS. Using Charles, the populate iOS proxy, I was able to capture some of the traffic from my phone whilst using the Den app.

Unfortunately, all the requests were targeted at Den’s cloud. I was able to see the details of my deployment, the various switches and sockets, but I wasn’t too interested in taking this approach, since I had Google Home and Alexa integrations already setup.

I fired up WireShark and used Apple’s guide on how to intercept all traffic from my iPhone – https://developer.apple.com/documentation/network/recording_a_packet_trace

I immediately spotted a HTTP request, which was a protocol switch request. More digging and some Google-fu and I leaned that this was just MQTT over websockets on port 1884

I loaded up MQTT Explorer and fed in the details (IP address of my hub and Port 1884) but got a 401 (Authentication failed). This was progress as I knew some thing was listening.

Fast forward a few hours and I’d extracted this packet. Thanks to this protocol document ( http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/MQTT_V3.1_Protocol_Specific.pdf ) I was able to determine this was a connect request (0001), which contained a username and password.

The request to login, with a client name of MQTT and the username and password

Armed with some credentials, I fired up MQTT Explorer and was connected.

After a few minutes, I started to get information over the connection! MQTT Explorer includes the # and $SYS/# subscriptions, so it basically gets everything published by the hub.

I knew that the device was one of the double sockets (the TV was on) and it was sending measurements!

{“43de2329-8808-57bc-af5b-ea01bd6a6d11”:{“online”:true},”c8de59a4-29e5-52ad-84b5-2574f005544b”:{“software”:[{“type”:”USER_1″,”version”:”1.3.0″}],”battery_level”:29,”online”:false,”status”:{“0”:{“last_state_changed”:”2019-08-09T15:46:24Z”},”1″:{“last_state_changed”:”2019-08-09T15:46:24Z”}}},”7039ee52-d25a-59c5-b957-844ff067233f”:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:false,”status”:{“0”:{“state”:0,”shutter”:1,”nfc_tag”:{“uid”:”04:cd:be:d2:35:5e:80″,”atqa”:”0x44″,”sak”:”0x00″,”version”:0,”name”:””,”type”:”APPLIANCE”,”keep_on”:false,”timeout”:{“enabled”:false,”duration”:0}},”last_state_changed”:”2019-10-09T11:18:17Z”},”1″:{“state”:0,”shutter”:1,”nfc_tag”:{“uid”:”04:f2:97:ca:35:5e:80″,”atqa”:”0x44″,”sak”:”0x00″,”version”:0,”name”:””,”type”:”APPLIANCE”,”keep_on”:false,”timeout”:{“enabled”:false,”duration”:0}},”last_state_changed”:”2019-10-09T10:59:42Z”}}},”1cef1fcd-3aa7-5a49-80b4-ea98b85c5827″:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:true,”status”:{“0”:{“state”:1,”shutter”:1,”nfc_tag”:{“uid”:”04:61:8a:7a:33:5e:80″,”atqa”:”0x44″,”sak”:”0x00″,”version”:0,”name”:””,”type”:”APPLIANCE”,”keep_on”:false,”timeout”:{“enabled”:false,”duration”:0}},”last_state_changed”:”2019-10-09T18:04:06Z”},”1″:{“state”:1,”shutter”:1,”nfc_tag”:{“uid”:”04:5a:b2:d2:35:5e:81″,”atqa”:”0x44″,”sak”:”0x00″,”version”:0,”name”:””,”type”:”APPLIANCE”,”keep_on”:false,”timeout”:{“enabled”:false,”duration”:0}},”last_state_changed”:”2019-10-09T18:04:08Z”}}},”c0713ae1-1913-5ab7-944b-d94ea4b5477d”:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:false,”status”:{“0”:{“state”:0,”is_ready”:true,”last_state_changed”:”2019-10-09T17:30:34Z”}}},”4f832cb2-824f-58dc-87ad-5a89e121b653″:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:true,”status”:{“0”:{“state”:0,”is_ready”:true,”last_state_changed”:”2019-10-07T17:23:55Z”}}},”59607912-dda1-5e97-9e3d-6b065b9288db”:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:false,”status”:{“0”:{“state”:0,”is_ready”:true,”last_state_changed”:”2019-10-08T18:59:02Z”}}},”5a008677-5c2e-5405-a6b8-357a7e64642a”:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:true,”status”:{“0”:{“state”:0,”is_ready”:true,”last_state_changed”:”2019-10-08T21:19:37Z”}}},”8016c79f-8502-5b5c-b480-bb6361af1398″:{“software”:[{“type”:”USER_1″,”version”:”1.4.0″}],”online”:true,”status”:{“0”:{“state”:0,”shutter”:0,”nfc_tag”:null,”last_state_changed”:”2019-10-07T11:13:41Z”},”1″:{“state”:0,”shutter”:0,”nfc_tag”:null,”last_state_changed”:”2019-09-29T12:54:36Z”}}}}

Switching stuff on and off

After more digging around, I eventually capture the Publish command used to control the state of the individual devices.

A 142 bytes WebSocket message sent when I turned on a switch
The content is an MQTT publish (0011)

This enabled me to update the state and thus turn the switches on and off.

I’ve created a simple web app – https://denclient.azurewebsites.net/ – which can be used to control the connected hardware. You just need to use the same process as I did to sniff the packets.

At the time of writing this, the Den app was still working for me, which enabled me to perform some basic packet sniffing. Since I performed this investigation, the Den App is no longer working. I presume that AWS/Azure have killed the Den backend due to unpaid bills.

TS MK II – Mesh Network Pt.2

As part of my Temperature Sensor upgrade, I’ve started looking into Mesh Networking support.

Documenting my progress

Rather than try and put a post together at the end, I’m going to try and document my progress as I go, with additions to this post each time I do something significant.

The beginning

I recommend you read the first post before picking up here.

31th July

I carried on through the developer study guy and implemented the basic Light. The messaging flow is now making sense to me. I knew the Microbit’s 16KB of RAM wasn’t going to be enough for any real project, so I decided to bite the bulled and pick up a Adafruit nRF52832 device.

Ordered from www.coolcomponents.co.uk, they arrived pretty quickly. I plugged it in, downloaded the Bluefruit app from the app store and was able to discover and connect to the device pretty quickly. So far, so good.

Unfortunately, I couldn’t flash it my Zephyr project. More digging around, using the Nordic Tools and I discovered that the West tool couldn’t detect the device. Strange, as I could flash it with simple Arduino sketches. I trawled the documentation. Tried changing drivers. Updated the bootloader. Nothing had any effect.

During the umpteenth reading of the Zephyr docs, I finally realised why:

Flashing Zephyr onto the nrf52_adafruit_feather board requires an external J-Link programmer. The programmer is attached to the X1 SWD header.

https://docs.zephyrproject.org/latest/boards/arm/nrf52_adafruit_feather/doc/index.html

I have emphasized external in the quote above. I totally missed that key piece of information. More googling and I learn that Segger J-Link is a special protocol and that a small unit is required to interface your computer with the board! Everything fell into the place. The different drivers and the talk of the SWD header.

Unfortunately, my Adafruit boards don’t have a SWD header. Fortunately, the board has space and solder pads for one to be added.

The SWD header goes here

Adafruit sell the headers, but nobody in the UK had them available. I searched around and found something on RS that looked right.

I wasn’t prepared for the tininess of the header

I found a J-LINK device on The PiHut which I ordered, along with a short cable. I’ll wait for that to arrive before I attempt to solder this header onto the board. I’m going to need some steady hands!!

3rd August

J-Link Device arrived and connected up. Managed to solder on the header without any issue.

Added the RS 10 pin SWD header to my Adafruit nRF52

I think hooked up my newly purchased J-LINK device. Side node here – I ordered a serial cable with this as PiHut said it was recommended. One came in the box, so the extra one I ordered was redundant! I’ll be writing to them about that.

Hooked up (I should point out that this shows the correct cable orientation – see below)

Running the nordic command shows it has found the J-LINK connector!

Sadly, nothing is easy…

Big cryptic error trying to flash the device

My attempt to flash the hello_world sample failed. Ah man – with some help from this https://embeddedcomputing.weebly.com/segger-j-link-with-adafruit-feather-nrf52.html I realised I had the cable connected the wrong way around. I could find no information on the orientation of the header or cable, but I reversed it based on that pic and boom…..

SUCCESS!

Zephyr hello_world sample successfully written to device

It appeared that I had successfully flashed the device with the sample code.

Terminal output shows hello_world is working

Bloomin’ heck. It did actually work!

With the flash process *finally* working, I create a new “light” project, using what I’d learned from the BBC Microbit. I was then able to reuse the BBC Microbit switch.

I’d consider that a success! I think I’ve enough of a basic grasp of BLE Mesh networking to park this part of the project and move on!

TS Mk II – Mesh Network Pt.1

As part of my Temperature Sensor upgrade, I’ve started looking into Mesh Networking support.

I was aware that the Expressif ESP32 supports the new Bluetooth LE Mesh protocol, and as Bluetooth Low Energy would help stretch battery life, it seemed like the natural place to start my investigations.

I was initially going to try the WiFi mesh, but it didn’t really seem to suit devices like temperature sensor as they are actually asleep most of the time. If nodes in the mesh keep turning on and off, the other devices in the mesh would be constantly reorganizing themselves to plug the gaps. I do want to learn about ESP’s WiFi mesh, but for now, I’ll focus on Bluetooth

Documenting my progress

Rather than try and put a post together at the end, I’m going to try and document my progress as I go, with additions to this post each time I do something significant.

Getting started

I’ve already got the ESP-IDF environment setup and have two different dev boards. I’m going to try and approach this in stages

  • Copy one of the sample projects and get it running
  • Add another node
  • Send data from one node to another
  • Add MQTT and WiFi to get data out of the mesh

9th July 2019

I pulled down the ESP-IDF code from their mesh branch and spun up on the samples. After faffing around with the menuconfig settings, I got the Node example compiled and deployed onto the device. It also appeared appeared in the Nordic iOS Mesh App.

I found a good post on the provisioning process, https://www.novelbits.io/bluetooth-mesh-tutorial-part-3/ and this helped me make sense of what I was seeing.

Once I tapped on the node, I was able to tap “Identify”, which pulled back more information. I think this is what novelbits called “Invitation”. I then tapped “Provision” and after a few seconds, and a lot of logging from the device, I had the first Node in my mesh!

11th July

Now that I had my first mesh node, I quickly realised why the Expressif code was only at version 0.6 – none of the provisioning data was being saved. This meant that each time I flashed the device, I had to provision it again. This became annoying after the fifth time, so I started digging into the actual provisioning process, to see if I could put in something temporary.

I was digging around on google and I came across the Zephyr RTOS. In their examples, they had a simple 2 node mesh demo. The readme page states that no provisioning is required, and whilst it’s insecure, it’s sufficient for the demo. In their code, I could see a few lines of code where they manually provision the device with their own network and device keys.

static const u8_t net_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};
static const u8_t dev_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};
static const u8_t app_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};

err = bt_mesh_provision(net_key, net_idx, flags, iv_index, addr, dev_key);

This seemed ideal for my tinkering. I also found reference to the Zephyr code in Expressif’s news release about their BLE Mesh, where they state they are basing their implementation on top of the Zephyr . This meant I could probably find simple code in the esp-idf repo.

Running this code on the device resulted in it saying that provisioning was complete.

19th July

After more digging around the ESP-IDF examples, I came across some testing code, which appeared to provision using the esp code.

bt_mesh_device_auto_enter_network()

Try as I might, I couldn’t get that to work. Just kept reporting an error of -22.

UPDATE 13th August: During some experimenting with how to bridge the mesh network with MQTT, I managed to get this code working eventually, meaning:

static const u16_t net_idx;
static const u16_t app_idx;
static const u32_t iv_index;

struct bt_mesh_device_network_info info = {
        .net_key = { 
                    0x01,
                    0x23,
                    0x45,
                    0x67,
                    0x89,
                    0xab,
                    0xcd,
                    0xef,
                    0x01,
                    0x23,
                    0x45,
                    0x67,
                    0x89,
                    0xab,
                    0xcd,
                    0xef,
                    },
        .net_idx = net_idx,
        .flags = flags,
        .iv_index = iv_index,
        .unicast_addr = 0x0002,
        .dev_key = {
                    0x01,
                    0x23,
                    0x45,
                    0x67,
                    0x89,
                    0xab,
                    0xcd,
                    0xef,
                    0x01,
                    0x23,
                    0x45,
                    0x67,
                    0x89,
                    0xab,
                    0xcd,
                    0xef,
        },
        .app_key = {
		0x01,
		0x23,
		0x45,
		0x67,
		0x89,
		0xab,
		0xcd,
		0xef,
		0x01,
		0x23,
		0x45,
		0x67,
		0x89,
		0xab,
		0xcd,
		0xef,
},
        .app_idx = app_idx,
        .group_addr = 0xc000
    };

    err = bt_mesh_device_auto_enter_network(&info);

20th July

After more attempts to provision the device directly, I’ve decided to kinda give up. As I mentioned, the Zephyr code is easier for me to understand at this stage, so I’m going to try and get a build of the Zephyr samples running on my ESP32 boards.

23rd July

After hours of playing around trying to get the Zephyr SDK compiling against the ESP32 toolchain, I decided to stop. I found an excellent resource at over on the Bluetooth SIG website, called “An Introduction to Bluetooth Mesh Networking“. The samples in their course use the Zephyr SDK and the BBC Microbit. I had a glance though it and most of the main concepts are covered, so I think I’ll order some BBC Microbits and run through it.

I also came across an interest post at https://hutscape.com/, which is worth checking out. The author, Sayanee, uses an Adafruit board to build a UV sensor. Pretty cool. At £26 (and no Friend Node support in the Nordic SDK), I decided against buying any for the time being.

24th July

I got a BBC Microbit and tried the Zephyr mesh_demo code. After more mucking around, I managed to flash it with their code.

Building the mesh_demo firmware
The output from the microbit shows it working

As suggested by the Bluetooth study guide, I turned on the logging and then

Enable the logging and I use up too much RAM

The damn firmware, with logging, was too big for the BBC Microbit’s 16KB of ram. Bloody hell. Nothing is every easy. Some googling revealed that the small amount of SRAM makes it almost impossible to run the mesh. The Apollo Computer ran on 4KB of RAM 🙂

Hardware development is tough! That said, these microbits are perfect for me to dabble in the fundamentals of Bluetooth Mesh and to get to grips with the models. Hopefully there is enough space for me to try my own code. Perhaps it’s possible to compress the Zephyr RTOS runtime down a little??

25th July

Been reading about how to figure out the size of the runtime. You just run the command ninja rom_report

The mesh_demo code takes up just over 3% of the total size
The bluetooth code takes up 66%

I don’t know what those numbers actually mean. I had assumed bytes, but that would mean the bluetooth library, if measured in bytes, was around 50KB and I know that’s not right.

28th July

I experimented with the Zephyr mesh_demo sample, but couldn’t make it work. Kept getting error codes when trying to send messages. I tried to tweak it here and there, but I was just strumbling around in the dark. I returned, instead, to the Bluetooth developer guide and gave that go. The guide starts at the Switch module.

The sample code wasn’t up-to-date with the latest Zephyr code, so I had to tweak it somewhat, but I got it to compile and, amazingly, it worked.

One Microbit is the light, the other the switch
Turning the light on and off over BLE Mesh!

You can continue following my adventures here