TS – Mk II – Mesh Networking

As part of my Temperature Sensor upgrade, I’ve started looking into Mesh Networking support.

I was aware that the Expressif ESP32 supports the new Bluetooth LE Mesh protocol, and as Bluetooth Low Energy would help stretch battery life, it seemed like the natural place to start my investigations.

I was initially going to try the WiFi mesh, but it didn’t really seem to suit devices like temperature sensor as they are actually asleep most of the time. If nodes in the mesh keep turning on and off, the other devices in the mesh would be constantly reorganizing themselves to plug the gaps. I do want to learn about ESP’s WiFi mesh, but for now, I’ll focus on Bluetooth

Documenting my progress

Rather than try and put a post together at the end, I’m going to try and document my progress as I go, with additions to this post each time I do something significant.

Getting started

I’ve already got the ESP-IDF environment setup and have two different dev boards. I’m going to try and approach this in stages

  • Copy one of the sample projects and get it running
  • Add another node
  • Send data from one node to another
  • Add MQTT and WiFi to get data out of the mesh

9th July 2019

I pulled down the ESP-IDF code from their mesh branch and spun up on the samples. After faffing around with the menuconfig settings, I got the Node example compiled and deployed onto the device. It also appeared appeared in the Nordic iOS Mesh App.

I found a good post on the provisioning process, https://www.novelbits.io/bluetooth-mesh-tutorial-part-3/ and this helped me make sense of what I was seeing.

Once I tapped on the node, I was able to tap “Identify”, which pulled back more information. I think this is what novelbits called “Invitation”. I then tapped “Provision” and after a few seconds, and a lot of logging from the device, I had the first Node in my mesh!

11th July

Now that I had my first mesh node, I quickly realised why the Expressif code was only at version 0.6 – none of the provisioning data was being saved. This meant that each time I flashed the device, I had to provision it again. This became annoying after the fifth time, so I started digging into the actual provisioning process, to see if I could put in something temporary.

I was digging around on google and I came across the Zeyphr RTOS. In their examples, they had a simple 2 node mesh demo. The readme page states that no provisioning is required, and whilst it’s insecure, it’s sufficient for the demo. In their code, I could see a few lines of code where they manually provision the device with their own network and device keys.

static const u8_t net_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};
static const u8_t dev_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};
static const u8_t app_key[16] = {
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
	0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef,
};

err = bt_mesh_provision(net_key, net_idx, flags, iv_index, addr, dev_key);

This seemed ideal for my tinkering. I also found reference to the Zeyphr code in Expressif’s news release about their BLE Mesh, where they state they are basing their implementation on top of the Zeyphr. This meant I could probably find simple code in the esp-idf repo.

19th July

Advertisements

Temperature Sensor – Mk II

My first go at creating a simple battery powered temperature sensor was really interesting and fun to do, but, like any project, there are always way to make it better!

For my Mark II sensor, there were a few things I wanted to improve:

  • Use of Wifi Mesh or Bluetooth mesh
  • Support MQTT discovery in Hass.io
  • Longer batter life
  • Better housing of the electronics
  • Low battery notifications

Mesh Networking

I’m interesting in exploring the concept of mesh networking. Within the Expressif family of devices, there are two flavors that I’m aware of; WiFi mesh and Bluetooth Low Energy Mesh. The ESP32 platform supports both of these. My current Temperature Sensor is built upon the ESP8266, but moving to the ESP32 should be straight forward.

In Progress – .

MQTT Discovery

I use the Hass.io platform to manage my automation at home. It supports a simple MQTT interface for receiving data and sending commands. Typically, you would configure the devices within a config file, but it does support a discovery protocol, which allows devices to make themselves known to Hass and to provide all the information required for Hass to use them.

I would like my temperature sensors to support this mode, so that when they fire themselves up for the first time, they would register themselves automatically.

Not started.

Battery Life

Not started.

Housing

Not started.

Low battery notifications

Not started.

Set extractor fan to automatic – Part 1

A few months back, after we discovered mould growing on our bathroom ceiling, I replaced the wall mounted extractor fan with an in-line model, mounted in the loft. I never grasped how important the position of the van was in relation to the source of steam and the source of fresh air. Without going into too much detail, the new vent, directly over the shower, is doing its job.

My extractor fan, like most people’s, is wired to the lighting of the bathroom, so turning on the light also turns on the fan. I have two problems with this:

  • You need to have the light on, even if it’s the middle of the day.
  • The fan runs in the middle of the night after a trip to loo.

The first point is my preference. I like natural daylight, so it does annoy me having to turn on the lights. Opening the window just isn’t as effective as the fan. And whilst the in-line unit I purchased is very quiet, in the dead of night, you can hear it roaring away .

It would be nice to only have the fan on when needed e.g. having a shower. Some fans use humidity sensors, but inline fans don’t have that option. Besides, they require the humidity level to rise enough to trigger them, which kinda defeats the point. My wife suggested we add an extra switch to control it, but it’s not practical as I’m not able to get a wire down into the socket. I’ve started looked at some sort of smart alternative.

The plan

I’ve been thinking about a solution and I’ve got an idea that I think will work. There are three parts to it.

#1 Add a temperature sensor to the hot water pipe that feeds the shower. I’ve built one for my son’s room, battery powered, so we can see how hot/cold his room gets during the night. The temperature sensor could tell when the shower is running as the pipe would get up to 50 or 60 degrees Celsius.

#2 Disconnect the fan from the bathroom light and control it using an ESP8266 and a relay. This will allow it to be turned off and on via an MQTT instruction. I will also reduce the fans overrun to zero and control that via software instead.

#3 Create an automation to engage the extractor fan when the temperature sensor reaches a certain temp and to turn it off when the temperature drops back down.

The Temperature Sensor

For this project, I’m planning on using a variation on my own home made temperature sensor.

In order to have a fast response response to changes in temperature, the unit is going to have to be powered constantly. Fortunately, there is an easy access between the shower pipes and the cupboard under the stairs. I’ll be able to run a cable up there with ease.

The Fan Control

The fan that I’m using in my bathroom is a bog standard inline extractor. It takes three inputs. Mains power, live and neutral and a switch live, which is tied to the bathroom lights.

My plan was to use the existing mains power to power my control circuit and to put the switched live under the control of a simple relay, rather than the light switch. I will also use Hass to manage the fan’s overrun, rather than using the inbuilt circuit. This gives me more control and flexibility.

I have a few Hi Link transformers, which take mains voltage and spit out 5v. The Wemos D1 boards I have will take a 5v input and the small relays I have are also powered using 5v.

Converts mains voltage to a steady 5V DC, perfect for Wemos

The operation of the control would be pretty basic. Using MQTT, it would listen for a run command, the body of which would hold a count in minutes. When received, it would engage a relay to the switched live input of the fan. Once the specified number of minutes elapsed, it would disengage the relay and the fan would stop. If a run command was received whilst the fan was already running, the time would just be reset. If a stop command was received, the relay would automatically disengage.

The controller would also publish its current state when it was changed, something that isn’t necessary, but useful for keeping Hassio up-to-date with changes it didn’t cause!

I assembled a small prototype using some veroboard and a relay I had. It is a low trigger relay, which means it’s engaged when the the indicator input has a low voltage. This is a little annoying (it’s using power to engage the contact). I ordered a high trigger relay, but pressed on with the prototype just the same.

The initial prototype

Sadly, this protoype didn’t work! Everything powered up, but the program on the ESP8266 didn’t appear to boot up. Experience taught me this was because of a lack of power, but I expected no more than 300mA to be drawn by both the relay and D1 board.

I attached my bench supply to get a better view of the power consumption.

W1 and the relay (engaged) draw 347mA, well below the 0.6A rating

So it’s wasn’t a power issue. I poked around a little more and after reviewing the MQTT logs, I was happy that the code on the D1 was actually running correctly. This was puzzling. All information pointed to the relay being a problem.

I reconnected to my PC via USB and ran the code again. Everything worked perfectly.

As I’m using a low trigger relay, this means that the relay engages when the output PIN is set to ground. Using my multi meter, I measure the voltage across the PIN and got a value of 0.095V

USB power gives 0.095v when the output pin is set to LOW

Reconnecting my rectifier, I tested the voltage again and got a different value!


Bench power gives 0.11v when the output pin is set to LOW

The penny then dropped. What if the cut off was 0.1v? I quickly popped a resister in series and bingo, the relay engaged!

As I already knew, the low trigger relay didn’t align with my requirements, so aside from a fun diversion, this wasn’t a problem I needed to solve.

HASS control

With the fan control hardware prototype up and running, I turned my attention to Hass control. As I was using the MQTT protocol, this was very straight forward.

After some experimenting, I had to bin my approach of passing the running time as part of the body because of how MQTT works in HASS. Whilst HASS will issue MQTT commands for a particular device, it also listens for status changes from the device. The body of these two messages must match up. For example, if I issued START5, my board would respond with START5 and HASS would turn the fan yellow to indicate its running. If I send the MQTT command outside of HASS (MQTT directly, for example) with a body of START10, HASS won’t identify the fan has started and its state would be wrong. I don’t want to force myself into relying only on HASS to start the fan. Commands like START and STOP are sufficient enough for my needs.

It’s not the end of the world. The fan already has a fixed overun time, so I won’t be any worse off. In the future, I could look at another MQTT command that could dynamically change the overrun time.

Tapping this runs the fan

More to do

The idea was sound, but I have much more work to do.

  • Replace the replay with a high-trigger one
  • Connect the control unit to the extractor fan
  • Connect the temperature sensor to the hot water pipe

I will cover these in future posts on the subect!

Adventures in deep sleep with ESP8266 and a temperature sensor

One of the small ESP8266 projects I did was the creation of a simple temperature sensor.

I build a simple circuit, following the guidance from the following articles:

I opted to use a probe on a long wire rather than using the little component. I ordered some of these from Amazon – https://amzn.to/2IiYEQs. I thought these would give me more flexibility when it came to positioning the probe.

The plan is to build a couple of these sensors and put then around my house. Mostly out of curiosity. I want to see how the house warms and cools during the coarse of the day and what impact the heating & occupancy has (more sensors!)

I had come across this article – https://openhomeautomation.net/esp8266-battery – which describes how an ESP8266 can run for years on a large battery. Seemed very straight forward and I figured that I could use a 9v battery to power one of these units for a few months at the very least.

I built the circuit and wrote the code using VSCode and the amazing PlatformIO plug-in. Essentially, my code connects to my WiFi, connects to an MQTT server (running on Hassio) and sends the temperature value. It will then go into a deep sleep for five minutes and repeat the process. This felt straight forward.

I put the circuit together on a breadboard to ensure I got the temperature reading working. I then moved the assembly onto a small circuit board and soldered it all in.

The 8266 comes requires 3.3v to operate, but comes with an in-built power regulator allowing you to power it with anything between 3.3v and 12v.

This is what I connected my 9v battery to. Everything seemed to work perfectly and my Hassio instance was recording the temperature coming from the sensor.

Temperature logged every 30 seconds until about 6am

Leaving it over night, I checked it in the morning. The temperature was logged every 30 seconds (I initially went for this to test its behaviour) until about 6am, when it went dead.

Some quick investigation showed the 9v battery had drained too much and the ESP8266 was no longer able to connect to the WiFi. That wasn’t what I expected. I knew it would use power when it was connected to the WiFi etc. but to drain a battery in a day? I decided to try and work it out.

First, let’s assume it’s running all the time. According to my multi meter, it’s drawing ~80mA.

This didn’t seem like a whole lot, until I checked this page – http://www.techlib.com/reference/batteries.html – and discovered that there are only 500mA hours in one of those batteries. So, at 80mA draw, the battery would supply the ESP8266 for 500/80 = 6.

Wow. Six hours wasn’t what I had in mind!

I’m not reading the temperature all the time, using a deep sleep as I mentioned before, to reduce its power usage between temperature reads. So if we assume it draws next to nothing when its asleep, let’s work out the power consumption over the course of an hour.

It takes about ten seconds to connect to the WiFi, connect to MQTT and send the temperature reading. This happens every thirty seconds, to in total, it’s essentially forty seconds between each read Read (10) | Sleep (30) | Read (10) etc.

So in an hour, we’ll take 90 readings.

This means we run at ~80mA for 90 x 10s or 15 minutes, per hour. This means our 500mA battery should run for about 24 hours. Except, it didn’t.

Using the multi meter again, I discovered the 8266 was drawing ~11mA when it was in deep sleep.

This was much, much higher than the power consumption outlined by the articles I had read online. Some further researching showed other people have the same problem.

It means the 8266 draws 11mA for 45 minutes of every hour. Out 500mA battery could only run for about forty five hours. That’s not even two days. It should be able to run for over a year on a battery when it’s asleep.

More reading on t’internet about the NodeMCU and one of the comments in this post (https://openhomeautomation.net/esp8266-battery) mentions the voltage regulator on the board draws ~11mA. Something called
Quiescent Current. I was supplying the board with 9v input to the GND/VIN pins, which would require the voltage regulator to kick in. Could this be the problem?

Using my bench power supply, I connected it directly to the 3.3v input and set the voltage to a constant 3.3v. I also removed the temperature sensor hardware and replaced it with a fixed software measurement to ensure no current was drawn through the board.

Zero Effect!

Still drawing 11mA in deep sleep. Note discrepancy between bench and multi-meter

Using the bench power supply did make me aware of one thing. The current readings were different across the two devices.

I had received delivery of some Wemos D1 modules, which other people had reported some success with. I wired up one of these and loaded on the same program (with temperature reading code completely removed).

According to my multi meter, it was using over 150mA, but wasn’t actually working (no message being sent via MQTT). This threw me completely!

In the spirit of clueless experimentation, I moved the input on the multi-meter from mA to 10A and, lo and behold, the current dropped and my MQTT subscriber started getting data. I got some help from @themainframe via Twitter and he told me this down to impedance and quite normal. I made a note to research this topic a little further.

Sending the temperature uses some power

The deep sleep also dropped the current being drawn to a level so low that it didn’t register on my bench supply. Result!!

Nothing being consumed in Deep Sleep

The next step was to re-enable the code and circuity that actually took the temperature reading. In my original design, I was feeding power to the probe from a 3.3v output on the ESP dev board. Following the circuits others designed, this time around I would power the circuit from the power supply directly.

Temperature probe connected to directly to power supply

This worked like a treat! Measuring the current, I could see the draw whilst it was measuring and transmitting the temperature was very low.

Really small power draw

The next step was moving from bench power to battery power. I recalled another blog post where the author had used a very large lithium ion battery to power his IoT device. I found the same battery and ordered a pair and also got some holders for them. It’s an 18650 type, which is 3.7v with a large 2000mAh capacity.

I also needed a way to drop from 3.7v to 3.3v. I had used a voltage regulator on my smart lamp project (blog post pending) so that seemed like what I wanted. I came across the MCP1700-3302E regulator, which seemed perfect. They seemed like they would deliver the 3.3v I needed from the 3.7v battery. They were also able to handle 250mA, which was more than enough for my setup.

Putting it all together and I had a working circuit!

Battery powered temperature with deep sleep (finally working!)

Next step was reducing this down to fit in the little box. I bought some veroboard and set to work.

Using some vero board to create a complete circuit.
I used pin holders rather than soldering the D1 directly to the board.
With battery and sensor attached
The board and battery back into the original box.

Leaving the unit over night and logging to the same Hass.io queue, I was delighted to discover when I came down in the morning that the device was still transmitting. It was interesting to see my kitchen falling to around 15 degrees.

The raw data being written
A smoother graph in Hass, which looks nicer!

With it all packaged up, the last step is to charge the LiPo battery fully before I “deploy” the device.

Once it’s been running for a week or so, I’ll have a much better idea of how reliable it is and then I will look at the next steps. I want to design a PCB for the sensor and get a better box to house it all. The software could also be improved, for example, only sending an updated temperature if it has actually changed. I would also like a way to dynamically alter the sleep period, for example, checking very 30 minutes between midnight and 5am.

It has been interesting to get back into some electronics. I forgot how much I really enjoy experimenting and learning with physical stuff, not just software!

Den – Installing a smart socket

As part of the Den Automation range, they ship a 2 gang smart socket. It has the ability to be switched on remotely, to determine what’s plugged in and to even monitor the energy usage of the item plugged in.

I pre-ordered a few of these units as I saw them being used to turn my TVs off at the wall, rather than use their own standby. I currently achieve this using a few Sonoff Plugs flashed with Tasmota firmware. These are great, but are ugly as hell.

To get the ball rolling, I wanted to test out Den’s Smart Tag feature, the one that knows what is plugged in. I thought this best on a socket that is regularly used. For me, that’s one in my bedroom where my wife plugs in her hair dryer and hair straighteners. I thought this would be a good testing ground!

Installation was very simple. **** DON’T ATTEMPT THIS UNLESS YOUR CONFIDENT YOU KNOW WHAT YOU’RE DOING! REMEMBER TO ISOLATE EVERYTHING AT THE FUSE BOARD AND CHECK THE SOCKET IS DEAD BEFORE TAKING THE COVER OFF. GET A QUALIFIED ELECTRICIAN IF YOU HAVE ANY DOUBT ****

A classic, but dumb, double gang socket!
Simply wiring with two lives, two neutrals and two earths
The inputs on the Den Socket
The socket installed and looking good!

With the socket installed, I tested the manual operation to ensure everything was okay.

The app doesn’t really deal with empty screens all that well!

The pairing process starts the same as the switch. Hold the Den button for a few seconds until the light starts blinking and start the process in the app. It paired very quickly.

Once it was connected, it showed that two sockets individually. Tapping the little socket flicked the physical switch on and off. The sockets don’t have the same 45 second recharge time that the light switches have since these sockets have constant power. A limitation of the lack of neutral in UK light switches. Den still deserves credit for solving that in their light switches.

Once I’d finished this part, the app prompted me to add some smart tags.

The Den Socket comes with five Smart Tags. You can see this in my unboxing post.

I found my wife’s hair dryer and straighteners and popped a tag onto each plug.

I think plugged in one of them.

The part at the bottom is interesting – if the appliance is left on for a specified number of minutes, Den can turn it off automatically. They use hair straighteners as their example for this, so I turned it on, setting ten minutes.

I repeated the process for a hair dryer.

These two items then show up in the Appliance section of the app. It knows *where* it’s plugged in and when something is unplugged.

I haven’t tested the notify if left on feature yet, so I’ll post something when I get a chance to test that.

I was also disappointed to find out that the energy usage feature isn’t currently available. I would have expected them to include something pretty basic (current watt consumption or similar). I’ll update this post when they release that.

I have two more of these units to install. The next one goes onto the TV in the sitting room. I’m going to pair my Den Remote with those sockets, making it easier to control the TV.

I’ll do another post on that in the future.

Den – Installing a light switch

In a previous post, I covered the Den Automation Smart Hub setup. Next up, I’d like to cover the installation and setup of their 1 gang light switch.

I ordered four of these, to cover the rooms upstairs in my house. I’ll post the processing I went through to install one of the switches.

WARNING – CHANGING THESE SWITCHES REQUIRES EXPOSE TO MAINS VOLTAGE, WHICH IS EXTREMELY DANGEROUS. TAKE THE NECESSARY PRECAUTIONS. I AM NOT RESPONSIBLE FOR ANY INJURY THAT MAY RESULT. REMEMBER TO ISOLATE YOUR SWITCHES AT THE FUSE BOARD. IF IN DOUBT, GET AN ELECTRICIAN!

The humble light switch. Please ignore the filth of my walls!

Having turned off the lights at the fuse board (see my stern warning above) I removed the face plate.

A simple switch. Brown is the live and the blue is the switched live. The box is earthed too.

Den Switches require an earth connection to function (I’m guessing they use the voltage across live and earth to charge their capacitor, keeping the leaked current so low it doesn’t trip the RCD – clever sods!)

The Den switch has four inputs, Earth, Live, Switched Live 1 and Switched Live 2.

Den provide a small Earth cable that acts as an extensions, so I wired that into the backing box.

I added the Den earth extension into the earth connector of the backing box.

The backing box on my switches isn’t deep enough to accommodate the Den switch, so I had to use the provided spacer. I hung that over the wires.

I then connected it up and screwed it in place (using the longer screws provided by Den)

The Den Switch after installation

After restoring the power at the fuse board, I was able to turn the light on and off. Pretty useful to know that still worked 🙂

To pair the switch with the Hub, I launched the Den App and, under the settings part, opted to add a new light switch.

The app starts to guide you through the process, similar to the Hub setup
A soft light starts to blink on the switch after holding down the button

The pairing process took about fifteen seconds.
Customising your new light switch

Name the light using their own names or create your own
Add it to a new room….
Choose an existing room
Done

The installation process was quick and painless.

Unfortunately, after I’d installed mine and I was cleaning it down, I noticed it didn’t quite fit correctly:

An imperfection in the switch prevented it sitting flush against spacer

I wrote to Den’s support email and, whilst it took two days to get an actual response, they dispatched a replacement which I received the next day. I swapped them out and the replacement was perfectly fine. I would have liked a faster reply from support, but they did resolve the issue without any question (I just included the above picture).

I swapped out four light switches upstairs, so I can now control all the bedroom lights.

Firmware update messages are kinda useless as you don’t know the room!

One point of complaint that I still has is that one of the switches hasn’t updated its firmware yet. Three updated to the 1.3 firmware without any issues, but the forth is still stuck on the 1.1 firmware. I need to contact support about this.

Den – The Smart Hub

The Den Smart Hub is the core component of Den’s smart home offering. You need to have one in order to use any of their other products.

The Hub is responsible for remote access and firmware upgrades. It’s worth noting that the Den products will operate *without* an internet connection (once initially setup). This is something that’s important to me. I find it really frustrating that an internet outage renders most smart home stuff useless. Den have addressed that and state the app etc. will work when on the same network as the Hub. I haven’t tried this out myself, but intend to.

Setting Up

Once you’ve signed into the Den app, the first thing it asks you to do is setup your Hub. I’ve covered the unboxing of the Hub in an earlier post.

I plugged my Hub in and connected the Ethernet cable to one of my Google WiFi units. For most users, I expect they’ll just plug the Hub directly into their WiFi router.

Of course, I immediately tried to pair and failed miserably.

I then read the letter that Den supplied with my order and there, they clearly state, that I should give it 10 minutes after plugging it in to update itself. So I made a cup of tea and sat at my table to drink it.

After scanning again, it found the Hub and asked me what room it was in. I just chose Kitchen as I was there and hit finish.

Conclusion

Simple as that. The process was simple. A small indicator within the app informing me of a firmware update might have been a useful and it’s something I’ve suggested to the Den team.

Next up, install a Den light switch!