Laundry Spy, Part 2: The infrastructure

Posted in Articles, ESP8266, Raspberry Pi, Tutorials

When we left off, we had built a weird two‐tentacled sea creature without a brain. Despite my hasty promise at the end of Part 1, I’m not going to give it a brain just yet, because I can always turn a two‐part series into a four‐part series merely by overthinking it.

To refresh your memory: the thing we built last time is what I’ll be calling a “laundry spy” for short. It’s an ESP8266 (an Arduino‐like microcontroller with built‐in WiFi) with two accelerometers tethered to it — one for the washing machine and one for the dryer. It will make intelligent guesses about when a load of laundry goes through those appliances based on how much vibration it detects from those two sensors.

To help the laundry spy notify me when a cycle is done, I’m going to enlist an intermediary: a Raspberry Pi running Node‐RED home automation software. This will take some work off the laundry spy’s hands; instead of needing to know how to notify my phone, it can simply report data to the Pi using a simple messaging protocol called MQTT. In effect, all the laundry spy will do is say “hey, Pi, I think the dryer’s done,” and then the Pi can decide what the next steps are.

Today I’m going to try to give you an abridged primer on Node‐RED and MQTT. The seasoned IoT veterans might want to skim this article.

How does the laundry spy talk to my phone?

Let’s go back to the beginning. How do I make my own phone buzz programmatically? I’ve got options. I could use Twilio and send my phone an SMS, for instance. But instead I landed on Pushover, which does one thing incredibly well: it shows arbitrary notifications of your choosing on your iPhone, Android phone, or computer (or all three).

It’s got an API, but odds are good that your platform of choice already has some sort of Pushover integration. Sending notifications is free; you pay a one‐time fee of $4.99 for the app that receives them on a particular platform. (For instance: spending $4.99 on iOS Pushover enables notifications across any iOS devices you own, and spending $4.99 on Pushover’s website for the desktop app enables them across any desktop computers you own.)

Notifications themselves are free as long as you don’t send more than 7,500 a month, and if you’re using Pushover just for household life hacks like these, you won’t come anywhere near this limit.

External service

So now the problem becomes: how do we connect the laundry spy to Pushover? Well, IFTTT or Zapier would be one way to go. In fact, the very first version of the laundry spy used a Zapier workflow that was triggered by a webhook; the ESP8266 would request a specific URL with one parameter (“washer” or “dryer”) that Zapier would use to build the text of a notification.

This approach isn’t as flexible, but if you’re already an enthusiastic user of one of these services, it might be closer to your wheelhouse.

It’s even possible (if not ideal) for the ESP8266 to talk to the Pushover API directly — it can make HTTP requests, after all. Later on I’ll elaborate on why I decided against this, but it’s a legitimate option.

A server on your network

But if you’re like me, you’ve already got a tiny computer somewhere in your house acting as an IoT hub — and if you’re not like me, you ought to be. I’ve got a Raspberry Pi 3 in my office running Node‐RED. I love it.

Node‐RED, as the name implies, is written in JavaScript, but you don’t need to know JavaScript to use it. You can build IFTTT‐like workflows inside it with a flowchart‐like UI that makes me nostalgic for the departed Yahoo! Pipes.

If it’s not your cup of tea, or if you’ve already thrown your lot in with a different IoT platform, then the implementation details will be different, but the approach will be very similar. You’ll be able to do all this in HomeAssistant, OpenHAB, Domoticz, or even just ~50 lines of your favorite programming language running as a daemon.

MQTT

OK. Fine. The goalposts move once again: how will the laundry spy talk to our IoT hub?

I don’t know how or why MQTT got itself established as the go‐to protocol for IoT things, but I do know it’s easy as hell to use. On ESP8266 you can use the PubSubClient library to connect to an arbitrary MQTT server and publish data to a certain channel that other clients can subscribe to.

Why use MQTT instead of, say, HTTP? After all, with Node‐RED you could pretty easily build a simple API for the laundry spy to make requests to. But MQTT is somehow even easier to use. I’ll illustrate.

Publishing to channels

Say I’m running an MQTT server at 10.0.0.1, and that I’ve configured the server to want a username of some_user_name and a password of some_password. (Don’t do this.) For testing purposes, I can install Mosquitto on my Mac with Homebrew (brew install mosquitto) and then publish a message to my server with a command like this:

mosquitto_pub -p 1883 -h 10.0.0.1 \
  -u some_user_name -P some_password \
  -t "some/channel" -m "aha" -r

That’ll publish the string aha to the channel named some/channel. Nothing will happen because nothing is subscribed to that channel, but if it doesn’t error out on you, you know that your MQTT server is up and running.

Now open a second terminal window and run this command:

mosquitto_sub -p 1883 -h 10.0.0.1 \
  -u some_user_name -P some_password \
  -t "some/channel"

You should immediately see the aha you sent a minute ago.

Arrange your two terminal windows so you can see both at the same time. Then, in the first window, run:

mosquitto_pub -p 1883 -h 10.0.0.1 \
  -u some_user_name -P some_password \
  -t "some/channel" -m "oho"

That oho message should appear instantaneously in the second window.

Node‐RED

Node‐RED has a specific documentation page for installation on Raspberry Pi. If you use a recent version of Raspbian, it’ll already be installed on your Pi, and you’ll merely need to schedule it to run on boot.

The visual environment of Node‐RED makes it intuitive to connect triggers to outcomes. Let’s play around a little bit.

Subscribing to MQTT topics
Node-RED MQTT node
An MQTT node.

MQTT support is built into Node‐RED. In the sidebar, under the “Input” heading, you’ll see an “MQTT” node. Drag it into the workflow, then double‐click it to configure it:

Configuration for the MQTT subscriber node.
  • The first time you configure an MQTT node, you’ll have to add an MQTT server. In our case, Node‐RED is running on the same computer as the MQTT server, so you’d specify localhost and the default port of 1883, assuming you didn’t change it.
  • After setting the server, you’ll set the topic (or channel, as I sometimes call it absent‐mindedly) for the node to listen on. The topic is just a string of one’s choosing; all that matters is that the publisher and subscriber agree on it. Earlier we used some/channel in our example.
  • The QoS field can be left on its default; the intricacies of at‐least‐once versus at‐most‐once are not interesting for our purposes.
  • The name field just determines how the node is labeled in the workflow view.

This node has a handle on its right side, which means it can output data. When messages come across the configured topic, this node will pass them along to other nodes in the workflow. One node can be connected to any number of other nodes.

Integrating with Pushover

For Pushover integration, we need to install a plugin. Click on the hamburger menu at top‐right, select “Manage palette,” click on the “install” tab, and search for pushover. After installing the node-red-node-pushover package, click on Done and you’ll have a new node in your palette under the “mobile” heading.

Pushover plugin for Node‐RED.

Drag it into your workflow, then double‐click on it.

You’ll need to configure two things here:

  • To send notifications via Pushover, you’ll need to create an app on Pushover’s site. Once you do, you’ll get an API key that you can pass along to Node‐RED. (Mine is just called “Node‐RED” — it’s not going to be visible to anyone, and I doubt its even needs to be globally unique.)
  • To receive notifications via Pushover, you’ll need to give it your user key, which is visible on the Pushover home page once you’re logged in.

You can notify several different users with the same app. For instance, I’m notifying two people in my household, so I’d create two Pushover nodes — one for me and one for my girlfriend. The nodes will have different user tokens but can use the same API key. (It doesn’t matter which of us owns the app.)

Now you’ve got two nodes: one that produces output and one that accepts input. Do you see where I’m going with this? Connect the MQTT node’s output to the Pushover node’s input. The Deploy button in the corner of the screen will activate the workflow. Once it’s deployed, you can publish text to the topic you configured and make it show up on screens of your choosing!

The fun stuff

Node‐RED workflows can be serialized and shared, so let me just dump some JSON in your lap. Here’s a workflow that describes what we want to happen. To import it, go to hamburger menuImportClipboard… and paste the contents into the text field. Then you can double‐click on each node to change the values as necessary.

Be sure to name your nodes descriptively as a favor to your future self.

Some notes:

  • The “done?” node will decide whether to pass a message along based on its contents. You’ll notice that we’re only letting messages through whose payload is 4. Why 4? Because the state machine we’ll build in part three has five states, numbered from 0 to 4, and the last state is “Done.” Just go with it for now.
  • The “Format message” node is a function node. Function nodes can accept arbitrary JavaScript, so I abuse the hell out of them. Inside, you’ll have access to a msg object which represents the incoming message, and if you return an object, it’ll be sent to the next node in the flow. You can modify the existing message or build one from scratch as we’re doing here.
  • The Pushover node will use the topic and payload properties of the message it receives as the title and body of the notification, respectively. We don’t need to set any of the other fields. But don’t forget to fill in your user key and API token as we discussed earlier.

Once you’ve got it configured, click on Deploy and the workflow will be live. Node‐RED will subscribe to two topics named laundry/washer/state and laundry/dryer/state. Now, we haven’t set up anything to publish to those topics yet, but that’s fine. MQTT lets you listen to a topic even if nothing has ever published to it before, and even if it never will, and that’s pretty sad and poignant if you think about it.

But I digress! You can test whether the workflow does what you expect by publishing a message to one of those channels, much like we did before:

mosquitto_pub -p 1883 -h 10.0.0.1 \
  -u some_user_name -P some_password \
  -t "laundry/washer/state" -m "4"

If you’re one of those annoying people who doesn’t screw anything up, running this command will make your phone buzz. If it doesn’t, then you can wade back into the workflow to find out what went wrong. There’s a “debug” node in the palette; attach it to various places in the workflow to see if something is getting stuck. The “Debug” tab in the right‐hand pane will display any messages that arrive at active debug nodes. (Once you connect a debug node to the workflow, you have to click Deploy for it to take effect. I still forget to do it half the time.)

If we attach the debug node here, we can spy on everything that the laundry/washer/state MQTT node emits.

Why don’t we just talk to Pushover directly?

The more pragmatic among you might think I’m overcomplicating this task. I mean, the ESP8266 can see the internet, can’t it? Why have an intermediary at all?

A few reasons:

  1. Decoupling is good. Today I want a notification on my phone when the cycle is done; tomorrow I might want to add a sound effect played over a speaker. I can manage those side effects in Node‐RED just by opening a web browser. If I managed them on the laundry spy itself I’d have to re‐flash its firmware every time I changed something.
  2. Even with the help of libraries, an HTTP request is a several‐step process on an Arduino or ESP8266, whereas an MQTT publish is a one‐liner. HTTPS, especially, comes with a lot of boilerplate, and Pushover’s API is HTTPS‐only (as all APIs should be). Making an HTTPS request on ESP8266 requires us to either hard‐code a certificate fingerprint in our firmware sketch or choose not to verify the server’s fingerprint altogether, and neither of those options seems prudent to me. (This is one of the reasons I moved away from the Zapier approach.)
  3. We won’t just publish machine state; we’ll have channels for raw data as well. Vibration detection is a bit tricky, and once we install the laundry spy, we’ll have to “tune” it to recognize what level of force signifies a cycle versus, say, someone stomping down the hall. Eavesdropping on raw force data over MQTT allows us to do this quite painlessly.

Now that we’ve got all this infrastructure in place, we’re ready to give our laundry spy a brain. Next time we’ll do the hard work of turning raw vibration data into meaningful inferences about whether our clothes are clean. Part Three will have all the stuff I had promised would be in Part Two, so I suppose there’s also the suspense of finding out if I’ve lied to you again.

Nostalgia-Tron, Part 10: Metadata

Posted in Articles, Raspberry Pi, Tutorials

You might have noticed that your list of installed games looks a bit bland in EmulationStation without artwork, game descriptions, and the like. You could use the metadata scraper that comes with RetroPie, but for MAME games I think it’s better to leverage the pedantry of the community and fill your game lists with more reliable metadata.

Read more →