How to instantly replace existing Raspberry Pis by the ESP8266

For my outdoor sensors I used an RaspberryPi with BMP180 + TSL2561 via I2C transferring the sensor data via MQTT messages.

During an IoT Meetup Session at codecentric office in Frankfurt we discussed the usage of an esp8266 and I was thrilled to get one and play with it.

Adafruit HUZAAH ESP8266 breakout highlights

  • 80-MHz-Mikrocontroller
  • Wi-Fi Direct (P2P), soft-AP
  • Integrated TCP/IP protocol stack with DNS Support
  • Wake up and transmit packets in < 2ms
  • 4 MB Flash-Chip
  • Onboard-Antenna
  • Status LEDs
  • Reset button
  • User button that can also put the chip into bootloading mode,
  • I/O:
    • 1 x Analog input (1.8V max)
    • 9 x GPIO (3.3V logic), which can also be used for I2C or SPI
    • 2 x UART pins
    • 2 x 3-12V power inputs, reset, enable, LDO-disable, 3.3V output

It turns out very quickly that the esp8266 is a really great piece of hardware and I planned to replace my Raspberry Pis acting as sensor hosts in my network. With its wifi capabilities the esp8266 lets me still run the I2C sensors with my MQTT approach.

E. g. the current Raspberry Pi installed outdoors sends values to these topics

  • sweethome/sensors/outdoor/temperature
  • sweethome/sensors/outdoor/pressure
  • sweethome/sensors/outdoor/lux

and if get the esp8266 to publish to these topics too the rest of my network won’t event recognize there was a change…

The Plan: Keep the I2C sensors but replace the “brain”.

replace_this by_that
Raspberry Pi HUZAAH ESP8266 breakout
40€ 10€
SD-Card: 4€
WLAN-Stick 8€
Total: 52€ 10€
Power Consumption: 3W 1W

Hardware

IDE

Arduino IDE

Libraries

The Libraries almost all there, can be easily installed via the Arduino IDE Library Manager.
Except for the esp8266 support the an additional Board Manager URL has to be set:
http://arduino.esp8266.com/stable/package_esp8266com_index.json

prefs_additional_board_managers_url

  • ESP8266WiFi
  • Adafruit BMP085 Unified
  • Adafruit TSL2561 Unified
  • Adafruit MQTT Library

library_manager_esp8266

Software

The esp8266 breakout has a build-in support for LUA, but I decided to flash it to be ready for usage with Arduino IDE and C++ (mostly because the drivers are all well supported by Adafruit).

#include <ESP8266WiFi.h>
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BMP085_U.h>
#include <Adafruit_TSL2561_U.h>

/************************* WiFi Access Point *********************************/

#define WLAN_SSID       "Lummerland"
#define WLAN_PASS       "****************"

/************************* MQTT Broker Setup *********************************/

const int MQTT_PORT = 1883;
const char MQTT_SERVER[] PROGMEM    = "192.168.0.61";
const char MQTT_CLIENTID[] PROGMEM  = "ESP-PUBLISHER-SERVICE";
const char MQTT_USERNAME[] PROGMEM  = "********";
const char MQTT_PASSWORD[] PROGMEM  = "********";

// Create an ESP8266 WiFiClient class to connect to the MQTT server.
WiFiClient client;

Adafruit_MQTT_Client mqtt(&client, MQTT_SERVER, MQTT_PORT, MQTT_CLIENTID, MQTT_USERNAME, MQTT_PASSWORD);

/****************************** Feeds ***************************************/
const char TEMPERATURE_FEED[] PROGMEM = "sweethome/sensors/outdoor/temperature";
Adafruit_MQTT_Publish temperature_topic = Adafruit_MQTT_Publish(&mqtt, TEMPERATURE_FEED);

const char PRESSURE_FEED[] PROGMEM = "sweethome/sensors/outdoor/pressure";
Adafruit_MQTT_Publish pressure_topic = Adafruit_MQTT_Publish(&mqtt, PRESSURE_FEED);

const char LUMINOSITY_FEED[] PROGMEM = "sweethome/sensors/outdoor/lux";
Adafruit_MQTT_Publish luminosity_topic = Adafruit_MQTT_Publish(&mqtt, LUMINOSITY_FEED);

Adafruit_BMP085_Unified bmp = Adafruit_BMP085_Unified(10085);
Adafruit_TSL2561_Unified tsl = Adafruit_TSL2561_Unified(TSL2561_ADDR_FLOAT, 12345);

/*************************** Sketch Code ************************************/

void setup() {
  Serial.begin(115200);
  delay(10);

  Serial.println("Sensor Test");
  if (!bmp.begin())
  {
    Serial.print("Ooops, no BMP180 detected ... Check your wiring or I2C ADDR!");
    while (1);
  }
  else {
    Serial.println("BMP180 ready.");
  }
  if (!tsl.begin())
  {
    Serial.print("Ooops, no TSL2561 detected ... Check your wiring or I2C ADDR!");
    while (1);
  }
  else {
    Serial.println("TSL2561 ready.");
  }

  configureTSL2561();

  // Connect to WiFi access point.
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(WLAN_SSID);

  WiFi.begin(WLAN_SSID, WLAN_PASS);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println();

  Serial.println("WiFi connected");
  Serial.println("IP address: "); Serial.println(WiFi.localIP());
}

void loop() {
  MQTT_connect();

  /* Get a new sensor event */
  sensors_event_t bmpEvent;
  bmp.getEvent(&bmpEvent);

  if (bmpEvent.pressure)
  {
    float temperature;
    bmp.getTemperature(&temperature);
    Serial.print("Temperature: ");
    Serial.print(temperature);
    Serial.println(" C");
    Serial.print("Publish Temperature: ");
    if (! temperature_topic.publish(temperature)) {
      Serial.println("Failed");
    } else {
      Serial.println("OK!");
    }

    float pressure = bmpEvent.pressure;
    Serial.print("Pressure:    ");
    Serial.print(pressure);
    Serial.println(" hPa");
    Serial.print("Publish Pressure: ");
    if (! pressure_topic.publish(pressure)) {
      Serial.println("Failed");
    } else {
      Serial.println("OK!");
    }
  }
  else
  {
    Serial.println("Sensor error");
  }

  sensors_event_t tslEvent;
  tsl.getEvent(&tslEvent);
  unsigned int luminosity = 0;
  if (tslEvent.light)
  {
    luminosity = tslEvent.light;
  }

  Serial.print("Luminosity:   ");
  Serial.print(luminosity);
  Serial.println(" lux");
  Serial.print("Publish Luminosity: ");
  if (! luminosity_topic.publish(luminosity)) {
    Serial.println("Failed");
  } else {
    Serial.println("OK!");
  }

  delay(5000);
}

void configureTSL2561()
{
  /* You can also manually set the gain or enable auto-gain support */
  // tsl.setGain(TSL2561_GAIN_1X);      /* No gain ... use in bright light to avoid sensor saturation */
  // tsl.setGain(TSL2561_GAIN_16X);     /* 16x gain ... use in low light to boost sensitivity */
  tsl.enableAutoRange(true);            /* Auto-gain ... switches automatically between 1x and 16x */

  /* Changing the integration time gives you better sensor resolution (402ms = 16-bit data) */
  tsl.setIntegrationTime(TSL2561_INTEGRATIONTIME_13MS);      /* fast but low resolution */
  // tsl.setIntegrationTime(TSL2561_INTEGRATIONTIME_101MS);  /* medium resolution and speed   */
  // tsl.setIntegrationTime(TSL2561_INTEGRATIONTIME_402MS);  /* 16-bit data but slowest conversions */
}

void MQTT_connect() {
  int8_t ret;

  // Stop if already connected.
  if (mqtt.connected()) {
    return;
  }

  Serial.print("Connecting to MQTT... ");

  while ((ret = mqtt.connect()) != 0) { // connect will return 0 for connected
    switch (ret) {
      case 1: Serial.println("Wrong protocol"); break;
      case 2: Serial.println("ID rejected"); break;
      case 3: Serial.println("Server unavailable"); break;
      case 4: Serial.println("Bad user/password"); break;
      case 5: Serial.println("Not authenticated"); break;
      case 6: Serial.println("Failed to subscribe"); break;
      default: Serial.print("Couldn't connect to server, code: ");
        Serial.println(ret);
        break;
    }
    Serial.println("Retrying MQTT connection in 5 seconds...");
    mqtt.disconnect();
    delay(5000);  // wait 5 seconds
  }
  Serial.println("MQTT Connected!");
}

To get the program running on the esp board you have to

Get the esp8266 into flash mode

press + hold “Reset” button
then
press + hold “GPIO0” button
then
release “Reset” button
then
release “GPIO0” button
-> the red LED turns on (with lower brightness)

flash

Ready for uploading

Open the Serial Monitor to see what’s going on:

arduino_ide_open_serial_monitor

Note the “Board”, “CPU Frequency”, “Upload Speed” and the “Port” settings! 

Before uploading you can build the program by click on “Verify”:

click_upload output_verify

Then start uploading:

click_upload output_uploadThe program starts immediately and the Serial Monitor shows whats going on:

serial_monitor

To debug/monitor the MQTT messages I am using MQTT.fx (of course ;-)):

mqttfx

No changes have to be made to other components of my networks. The replacement was transparent to the (Mobile) Clients also:

iphone

SweetHomeHub: Home Control with Raspberry Pi and MQTT – Part 1

Since quite a long time I am working on my universal Raspberry Pi based Intertechno-Remote (see former posts 1 2 3 4):

intertechno remote

I tried different approaches to trigger/control my remote control service via a custom HTTPServer/-Handler and a simple Vert.x verticle.
Since MQTT v3.1.1 turns out as on of the de-facto standard protocols for the IoT I also implemented an MQTT client.

This MQTT client basically follows two design patterns:

1. One topic for each device
For each device a topic is defined. Its state can be controlled by publishing a message with payload “ON” or “OFF”.
Pro:

  • the user must not know about the address code of the Intertechno device
  • changes of the address must not be published
  • the message is simply “ON” or “OFF to control the device

Contra:

  • the user must know the topic for each device
  • the user can only control configured devices

2. One topic for a JSON message
Pro:

  • very flexible to control the devices

Contra:

  • the user must know about the syntax of the JSON and the coding of devices

Solution:
Provide both options 😉

One topic for each device

topics_table

My configuration is very simple
On start-up the Client is searching for sweethomehub-config.xml in the users home directory which is then unmarshalled from JAXB.
This configuration contains the codes and the topic for each device and the MQTT settings for the broker connection:

<configuration>
    <devices>
        <device>
            <houseCode>a</houseCode>
            <groupId>1</groupId>
            <deviceId>1</deviceId>
            <name>Light Front-Door</name>
            <mqttTopic>front/lights/door</mqttTopic>
        </device>
        <device>
            <houseCode>a</houseCode>
            <groupId>1</groupId>
            <deviceId>2</deviceId>
            <name>Light Terrace</name>
            <mqttTopic>garden/lights/terrace</mqttTopic>
        </device>
        <device>
            <houseCode>a</houseCode>
            <groupId>1</groupId>
            <deviceId>3</deviceId>
            <name>Fountain</name>
            <mqttTopic>garden/devices/fountain</mqttTopic>
        </device>
        <device>
            <houseCode>a</houseCode>
            <groupId>1</groupId>
            <deviceId>4</deviceId>
            <name>Light Garden</name>
            <mqttTopic>garden/lights/ambiente</mqttTopic>
        </device>
        <device>
            <houseCode>a</houseCode>
            <groupId>1</groupId>
            <deviceId>3</deviceId>
            <name>Light Living Room</name>
            <mqttTopic>livingroom/lights/ambiente</mqttTopic>
        </device>
    </devices>
    <mqttClientConfiguration>
        <mqttClientId>SweethoemMQTTClientId</mqttClientId>
        <mqttBrokerAddress>sweethome</mqttBrokerAddress>
        <mqttBrokerPort>1883</mqttBrokerPort>
        <mqttMessagesBaseTopic>sweethome</mqttMessagesBaseTopic>
    </mqttClientConfiguration>
</configuration>

And there is one additional topic awaiting the JSON commands:
sweethome/devices/jsoncommand

{
  "devices":[
    {
      "device":{
        "name": "Light Front-Door",
        "houseCode": "a",
        "groupId": "1",
        "deviceId": "1"
      },
      "command":"ON"
    },
    {
      "device":{
        "name": "Light Terrace",
        "houseCode": "a",
        "groupId": "1",
        "deviceId": "2"
      },
      "command":"ON"
    },
    {
      "device":{
        "name": "Light Living Room",
        "houseCode": "a",
        "groupId": "1",
        "deviceId": "3"
      },
      "command":"ON"
    }
  ]
}

The central method to handle arrived messages:

MqttMessageReceiver.messageArrived

The JsonDeviceCommandProcessor:

JsonCommandProcessor

And the doSwitch methods:

DeviceControl

MQTT Client running on the Raspberry Pi waiting for messages:
mqtt-receiver

… and receiving command messages:

mqtt-receiver-commands

Testing the receiver with MQTT.fx

mqtt-fx

Complete code can be found at BitBucket.

MQTT.fx 0.0.10 released

MQTT.fx 0.0.10 was released.

Download latest binaries in all flavours at mqttfx.org.
Bugs and feature requests can be reported via the issue tracker.

New in this version:
MQTT-Message Clipboard

Option to copy the message payload to clipboard
copypayload
Service menus at publish and subscription panel: now you can clear received messages buffer and the recent topics lists
publish_service
subscribe_service

Extended logging to mqttfx.log
Added Application Window Icon (for Windows)
Now running/bundled with Java 8u20
Updated 3rd-party libs (e.g. ControlsFX 8.20.7)

MQTT.fx 0.0.9 released

A new version of MQTT.fx is available (probably the last release before JavaOne)

Download latest binaries in all flavours: mqttfx.org.
Bugs and feature requests can be reported via the issue tracker.

This release comes with some new features:

Message Font Size

(in respect to Gerrit Grunwald @hansolo_)
To improve ledibility of large message bodies now the font size of messages can be set to a fixed value (0.6em – 1.5em) or a dynamic size depending on message size (1.0em – 0.6em).
You can open the new Settings Dialog via Menu Extras / Settings:

settings

Dynamic message font size:

fonts

Clear History

(also in respect to Gerrit Grunwald)
The Connection Profile Dialog (Section “General”) and the Publish/Subscribe panels now the history of last topics can be cleared.

clear_publish

clear_sub

clear_profile

“Never again drive me crazy with silly questions”

(in respect to David Janes @dpjanes)
The Exit Dialog now offers an option “Do not ask me again!” to get not asked again when closing the app.

exit

Bugfixes

Also I again have fixed some bugs (typos, usability, layout + style issues).

“PiMac unibody” or a new home for my touch screen

In preparation for the JavaOne 2014 Session my 7″ touch screen from Chalk-Elec finally found a new home.
I bought a 1 mm sheet of aluminium and treated it with my jig saw and a rasp and folded a triangle stand for the screen. Basically there would be room enough for e.g. a Rapsberry Pi and it also stand touching the display:

DSC_9673

DSC_9675

DSC_9672

DSC_9668

Some anti slip pads on the button side

IMG_3589

IMG_3587

The Sketch

PiMac Scetch

MQTT.fx 0.0.3 released

Download latest binaries HERE.

Most significant change: I have added a pane to show some status values of the broker:

mqtt-0.0.3

In v0.0.3 these $SYS topics are recognised (Tooltips are showing the descriptions):

Version: $SYS/broker/version
Build: $SYS/broker/timestamp
Uptime: $SYS/broker/uptime
Subscriptions Count: $SYS/broker/subscriptions/count
Changeset: $SYS/broker/changeset
Clients Connected: $SYS/broker/clients/active
Clients Expired: $SYS/broker/clients/expired
Clients Disconnected: $SYS/broker/clients/inactive
Clients Maximum: $SYS/broker/clients/maximum
Clients Total: $SYS/broker/clients/total
Connection: $SYS/broker/connection/#
Messages Inflight: $SYS/broker/messages/inflight
Messages Received: $SYS/broker/messages/received
Messages Sent: $SYS/broker/messages/sent
Messages Stored: $SYS/broker/messages/stored
Messages Publish Dropped: $SYS/broker/publish/messages/dropped
Messages Publish Received: $SYS/broker/publish/messages/received
Messages Publish Sent: $SYS/broker/publish/messages/sent
Messages Retained Count: $SYS/broker/retained/messages/count
Bytes Received: $SYS/broker/bytes/received
Bytes Sent: $SYS/broker/bytes/sent
Load Connections: $SYS/broker/load/connections/+
Load Bytes Received: $SYS/broker/load/bytes/received/+
Load Bytes Sent: $SYS/broker/load/bytes/sent/+
Load Messages Received: $SYS/broker/load/messages/received/+
Load Messages Sent: $SYS/broker/load/messages/sent/+
Load Publish Dropped: $SYS/broker/load/publish/dropped/+
Load Publish Received: $SYS/broker/load/publish/received/+
Load Publish Sent: $SYS/broker/load/publish/sent/+
Load Sockets Count: $SYS/broker/load/sockets/+

NetBeans 8 – Shortcut to Remote Platforms

The recently released NetBeans 8 holds a great gift for all Java Embedded Developers:

Build-in support for Remote Java Standard Edition platform.

To introduce this great and handy feature I created some screencasts:

Part1: headless

Part2: JavaFX on Pi

Part 3 – Remote Platform Debugging

José Pereda has also blogged about this feature in action: He and Mark Heckler used it in a distributed environment for their talk G-Force! Self-Powering Global Sensor Nets for IoT (JFokus 2014).

In addition to the support for a full JRE Environment its also worth to mention there is a nice editor as a fronted to the “recreate”-command provided by the all new EJDK 8:

ejdk

Update/Hint
To launch Raspberry PI Java applications from within Eclipse a project called LaunchPI has been started.

For Intellij/IDEA there is a plugin called “sourcesync” that might be helpful for even remote deployment.

But by now I can’t find other approaches that are such seamless and well integrated as NetBeans Remote Platform support.

Between 0 and 1 – PWM with Raspberry Pi

Switch on and off an LED and let run a motor at one speed and direction is easy, but what if I want to e.g. control the brightness of an LED or control motor-speed?

A kind of analog signal is needed:
This is done with Pulse-width modulation (PWM). PWM is a “modulation technique that conforms the width of the pulse, formally the pulse duration, based on modulator signal information.” (Wikipedia).

In a nutshell:
The average value of voltage is depending on the frequency the load is switched on and off.

Basically there are two ways to create PWM:

Hardware PWM

  • very fast (max. possible frequency / period)
  • software independent, independent from program flow
  • clean signal

BUT:

  • not all pins might support HW PWM

Software PWM

  • each I/O pin can be used for SW PWM

BUT:

  • software controlled timing has to be implemented
  • implementation and timing have to respect CPU usage
  • not a real clean signal

In other words: HW PWM is clearly prefered!

But unfortunately there is only one HW PWM pin (#18) available on the Rasperry Pis GPIO header (also its shared with the Audio system. That means that you can’t use PWM or Servo output and play audio through the 3.5mm jack at the same time).

If a RasPi project needs more than 1 PWM it has to use soft-pwm.
However thank god all the hard work is almost done by using Gordon Henderson’s wiringPi-lib and its Java-API Pi4J.

Pi4J provides a class called “SoftPwm”. As it uses wiringPi functions, the GPIO subsytem has to be initialzed first.
Therefore the programm has to executed as “sudo”!

Here is a simple example:

public class WiringPiSoftPWMExample {
    
    public static void main(String[] args) throws InterruptedException {
        
        // initialize wiringPi library
        Gpio.wiringPiSetup();

        // create soft-pwm pins (min=0 ; max=100)
        SoftPwm.softPwmCreate(0, 0, 100);

        IntegerProperty value1 = new SimpleIntegerProperty();
        value1.addListener((ObservableValue<? extends Number> observable, Number oldValue, Number newValue) -> {
            SoftPwm.softPwmWrite(0, newValue.intValue());
        });

        // continuous loop
        while (true) {
            // fade LED to fully ON
            for (int i = 0; i <= 100; i++) {
                value1.set(i);
                Thread.sleep(25);
            }

            // fade LED to fully OFF
            for (int i = 100; i >= 0; i--) {
                value1.set(i);
                Thread.sleep(25);
            }
        }
    }
}

LED blink

Works!
What’s next?

IMG_3176

3 x optical coupler + RGB LED stripe triggered by JavaFX UI contolled SoftPwms: