Magic-Wand Gesture Trigger — Arduino L4

Learning Goals 5 min

Cluster F's capstone: mount the Nano 33 BLE Sense + gesture model into a real "wand" you can wave to trigger different circuits — a smart lamp toggle, a buzzer, an MQTT publish. The combination of edge AI + WiFi/BLE + physical world that's genuinely magical. By the end of this lesson you will:

Mount the Nano + battery + LED + buzzer in a wand-shaped enclosure.
Tie each recognised gesture to a real-world action: BLE characteristic update, MQTT publish, or physical effect.
Demo the wand: wave → lamp on; punch → music; circle → emergency stop.

Warm-Up 10 min

Hardware:

Nano 33 BLE Sense.
3.7 V LiPo + boost converter to 5 V (or USB power bank if you don't mind the form factor).
Wand-shaped enclosure: PVC pipe, 3D-printed wand, cardboard tube.
Tiny LED + buzzer for feedback.
Optional: WiFi-capable secondary device (ESP) to be the "target".

The system architecture

Wand (Nano 33 BLE Sense): detects gestures, publishes them.
Optional gateway (ESP): subscribes to wand events, drives downstream actions.
Targets (lamp, IoT cloud, MQTT topic): receive commands from the gateway.

For a simple all-in-one demo, the Nano can drive a few outputs directly. For a polished build, BLE / MQTT / WiFi lets the wand control devices across the room.

New Concept · Wand → world 25 min

Output options

Output	How
Local LED + buzzer	On the wand. Confirms gesture detected.
BLE characteristic	Phone or another ESP reads the gesture; reacts. Modern, iOS-friendly.
MQTT publish	If wand has WiFi (Nano ESP32, not Nano 33 BLE) — direct publish to broker.
Arduino IoT Cloud Property	Wand updates a property; cloud dashboards / phone app react.

Nano 33 BLE Sense doesn't have WiFi, only BLE. For WiFi, use a 2-board pattern: Nano BLE detects gesture, BLE-sends to an ESP, ESP publishes MQTT.

The combined sketch (BLE output)

#include <TensorFlowLite.h>
#include <Arduino_LSM9DS1.h>
#include <ArduinoBLE.h>
#include "gesture_model.h"
// ... TFLite includes ...

BLEService gestureService("19B10000-E8F2-537E-4F6C-D104768A1214");
BLEByteCharacteristic gestureChar("19B10001-E8F2-537E-4F6C-D104768A1214",
                                  BLERead | BLENotify);

const int LED_PIN = LED_BUILTIN;
const int BUZZER  = D9;
const char* LABELS[] = {"wave", "punch", "circle"};

void setup() {
  Serial.begin(9600);
  pinMode(LED_PIN, OUTPUT);
  pinMode(BUZZER,  OUTPUT);

  if (!IMU.begin()) while(1);

  if (!BLE.begin()) while(1);
  BLE.setLocalName("Wand");
  BLE.setAdvertisedService(gestureService);
  gestureService.addCharacteristic(gestureChar);
  BLE.addService(gestureService);
  gestureChar.writeValue((byte)0);
  BLE.advertise();

  // ... TFLM init ...

  Serial.println("# Wand ready");
}

void doGestureAction(int g) {
  digitalWrite(LED_PIN, HIGH);
  tone(BUZZER, 660 + g * 220, 200);
  delay(200);
  digitalWrite(LED_PIN, LOW);

  gestureChar.writeValue((byte)g);   // notifies subscribers
}

void loop() {
  BLEDevice central = BLE.central();
  // ... motion-triggered capture + inference ...

  if (bestScore > 0.85) {
    Serial.print("Gesture: "); Serial.println(LABELS[best]);
    doGestureAction(best);
  }
}

The phone (or another ESP) subscribed to gestureChar sees a byte arrive on every detected gesture: 0 = wave, 1 = punch, 2 = circle. From there, the receiver decides what to do.

The downstream ESP

On a Nano ESP32 or ESP8266 + ArduinoBLE: scan for "Wand", connect, subscribe to gestureChar, and react. For each gesture:

Wave (0) → MQTT publish "ON" to your smart lamp topic.
Punch (1) → MQTT publish a chord to your sound device.
Circle (2) → MQTT publish "STOP" to a robot car.

Worked Example · Build the wand 30 min

Step 1 — assemble

30 cm PVC pipe or cardboard tube. The Nano sits in the middle, LiPo at the base.
Bring out an LED + 220 Ω to the tip. Buzzer mid-shaft.
USB-rechargeable: include a TP4056 charging circuit. (Or just removable LiPo + charger.)
Power switch on the side.

Step 2 — upload the wand sketch

Combine TFLM (L04-35) + BLE characteristic + local feedback (LED/buzzer). Verify each gesture lights the LED + fires a unique tone.

Step 3 — set up the receiver

Second device — your phone (Bluefruit Connect / nRF Connect) — connects to "Wand" over BLE. Subscribes to the gesture characteristic. Watch the gesture byte change live as you wave.

Step 4 — pair to a real downstream

Either:

An ESP32 with ArduinoBLE that subscribes and publishes MQTT to your existing CloudLamp setup.
Home Assistant's BLE integration (advanced).
An IFTTT applet triggered via the receiver ESP.

Step 5 — full demo

Wave the wand toward your CloudLamp. Lamp turns on.
Punch toward your robot car. Car stops.
Circle in the air. Buzzer plays a melody on a connected speaker.

Step 6 — calibrate sensitivity

If gestures misfire on incidental motion (walking, putting down the wand), raise the motion trigger threshold and confidence threshold. Trial and error.

Try It Yourself 15 min

🟢 Easy

Goal: Add a 4th gesture (e.g. "tap"). Re-capture, re-train, re-deploy. Bind to a 4th action.

🟡 Medium

Goal: Have the wand "disarm" when held still for 3 seconds (motion trigger doesn't fire). Re-arm on movement. Avoids false positives when wand is in someone's lap.

🔴 Stretch

Goal: Train on data from 3 different people. Verify the wand works across users. Compare accuracy vs single-user-trained.

Mini-Challenge · Ship the wand 10 min

Polished enclosure.
3 gestures, 3 distinct downstream actions.
Demo to a non-coder. Can they understand what each gesture does without explanation?
Battery-powered, no USB.
Video of the full demo.

Ship-ready test: a parent / classmate can pick up the wand and trigger actions. Edge AI + BLE in their hand. Cluster F done.

Recap 5 min

Magic wand = Nano 33 BLE Sense + TFLM gesture model + BLE characteristic + downstream ESP / phone / cloud. The full pipeline: capture data → train Colab → convert to TFLite → embed in Arduino → infer live → react via BLE / MQTT / cloud. Cluster G (Power and Low-Power Design) starts tomorrow — making this same wand last weeks on a battery instead of hours.

Wand: Anything wand-shaped that detects gestures. PVC pipe, 3D-print, or a cardboard tube.
BLE characteristic gesture broadcast: Wand publishes detected gesture as a byte on a BLE characteristic. Any device that connects + subscribes sees it.
Gateway pattern: Wand → BLE → gateway ESP → MQTT → home automation. Bridges BLE-only devices to WiFi.
Motion trigger: Start capturing only when motion exceeds threshold. Saves CPU + avoids false inferences on idle data.
Confidence threshold: Min model confidence before accepting the gesture. ~0.85 typical.
Cross-user training: Including data from multiple people during training. Required for products that'll be used by anyone.
Edge → BLE → cloud chain: Standard IoT product architecture: small device infers locally, broadcasts via BLE, gateway forwards to cloud.

Homework 5 min

Finish the wand. Demo to someone. Record video.
Read ahead to ARD-L04-37 (Sleep Modes). Bring the Nano + a multimeter capable of measuring µA.

Learning Goals 5 min

Mount the Nano + battery + LED + buzzer in a wand-shaped enclosure.
Tie each recognised gesture to a real-world action: BLE characteristic update, MQTT publish, or physical effect.
Demo the wand: wave → lamp on; punch → music; circle → emergency stop.

Warm-Up 10 min

Hardware:

Nano 33 BLE Sense.
3.7 V LiPo + boost converter to 5 V (or USB power bank if you don't mind the form factor).
Wand-shaped enclosure: PVC pipe, 3D-printed wand, cardboard tube.
Tiny LED + buzzer for feedback.
Optional: WiFi-capable secondary device (ESP) to be the "target".

The system architecture

Wand (Nano 33 BLE Sense): detects gestures, publishes them.
Optional gateway (ESP): subscribes to wand events, drives downstream actions.
Targets (lamp, IoT cloud, MQTT topic): receive commands from the gateway.

For a simple all-in-one demo, the Nano can drive a few outputs directly. For a polished build, BLE / MQTT / WiFi lets the wand control devices across the room.

New Concept · Wand → world 25 min

Output options

Output	How
Local LED + buzzer	On the wand. Confirms gesture detected.
BLE characteristic	Phone or another ESP reads the gesture; reacts. Modern, iOS-friendly.
MQTT publish	If wand has WiFi (Nano ESP32, not Nano 33 BLE) — direct publish to broker.
Arduino IoT Cloud Property	Wand updates a property; cloud dashboards / phone app react.

Nano 33 BLE Sense doesn't have WiFi, only BLE. For WiFi, use a 2-board pattern: Nano BLE detects gesture, BLE-sends to an ESP, ESP publishes MQTT.

The combined sketch (BLE output)

#include <TensorFlowLite.h>
#include <Arduino_LSM9DS1.h>
#include <ArduinoBLE.h>
#include "gesture_model.h"
// ... TFLite includes ...

BLEService gestureService("19B10000-E8F2-537E-4F6C-D104768A1214");
BLEByteCharacteristic gestureChar("19B10001-E8F2-537E-4F6C-D104768A1214",
                                  BLERead | BLENotify);

const int LED_PIN = LED_BUILTIN;
const int BUZZER  = D9;
const char* LABELS[] = {"wave", "punch", "circle"};

void setup() {
  Serial.begin(9600);
  pinMode(LED_PIN, OUTPUT);
  pinMode(BUZZER,  OUTPUT);

  if (!IMU.begin()) while(1);

  if (!BLE.begin()) while(1);
  BLE.setLocalName("Wand");
  BLE.setAdvertisedService(gestureService);
  gestureService.addCharacteristic(gestureChar);
  BLE.addService(gestureService);
  gestureChar.writeValue((byte)0);
  BLE.advertise();

  // ... TFLM init ...

  Serial.println("# Wand ready");
}

void doGestureAction(int g) {
  digitalWrite(LED_PIN, HIGH);
  tone(BUZZER, 660 + g * 220, 200);
  delay(200);
  digitalWrite(LED_PIN, LOW);

  gestureChar.writeValue((byte)g);   // notifies subscribers
}

void loop() {
  BLEDevice central = BLE.central();
  // ... motion-triggered capture + inference ...

  if (bestScore > 0.85) {
    Serial.print("Gesture: "); Serial.println(LABELS[best]);
    doGestureAction(best);
  }
}

The phone (or another ESP) subscribed to gestureChar sees a byte arrive on every detected gesture: 0 = wave, 1 = punch, 2 = circle. From there, the receiver decides what to do.

The downstream ESP

On a Nano ESP32 or ESP8266 + ArduinoBLE: scan for "Wand", connect, subscribe to gestureChar, and react. For each gesture:

Wave (0) → MQTT publish "ON" to your smart lamp topic.
Punch (1) → MQTT publish a chord to your sound device.
Circle (2) → MQTT publish "STOP" to a robot car.

Worked Example · Build the wand 30 min

Step 1 — assemble

30 cm PVC pipe or cardboard tube. The Nano sits in the middle, LiPo at the base.
Bring out an LED + 220 Ω to the tip. Buzzer mid-shaft.
USB-rechargeable: include a TP4056 charging circuit. (Or just removable LiPo + charger.)
Power switch on the side.

Step 2 — upload the wand sketch

Combine TFLM (L04-35) + BLE characteristic + local feedback (LED/buzzer). Verify each gesture lights the LED + fires a unique tone.

Step 3 — set up the receiver

Second device — your phone (Bluefruit Connect / nRF Connect) — connects to "Wand" over BLE. Subscribes to the gesture characteristic. Watch the gesture byte change live as you wave.

Step 4 — pair to a real downstream

Either:

An ESP32 with ArduinoBLE that subscribes and publishes MQTT to your existing CloudLamp setup.
Home Assistant's BLE integration (advanced).
An IFTTT applet triggered via the receiver ESP.

Step 5 — full demo

Wave the wand toward your CloudLamp. Lamp turns on.
Punch toward your robot car. Car stops.
Circle in the air. Buzzer plays a melody on a connected speaker.

Step 6 — calibrate sensitivity

If gestures misfire on incidental motion (walking, putting down the wand), raise the motion trigger threshold and confidence threshold. Trial and error.

Try It Yourself 15 min

🟢 Easy

Goal: Add a 4th gesture (e.g. "tap"). Re-capture, re-train, re-deploy. Bind to a 4th action.

🟡 Medium

Goal: Have the wand "disarm" when held still for 3 seconds (motion trigger doesn't fire). Re-arm on movement. Avoids false positives when wand is in someone's lap.

🔴 Stretch

Goal: Train on data from 3 different people. Verify the wand works across users. Compare accuracy vs single-user-trained.

Mini-Challenge · Ship the wand 10 min

Polished enclosure.
3 gestures, 3 distinct downstream actions.
Demo to a non-coder. Can they understand what each gesture does without explanation?
Battery-powered, no USB.
Video of the full demo.

Ship-ready test: a parent / classmate can pick up the wand and trigger actions. Edge AI + BLE in their hand. Cluster F done.

Recap 5 min

Wand: Anything wand-shaped that detects gestures. PVC pipe, 3D-print, or a cardboard tube.
BLE characteristic gesture broadcast: Wand publishes detected gesture as a byte on a BLE characteristic. Any device that connects + subscribes sees it.
Gateway pattern: Wand → BLE → gateway ESP → MQTT → home automation. Bridges BLE-only devices to WiFi.
Motion trigger: Start capturing only when motion exceeds threshold. Saves CPU + avoids false inferences on idle data.
Confidence threshold: Min model confidence before accepting the gesture. ~0.85 typical.
Cross-user training: Including data from multiple people during training. Required for products that'll be used by anyone.
Edge → BLE → cloud chain: Standard IoT product architecture: small device infers locally, broadcasts via BLE, gateway forwards to cloud.

Homework 5 min

Finish the wand. Demo to someone. Record video.
Read ahead to ARD-L04-37 (Sleep Modes). Bring the Nano + a multimeter capable of measuring µA.