From Idea to IoT

Blog

  • Bridging the Gap: Connecting Huawei ESM Batteries to Deye Inverters with ESP32

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management.
      • Modbus Master: Polls multiple Huawei ESM packs for voltage, current, temperature, and cell-level data.
      • Protocol Emulator: Acts as a Pylontech BMS slave to the inverter, “tricking” it into seeing a compatible battery bank.
      • IoT Gateway: Monitors the inverter itself via the Solarman V5 TCP protocol and pushes data to Home Assistant.

    Technical Deep Dive: Solarman V5 Protocol

    One of the more interesting aspects was implementing the Solarman V5 protocol over TCP. Unlike standard Modbus TCP, Solarman wraps Modbus frames in a proprietary header (starting with 0xA5) and uses a custom checksum. 

    Home Assistant Integration

    To make the system user-friendly, I implemented MQTT Auto-Discovery. The bridge registers itself with Home Assistant the moment it connects to the network, creating sensors for everything from “Grid Power” to “Battery Cell Temperature” without any manual YAML configuration. This allows for beautiful dashboards and automation—like notifying me if a specific battery cell shows a voltage imbalance, or automatically adjusting load based on real-time solar production retrieved directly from the inverter’s logger.

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management. In the world of residential solar and energy storage, mixing and matching components often leads to a common headache: communication protocol mismatch. Recently, I tackled a project to bridge the gap between high-capacity industrial Huawei ESM-48100 battery packs and a Deye Hybrid Inverter. The result is an ESP32-powered “BMS Bridge” that not only translates battery telemetry but also integrates deeply with Home Assistant via MQTT.
    The bridge acts as a multi-protocol translator in real-time.

    The Challenge: Industrial vs. Residential

    Huawei ESM series batteries are robust, industrial-grade units, but they speak a specific flavor of Modbus RTU. On the other side, residential inverters like Deye or Sunsynk typically expect a Pylontech-compatible CAN or RS485 protocol to manage charging parameters (SOC, Voltage limits, etc.). Without a bridge, the inverter is “blind” to the battery’s state, leading to inefficient charging and safety risks.

    The Solution: An Intelligent Protocol Bridge

    Using the ESP32 and the ESP-IDF framework, I developed a firmware that performs three simultaneous roles:
      • Modbus Master: Polls multiple Huawei ESM packs for voltage, current, temperature, and cell-level data.
      • Protocol Emulator: Acts as a Pylontech BMS slave to the inverter, “tricking” it into seeing a compatible battery bank.
      • IoT Gateway: Monitors the inverter itself via the Solarman V5 TCP protocol and pushes data to Home Assistant.

    Technical Deep Dive: Solarman V5 Protocol

    One of the more interesting aspects was implementing the Solarman V5 protocol over TCP. Unlike standard Modbus TCP, Solarman wraps Modbus frames in a proprietary header (starting with 0xA5) and uses a custom checksum. 

    Home Assistant Integration

    To make the system user-friendly, I implemented MQTT Auto-Discovery. The bridge registers itself with Home Assistant the moment it connects to the network, creating sensors for everything from “Grid Power” to “Battery Cell Temperature” without any manual YAML configuration. This allows for beautiful dashboards and automation—like notifying me if a specific battery cell shows a voltage imbalance, or automatically adjusting load based on real-time solar production retrieved directly from the inverter’s logger.

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management.

  • Building a Real-Time AI Wildlife Monitor with ESP32-CAM and TensorFlow

    Developing a real-time wildlife monitoring system is a challenging yet rewarding project that combines hardware, machine learning, and web development. My latest project, WildLife Monitor, is an end-to-end solution designed to monitor safety conditions using an ESP32-CAM and a custom-trained Convolutional Neural Network (CNN).

    The Vision

    The goal was to create a system that doesn’t just “watch” but “understands.” Whether it’s monitoring a backyard for curious pets or ensuring an area is clear of hazards (like snakes), the WildLife Monitor provides real-time visual feedback and automated alerts based on what it sees.

    The Tech Stack

    • Hardware: ESP32-CAM (for low-cost, Wi-Fi-enabled video streaming).
    • Backend: Flask (Python) to manage the stream, data, and dashboard.
    • Computer Vision: OpenCV for frame processing and TensorFlow/Keras for the CNN model.
    • Messaging: MQTT for real-time safety alerts.
    • Frontend: A responsive web dashboard for monitoring and active learning.

    Key Features

    1. Real-Time AI Inference

    The system captures an MJPEG stream from the ESP32-CAM. Each frame is preprocessed and passed through a CNN model (model_cnn.h5). The dashboard overlays real-time predictions—identifying if a subject is present and whether the situation is “Safe” or “Unsafe.”

    2. Active Learning Pipeline

    One of the most powerful features of this project is the Active Learning workflow. When the model encounters a frame it isn’t confident about, or when I manually flag a frame, it’s saved into a review folder.

    • Manual Labeling: I built a dedicated interface to quickly categorize these “unknown” frames.
    • One-Click Retraining: Once labeled, I can trigger a model retraining session directly from the dashboard. This creates a feedback loop that constantly improves the model’s accuracy.

    3. Real-Time Alerting with MQTT

    Safety is the priority. When the system detects an “Unsafe” condition, it immediately publishes a message to an MQTT broker. This can be used to trigger physical alarms or send phone notifications.

    The Code Behind the Magic

    The backend handles the stream and runs inference in real-time. Here is a look at how the frames are processed:

    
if model:
    try:
        input_data = preprocess(frame)
        preds = model(input_data, training=False).numpy()
        idx = np.argmax(preds[0])
        conf = np.max(preds[0])

        if idx < len(TRAIN_CLASSES):
            label = TRAIN_CLASSES[idx]
            color = (0, 255, 0) if "safe" in label else (0, 0, 255)
            cv2.putText(frame, f"{label} ({conf:.2f})", (10, 30), 
                        cv2.FONT_HERSHEY_SIMPLEX, 1, color, 2)
    except Exception as e:
        print(f"Inference error: {e}")

    What’s Next?

    Building this project highlighted the importance of data quality over quantity. In the future, I plan to:

    • Optimize for Edge: Move some of the inference directly onto the ESP32 to reduce latency.
    • Async Processing: Implement background threading for inference to maintain a high-FPS video feed.

    Conclusion

    WildLife Monitor is more than just a camera; it’s a smart assistant that learns from its environment. It bridges the gap between simple video surveillance and intelligent, reactive monitoring.

  • Building a Snake Deterrent with ESPHome

    Snakes are highly sensitive to ground vibrations, interpreting them as signs of nearby predators or large animals. This natural behavior can be leveraged to create a humane deterrent system: instead of relying on chemicals or barriers alone, we can generate irregular seismic signals that encourage snakes to move away.

    This project explores how to build a low‑cost, programmable vibration driver using an ESP8266 microcontroller and ESPHome firmware. By switching a 12 V hammer actuator in randomized bursts, the system produces unpredictable vibration patterns that mimic footsteps or animal movement. The result is a practical demonstration of how embedded systems, simple electronics, and open‑source firmware can be combined into a real‑world solution.

    Why Random Bursts?

    A steady metronome‑like vibration quickly becomes background noise. Random clusters of strikes, separated by unpredictable pauses, feel more “alive” and unsettling. This project uses ESPHome on an ESP‑01S relay board to generate those bursts automatically.

    Why 12 V Matters

    One important design choice was to keep the system at 12 V DC. Many garden devices unfortunately operate directly from 230 V mains, which introduces unnecessary electrical hazards in outdoor environments. By standardizing on 12 V, the system remains low‑voltage and touch‑safe, significantly reducing the risk of shock during installation, maintenance, or accidental contact.

    Hardware Setup

    • ESP‑01S WiFi relay board with external voltage regulator 12V -> 5V.
    • 12 V Selenoid Lock, used as hammer actuator mounted to a metal stick pushed into the ground
    • Shared 12 V supply: powers both the hammer and the relay board

    The relay switches the hammer coil, while ESPHome firmware handles the random timing.

    ESPHome Configuration

    Here’s the YAML configuration that runs the random burst script at boot, with a manual switch to enable/disable it:

    yaml
    esphome:
  name: esphome-relay2
  friendly_name: esphome relay2 snake hammer
  on_boot:
    priority: -10
    then:
      - if:
          condition:
            switch.is_on: random_enabled
          then:
            - script.execute: random_burst

esp8266:
  board: esp01_1m

switch:
  - platform: gpio
    pin: GPIO0
    name: "Relay2"
    inverted: true
    id: relay2

  - platform: template
    name: "Random Burst Enabled"
    id: random_enabled
    optimistic: true
    restore_state: true
    turn_on_action:
      - script.execute: random_burst
    turn_off_action:
      - script.stop: random_burst

script:
  - id: random_burst
    mode: restart
    then:
      - while:
          condition:
            switch.is_on: random_enabled
          then:
            - lambda: |-
                int burst = random(2, 5); // 2–4 strikes
                for (int i = 0; i < burst; i++) {
                  id(relay2).turn_on();
                  delay(200); // hammer ON
                  id(relay2).turn_off();
                  delay(random(300, 1000)); // random pause
                }
            - delay: !lambda "return random(5000, 15000);" // random gap before next burst

    How It Works

    • Burst clusters: 2–4 relay activations per cycle
    • Random intra‑burst delay: 300–1000 ms between strikes
    • Random inter‑burst gap: 5–15 s before the next cluster
    • Control switch: Exposed in Home Assistant and the ESPHome web UI, so the system can be disabled remotely

    Results

    The hammer delivers irregular seismic signals through the ground near entry points. Combined with perimeter lighting and physical barriers, this creates a layered deterrent system. It’s not a silver bullet, but it reduces surprise encounters and adds peace of mind.

    Final Thoughts

    This vibration deterrent system is no longer just a concept on paper — it is currently undergoing real‑world testing. The ESPHome‑driven relay board and hammer actuator are deployed and running continuous randomized bursts, allowing me to observe how the design performs outside of the lab. Early results are promising, but the ongoing field trials will provide the most valuable insights: durability of the hardware, consistency of the randomization, and, most importantly, the behavioral response of snakes to irregular seismic signals.

    By documenting this project as it evolves, I aim to demonstrate not only the technical implementation but also the practical effectiveness of combining embedded systems with behavioral cues in a real environment. This piece highlights the journey from idea to deployment — and the testing phase is where theory meets reality.

  • Reading a Huawei Battery BMS with an ESP32 and ESPHome

    A deep dive into reverse-engineering RS485 Modbus communication to bring industrial battery telemetry into Home Assistant.

    Background

    The Huawei ESM-48100 is a 48V 100Ah LiFePO4 energy storage module originally designed for telecom sites. It’s a well-built, high-density battery — but like most industrial hardware, it was never meant to talk to a home automation system. It exposes data over a 4-wire RS485 interface using Modbus RTU, and Huawei’s official protocol documentation sits behind an enterprise support portal.

    This project was about bridging that gap: taking a piece of industrial-grade battery hardware and making its internal data — state of charge, cell voltages, temperature, alarms — visible in Home Assistant through a $5 ESP32 and a $1 MAX485 transceiver.

    The Hardware Stack

    The physical layer is straightforward but required careful attention to the details.

    Components used:

    • ESP32-WROOM development board
    • MAX485 RS485 transceiver module
    • RJ45 cable terminated to the ESM-48100’s COM port

    The ESM-48100’s COM port is a standard RJ45 connector carrying 4-wire RS485 on pins 1, 2, 4, and 5, alongside CAN bus signals on pins 7 and 8. Only the RS485 lines were used for this project.

    Wiring the MAX485 Module

    The MAX485 chip has two sides — one facing the microcontroller, one facing the RS485 bus. A naming confusion that trips up many people: the module’s pins are labeled DI (Driver Input) and RO (Receiver Output), which reflect the chip’s own perspective, not the ESP32’s. DI connects to the ESP32’s TX line, and RO connects to the ESP32’s RX line.

    Some MAX485 modules expose separate DE and RE pins for flow control — these need to be tied together and driven by a GPIO on the ESP32. Others tie them internally, handling flow switching automatically. Both variants work at 9600 baud with Modbus RTU, though the manual DE/RE approach is more deterministic at higher speeds.

    A 120Ω termination resistor was placed across the A and B lines at the far end of the RS485 bus, as required by the Modbus standard to prevent signal reflections.

    Wiring Summary

    ESP32 GPIO16 (TX) → MAX485 DI
ESP32 GPIO17 (RX) ← MAX485 RO
ESP32 GPIO4       → MAX485 DE + RE (if exposed)
3.3V              → MAX485 VCC
GND               → MAX485 GND
MAX485 A+  →  ESM-48100 RJ45 Pin 1 (RS485 T+)
MAX485 B-  →  ESM-48100 RJ45 Pin 2 (RS485 T-)

    The Software: ESPHome and Modbus RTU

    ESPHome is a firmware framework for ESP32/ESP8266 that lets you define hardware integrations entirely in YAML. It natively supports Modbus RTU via its modbus_controller component, which handles framing, CRC, request queuing, and register parsing — so there’s no need to implement the protocol manually.

    The configuration defines:

    • A UART component on GPIO16/GPIO17 at 9600 baud, 8N1
    • A Modbus component that references the UART
    • A Modbus Controller targeting slave address 0x01 with a 10-second poll interval
    • Individual sensors mapped to each holding register

    Register Map

    Sensor Register Scale
    Battery Voltage 0x0000 0.01 V/bit
    Battery Current 0x0001 0.01 A/bit (signed)
    State of Charge 0x0002 0.1 %/bit
    State of Health 0x0003 0.1 %/bit
    Remaining Capacity 0x0004 0.1 Ah/bit
    Max Cell Voltage 0x0005 1 mV/bit
    Min Cell Voltage 0x0006 1 mV/bit
    Max Cell Temp 0x0007 0.1 °C/bit
    Min Cell Temp 0x0008 0.1 °C/bit
    Average Cell Temp 0x0009 0.1 °C/bit
    Status Flags 0x000A bitmask
    Alarm Flags 0x000B bitmask
    Cycle Count 0x000C 1 cycle/bit

    ESPHome Configuration (excerpt)

    uart:
  id: uart_modbus
  tx_pin: GPIO16
  rx_pin: GPIO17
  baud_rate: 9600
  stop_bits: 1
  data_bits: 8
  parity: NONE
  flow_control_pin: GPIO4

modbus:
  id: modbus_esm
  uart_id: uart_modbus
  send_wait_time: 250ms

modbus_controller:
  - id: esm_bms
    address: 0x01
    modbus_id: modbus_esm
    update_interval: 10s

sensor:
  - platform: modbus_controller
    modbus_controller_id: esm_bms
    name: "Battery SOC"
    register_type: holding
    address: 0x0002
    unit_of_measurement: "%"
    device_class: battery
    value_type: U_WORD
    filters:
      - multiply: 0.1

    Derived Sensors

    Beyond the raw registers, template sensors were added to derive useful secondary values:

    • Battery Power — calculated as voltage × current (W)
    • Cell Voltage Delta — max minus min cell voltage (mV), a key indicator of cell imbalance
    • Battery State — text sensor decoding the status bitmask into Charging / Discharging / Idle
    • BMS Alarm Active — binary sensor that fires whenever any alarm bit is non-zero

    The Hard Part: Finding the Register Map

    This was the most significant challenge of the project. Huawei’s Modbus protocol document for the ESM-48100 series is not publicly accessible without an enterprise support account. Community documentation is sparse, and what exists is scattered across energy storage forums.

    The ESM-48100 is documented as complying with the YDN1363 protocol over RS485. The register layout used in this project was sourced from community reverse-engineering efforts, not from the official Huawei document — which means it carries some uncertainty. A key caveat discovered during research: some firmware revisions use a 0x1000 base offset for all registers. If sensors return zeros or errors, shifting every address up by 0x1000 is the first thing to try.

    This kind of ambiguity is common when working with industrial hardware in hobby contexts. The approach taken here was to be explicit about what is verified and what is inferred, rather than presenting unverified information as confirmed fact — a discipline that matters when the output is a real electrical system.

    What This Enables in Home Assistant

    Once running, the ESP32 joins the home network and appears in Home Assistant as a device with 15+ entities. From there, the data feeds into:

    • Energy dashboards tracking charge and discharge cycles over time
    • Automations that respond to low SOC, high cell temperature, or active alarms
    • Long-term statistics on SOH degradation and cycle count growth
    • Notifications when the BMS alarm register goes non-zero

    The cell voltage delta sensor is particularly useful — a widening gap between max and min cell voltages over time is an early indicator that cells are drifting apart, which typically precedes a significant SOH drop.

    Reflections

    This project sits at the intersection of embedded systems, industrial protocols, and home automation. The hardware side — RS485 wiring, termination, flow control — required understanding the physical layer of the Modbus standard. The software side required navigating ESPHome’s component model, register types, and value scaling. And the research side required distinguishing between what Huawei officially documents and what the community has inferred.

    The result is a reliable, OTA-updatable, WiFi-connected BMS monitor that costs under $10 in hardware and runs indefinitely without any cloud dependency.

    Built with ESPHome and Home Assistant on an ESP32 with a MAX485 RS485 transceiver.
    Hardware: Huawei ESM-48100B1 LiFePO4 48V 100Ah energy storage module.

  • [SOLVED] SRFPROG.exe sometimes stuck or crashed

    You might have experienced this same issue, presented on e2e.ti.com forum, but TI has “locked” my thread without answer. Here’s how to solve that for a very fast and reliable flashing usable in production.

    We are trying to use srfprog.exe to flash the CC2640R2F, and it gets stuck very often, on any step or any argument, under Win-10 64, and using the JTAG from LaunchXL-CC2640R2. Note that SmartRF Flash Programmer 2 also gets stuck or crashes sometimes. I believe for the same reason.

    This, for example, is an output of srfprog where it got stuck and never returned:

    srfprog -ls auto

Texas Instruments SmartRF Flash Programmer 2 v1.7.5-windows
-------------------------------------------------------------------------------

    … stuck here.

    Here is the solution that has been proven to get srfprog.exe to work reliably without crashing for flashing TI’s CC2640 BLE chip family:

    • Wrap the srfprog calls into another program. Let’s call this wrapper srfwrap.
    • srfwrap will detect output activity from srfprog.exe .
    • If srfprog.exe is doing it’s job, it just goes on.
    • If srfprog.exe crashes or gets stuck, srfwrap will kill srfprog and restart the last command. Until it is successfully done.
    • Using this technique, I have been able to reliably program CC2640R2F in production.
    • Combined with other tools, programming the CC2640R2F can be as fast as under 40 seconds per chip, including the MAC address, a Serial Number, and securely locking the JTAG for unwanted access.

    I hope this helps, let me know in the comments 🙂

    Jerome

  • CC2640R2F: Secure bootloader for encrypted OTA update

    CC2640R2F: Secure bootloader for encrypted OTA update

    Development of a secure bootloader for encrypted OTA update via BLE.

    The bootloader has been developed in C in order to allow the OTA update of the Texas Instrument CC2640R2F, securely with an encrypted firmware image.

    The client controls the private keys and thanks to encryption techniques it is not possible to disassemble the firmware from it’s publicly shared binary.

    This secure bootloader is loaded on a genuine product and third parties cannot use the encrypted application image thus protecting the Intellectual Property from unwanted copies.

  • TI CC2640R2F: Start-up interrupt vector table explained

    TI CC2640R2F: Start-up interrupt vector table explained

    The TI CC2640R2F has a unique start-up sequence, based on the ARM core with TI’s ROM loader on top, here’s how it works!

    Prerequisite, General Facts

    • The BIM (bootloader) or application vector table is set by the linker to any known address.
    • The vector table holds the pointers to 15 ISRs out of 50.
    • On next hardware interrupt, the CPU jumps to the correct ISR using the vector table.
    • In OAD (over-the-air-download), the flash interrupt vectors of an image are located at the beginning of the image’s flash region right after the image’s header.  There are 15 interrupt vectors.
    • The CCFG contains the address of the user program Vector Table. In BIM the start address of the interrupt vector table is modified as follow:#define SET_CCFG_IMAGE_VALID_CONF_IMAGE_VALID 0x1F000Which is stored intoDEFAULT_CCFG_IMAGE_VALID_CONF @ CCFG_END – 5 * sizeof(uint32_t).

    How does the CPU know where the vector table is?

    Normally the Vector table must be located at a fixed absolute address (i.e. ‘reset’ address). However, Cortex-M3 allows to move the table using special register VTOR. In XDC this is

    “M3Hwi.vectorTableAddress = 0x20000000;”

    The XDC script initializes at compile-time a structure ‘Hwi_module’ with the new Vector Table address.
    The assignment is done in the function Hwi_initNVIC().

    “Hwi_nvic.VTOR = (UInt32)Hwi_module->vectorTableBase;”

    Hwi_initNVIC() is hidden and located in ROM at 0x1001a699.
    “\packages\ti\sysbios\family\arm\m3\Hwi.c”

    Where is the vector table on Reset?

    On reset, the vector table location register VTOR points to the ROM, @ 0x10000000 ! TBC

    The CC2640R2F has a ROM that contains a basic loader. There is more but that won’t be covered here.

    Boot procedure flow of operations

    1. ARM CPU reset, the vector table location register VTOR is pointing to ROM @ 0x10000000 ! TBC
    2. ARM CPU jumps to the ROM entry, the address is in the Vector Table.
    3. ROM loader copies the user program Vector Table to RAM @ 0x20000000. The address of the user program Vector Table is stored in the CCFG.
    4. The new Vector Table is in RAM at address 0x20000000 but VTOR has not yet been updated.
    5. ROM modifies the vector table location register VTOR to point to RAM @ 0x20000000.
    6. ROM reads the ResetISR from the RAM Vector Table and jumps to it.
    7. FLASH user program starts at ResetISR.
    8. FLASH user program running.

    with BIM OAD

    1. The first FLASH user program to start at ResetISR in the step 7 is the bootloader (BIM).
    2. BIM knows where or how to find an application Vector Table.
    3. BIM decides if the application is valid and if so will continue
    4. BIM jumps to the Application ResetISR.
    5. Application Starts.
    6. The current Vector Table is in RAM and still contains the BIM vector table copied in step 3.
    7. Application’s startup will copy its 15 own Interrupt Vectors from Flash into RAM and overwrite the vectors table.
    8. To specify the RAM address to copy the vectors to, this is done “indirectly” via the XDC script.
      “M3Hwi.vectorTableAddress = 0x20000000;”
    9. After this point, all interrupts and exceptions should be handled by the Application.
    10. Application Running.
    11. A reset will start this process again from the very first step.

    How to Debug Step Through with OAD BIM bootloader

    For development, it is always preferred to have a firmware as close as the release but with the capability of debugging it, stepping through it etc. There are several possibilities for achieving this:

    1. Stock bootloader

    The debug application firmware can have an empty header, that is not filled in. The application firmware is loaded in it’s memory space without overwriting the bootloader. The program pointer is altered via JTAG to point directly to the application entry point, effectively bypassing the bootlader. The CCFG is not overwritten.

    If an external system occurs, the bootloader will see the debug application firmware as invalid because its header is blank. As a result it will perform the necessary action for “missing” firmware. With Off-chip OAD this corresponds to flashing the latest image present on the off -chip flash memory.

    2. Debug Bootloader with NO_COPY

    To avoid the bootloader to reject the debug application firmware after a reset, it is possible to compile a modified bootloader with the definition NO_COPY enabled. This bootloader is skipping all check and jumping to the application firmware immediately.

    3. No bootloader

    For development purposes where the bootloader is not needed, best is to totally remove it.

    Make a new target without booloader, allocating that flash space for the Application Firmware. This extra flash space is helpful to be used for debug purposes, like extra debug functions, extra print string messages, or lower code space optimisation.

    The CCFG needs to be part of the application in order to contain the correct address of the user program Vector Table.

    BIM OAD extend size to more than one page

    To extend the flash size of the BIM OAD, it is necessary to change linker script. But doing so will fail to boot because the vector table will move forward to the new BIM starting page. This is why is it necessary to manually update the following definition:

    #define SET_CCFG_IMAGE_VALID_CONF_IMAGE_VALID 0x1F000

    This will tell the ROM bootloader what the the BIM starting address is, which is also where the bootloader program Vector Table is.

    CC2640R2F doesn’t start without JTAG Debug Probe

    This problem can be a direct consequence of the previous points. If CC2640R2F doesn’t start without JTAG Debug Probe it could be due to the reset address stored in CCFG being wrong.

    When loading a firmware in debug mode, the debugger will overwrite the default loading sequence and start directly at the image Startup address. So it will effectively start with JTAG.

    When using BIM OAD, after reset, the ROM loader will use the address provided in “ccfg_app_ble.c” and placed into CCFG:
    #define SET_CCFG_IMAGE_VALID_CONF_IMAGE_VALID 0x1F000
    Make sure this is the address of the bootloader start page.
    —
    Sources: TI e2e forum + own experimentation and analyses.
  • MCHP PIC p18f4431: Motor Control, 3-Ph, IGBT

    MCHP PIC p18f4431: Motor Control, 3-Ph, IGBT

    Development of a 3-Phases motor speed control via PWM controlling IGBTs.
    + Technology watch and R&D.
    + Electronics Hardware (schematics, BOM, PCB layout including SMPS) .
    + Embedded software in bare metal C on Microchip PIC p18f4431.
    + 1KW 3-phases motor.
    + Current sensing.
    + H-bridge regenerative breaking.
    + Flyback DCDC power supply.
    (2006 – 2008)

  • CC2640R2F: Smart Bluetooth Low Energy BLE Central (client)

    CC2640R2F: Smart Bluetooth Low Energy BLE Central (client)

    Development of a BLE central; The device is setup as a client, it connects to one or multiple Bluetooth sensors at the same time and without time multiplexing. The central gathers the information, processes them and takes action accordingly. With the TI CC2640R2F it runs on a single AAA battery.
    [Project is still confidential, more details to come later]
  • Cypress PSoC Central for multiple peripherals

    Cypress PSoC Central for multiple peripherals

    Are you considering using Cypress’s PSoC 4 BLE to run as a central?

    Cypress offers a wide range of BLE 4.x System on Chip (PSoC) and Radio on Chip (PRoC) as well as plenty of certified modules for a faster time to market.

    They also offer various basic embedded software example codes, which are very important for a successful development.

    Many BLE systems are used as Peripherals, running a GATT Server. A client can connect to the peripheral and typically performs read / writes on its characteristics.

    One type of popular Client are smartphones, they are usually used for user setup, gathering data and general interaction with the Bluetooth peripheral.

    It is also possible for Cypress’s PSoC 4 BLE to act as a client. The IoT chip can scan for advertisement data, get the scan response, connect to the peripheral, discover its characteristics, and perform read/write/notify on them.

    Multiple peripherals

    Cypress’s PSoC 4 BLE does not support simultaneous connection to multiple peripherals.

    They however advertise what they are calling “time multiplexing”.

    What is Cypress’s PSoC connection time multiplexing? 

    Behind this marketing term “time multiplexing” is a very basic concept : because PSOC 4 BLE used as a central cannot connect to multiple peripheral at once, the central will connect to one peripheral after another, one at a time… as easy as it sounds.

    A typical application for a PSoC 4 BLE Time Multipled Central (TMC) could follow a sequence like this :

    1. Scan for all peripherals.
    2. Connect to the first peripheral and discover its characteristics.
    3. Disconnect.
    4. Connect to every other peripheral and discover their characteristics in a similar manner.
    5. Bond to all peripherals the system is interested with.
    6. Connect to one of the peripheral.
    7. Do the necessary Read/Write operations.
    8. Disconnect.
    9. Repeat operations #6 to #8 in a loop with every peripheral.

     

    Why is Cypress’s Time Multiplexing a bad approach? 

    While a BLE Time Multiplexed GAP Central seems easy and works fine, there’s a couple of reasons why this approach is very bad.

    Power (in)efficiency

    First, BLE stands for Bluetooth Low Energy, and as the name implies, it has been designed to be energy efficient. The time multiplexing does not allow to take profit of the design and thus is very inefficient.

    In BLE, the connection is maintained open by periodical short messages (Connection Interval), this is handled at the low level (Link Layer), allowing the application to sleep. Cypress’s solution for a multi peripheral central implies CPU and BLE overhead due to the connection /disconnection. This affects both central and the peripheral.

    Notification/Indication 

    BLE has a Notification/Indication system that can notify the client of a change in the peripheral’s value. Doing so there’s no need of constantly polling the data. Both central and peripheral can sleep as long as there’s no data update.

    These flags are disabled by default and it is required to activate them by performing a write operation to the CCCD (Client Characteristic Configuration Descriptor) of the corresponding characteristic.

    Most peripherals reset all flags on disconnection, thus making it useless with Cypress’s time multiplexing.

    Stolen peripheral 

    BLE is now widely used and everyone has a smartphone capable of connecting to any peripheral that is advertising. Once connected, the peripheral will stop advertising because it is already connected and cannot accept a new connection.

    This means that if you have 4 devices you connect to in a time multiplexed manner, each device will be available and connectable 3/4 of the time.

    During this time, should anyone connect to the device, it’ll be invisible to the central and impossible to connect to. Connecting to a peripheral could be intentional or unintentional. What I’ve learned from experience and very concerning in an industrial environment is that bugs in some Android devices lead Android not willing to disconnect from the peripheral even though the Bluetooth is switched off in the UI! The only reliable solution found is to reboot the Android phone.

    Placeholder for research : would BLE authentication improve this issue by avoiding a hacker to connect to the peripheral?

    Could the BLE component be improved 

    Cypress’s BLE component could certainly be improved in various manners, but the limiting RAM space and lack of low layer documentation seems to make this solution very difficult. I think this development time should be spent doing a better user application rather spending time on developing a proprietary BLE driver

    The advantages of BLE Time Multiplexed Connection

    The main advantage by far of such a system is to be able to connect to an unlimited number of peripherals. Whether there are 4 or 40 sensors in the system, there is not much difference from the central’s point of view. The connection time will of course be shared between all sensors with all the downside listed above.

    Most sensors are battery powered, it is possible for the user application to sleep while no client is connected and wake-up only once a connection request is received. Advertisement is handled by the link layer, no need for the user application to run.

    Other solutions 

    I have successfully developed a BLE central client, able to connect with multiple simultaneous peripherals at once. The central is maintaining connections for 4 peripherals while advertising and allowing connection of a client to its own GATT server. The low power client also supports all the benefits of BLE including Notifications and CPU sleep. However, the BLE central is based on a TI CC2640R2F instead of Cypress’s solution.

    Are you looking to create your own BLE central device, taking advantage of the low power design of the wireless connection? I am looking forward to discuss your needs, contact me now: jerome at yupana-engineering.com!