From Idea to IoT

Category: Home Assistant

  • Bridging the Gap: Connecting Huawei ESM Batteries to Deye Inverters with ESP32

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management.
      • Modbus Master: Polls multiple Huawei ESM packs for voltage, current, temperature, and cell-level data.
      • Protocol Emulator: Acts as a Pylontech BMS slave to the inverter, “tricking” it into seeing a compatible battery bank.
      • IoT Gateway: Monitors the inverter itself via the Solarman V5 TCP protocol and pushes data to Home Assistant.

    Technical Deep Dive: Solarman V5 Protocol

    One of the more interesting aspects was implementing the Solarman V5 protocol over TCP. Unlike standard Modbus TCP, Solarman wraps Modbus frames in a proprietary header (starting with 0xA5) and uses a custom checksum. 

    Home Assistant Integration

    To make the system user-friendly, I implemented MQTT Auto-Discovery. The bridge registers itself with Home Assistant the moment it connects to the network, creating sensors for everything from “Grid Power” to “Battery Cell Temperature” without any manual YAML configuration. This allows for beautiful dashboards and automation—like notifying me if a specific battery cell shows a voltage imbalance, or automatically adjusting load based on real-time solar production retrieved directly from the inverter’s logger.

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management. In the world of residential solar and energy storage, mixing and matching components often leads to a common headache: communication protocol mismatch. Recently, I tackled a project to bridge the gap between high-capacity industrial Huawei ESM-48100 battery packs and a Deye Hybrid Inverter. The result is an ESP32-powered “BMS Bridge” that not only translates battery telemetry but also integrates deeply with Home Assistant via MQTT.
    The bridge acts as a multi-protocol translator in real-time.

    The Challenge: Industrial vs. Residential

    Huawei ESM series batteries are robust, industrial-grade units, but they speak a specific flavor of Modbus RTU. On the other side, residential inverters like Deye or Sunsynk typically expect a Pylontech-compatible CAN or RS485 protocol to manage charging parameters (SOC, Voltage limits, etc.). Without a bridge, the inverter is “blind” to the battery’s state, leading to inefficient charging and safety risks.

    The Solution: An Intelligent Protocol Bridge

    Using the ESP32 and the ESP-IDF framework, I developed a firmware that performs three simultaneous roles:
      • Modbus Master: Polls multiple Huawei ESM packs for voltage, current, temperature, and cell-level data.
      • Protocol Emulator: Acts as a Pylontech BMS slave to the inverter, “tricking” it into seeing a compatible battery bank.
      • IoT Gateway: Monitors the inverter itself via the Solarman V5 TCP protocol and pushes data to Home Assistant.

    Technical Deep Dive: Solarman V5 Protocol

    One of the more interesting aspects was implementing the Solarman V5 protocol over TCP. Unlike standard Modbus TCP, Solarman wraps Modbus frames in a proprietary header (starting with 0xA5) and uses a custom checksum. 

    Home Assistant Integration

    To make the system user-friendly, I implemented MQTT Auto-Discovery. The bridge registers itself with Home Assistant the moment it connects to the network, creating sensors for everything from “Grid Power” to “Battery Cell Temperature” without any manual YAML configuration. This allows for beautiful dashboards and automation—like notifying me if a specific battery cell shows a voltage imbalance, or automatically adjusting load based on real-time solar production retrieved directly from the inverter’s logger.

    Scaling to Remote Sites: Solving the Local Server Problem

    One common limitation with DIY energy monitoring is the assumption that the hardware and the Home Assistant (HA) server live on the same local network. But what about rental properties? To solve this, I designed the bridge to work over a Remote MQTT Architecture. By leveraging MQTT’s “Hub and Spoke” model, the bridge can be deployed anywhere with a simple Wi-Fi connection, while the management server remains at a central location.

    1. Firewall-Friendly Outbound Connections

    Instead of requiring complex VPNs or port forwarding at the remote site (which is often impossible on LTE/5G CGNAT gateways), the ESP32 initiates an outbound connection to a central broker. In the init_mqtt() function, we simply point the client to a cloud-hosted URL: 
    mqtt_cfg.broker.address.uri = "mqtts://central.my-domain.com:8883"; mqtt_cfg.broker.verification.crt_bundle_attach = esp_crt_bundle_attach;

    2. Security via TLS Encryption

    Sending battery data over the open internet is a security risk. To mitigate this, I utilized the ESP-IDF CRT Bundle. This allows the ESP32 to verify the server’s SSL certificate using the same root CA logic as a modern web browser, ensuring the telemetry is encrypted and the source is authenticated.

    3. “Zero-Touch” Remote Maintenance (OTA)

    Deploying at a remote site means you can’t just plug in a USB cable to fix a bug. I implemented a remote Over-The-Air (OTA) update trigger via MQTT. By publishing a signed firmware URL to a specific command topic, the bridge can update itself anywhere in the world: 
    if (strncmp(event->topic, "BMS/command/ota", event->topic_len) == 0) { // URL received via MQTT, trigger update task xTaskCreate(ota_task, "ota_mqtt_task", 8192, (void*)ota_url, 5, NULL); }

    4. Global Discovery

    Because Home Assistant’s MQTT Discovery is topic-based rather than IP-based, the remote bridge “appears” in the HA dashboard as soon as it connects to the central broker. This creates a seamless experience where data from multiple remote sites can be aggregated into a single, unified energy dashboard. This architecture transforms the project from a simple local converter into a professional-grade, scalable monitoring solution for distributed energy assets.

    Future Work

    The project is currently stable and running in a production environment.  Check out the source code and documentation on my GitHub.
    Keywords: ESP32, Huawei ESM48100, Deye Inverter, Solarman V5, Modbus RTU, Pylontech Emulation, Home Assistant, Energy Management.