How a stubborn P2P login error, a mislabeled H.265 stream, and five bytes of leaked internal SDK metadata turned into a proper local streaming bridge — with an AI pair-programmer along for the whole ride.
The problem
I had a cheap cloud-connected CCTV camera — the kind you add to a phone app by typing in a “CloudID,” a username, and a password, no IP address in sight. It works fine in the vendor’s own VMS desktop app, but I wanted the actual video stream available locally: something I could point VLC at, feed into Blue Iris, or eventually wire into other tooling. No official API, no documentation, just a Windows app and a folder full of DLLs.
So I went digging into the DLLs the VMS app itself uses, and worked through the whole thing with Claude.ai as a genuine collaborator — not just autocomplete, but someone to reason through hex dumps, packet captures, and dead ends with in real time, over what turned into a pretty long debugging session.
Step one: finding the SDK underneath the app
A quick look inside the VMS install folder turned up NetSdk.dll, exporting a classic H264_DVR_* C API — the same “NetSDK” family used by a huge number of white-label DVR/NVR/camera brands. No headers were included, but a public open-source project on GitHub happened to ship the matching netsdk.h, which meant I could write a proper ctypes wrapper in Python: log in with the CloudID, start real-time play, and register a callback to receive raw frame data — all without touching the vendor’s own app.
First hurdle, immediately: login failed. Every time. Error -12003, undocumented, no explanation.
Step two: proving the network wasn’t the problem
Rather than guess at firewall settings or NAT configuration, we captured the actual network traffic during a login attempt and looked at it byte by byte. What we found was genuinely surprising: real video-sized data packets were flowing over the connection within 0.6 seconds of the login attempt — the P2P tunnel to the camera was working perfectly. The DLL’s own internal “did the login succeed” bookkeeping just never recognized it.
That reframed the whole problem. It wasn’t a network issue, or a credentials issue — it was a bug specific to that particular build of the DLL. Swapping in an older version of the same SDK (sourced from that same open-source reference project) sidestepped the bug entirely. Login succeeded, and real-time play started pulling actual frames.
Step three: the video that wasn’t there
Progress — except the video was completely blank. No errors, no crash, just black.
FFmpeg’s logs kept complaining about invalid NAL units — a NAL unit being the basic chunk video data is packaged into in H.264/H.265. The pattern was oddly consistent: always the same two types, over and over, in bursts. That regularity was the clue that this wasn’t random corruption from a flaky connection.
Pulling a raw capture and inspecting it byte-by-byte turned up the actual cause: small 5-to-13-byte chunks were showing up between real frames, disguised just enough to look like a NAL unit but with an impossible header byte that no real video codec would ever produce. Decoding those bytes directly revealed exactly what they were: internal packet-length metadata from the SDK itself, leaking into the buffer meant for pure video data. The trailing four bytes of each one, read as a plain integer, matched the size of the very next real frame almost exactly.
Once identified, the fix was simple — filter out any chunk with that impossible header byte before it ever reaches the video pipeline. The invalid-NAL warnings dropped to zero, and real video finally appeared.
Step four: the wrong codec, the wrong ffmpeg, and a queue
A few more rounds of debugging turned up:
- The stream was H.265 (HEVC), not H.264 — decoded fine once told, but every default assumption needed correcting.
- A stripped-down ffmpeg build (accidentally copied in from a video editor during testing) that could read the stream fine but could never actually open a network listener for VLC to connect to — swapped for a proper build and serving worked immediately.
- A threading bug of my own making: the camera SDK delivers frames on its own internal thread, and if ffmpeg’s input pipe ever backed up, that thread would block — silently freezing frame delivery from the camera entirely. Decoupling the two with a queue and a dedicated writer thread fixed it for good, and let frames drop gracefully under load instead of the whole pipeline seizing up.
Where it ended up
What started as “why won’t this login” became a proper local streaming bridge:
- Multiple cameras, each independently configured
- Both main and sub streams available simultaneously
- A single shared port option via MediaMTX, with genuine on-demand streaming — the camera session only starts when someone actually connects, and stops again shortly after they leave
- Clean shutdown handling, auto-restart when a viewer disconnects, and accurate stream timing
All from a device that offers precisely zero official ways to do any of this.
The AI-pair-programming part
I want to be specific about how Claude.ai actually helped, because it wasn’t “write me a script” — it was closer to working alongside someone who could read a hex dump, reason about NAL unit structure, cross-reference an open-source SDK header against an undocumented DLL, and change theories cleanly when the evidence pointed somewhere new. Several points in this project turned entirely on inspecting raw bytes from packet captures and file dumps rather than guessing — and having something that could do that analysis quickly, and then just as quickly admit when a theory was wrong and move to the next one, made the difference between this being a weekend project and an unsolved mystery.
Takeaways
- When something “just doesn’t work,” look at the actual bytes on the wire. Every real breakthrough here came from inspecting raw captures directly rather than reasoning from documentation that didn’t exist.
- A consistent, repeatable failure pattern is a clue, not noise. The same two NAL types corrupting over and over was the signal that led straight to the fix.
- Don’t trust bundled binaries blindly. Two separate “this is broken” dead ends turned out to be a broken DLL build and a swapped-in wrong ffmpeg copy — not the architecture.
- Cheap cloud hardware isn’t a black box if you’re willing to open it up. No API, no docs, no problem.
Got a similar reverse-engineering project or IoT integration challenge? That’s exactly the kind of hands-on problem-solving we work through
Leave a Reply