I'm working on reverse engineering a binary protocol and I think I have the basics of it now working. The bulk of the actual data analysis is done by Claude Code which is doing a terrific job. It has limitatins but I'm finding that it is quite useful in this situation.
I did the initial investigation without any AI tooling, this gave me a foundation that I kept referring back to as I needed to keep things simple so that Claude could simply do the data analysis.
The first thing I did was use tcpdump to capture the packets and then I wrote a python routine that would simply replay these messages to the server. I was able to get this working and could see the data in the stream. Changing the data would work and then I figured out the length field and was able to make at least that part dynamic. This proved that my strategy would work. This simple example is something I kept around as it gave Claude a base to use when it got lost in the weeds.
I used tcpdump and test programs to talk to the server and captured all the traffic. I made clean copies of the packet traces for various commands and variations. This gave me about 10-15 packet traces that I then fed into Claude and had to deduce the protocol.
I initially had it write me a SPEC document for the protocol but quickly realized that having it update wasn't worth it especially with how much things were changing. Instead having a summary file was far more useful, especially with Claude compacting issues. Claude would compact at the worst times and so I think a good move is to update the summary file and then reset the context.
Claude was able to figure out the basic structure of things but I needed to write more tests and do more packet captures to iron out bugs and issues that I found. I can't speak to the code yet because I've only glanced at it but my test routines are working and I've tested the weirder edges now. These edge cases is where I had to generate both good traces from the official client and the bad traces from my own client. having these helped Claude figure out the differences and then it would fix them up.
I think this was a pretty good workflow though maybe I didn't need to be involved and had I given claude the abilty to talk directly to the server it could have done a better job. However being involved made me more aware of what it was doing and I read the notes that Claude was leaving. Some bugs Claude would lose the thread and I had to reset the repo and clear the context. But this still much better than trying to figure out everything myself.
I'm pretty happy with how much Claude did and how much of the protocol is currently working. I'll need to actually look at the code as I want to write a client myself. This would be the true test I think of this entire endevaor. The goal was to basically get a spec and reference implementation in python so that I can then build something in another language.
I think Claude might be able to build out a testing emulator and set of tests that I can then use for my own program. Overall this has been a pretty good experience so far.
Have Claude generate a PROJECT-SUMMARY.md for major things as it does them.
Clear the context early and often and have Claude work off the summary document.
Create the initial tests to give Claude something to point back to.
Use git to make sure you have versions to roll back to
Write scripts to simplify gathering and generating data
Use Ctrl G to open vim to edit the prompt, the more descriptive the better
Organize the folders and make sure that's part of the summary document