Once I finished up the secure sockets logic in ScarletDME, I then updated my webserver, SERAPHIM, to start using the secure sockets. This step was very simple as I had kept the interface of the secure functions the same as the regular functions. This meant that it was largely just copy paste to add support for secure sockets.
I got everything working and pretty quickly I had my blog being served out over https and I could browse my blog properly. This was a great sign. I loaded this to my real server and then I started SERAPHIM properly.
I refreshed the browser and quickly ran into errors on my server.
I quickly realized what the issue was as it was painfully obvious what was happening and I'm surprised I didn't catch it earlier.
First, let's go back to when I first wrote SERAPHIM. I had written SERAPHIM as a single threaded web server that handled things sequentially. In an effort to add some form of multithreading, I added logic to phantom off a process so that the server could process a request while also listening to a request. However, Pick doesn't have real multithreading, it has PHANTOMs. The way I was using them was that I would start the server socket, wait for a client to connect, kill the server socket, phantom off a new process that will go and start a new server socket elsewhere, and finally process the client that connected.
This logic worked but there is a moment where there is no one listening and so I put caddy in front of my server to hold on to connections that come during the very small periods of time in between the server socket being turned off and then being turned on.
Now this logic works fine with regular http, there isn't any real state so one server socket is no different from the other.
However for secure sockets, now there is some state. When the server is started for the first time, all the SSL logic is set up and pointers are maintained. The server socket listens for a client and when a client connects, they need that SSL context that was already set up so that they can communicate with the server and the server can communicate with the client.
The snag is that if I shutdown my server socket, that context is free'd up. The server socket is closed, the phantoming process starts a new server socket somewhere else and the client socket is then left with an unavailable SSL context. The errors were very much about this.
I don't think I have an out for this issue, I could keep the SSL context on the client socket but that would mean that each client will get their own context and each client will close out their ssl context. This places the onus on the handling of the client socket which doesn't feel right. This is server socket logic infecting the client socket because of how I am building an application much higher up.
The solution that I ended up reaching was to remove my phantoming logic. I always thought this logic was hacky and so removing it now seems like the best choice. If I make the server truly single threaded, the issues with the SSL context disappear. The problem now is that my server is single threaded. However I could deal with that in other ways that make much morse sense than trying to get a fake version of multithreading working.
So now my blog is working and using the secure sockets code that I wrote up!
We'll see how long the site actually stays up before it get's taken down. I would like to do the ssl termination in SERAPHIM but I could always leave it in nginx if I have to.