Author's Note - Author thought he would have a write up before this outlining how to create a streaming site. It currently doesn't exist and may never exist. This is a guide to write a web server in general!
Stream-V1 was my hacky solution to have my own streaming site. I largely relied on h264 and aac so that I didn't have to do anything fancy. My current streaming site is only 150 lines of code but it does have a few dependencies.Currently I basically get a directory and list all the files in it, if you click a file, it will go to a new web page that will display the video in the browser and you can then then watch it.
This was a pretty simple method of getting all of my content easily accesible in my home network.
Now in this chapter we're going to remove the dependencies!
The current dependencies are actix_web, tera and serde. I have some ideas about how to rewrite tera but not so much actix_web so I'm going to skim the rust book's tutorial. For some reason the rust book just does not click with me and it puts me off. Serde we may not need because we are going to manually handle things.
One large reason for this is so I can bring the compile times down. Right now they seem to be tooooo long.
Let's get started!
As with all servers the first step is surprisingly the easiest! We need to open a socket and start listening to it.
use std::net::TcpListener;
fn main() {
let ip_address = "0.0.0.0";
let port = "7083";
let socket = TcpListener::bind(format!("{}:{}", ip_address, port)).unwrap();
println!("Server is running at {}:{}", ip_address, port);
for data in socket.incoming() {
let data = data.unwrap();
println!("{:?}", data);
}
}
Here, the first thing to notice is that we are going to use the standard library and use the net package. This is what is going to let us bind to a TCP socket. In our main function we specify that we want to bind to address 0.0.0.0, this way we don't have to be on the local machine to test.
I am building on a dev server that is elsewhere. Had I put 127.0.0.1, our server would only be accesible from within the machine. We also specify the port we want to listen on. This port also needs to be opened on the OS level, I opened all ports between 7000-8000 as that way I don't have to keep manually opening ports.
Once we have bound a TCP port to a variable, we now begin the blocking call. socket.incoming() sits and waits until data comes in on it. We read the data in and we print it directly out.
cargo-watch -x run
[Running 'cargo run']
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
Running target/debug/b-server
Server is running at 0.0.0.0:7083
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58714, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58715, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58716, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58717, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58718, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58719, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58720, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58721, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58722, fd: 4 }
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58723, fd: 4 }
We can not navigate to the ip address of the machine running the server and go to the port. In my case this was 192.168.7.41:7083. Once we hit that page we should see a connection reset in our browser. However on our web server we should see the above data.
The above shows that we received a message from 192.168.7.7 through various ports, this is because the browser is making multiple requests to the site and may even be trying to guess what is on the server. For instance it may be requesting a favicon automatically.
The data object we are reading in now is actually a TcpStream. Let's read the data that is inside.
use std::io::prelude::*;
use std::net::TcpListener;
fn main() {
let ip_address = "0.0.0.0";
let port = "7083";
let socket = TcpListener::bind(format!("{}:{}", ip_address, port)).unwrap();
println!("Server is running at {}:{}", ip_address, port);
for session in socket.incoming() {
let mut session = session.unwrap();
let mut buffer = [0;2048];
session.read(&mut buffer).unwrap();
let req = String::from_utf8_lossy(&buffer);
println!("{:?}", session);
println!("{}", req);
}
}
The first thing we do is we want to bring the read and write traits into scope. This is because the TcpStream type uses those traits and we want to use them as well. The data variable is a TcpStream object and once we have the read and write traits, we can easily manipulate and use our sockets and data.
The first thing we do inside our loop is set up our buffer. HTTP request sizes usually range from 2kb to 8kb. For our server, we're going to set it to 2kb. So we start by setting up a buffer which is an array of u8, unsigned 8 bit integers, to 0. We then read in 2kb of data from the session. We then convert the binary we read in to a string. We use the loss version of the utf8 conversion function because if for some reason the text isn't utf8 we'll replace the offending characters with ?.
Lastly, we print out the request.
Now let's to the server's url again.
Server is running at 0.0.0.0:7083
[Running 'cargo run']
Compiling b-server v0.1.0 (/home/nivethan/bp/b-server)
Finished dev [unoptimized + debuginfo] target(s) in 0.61s
Running target/debug/b-server
Server is running at 0.0.0.0:7083
TcpStream { addr: 192.168.7.41:7083, peer: 192.168.7.7:58841, fd: 4 }
GET / HTTP/1.1
Host: 192.168.7.41:7083
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: compact_display_state=false; filter=all
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Voila! We can now see the raw data that the browser is sending to our server. That's all the internet is!
Now let's return something!
...
for session in socket.incoming() {
let mut session = session.unwrap();
let mut buffer = [0;2048];
session.read(&mut buffer).unwrap();
let req = String::from_utf8_lossy(&buffer);
println!("{:?}", session);
println!("{}", req);
let response = "HTTP/1.1 200 OK\r\nContent-Length: 2\r\nContent-Type: text/plain\r\n\r\nOK";
session.write(response.as_bytes()).unwrap();
session.flush().unwrap();
}
...
So now instead of just reading from our session, we are also going to write out a response. In this case HTTP has a specific format we need to adhere to. This is why we have the various parts of the responses. We then write this out to the session as bytes.
HTTP Responses are delimited by carriage-return line-feeds. The first part of the response is always the status, then followed by it are the headers seperated by cr-lf. Finally 2 CR-LF signify the end of the headers and the start of the body. The body itself has no terminating character.
So our response above is saying the HTTP request succeed, we are sending back a response where the body is 2 characters. We set the content type to text/plain and finally at the end our message is OK.
If we navigate to our browser we should now see data being printed on the browser!
Now this is the kernel of our web server! You can see here the request response cycle happening. The browser makes a request and sends over text, we get that text and then send back text. HTTP is just the format of those messages!
Now let's parse the request so we can do different things based on what we get!
Our parser is going to be very barebones, we aren't focused on defensiveness yet, for now we are going to get the structure of our parser working and then we will go back to tighten things up.
This way we can focus on the forest before we get stuck in the weeds.
Let's get started!
The first thing to understand is parsing isn't anything special! We currently get a request from the browser and we are printing that to the screen. We just want to make this string a little bit easier to manipulate. Wouldn't be great to be able to just reference the method or URI in the request without have to do string slicing and using offsets to get what we want. That's what parsing does!
We are going to take our request string and make an object out of it. This way we can easily manipulate the request.
The first thing we do is read the spec!
A HTTP request has very simple format, not quite as simple as Gopher or Gemini but simple nonetheless.
A request is delimited by carriage-return line-feeds. A double carriage-return line-feed marks the end of the headers and the start of the body. The body simply just ends.
The first part of the request is the request-line. This is the part with the method, the URI and the version.
GET / HTTP/1.1\
This is delimited by spaces.
The next part of the request is the headers. This will be a series of key value pairs. This can be any number of headers in any combination. This is where HTTP's complexity comes from!
Host: 192.168.7.41:7083\r\nUser-Agent: Mozilla/5.0
Our headers which are split by carriage-return line-feeds and then also split onthe colon. We will need some sort of hashmap for this.
Finally we have the body that will take up the rest after the double carriage-return line-feed. In our server, the body is what will get the junk characters. We are reading in a fixed amount of bits from out socket and many times HTTP requests will be smaller. So most of what we read may be junk and all of that junk will fall into the body.
For now we are going to let that be but in the future we are going to clean it up so we can handle getting data in the body as well!
Now that we have the layout of the request, let's get into the code!
...
#[derive(Debug)]
struct HttpRequest {
method: String,
uri: String,
version: String,
headers: HashMap<String, String>,
body: String
}
impl HttpRequest {
fn new(request_data: String) -> Self {
let r: Vec<&str> = request_data.splitn(2, "\r\n\r\n").collect();
let request_data = r[0];
let body = r[1].to_string();
let r: Vec<&str> = request_data.splitn(2, "\r\n").collect();
let status_line = r[0];
let s: Vec<&str> = status_line.split(" ").collect();
let method = s[0].to_string();
let uri = s[1].to_string();
let version = s[2].to_string();
let header_raw_data = r[1];
let header_data: Vec<&str> = header_raw_data.split("\r\n").collect();
let mut headers: HashMap<String, String> = HashMap::new();
for header in header_data {
let key_value: Vec<&str> = header.splitn(2,":").collect();
headers.insert(key_value[0].to_string(), key_value[1].to_string());
}
HttpRequest { method, uri, version, headers, body }
}
}
...
The first thing we do is build a HttpRequest struct, this is what is going to hold our request data.
Our new function is what will do the actual parsing. We know what the structure of the request is going to be so for now we will build strict for it. Any deviance from this structure will cause our application blow up! But that's alright!
We first split our request into 2 parts, we want the header of the request and the body of our request to be seperated.
Next we split on just the first carriage-return line-feed. This is because we want to split the header into the status line and the headers.
Next we split the status line on a single space. This will give us the method, the uri and the version.
We then split the header by carriage-return line-feed. Now we have a variable number of headers that we will loop through and add to our headers hashmap.
Voila!
We have now converted a request from a string to a useable object!
...
session.read(&mut buffer).unwrap();
let request_data = String::from_utf8_lossy(&buffer);
let request = HttpRequest::new(request_data.to_string());
println!("{:?}", request);
...
Let's update our main loop to use our HttpRequest object.
Now when we navigate to our browser we should see our request getting printed out as an object! The body will contain junk but the rest of the fields should match the request that came in!
Now that we have the parsing done and we have our HttpRequest object, let's have it serve up a file!
Now that we have our request, we want to send back a response based on it. The big picture is that we have a series of checks we do for a request. The first thing we need to do is use the method to figure out what the browser is asking for. If it is a GET, then we know that we need to send something back, if it is a POST however, we know we are receiving information. There are other methods like PUT, DELETE and UPDATE but for now let's just focus on GET.
The next thing we need to look at is the URI. If the request is a GET, we then need to find the file specified by the URI and return that the browser.
Thats it!
Let's jump into the code!
for session in socket.incoming() {
let mut session = session.unwrap();
let mut buffer = vec![0;2048];
session.read(&mut buffer).unwrap();
let request_data = String::from_utf8_lossy(&buffer);
let request = HttpRequest::new(request_data.to_string());
let base_dir = "./html";
let response = if request.method == "GET" {
let filename: String = if request.uri == "/" {
"index.html".to_string()
} else {
request.uri
};
let path = format!("{}/{}", base_dir, filename);
if Path::new(&path).exists() {
let content = fs::read_to_string(&path).unwrap();
let mime_type = Path::new(&path).extension().unwrap().to_string_lossy();
let mime_type = if mime_type == "js" {
"javascript".to_string()
} else {
mime_type.to_string()
};
let content_type = format!("text/{}", mime_type);
format!(
"HTTP/1.1 200 OK\r\nContent-Length: {content_length}\r\nContent-Type: {content_type}\r\n\r\n{body}",
content_length=content.len(),
content_type=content_type,
body=content
)
} else {
"HTTP/1.1 404 NOT FOUND\r\n\r\nNot Found".to_string()
}
} else {
"HTTP/1.1 500 Internal Server Error\r\n\r\nIn Progress".to_string()
};
session.write(response.as_bytes()).unwrap();
session.flush().unwrap();
}
Inside our server's main loop we have just converted a request string to a request object.
The next step is to now use that request to server the requested file.
We set the base_dir variable as we do need a starting location, this can be anywhere you want it to be.
Next we check the method on the request, if it's not a GET request, we are going to return a Internal Server Error message. This is because our server doesn't have an implementation for any other method!
Once we have a GET request we check the URI. Here we are checking to see if the URI is a single /, we are going to assume a signle / means that the browser is looking for index.html. If it's not a single /, then we will assume it is a path to a specific file.
Now the next step is to check if the file exists. If it doesn't we'll return a 404 Not Found error to the browser.
If we find the file, now we can read it in and set the content type, content length and body. The content type requires expects certain things for the various files that a website uses, html files are text/html, javascript files are text/javascript and css files are text/css.
Voila! We have our response. Now if we create a html file in our ./html folder we should start seeing things happening!
./html/index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Our First Page</title>
<link rel="stylesheet" href="/css/style.css">
</head>
<body>
<h1>Hello, world!</h1>
<script src="/js/script.js"></script>
</body>
</html>
If we navigate to our server, we should now see Hello World on the page! If we open the inspector tool, we should also see requests to the style and javascript saying 404 Not Found.
We can create those directories and files and everything should start working!
A little bit of clean up would be to move the response logic into it's own function. Currently we have all of it sitting in our main loop which is cluttering it.
fn handle(buffer: Vec<u8>) -> String{
let request_data = String::from_utf8_lossy(&buffer);
let request = HttpRequest::new(request_data.to_string());
let base_dir = "./html";
let response = if request.method == "GET" {
let filename: String = if request.uri == "/" {
"index.html".to_string()
} else {
request.uri
};
let path = format!("{}/{}", base_dir, filename);
if Path::new(&path).exists() {
let content = fs::read_to_string(&path).unwrap();
let mime_type = Path::new(&path).extension().unwrap().to_string_lossy();
let mime_type = if mime_type == "js" {
"javascript".to_string()
} else {
mime_type.to_string()
};
let content_type = format!("text/{}", mime_type);
format!(
"HTTP/1.1 200 OK\r\nContent-Length: {content_length}\r\nContent-Type: {content_type}\r\n\r\n{body}",
content_length=content.len(),
content_type=content_type,
body=content
)
} else {
"HTTP/1.1 404 NOT FOUND\r\n\r\nNot Found".to_string()
}
} else {
"HTTP/1.1 500 Internal Server Error\r\n\r\nIn Progress".to_string()
};
response
}
Here we moved everything to a handle function, we are also passing in the raw data we read in from the socket.
for session in socket.incoming() {
let mut session = session.unwrap();
let mut buffer = vec![0;2048];
session.read(&mut buffer).unwrap();
let response = handle(buffer);
session.write(response.as_bytes()).unwrap();
session.flush().unwrap();
}
Our main loop is alot cleaner now!
Alright! We have our request being made into an object and we have it being handled well enough for now. Our little web server is starting to get bigger! The next step is to make our server multithreaded. Currently we handle requests in sequence so if for some reason a request causes our server to slow down then it will slow down everyone who accesses our server. Multithreading will solve this by putting requests in their own threads on the server.
Let's get after it!
Okay! This chapter is a doozy! Multithreading is always a pain in the ass, up and till you get it working. At which point its actually very straightforward.
Let's go over what the general structure of threading and thread management looks like. We currently have requests coming in and we handle them sequenetially. This means if one request causes an expensive event then it will have reprecussions on the entire server.
For instance, if a request comes in and we need to ping another server for data, then every request needs to wait for that ping to finish. Instead what we should do is send requests to their own threads, that we take advantage of the full resources available to us on our server.
However we don't want to spin up a thread for every request, this is because if we spike in requests, our server could end up using all its resources spinning up threads and melt.
So we want to spin up threads but we want to have a sane number of threads, if we get more requests than threads, then the requests need to be queued up and processed.
Big picture is we want to maintain an array of threads that we can send requests to. Ultimately a request is just a string, but the processing of it is a function! This is the key idea, we want to create a function for each request and send each function to a thread that.
Once the function ends, we want the thread returned to use so we can use it again.
Now that we have the gist of multithreading, let's take a look at the code!
...
use std::{fs, thread};
use std::sync::{mpsc, Arc, Mutex};
...
fn main() {
let ip_address = "0.0.0.0";
let port = "7083";
let socket = TcpListener::bind(format!("{}:{}", ip_address, port)).unwrap();
println!("Server is running at {}:{}", ip_address, port);
let number_of_threads = 10;
let mut pool = Vec::new();
let (sender, receiver) = mpsc::channel::<Box<dyn FnOnce() + Send>>();
let receiver = Arc::new(Mutex::new(receiver));
for i in 0..number_of_threads {
let receiver = Arc::clone(&receiver);
pool.push(thread::spawn(move || loop {
let job = receiver.lock().unwrap().recv().unwrap();
println!("Executing...{}", i);
job();
})
);
}
for session in socket.incoming() {
let job = Box::new(|| {
let mut session = session.unwrap();
let mut buffer = vec![0;2048];
session.read(&mut buffer).unwrap();
let response = handle(buffer);
session.write(response.as_bytes()).unwrap();
session.flush().unwrap();
});
sender.send(job).unwrap();
}
}
There's alot going on but let's go line by line starting with the number_of_threads.
The number_of_threads is what is going to act as our limiter. For now we are going to stick to 10.
The next thing we do is intialize our thread pool variable. This is what will contain our list threads.
Now we get the crux of our threads. The channel. The channel is how we communicate to a thread. We are going to create a sender and a receiver and then inside each thread we are going to leave behind a receiver. This means that we can now send data to any of our threads.
The type of the channel is key. We want to send closures to our receiver so we set the type on the channel to FnOnce. I'm not sure why we need the Send trait but it likely has something to do with the way the channel works. The other thing to note is that we bundle this type as a Box. This is because we want this thrown to the heap. This is because we don't know how big a function is as it can be any size. So we can't stick it on the stack as it doesn't have a fixed size.
Now the next thing we need to do is allow for the receiver to by synchronized. We want to make sure the threads and the receivers in them are locked when in use. I'm not entire sure of why we the Arc portion of it as to me the Mutex appears to be enough. The mutex is what locks and unlocks the receiver.
The next thing we do is intialize the threads and store them in our pool array.
We spawn a thread and we move the an entire closure into it. The closure contains the receiver we created above. The receiver locks itself and then waits for data to come through. The .recv() is a block call so all our threads will be sitting on the line of code waiting for data!
When we get data on the recv() call, we will be getting a function which we will then assign to job and run it.
! We have our threads ready to work! We have a series of threads spun up with a receiver inside each one waiting for a function.
Next we get to our main loop.
This time instead of running the session handler logic right away, we are going to assign the logic to a variable. This way we can save the function and pass it to a thread.
This is where we use the sender that we created through channel. We send the function we just created and it will get automatically assigned to a receiver. That receiver is sitting in a thread just waiting to do something.
! Voila! We have just wrote out multithreading webserver. Now the sender portion I consider magic. It bothers me to say automatically assigned as something about it doesn't sound right. Somewhere inside the channel I'm guessing the sender and receiver logic is explicit. It also seems strange to call the library mpsc, multiple producers, single consumer as in my eyes it appears that we have a single producer, sender, and multiple consumers, receiver clones.
I also don't know what the Mutex is really doing to the receiver, I'm guessing its a decorator of some sort. A lot of things bother me about this chunk of code.
But for now I'll let it be!
On to the next piece.
Now that our server is multithreaded and it can parse requests, its time to start working on our server's core functions. The core of our web server is really the application server. We want to write functions to handle various end points which really means we want to run specific functions for specific requests.
If we get a request for /index.html, we want to serve the page, index.html, however if get a request for /index. We want to run a function for /index. This is actually straightforward. Currently we return a 404 if we can't find the file, but we can add another case, if we can't find the file, then let's check a dictionary of functions and see if theres an entry.
Because our server is now multithreaded, we're going to have to start thinking about how to share data across threads. In this case, our function dictionary is going to be a hashmap of strings and functions that we need all the threads to access. Luckily our dictionary will be immutable so we won't have to worry about locking before writing.
Let's look at some code!
...
fn index(request: HttpRequest) -> String {
println!("{:?}", request);
String::from("Hello from the Index!")
}
...
fn sleep(request: HttpRequest) -> String {
println!("{:?}", request);
thread::sleep(std::time::Duration::from_secs(5));
String::from("Hello from the Sleep!")
}
...
...
fn main() {
let mut function_dictionary: HashMap<&str, Box<dyn Fn(HttpRequest) -> String + Send + Sync>> = HashMap::new();
function_dictionary.insert("/index", Box::new(index));
function_dictionary.insert("/sleep", Box::new(sleep));
let function_dictionary = Arc::new(function_dictionary);
let ip_address = "0.0.0.0";
let port = "7083";
...
Here we start by defining out function dictionary witha specific type. We want to hold functions in our hashmap and so we set the type accordingly.
Now we don't know how big our functions are going to be so we're going to put inon the heap. This is why we Box up our function so that the size doesn't matter and we can toss it onto the heap.
We want our functions to take in an HttpRequest and return out String.
Finally, we want out function to have the thread safety traits, normally types would automatically get these traits but because we are putting things on the heap, we need specify them explicitly.
Voila! We have the type of our thread safe function dictionary now.
Next we insert in a few functions and their endpoints into our dictionary.
Lastly, we want to create a reference counted version of our function dictionary. This way we can pass it into threads and keep track of what's currently using the hashmap.
...
for session in socket.incoming() {
let function_dictionary = Arc::clone(&function_dictionary);
let job = Box::new(|| {
let mut session = session.unwrap();
let mut buffer = vec![0;2048];
session.read(&mut buffer).unwrap();
let response = handle(buffer, function_dictionary);
session.write(response.as_bytes()).unwrap();
session.flush().unwrap();
});
sender.send(job).unwrap();
}
...
Now we update our main loop, here we will clone our function dictionary, this has the effect of incrementing a reference counter held in Arc. We are also setting function dictionary to a pointer here.
We are then going to send our handle function the pointer to the function dictionary. This way we don't have duplicate dictionaries, rather every thread will use the same one. Because we aren't updating the function dictionary, we don't need to lock it and can just pass it using Arc.
Now let's use our dictionary in our handle function.
...
fn handle(buffer: Vec<u8>, function_dictionary: Arc<HashMap<&str, Box<dyn Fn(HttpRequest) -> String + Send + Sync>>>) -> String {
...
The first thing we do is update our function name so that we pass in the function dictionary correctly.
...
if Path::new(&path).exists() {
let content = fs::read_to_string(&path).unwrap();
let mime_type = Path::new(&path).extension().unwrap().to_string_lossy();
let mime_type = if mime_type == "js" {
"javascript".to_string()
} else {
mime_type.to_string()
};
let content_type = format!("text/{}", mime_type);
format!(
"HTTP/1.1 200 OK\r\nContent-Length: {content_length}\r\nContent-Type: {content_type}\r\n\r\n{body}",
content_length=content.len(),
content_type=content_type,
body=content
)
} else {
let uri = request.uri.clone();
let uri = uri.as_str();
if function_dictionary.contains_key(&uri) {
let f = function_dictionary.get(&uri).unwrap();
f(request)
} else {
"HTTP/1.1 404 NOT FOUND\r\n\r\nNot Found".to_string()
}
}
...
We first check to see if the path exists on the file system, if it does, we will send the file directly.
The new part is now, if we don't find a file that matches, then we check to see if the function_dictionary contains the request uri. If it does contain it, we retrieve the function associated with it and run it with the request we received.
Voila! We have just wired up function dictionary into all of our threads. We can now navigate to our browser and test this our by hitting our index or sleep end points. We can also run multiple sleep end points and see that everything is getting threaded properly!
Next, we'll work on making a HttpResponse object, that way we don't just return strings and have to write out all of the various HTTP statuses.
See you soon!
Now that we have our server and our application server running, let's clean up our responses. Currently we send back straight strings from our functions, which means that we need to construct and calculate the lengths of the content and set the statuses correctly.
We're going to make a response object that will do all the boilerplate for us.
Rewriting out responses from a string to HttpResponse involves making changes all over the place so they best place to start is at the end so we can see what we are working towards! It will be easier to implement the HttpResponse struct first and then follow the code in section 1 but we'll start here as it'll help to see what our end game is.
Let's get started!
This is how we'll be using the HttpResponse throughout our code!
...
if request.method == "GET" {
let filename: String = if request.uri == "/" {
"index.html".to_string()
} else {
request.uri.clone()
};
let path = format!("{}/{}", base_dir, filename);
if Path::new(&path).exists() {
HttpResponse::ok().file(path)
} else {
let uri = request.uri.clone();
let uri = uri.as_str();
if function_dictionary.contains_key(&uri) {
let f = function_dictionary.get(&uri).unwrap();
f(request)
} else {
HttpResponse::not_found()
}
}
} else {
HttpResponse::internal_server_error()
}
...
We've now replaced alot of our code so that the Response object will handle setting up the mime types and content length.
In the above code we have 3 main types, we have ok(), not_found() and internal_server_error() on the response object. All these functions return a HttpResponse. We can then add a file path to the response and the response will go in and get the mime type and content length and fill those in for us.
...
fn index(request: HttpRequest) -> HttpResponse {
println!("{:?}", request);
HttpResponse::ok().body("Hello from the Index!")
}
fn sleep(request: HttpRequest) -> HttpResponse {
println!("{:?}", request);
thread::sleep(std::time::Duration::from_secs(5));
HttpResponse::ok().body("Hello from the sleep!")
}
...
Our two end point functions we now return HttpResponses as well. Here instead of a file we use the body function to set some raw text.
Now, anywhere we return a String, we need to update to return HttpResponse. This is also true for our function dictionary type.
...
let mut function_dictionary: HashMap<&str, Box<dyn Fn(HttpRequest) -> HttpResponse + Send + Sync>> = HashMap::new();
...
....
fn handle(buffer: Vec<u8>, function_dictionary: Arc<HashMap<&str, Box<dyn Fn(HttpRequest) -> HttpResponse + Send + Sync>>>) -> HttpResponse {
...
Now that we have seen how the HttpResponses have been are going to be used, let's get started on the implementation!
In this section we'll look at the actual implementation of our HttpResponse object. This is going to be a bigger chunk of code than the request as we'll be constructing various different things responses for a browser's requests.
Let's get started!
...
#[derive(Debug)]
struct HttpResponse {
version: String,
status_code: usize,
status_text: String,
headers: HashMap<String, String>,
body: String
}
impl HttpResponse {
...
}
...
Here we have our HttpResponse with the core pieces it needs. This is a straightforward translation of the String we had constructed as a response.
Now let's look at the first type of response we'll send - the OK
...
impl HttpResponse {
...
fn ok() -> HttpResponse {
let version = "HTTP/1.1".to_string();
let status_code = 200;
let status_text = "OK".to_string();
let headers: HashMap<String, String> = HashMap::new();
let body = "".to_string();
HttpResponse {
version, status_code, status_text, headers, body
}
}
...
}
...
We initialize our HttpResponse here with some set values. The key is our status_code and status_text.
...
fn not_found() -> HttpResponse {
let version = "HTTP/1.1".to_string();
let status_code = 404;
let status_text = "NOT FOUND".to_string();
let headers: HashMap<String, String> = HashMap::new();
let body = "Page could not be found.".to_string();
HttpResponse {
version, status_code, status_text, headers, body
}
}
fn internal_server_error() -> HttpResponse {
let version = "HTTP/1.1".to_string();
let status_code = 500;
let status_text = "INTERNAL SERVER ERROR".to_string();
let headers: HashMap<String, String> = HashMap::new();
let body = "500 - Internal Server Error".to_string();
HttpResponse {
version, status_code, status_text, headers, body
}
}
...
Next we initialize our not_found and interal_server_error versions of HttpResponse. Here we want to set the body as well.
With that we have 3 http responses ready to be used. Now the next thing we need to do is write out body functions. We want to be able to do .body("some text") and send have our response have the body containing that text. We also want to be able to specifiy a file and have the response get the file and fill in the relevant information.
impl HttpResponse {
...
fn body(mut self, body: &str) -> Self {
let content_type = "text/html".to_string();
let content_length = body.len();
self.headers.insert("Content-Type".to_string(), content_type);
self.headers.insert("Content-Length".to_string(), content_length.to_string());
self.body = body.to_string();
self
}
...
}
...
Our body function is straightforward, we pass in a string slice and we set that the HttpResponse object's body. We also add 2 headers to our currently empty header hashmap.
...
impl HttpResponse {
...
fn file(mut self, path: String) -> Self {
let body = fs::read_to_string(&path).unwrap();
let mime_type = Path::new(&path).extension().unwrap().to_string_lossy();
let mime_type = if mime_type == "js" {
"javascript".to_string()
} else {
mime_type.to_string()
};
let content_type = format!("text/{}", mime_type);
let content_length = body.len();
self.headers.insert("Content-Type".to_string(), content_type);
self.headers.insert("Content-Length".to_string(), content_length.to_string());
self.body = body;
self
}
...
}
...
Our file function is similar to our body but instead of setting the mimetype by default we want to use the file's extension instead. We also need to read the file in here and set it to the HttpResponse's body.
With these 2 functions we now can send data to the browser using our HttpResponse!
The last thing we need to do is write a function that gives us a String of our HttpResponse object that we can send to the browser!
impl HttpResponse {
...
fn data(self) -> String{
if self.headers.is_empty(){
format!(
"{version} {status_code} {status_text}\r\n\r\n{body}",
version=self.version,
status_code=self.status_code,
status_text=self.status_text,
body=self.body
)
} else {
format!(
"{version} {status_code} {status_text}\r\n{headers}\r\n\r\n{body}",
version=self.version,
status_code=self.status_code,
status_text=self.status_text,
headers=self.headers(),
body=self.body
)
}
}
...
}
We have 2 versions of our response that we can create, we have one where if there are no headers, we leave the headers our. This can happen in the case of where we are sending back error pages. The other case is when we send back data we will need to format out headers as well.
...
fn headers(&self) -> String {
let mut headers_string = Vec::new();
for (key, value) in &self.headers {
headers_string.push(format!("{}:{}", key, value));
}
headers_string.join("\r\n")
}
...
Our headers() function will give us back a formatted string that we can then put in our HttpResponse String.
With that we have finished writing out HttpResponse object!
We now have a our object, we can now go back to section 1 and rewrite everywhere we use Strings to use HttpResponse.
This should simplify and make it easier to reason with our code!
The last thing we need to do is use the HttpResponse object when sending data back to the browser.
...
let job = Box::new(|| {
let mut session = session.unwrap();
let mut buffer = vec![0;2048];
session.read(&mut buffer).unwrap();
let response = handle(buffer, function_dictionary);
session.write(response.data().as_bytes()).unwrap();
session.flush().unwrap();
});
...
Now we can use response.data() to get the String from the HttpResponse object and we then send out converting it to bytes.
Voila! We are now done! We now have our HttpResponses wired in and working and we should be able to navigate to our browser and hit our index.html or our function end points and everything should be working.
At this point we have a very functional web server! This only took about 250 lines!
This chapter was alot of grunt work but it had to be done, the next chapter I hope is a little bit better as in that chapter we'll be looking at templating!
See you soon.
Now that we have our web server up and running and it's being multithreaded, it's time to focus on our application server. We did get it started with our function dictionary but in this chapter we're going to extend it with a templating engine.
Currently our web server is analogous to nginx, if it gets requests, it will respond with files if a file was requested or with data if an endpoint was requested.
Now we're going to make it so that our end points can merge data and a template together!
Let's get started!
I took a stab at writing a template engine as I went but as I thought I'm not entire capable yet. I got a very simple but fragile templating engine going but it can't handle many things and it just doesn't fit together like the crafting interpreters language I worked on. With that, as I went things fell into place and it led to easy extension. My version is hard to reason with and extend.
So I think i will end my templating language adventure and go to back and readthe book and then come back to this.