This is really an addon to my previous post of the same name.
Reading specific values into a buffer took longer than just adding the entire data to the buffer. Sort of makes sense but I'm curious what's going on. I didn't think the cost would be so much but reading just the first 2 values was stupidly expensive. One reason could be because universe uses string parsing to get the values. It could be that because I'm doing a read, I need to parse each thing versus MATREAD which would be far quicker for getting individual values but has the cost of setting up the variables.
This is a fun little point where READ is great to get the data quickly but hard to handle whereas MATREAD is slower to get the data but fast to handle.
This also opened my eyes to the fact that the amount of data I'm trying to send is actually quite a bit. How do people ship megabytes of data to the client. I could ship the data from the database to node quickly but then in node I need to strip it down and even stripped down theres a ton of data to send back. I think adding nginx would help as I could gzip the data there but the core problem is still that shipping data is expensive.
I'm guessing the solution might be to page the data but then I need to write backend code to handle pagination of the data which I thought frameworks were supposed to alleviate. The idea being I could do everything in the frontend once we have the data there. I'll need to think about what exactly I'm trying to do and display on the screen.