Interrupts and Request Memoization

Node Tips and Tricks for faster APIs (3)

4 min readApr 13, 2018

In the previous part of this article series, we discussed handling blocking operations in our APIs with child processes.

In this article, we’ll take a look responsiveness and efficiency, as areas we can focus on to improve our APIs.

Response Interrupts (Responsiveness)

Consider the following two conversations …

David: Hey Mark, can you convert these files to PDF for me?
Mark: *takes 30 mins*
David: Mark, are you there?
David: Mark???
Mark: Here, David … Here are your files

David: Hey Mark, can you convert these files to PDF for me?
Mark: Sure, it could take a while though
David: Okay, I’ll check on you every 10 minutes

In the real-world, some tasks can, and will take too long to execute. API clients like browsers have a request timeout which is typically 5 minutes.

You may not want to keep connections open that long.

In Node, it is possible to interrupt a long-running connection, and respond with a tentative location of the result.

For example, if our API endpoint were to parse a large XML document which could take 60 seconds to complete, we could respond with a 202 HTTP status code response after 3 seconds, and place the URI location for our result in the response body or header.

The client could then poll that URI location at intervals.

When polled,

If not ready still, respond with a 202.
If ready, respond with a 200
If complete, but with errors, respond with a 500, so the client can handle it.

This way, your connections are not kept open too long. It may not speed up your API, but it’d make it more responsive to clients.

For Example,
We’ll use a slightly different API. One that computes the factorial of a supplied number.
We will delay the computation by 10 seconds to simulate a large amount of work to be done, then interrupt the response at 500 milliseconds.

Located here.

So with this API running, when you load http://localhost:3030/factorial?value=15 in your web browser, you’d get a response that looks like:

{ "location": "http://localhost:3030/factorial/1" }

Which gives you information on where the result of the computation would be when finished.

You can then load http://localhost:3030/factorial/1 in the browser, and when it’s ready, it’d give you the result.

If you look through the code, I am using a simple array to store the results, but in production, you may need something much more sophisticated like a database.

Request Memoization (Efficiency)

In the factorial API used in the previous section, when a user requests the factorial of 15, gets the result, and requests it again, should we calculate the factorial again?

If we have access to the previously computed result of the factorial of 15, we don’t need to, and this is the basis of memoization.

If your API endpoints are pure (forgive the term), meaning that they will always give the same result for an input, you can afford to memoize the request.

For example,
We have an API endpoint that accepts large text files, we reverse the string on each line and return the result.
If we send the same text file more than once, it would have to process it every time, which is a waste of CPU time, since we can afford to remember the result for the last time it was processed and return it.

Located here

We added express-body-parser to help us with retrieving the text input from the request.

To test, run this program and POST data in plain-text to http://localhost:3030/. For example,

andela
meme
scooby-doo
leadership
africa

… it should respond with

aledna
emem
ood-yboocs
pihsredael
acirfa

While it is possible for us to maintain a decent map of input to output in memory, or in a persistent storage like a Database or cache service like Redis, we could leverage the power of hashes to identify request inputs.

Hashing scrambles plain text or bytes to produce a unique message digest, which means no two different inputs will produce the same hash.

A typical hash string is about 16 or 32 characters, which is much smaller than the potential input files which could be as large as 100MB depending on your server capabilities.

This means no matter how large our request input is, we can identify a duplicate anytime by using its hash, so we don’t have to store the entire input-output map.

Let’s see how that is done …

Located here

The calculateHash function computes the hash of the request body and stores it in the req.hash property.

For simplicity, we’re responding with the request hash. But this can be extended to compute the reverse result, store it by the request hash, and lookup future requests by its hash.

If your output is large, you should probably store the input-output map/dictionary in a persistent-storage service like a database.

I will leave you to comparing the performance of the string-reversal API endpoint with and without request-memoization. You can give feedback in the comments.

Dealing with “Impurities”

We’ve discussed request memoization in pure endpoints, but we hardly deal with pure endpoints in the real world, so if your API endpoint is “impure”, and you understand exactly what its “impurities” are, you could still use this method by implementing cache-invalidation mechanisms such as timed expirations.

Conclusion — What we have learned

Now, we know these methods for improving the performance of our APIs: