Promise-based REST API clients

In software development, the concept of promises (sometimes also called futures) is deceivingly simple.  There's a pretty good wikipedia article that explains it better than I do, and the opening line is about a good a summary as I can think of:

In computer science, future, promise, and delay refer to constructs used for synchronizing in some concurrent programming languages. They describe an object that acts as a proxy for a result that is initially unknown, usually because the computation of its value is yet incomplete.

What that means in normal people English is that you call a method to do something (i.e. compute a value, gather some data, etc) and instead of doing what you requested immediately and making you wait, it returns a "promise" that it will eventually do what you requested.  While this is generally done to make concurrency easier, I think the concept works well for hierarchical REST API clients as well.

Let me explain.  No, no, there is too much.  Let me sum up.

REST APIs are generally hierarchical.  You often see structures like:

GET /artists - get a list of artists
GET /artists/metallica - get the artist Metallica
GET /artists/metallica/albums - get a list of albums by Metallica

And so on.  What I wanted from an API client was to do something along the lines of this Python snippet:

for album in client.artists('metallica').albums:
    album.delete() # take that Metallica

What I didn't want to happen in the above snippets was to do all of these HTTP requests

GET /artists
GET /artists/metallica
GET /artists/metallica/albums
DELETE /artists/metallica/albums/killemall
DELETE /artists/metallica/albums/ridethelightning
...

But instead just do:

DELETE /artists/metallica/albums

(or if the API didn't support bulk-delete in that way, just do a single GET followed by a series of DELETE calls)

So, enter promises.  If each method in that chain simply returned the promise of fetching the underlying data, I could chain off of it and only actually fire off HTTP requests for the things I actually needed to get.

I should've taken the blue pill

So, that makes pretty good sense, but that's not all.  For your 3 easy payments of $29.95 you get not only this nice, easy-to-use method chaining and minimal HTTP request overhead, but also implicit parallelized requests as well.  What if, given the above example, you could also do something like:

for album in client.artists.albums:
     print album.tracklisting

And that just automatically did basically this:

1. GET /artists
2. for every $artist, GET /artists/$artist/albums (in parallel)
3. for every album from every $artist, print the tracklisting

Now, let's see how deep this rabbit hole can go:

for track in client.artists.albums.tracks:
    print "%d - %s" % (track.number, track.name)

1. GET /artists
2. for every $artist, GET /artists/$artist/albums (in parallel)
3. for every $album from every $artist, GET /artists/$artist/albums/$album/tracks (in parallel)
4. for every track for every $album from every $artist, print the number and name

I am serious, and stop calling me Shirley

I think this is a really powerful idea that could make an API client very easy to use and actually, you know, fun.  But maybe I'm alone here, since every API client ever is just the same old boring bunch of methods, sometimes in namespaces, sometimes with actual model-style classes for the resources, if you're lucky.  Granted, this idiom could be abused to fire off hundreds of HTTP requests in parallel and potentially overwhelm the server, either by accident or malice.  But, in my opinion the ease-of-use outweighs the potential for mass destruction.  I'd like to see a client that works this way, even if I have to build it myself (especially if I get to build it).  I've managed to build a client that does the first half of this, and does it pretty well IMO (blog post coming soon).  I have a good idea of how to accomplish the second part, but I haven't yet overcome the effort-to-payoff ratio on that one.  It would be cool, and potentially useful, but it's a lot of work for such a small benefit.  Still, that itch needs scratching, and one day it shall be scratched.  Even though I realize that maybe it means I'm insane.