How to efficiently query movies with characters via The Movie DB API?

Question

I have a sort of whiteboard style question about efficiently querying The Movie DB API to find a list of characters in movies.

My overall aim is to:

1. Search for a movie by title (1 request)
2. Look up all the characters played in that movie (1 request)
3. Look up which movies that character has been in, played by the same actor (1 request/movie)
4. Repeat with that list of movies via step 2, and filter down to only characters in the original movie, until no unexplored characters remain.

My issue is that the DB call for step 3 only brings back the films those actors are in, as opposed to characters (in fact you can only search by actor id, not character id). This means that, for instance, step 3 for the ~23 actors returned in step 2 for the lord of the rings, it then returns 844 movies each one needing a request to find the characters in that movie. The problem is that most of these are going to be false positives.

Is there a more efficient way to query the API? I've considered combining pairs of actor IDs to narrow it down, but I don't think that would narrow down the movies returned by enough.

How can I efficiently make this query?

Answer 1

Look on the bright side! Actors (and producers / directors) often accompany one another from project to project, for example to collaborate on a sequel.

So if you memoize movie results, storing them to disk or to redis, you will often get cache hits when examining the 2nd or 3rd actor who participated in a sequel.

---

A film may have many characters, many actors. But only the top few of them are "important" and will transition to a sequel.

Download e.g. http://files.tmdb.org/p/exports/person_ids_02_01_2023.json.gz

Each row has a popularity attribute, where higher values indicate greater box office draw.

Later on you will crawl lots of movies based on those 23 actors. Use popularity to prioritize the principal actors, sorted in descending order. Pick a cut-off where you choose not to crawl movies based on a minor supporting actor's link.

---

ADDENDUM

Here are the first two rows of that file.

{
  "adult": false,
  "id": 16767,
  "name": "Aki Kaurismäki",
  "popularity": 3.948
}
{
  "adult": false,
  "id": 54768,
  "name": "Turo Pajala",
  "popularity": 1.473
}