Caching Layer in libraries: what's the most correct design?

Question

Introduction

Let's say that you have such design in your application:

- application  (laravel)
  - web scraper (used by application)
    - query builder (used by web scraper)

The application is what actually uses the Web Scraper to fetch the data from the web, however since we use the Web Scraper as a search engine as well, we build the query string used from the Web Scraper based on user input.

The Web Scraper is the library that actually deals with extracting and parsing HTML files taken from the sites we work in.

The Query Builder makes use of some remote APIs (ie. google maps for coordinates based on user input) to be used in the Web Scraper.

Code Example

class WebsiteScraper 
{
    public function city($city)
    {
        $this->parameters['city'] = CityIdentifierFactory::retrieve($city);
    }
}

class CityIdentifier
{
    const API_URL = "http://maps.google.com/maps/api/geocode/json";

    public function __construct(\GuzzleHttp\Client $client)
    {
        $this->client = $client;
    }

    public function retrieve($city)
    {
        try
        {
            $coordinates = $this->getCoordinates($city);

            if (count($coordinates) <= 0) {
                return [];
            }

            return $coordinates;
        } catch (Exception $e) {
            throw new QueryBuilderCityFetchingException(
                "Whoops! Something wrong happened."
            );
        }
    }

    private function getCoordinates($city)
    {
        $response = $this->client->get(self::API_URL, [
            'query' => [
                'address' => $city,
                'region'  => 'IT'
            ]
        ]);

        $body = json_decode($response->getBody()->getContents());

        return [[
            'city'     => $city,
            'lat'      => $body->results[0]->geometry->location->lat,
            'lng'      => $body->results[0]->geometry->location->lng,
            'location' => $city
        ]];
    }
}

class CityIdentifierFactory
{
    public static function retrieveFromDisk($name)
    {
        $identifier = new FilesystemCityIdentifier(__DIR__ . "/../../assets/provinces.json");
        return $identifier->retrieve($name);
    }

    public static function retrieve($name)
    {
        $identifier = new CityIdentifier(new Client);
        return $identifier->retrieve($name);
    }
}

Problem

The problem is that the Web Scraper is ran thousands of times a day, and Google Maps API have limits that we should not reach (even though we reached them already! :( ). I would like to implement a caching layer in the CityIdentifier component of the Query Builder, so that I avoid to make thousands of queries related to the same city.

Question

Where should I implement this caching layer, knowing that all of these things are going to be dependency-injected inside Laravel (4.2)?

Should it be in the Web Scraper, in the query builder, or in the actual CityIdentifier component?

Where should cache be saved? In an internal folder of the query builder, or in some application folder that is configured in the library?

Answer 1

Preface: I know you are asking about PHP. I have no PHP experience but here is what I would do as a Java developer. I do not know how relevant this makes my answer.

You wrote:

I would like to implement a caching layer in the QueryBuilder

This sounds like a good case for a Decorator. Here you would wrap a "regular" QueryBuilder and add caching behavior. The code might look something like this:

class CachingQueryBuilder implements QueryBuilder {

    QueryBuilder basicQueryBuilder;
    Cache cache;

    @Override Query buildQuery(String input) {

        if(cache.contains(input)) {
            //returned cached contents when possible
            return cache.get(input);
        } else {
            //build query manually, cache it, and return it
            Query q = basicQueryBuilder.buildQuery(input);
            cache.put(q);
            return q;
        } 
    }
}

Google's Guava API (a Java resource) has some great caches at: Guava Caches.