Design that hides cache as an implementation detail

Question

I have a type which is fairly heavy (maintains an off-heap object), and which can be uniquely derived from a String. It requires cleanup, so of course the type has a close() method, but it is used in contexts where clients may not be sure when they are done with their object; or they may simply not be aware they are supposed to close it. Therefore the class also has a finalize() method as a last resort:

class Wrapper implements Closeable {
  private Heavy h;
  private Wrapper(String s) { this.h = initNative(s); }
  public static Wrapper fromString(String s) { return new Heavy(s); }
  public int doSomething() { return h.doSomething(); }
  public synchronized void close() { 
    if (h != null) {
      cleanupNative(h);
      h = null;
    }
  }
  public void finalize() { close(); }
}

Now I would like to introduce a layer of indirection, so that:

1. Two clients who call Wrapper.fromString with the same string can share the same Heavy (I don't care if they share the same Wrapper or not).
2. I can limit the number of Heavy objects live in the system at a time.

My plan is to keep track of which Wrapper objects have been used recently, and for those which have not been used for some time, I can deallocate their Heavy objects to make room for other requests. Then if the Wrapper turns out to be needed again, it will notice that its Heavy is null, and reinitialize it from the same String: a sort of lazy-loading where I also unload objects. Something like this:

class WrapperFactory {
  private Cache<String, Wrapper> cache = new Cache<>(/*capacity=*/10);
  public Wrapper fromString(String s) {
    if (cache.isFull()) {
      cache.evictOldest().close();
    }
    return cache.computeIfAbsent(s, Wrapper::fromString);
  }
}

class Wrapper {
  private String s;
  private Heavy h;
  public static Wrapper fromString(String s) { this.s = s; }
  public synchronized int doSomething() {
    if (h == null) {
      h = initNative(s);
    }
    return h.doSomething();
  }
  // same cleanup methods
}

The overall idea seems workable to me, but I'm not sure of some of the details.

• I'm not sure what kind of data structure to use for my Cache type: I think ben-manes/caffeine has the features I need, but I'd prefer not to take it on as a dependency if something simpler will work.
• How do I get the Wrapper objects to communicate back with the WrapperFactory so that I don't erase an in-use object, and the objects I do erase are the ones least often used?

Lastly it seems to me that what I'm doing is not so uncommon: there must be some library handling it, or perhaps design pattern(s) describing how to do it well. It's a bit like Flyweight, a bit like interning.

Answer 1

So first as a small point - finalize is depreciated. Don't use it. You need to rely on calling contexts to handle closing the object. Use try-with-resources where you can and maybe use reference counting to know when to de-allocate the underlying object.

Beyond that, what want sounds most like object pooling. You can take inspiration from libraries like Jedis where there is an interface that represents all the operations you can perform and an object pool you can ask to borrow something that implements that interface from. You then either have your operations interface implement closable or make some sub interface that does, and calling .close() on that returned object returns it to the pool.

In that example everything is using Apache Commons Pool which, while I have no experience using except as a consumer, seems to be in the right direction.

The same general technique is used by any sort of "resource pooling" object like HikariCP which pools database connections or java's Executors which pool access to threads.

To make everything work properly across multiple threads, I do suggest using a library like Commons Pool, but we can show the general pattern without it.

First, segregate out the operations of Heavy

public interface Heavy extends Closeable {
    int add(int x, int y);
}

If you are on the newest Java this might be a good place to use sealed interfaces.

Next, make your actual implementation that handles the native resource.

final class HeavyNative implements Heavy {
    private final Pointer pointer;

    HeavyNative(String name) {
        this.pointer = ...???...;
    }
    
    @Override
    public int add(int x, int y) {
        // oogabooga
    }

    @Override
    public void close() {
        pointer.free();
    }
}

Depending on how you are structuring things, you might see some benefit in making the actual implementations here package-private, which is what I am doing.

Then you can make your object that lets you borrow a Heavy. I am going to hand-wave this a bit and use synchronized for everything, but do look into Apache Commons Pool and similar libraries. That being said, if you are reluctant to use dependencies at all, this will (probably) work.

public final class HeavyFactory {
    // You could also wrap this logic into HeavyNative or make an entirely separate class
    // or make it not atomic since that doesn't super matter in an entirely synchronized
    // context.
    private static record RefCountedHeavy(Heavy heavy, AtomicInteger rc);
    
    private Map<String, RefCountedHeavy> givenOut = new HashMap<>();
    
    public synchronized Heavy getHeavy(String name) {
        if (givenOut.contains(name)) {
            final var refCountedHeavy = givenOut.get(name);
            refCountedHeavy.rc().addAndGet(1);
            return new BorrowedHeavy(name, refCountedHeavy.heavy(), this);
        }
        else {
            final var heavy = new HeavyNative(name);
            this.givenOut.put(name, new RefCountedHeavy(heavy, new AtomicInteger(1)));
            return new BorrowedHeavy(name, heavy, this);
        }
    }
    
    synchronized void returnHeavy(BorrowedHeavy heavy) {
        final var refCountedHeavy = givenOut.get(heavy.name);
        final var rc = refCountedHeavy.rc().addAndGet(-1);
        if (rc == 0) {
            // no one is using it
            refCountedHeavy.heavy().close();
            givenOut.remove(heavy.name);
        }
    }

}

So you can get a Heavy from this, and if you know you have a Heavy that is borrowed you can return it. Then for BorrowedHeavy:

 // This could be a static class in the factory if you wanted
final class BorrowedHeavy implements Heavy {
    // Or whatever other book keeping you want accessible to handle returns
    // you can make whatever the "key" is part of the interface or do some other
    // method of book keeping if you want.
    final String name; 
    private final Heavy heavy;
    private final HeavyFactory maker;

    BorrowedHeavy(String name, Heavy heavy, HeavyFactory maker) {
        this.name = name;
        this.heavy = heavy;
        this.maker = maker;
    }

    // delegate all functionality
    @Override
    public int add(int x, int y) {
        return this.heavy.add(x, y);
    }

    // and when closed, return to the pool
    @Override
    public void close() {
        this.maker.returnHeavy(this);
    }
}

Then, when all is said and done, you just need to maintain a reference to a single HeavyFactory everywhere.

In fact, you could even make HeavyFactory package private and make a singleton instance on the Heavy interface.

public interface Heavy extends Closeable {
    private static HeavyFactory factory = new HeavyFactory();

    static Heavy forName(String name) {
        return factory.getHeavy(name);
    }
    
    int add(int x, int y);
}

At which point the public interface into your code is just that single method and you are free to fiddle about with the specifics of pooling however you want to.

try (final var heavy = Heavy.forName("abc")) {
    System.out.println(heavy.add(1, 2));
}

Does that make sense? I know the code got kinda messy with BorrowedHeavy and HeavyFactory so I can clean up or clarify if needed, but to summarize the general points:

• What data structure to use? Probably hash map unless you want to include libraries at which point just use libraries made for object pooling.
• How do you get wrapper objects to communicate back? Give them a reference to the factory when they are made and communicate at close() time.
• Interfaces are your friends.