Entity objects with data external to the database

Question

I'm looking for design patterns to model (java) objects that are partially stored in a database and partially stored some other external source. In my case the external data is a whole lot of files which are read into memory and parsed into objects at application startup. The database contains the part of the data that changes more or less often and changing that part is one purpose of the application, while the external source contains data that rarely changes and never from inside the application.

Example: An object modelling a passenger plane flight might consist (among other things) of a flight plan object modelling where and when the plane starts/lands, and an airplane object modelling the passenger capacity, max speed, fuel capacity... of the physical plane assigned to do this flight. Of these two, the flight plan can change, while the physical airplane does not change. Only the association which physical plane is used on which flight changes.

In an earlier version of the application everything was stored in the database, only the airplane table was never modified after a plane was entered into the system. This turned out not to be efficient enough for various reasons and it was decided to partially ditch the database and instead keep the immutable data as completely assembled object-graph in memory.

The current solution looks something like this:

@Entity
public class Flight{
   @Id @Generated
   private int id;

   @OneToOne @JoinColumn(name="flightplan_id")
   private FlightPlan flightPlan;

   // previously:
   // @ManyToOne @JoinColumn(name="airplane_id")
   // private Airplane airplane;

   @Column(name="airplane_id")
   private int airplaneId;

   @Transient
   private Airplane airplane;

   public Airplane getAirplane(){
       if(airplane==null){
          airplane=ExternalDataSource.getAirplaneDataById(this.airplaneId);
       }
       return airplane;
   }

   // other getters and setters
}

In other words: The database stores only a unique identifier the external data, the external data itself is looked up from a cache inside the getter.

This construct works fine as far at the application itself is concerned, but is terrible for testing, because every time I need a Flight object in a test, I cannot just create an instance and fill it with data, I have to mock out a static method of ExternalDataSource first to get a functioning set of data. The current approach also duplicates some data: The airplane id is present twice - in Flight.airplaneId and in Flight.airplane.id. So either the setters need to contain non-trivial pieces of logic (i.e. yet another call to the code handling external data inside setAirplaneId) or one risks these two getting out of sync. Because one can never be sure if the code I'm testing uses (or will use after refactoring) flight.getAirplaneId() or flight.getAirplane().getId() or both, one really should duplicate this data when writing tests. Of course, I will probably forget that, set only one in the test data and wonder sometime later why half of the tests are failing. Often the primary key of the external data is not just a simple integer, but some composite key, so that multiple fields are duplicated so the potential for errors is multiplied as well.

My question is: What is a practical design pattern for this situation that is less error-prone and easier to use in tests? Ideally the Flight entity would still only have a FlightPlan field and an Airplanefield, plain getters and setters for them, and no only-for-technical-reasons additions (apart from whatever magic annotation one needs to achieve this). Then I would declare somewhere else how JPA has to transform between the complex Airplane object and its database representation as int (or multiple columns of primitive types for composite keys).

Answer 1

The duplicate airplaneId is not as big of a problem as the fact that your entity knows about an ExternalDataSource. It should not be responsible for fetching the Airplane, it's just an entity that doesn't know about caches or much about anything really.

This could be abstracted to the data access layer, so you would have something like the following

public Flight findFlightWithAirplaneById(int id) {
    Flight flight = // retrieve flight from database
    flight.setAirplane(cache.getAirplane(flight.getAirplaneId()));
}

The entity no longer knows about the cache, now the data access layer does which should be fine. You now have more control so you can also write methods that do not fill in the airplane (although if it comes from cache, it's probably not worth it), and it's readily testable by mocking cache.