Data access for composable applications

Question

I'm wondering if anyone can share their thoughts, experience and guidance on how to approach the data access layer for a composable application?

By composable application, I mean an application with the domain layer made up of modules. The goal is to make the application highly extensible by 'plugging in' additional (or replacement) modules. Each module encapsulates (and isolates) a set of related functionality.

My dilemma is the right approach for the data access layer to support this composability (modularity) while still providing isolation from the back-end database. Because each "module" can implement its own set of domain objects, there really is no way that I see an ORM like EF working unless I simply create one mega data context or make changes to the data access layer each time a new module is introduced (which isn't possible if the module is added by a 3rd-party or as a post-deployment extension).

For what it's worth, this is an enterprise-scale application with a database containing over 200 tables and a ton of sproc, udfs, etc.

What advice can you share?

UPDATE

I confirmed today that we will be using a legacy data store for the solution that utilizes stored procedures and user-defined functions for the majority of the data access operations. Our hope is to evolve away from this as time goes by, but the budget and timeline for the project dictate the we build on top of the existing databases.

Does that change your thoughts?

Answer 1

There isn't a hard and fast rule an application need use a single DbContext; I would separate things out into a core data context and other more focused DbContext. Or things that aren't even DbContexts at all when the data isn't coming from a relational database . . .

Answer 2

From my experience (because I've made a CMS using .NET Framework and LINQ to SQL ORM):

Any CMS should address 3 groups, namely developers, designers, and users. When you say module, it's like you talk about developers group. Now you have consider these items:

1. Provide an infrastructure and ask your users to use your infrastructure when developing their modules (for example, WordPress is like that and has a class called wp_options and whenever you want to store or retrieve an option, you're encouraged to use this class). This way, your only work could be to create some base classes like DataAccessBase or some generic classes, etc.
2. Any CMS is made of core + extensible parts. In some CMS applications, taxonomy is not extensible, thus it's part of the core. In many CMS applications, security is not extensible, thus it's part of the core. Whatever you do, you should implement a core. In this case, you can ask developers to use your core for non-extensible parts like security, membership, taxonomy, comments, options, etc. For example, you can have a class called UserFacade with a method called IsAuthenticated. Then developers can use your abstraction to get what they want from core.
3. Each CMS has a different implementation of database. Some use EAV model and let you create Content Types dynamically, while others simply ask you to build your related tables. If your CMS is from the second category, then module developer can only use your infrastructure to connect to the database, but other parts of data retrieval and storage are module specific and each developer should have his/her own implementation. However, if your CMS is from the first category, then you can simply provide an abstraction layer for module developers to never hit your database (for example they can write a line of code like: dynamic riddles = EntityProvider.GetList("Riddle"); or EntityPersister.Save(riddleKeyValueDictionary);

Answer 3

I've done that and it's really pretty simple with an ORM like (N-)Hibernate.

If an EF-Context is similar to a session in Hibernate, then a mega-context clearly is something you should avoid! However, Hibernate has got a Session-Factory, which reads all the mappings and creates the sessions (probably contexts in EF) when you need them. And this factory can be the mega-factory without being an anti-pattern.

Each module can contain its own mappings and entities. Hibernate does a great job of persistence ignorance! They key point here is, that the context is fed the mappings during startup time and not at design time.

I just create a SessionManager (which contains NHibernates SessionFactory). During startup, all the mappings from all the modules are registered. We have about 200 tables and nearly as many entities and that is not a problem at all, there're also no memory issues. The session manager creates database sessions as we need them. Usually we have here a one session per use-case approach (translates to one session per dialog for us pretty well).

As all the modules share the same host, they have access to an IoC container, so they can resolve the session factory and get as many sessions as they want.

The only problems pose intermodular inheritance mappings from time to time. Intermodular inheritance mappings can be quite tricky and some entities could not be inheritance mapped as we would have liked them to. But these cases were rare (maybe one, two entities of those 200) and then we mostly did HasOne relations etc.

Answer 4

Some ideas:

Create a mapping layer, which maps module level domain objects to the objects that the DAL consumes. The mapping layer would represent the transformation on domain modules data to DAL consumable data. Internally, the domain module could use it's own data structures, but when it came time to talk to the DAL, those data structure would have to be mapped to DAL comsumable objects.

Module -> Domain objects -> Mapper -> Dal Objects -> DAL

If these modules actually extend the database, then some sort of extendable database schema would be necessary and the modules would call the generic extendable schema methods to save the data.

Additionally, you could explose the internals to communicate with the DAL with an interface that might expose ExecuteReader, ExecuteSclare, ExecuteNonQuery, etc. That means the domain module can talk directly with the DAL, but I think that is too much direct exposure.