Architecture : am I doing things right?

Question

I'm trying to use a '~classic' layered arch using .NET and Entity Framework. We are starting from a legacy database which is a little bit crappy:

• Inconsistent naming
• Unneeded views (view referencing other views, select * views etc...)
• Aggregated columns
• Potatoes and Carrots in the same table
• etc...

So I ended with fully isolating my database structure from my domain model. To do so EF entities are hidden from presentation layer. The goal is to permit an easier database refactoring while lowering the impact of it on applications.

I'm now facing a lot of challenges and I'm starting to ask myself if I'm doing things right.

1. My Domain Model is highly volatile, it keeps evolving with apps as new fields needs are arising. Complexity of it keeps raising and class it contains start to get a lot of properties.

2. Creating include strategy and reprojecting to EF is very tricky (my domain objects don't have any kind of lazy/eager loading relationship properties):

   DomainInclude<Domain.Model.Bar>.Include("Customers").Include("Customers.Friends")
   // To...
   IFooContext.Bars.Include(...).Include(...).Where(...)
   

3. Some framework are raping the isolation levels (Devexpress Grids which needs either XPO or IQueryable for filtering and paging large data sets)

I'm starting to ask myself if :

• the isolation of EF auto-generated entities is an unneeded cost.
• I should allow frameworks to hit IQueryable? Slow slope to hell? (it's really hard to isolate DevExpress framework, any successful experience?)
• How can I lower the volatility of my domain model?

Precisions

EF Isolation and fields that should or should not be included on service calls

Here is how I isolate EF Entities. Presentation layer is an isolation layer as no presentation can access to repositories class.

// Inside a service class. For this sample Customer is defined in the service model
// And Person is autogenerated by Entity Framework
/// Get the customers who bought an item ///
IEnumerable<Customer> GetCustomers(Item item){
    var query = from person in _personRepository.GetItemBuyers(item.Id)
                select new Customer {
                  Id = person.CustomerId
                  , FirstName = person.FirstName
                  , LastName = person.LastName
                  , Sex = person.Sex == 'M' ? Sex.Male : Sex.Female};
    return query;
}

It can get time consuming especially when you need to include (or not) some relationship properties. There is multiple solutions for including:

 1. GetCustomersWithOrders(Item item); 
 2. GetCustomers(Item item, IncludeStrategy<Customer> include);
 3. GetCustomers(Item item, params string[] fieldsIncluded);

1. Will obvioulsy result in service class containing thousands of GetCustomersWith...With...
2. Will create a lot of IncludeStrategy inherited class. I'm still working on a dynamic fluent approach DomainInclude<Customer>.Include("Orders").Include("Friends")
3. Will need the creation of some mapping dictionnary "Orders" => "CustomerAccount.PayedOrders"

Framework hitting IQueryable

Repositories don't expose IQueryable, the goal is to hide entity framework. Devexpress datagrids need an IQueryable to allow paging/filtering without loading the whole table in memory.

Finding a workaround is nearly impossible as it will be really time consuming and the portability of the workaround is not assured with new devexpress releases.

My question here is: Is it a big deal to allow devexpress going through the layers up to the EF context class. Should I drop Devexpress instead and use a more flexible datagrids (Sencha ExtJs, ...)

Answer 1

You've got a number of questions in your overall question. I'll try to touch upon each of them.

Overall, the answer is:

"Yes, you're doing things right."

Rarely are we given a clean and ideal architecture to work from. Heck, rarely are we even given an architecture, period. Small consolation, I realize. But you're asking the right questions and taking the right approach.

One thing you may not have realized is that isolating the EF layer and the database access is a step towards encapsulating the changes the application is seeing. Juval Lowy of iDesign is a huge advocate of encapsulating the change. Change is good because it keeps us employed and keeps our applications alive, but we want to limit the impact any change can have upon the rest of the application. So we encapsulate it.

You can't lower the volatility of your domain (note, I didn't say domain model) and you wouldn't want to even if you could. All you can do is encapsulate that volatility. Once you isolate the volatility of your domain, then your domain model will fall in line.

Identify the areas of your database that seem prone to most frequent change. Isolate out those aspects so you can more easily make the changes while minimizing downstream impact. Using a data access layer is a good start. It might be time to consider a sub-layer for data access that allows you to create a composite of those for the main DA layer.

Some people love EF, some people really don't see much advantage to it when weighed against the costs. Of those who use EF, they tend to work with the latest release of that toolset. The recent (v4.5?) updates have made EF more usable / workable. Wether or not it's a good technological decision for you is something that only you can decide. Look at your alternatives; examine the cost to switch to those; and move forward. As a suggestion, create a test branch in your repository, migrate away from EF on that branch and work through some of the challenges. Then you'll know.

The performance issues within your database imply that you need to consider some restructuring. Normalization, in moderation, is a good thing. It's easy to go overboard with it, but restructuring your tables so that they are more easily indexed will likely help with the performance issues. This could also help with encapsulating some of the change you're seeing from updates to the applications.

For example, you may have a table CustomerInfo with a primary key of CustomerID. If App1 is adding / changing ancillary columns off of CustomerInfo then you can give App1 it's own subtable CustomerInfoExtensions keyed back to CustomerInfo with the CustomerID.

I don't know of any data access approach that will fix bad structure within the database tables. So you may be "blaming" EF in this case when the root issue lies in the DB. And there are no silver bullets when it comes to crunching huge data sets. If DomainInclude<Customer>.Include("Orders").Include("Friends") creates a cartesian product across your largest tables, well, then that will simply be your least performant query. Fix the query before worrying about the data access method.

Exposing an IQueryable isn't necessarily a bad thing. If you need (want) a more concrete expression of that Interface, then pick an appropriate generic container. We pass collections around like that all the time, and it makes using the MVVM pattern a lot simpler to implement when displaying the data. I think your concern here is that you feel there should be something "more" to what the EF layer is returning. If an IQueryable is sufficient, then use one. If you need a first class data object on the return, then create one. But don't create a first class data object just to have one if the IQueryable is good enough. That would be extra work.

The question of DevExpress vs. other datagrids leads me to an answer of "meh." You've got bigger challenges to resolve first before worrying about that one. Consider the UI (View) as an area that will change. Encapsulate it. When you decide you hate FooBar-Grid (or whatever is your current flavor of UI) then you can get rid of it more easily. Personally, I would worry about enhancing the DB and data access layers first, but I'm an engineer who appreciates the simple elegance of the command line. Call me biased.