6

in a simple multi layer architecture, in which layer do I have to implement something like parsing file. For example: I have a file and I have to extract specific information into an object. I think that this stuff is data access, because the business layer need data. How and where the data come from is not the point. do you agree?

5
  • The Data Access Layer normally contains your ORM and accesses a database. Parsing a file would normally occur somewhere else. – Robert Harvey Oct 9 '15 at 5:14
  • That means in the data access layer I will read my file and in the business layer I have to parse the content, right? – Marcel Hoffmann Oct 9 '15 at 5:23
  • You're overthinking this. Read and parse the file in the layer where it is most convenient to do so. – Robert Harvey Oct 9 '15 at 5:28
  • Oh, I I thought that everything work with files and databases should be in the data access layer. Okay thanks, than I will implement this in the business layer. – Marcel Hoffmann Oct 9 '15 at 5:33
  • 2
    I disagree with idea that Data access layer should not handle files. Quite opposite. Data access layer should be for ALL kind of data, not only database. – Euphoric Oct 9 '15 at 7:18
6

I would recommend to stop thinking in terms of "layers" and start thinking in terms of "modules". Right now, your Data Access layer contains only Database access module. So right next to it, you would put File Access module. This module would be in same layer as Database access module in terms of which modules it can reference and which modules can reference it. But it would probably be separate library, just like Database access module is it's own library (or I strongly hope so).

Even better case would be if you had an abstraction for saving and loading business data. The Data access layer (and its including modules) would then contain implementations of this abstraction. So you would have SaveDataInDatabase, SaveDataInFileX and SaveDataInFileY classes. The business code would then use this abstraction without knowing or caring how the data will be persisted.

3
  • I think this is a pretty solid answer in cases where your files are essentially a form of data persistence. If, on the other hand, your files are only data input then I don't see anything wrong with doing your file parsing in your business layer (or a UserServicesLayer as mentioned in another post). – MetaFight Oct 9 '15 at 9:03
  • Though, in my experience, if the business wants to read from certain files now it's almost certain they'll want to write back to them later. So, if I had to implement this I would definitely follow your suggested approach. – MetaFight Oct 9 '15 at 9:05
  • 1
    @MetaFight I would say it's more certain that they will want to write something back somewhere now or in the future, Input is a subset of access – Caleth Oct 9 '15 at 10:51
3

From what you describe at a basic level, you are:

  1. Pulling data from some persistent storage

  2. Parsing it into some sort of generic format

  3. Mapping it to objects and properties

The Repository Pattern solves this problem. It involves three main types of classes:

  1. The "Repository" class, which just deals with your object model, e.g. your "entity classes" or "domain model" in your application. It can provide both read and write access for this data

  2. The "Gateway" class, which is responsible for reading and writing data to a persistent storage, be it a database, web service or file. As we will see later, this is where file parsing should exist.

  3. The "Factory" class, which creates instances of your Domain Model and maps data from the generic data the Gateway returns to your Domain Model.

They key to this pattern is implementing interfaces for each type of class. Let's examine how a blog might be modeled and persisted in this manner.

Repository Pattern Domain Model and Interfaces

Let's start out with the Domain Model:

public class Blog
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}

Nothing too fancy. Next, we need to define our interfaces, first by defining the interface for the blog repository.

public interface IBlogRepository
{
    Blog Find(int id);
    void Create(Blog blog);
    void Update(Blog blog);
    void Delete(Blog blog);
}

The IBlogRepository interface is focused mainly on your Domain Model with no mention of how persistence is actually achieved. This provides a layer of decoupling for the rest of your application.

Next, the blog gateway interface allows the repository to preform specific CRUD operations (Create, Read, Update, Delete) on your data store.

public interface IBlogGateway
{
    IList<IDictionary<string, object>> Load();
    void Create(int id, string name, string description);
    void Update(int id, string name, string description);
    void Delete(int id);
}

It's best to keep the "gateway" devoid of any knowledge about your Domain Model, as this will provide maximum flexibility to change your domain model with limited affect on the "gateway", and to change your "gateway" with limited affect on your domain model.

The Load method just returns a list of dictionary objects, a nice, generic format for the data.

The last interface is the blog factory object responsible for creating instances of your Domain Model and mapping your generic data to the Blog objects.

public interface IBlogFactory
{
    Blog CreateInstance(IDictionary<string, object> data);
    void UpdateInstance(IDictionary<string, object> data, Blog instance);
}

Repository Pattern Concrete Classes

Now that we have our interfaces defined, let's start by creating a BlogRepository class that implements the IBlogRepository interface:

public class BlogRepository : IBlogRepository
{
    private IBlogGateway gateway;
    private IBlogFactory factory;
    private IList<Blog> cache;

    private IList<Blog> Cache
    {
        get
        {
            if (cache == null)
            {
                var data = gateway.Load();

                cache = new List<Blog>();

                foreach (var record in data)
                {
                    cache.Add(factory.CreateInstance(record));
                }
            }

            return cache;
        }
    }

    public BlogRepository(IBlogGateway gateway = null, IBlogFactory factory = null)
    {
        this.gateway = gateway ?? new BlogFileGateway(@"C:\Path\To\Blogs.txt");
        this.factory = factory ?? new BlogFactory();
    }

    public Blog Find(int id)
    {
        Cache.SingleOrDefault(b => b.Id == id);
    }

    public void Create(Blog blog)
    {
        Cache.Add(blog);
        gateway.Create(blog.Id, blog.Name, blog.Description);
    }

    public void Update(Blog blog)
    {
        gateway.Update(blog.Id, blog.Name, blog.Description);
    }

    public void Delete(Blog blog)
    {
        Cache.RemoveAll(b => b.Id == blog.Id);
        gateway.Delete(blog.Id);
    }
}

Now we can start to see how each piece fits together. The constructor for BlogRepository allows you to pass in your own gateway and factory object, falling back to a default implementation of your choosing.

To find a single Blog the repository first loads all the data from the gateway, and maps it to objects using the factory. The Find method then returns the first Blog that matches the Id. Admittedly you might want to create a method on the "gateway" object to return a single record, but the repository is also a good place to implement caching.

Now we get into the part you are interested in: parsing the file.

public class BlogFileGateway : IBlogGateway
{
    private string filePath;

    public BlogFileGateway(string filePath)
    {
        this.filePath = filePath;
    }

    public IList<IDictionary<string, object>> Load()
    {
        // Read entire file and return the parsed data
    }

    public void Create(int id, string name, string description)
    {
        // Append the new data to the file
    }


    public void Update(int id, string name, string description)
    {
        // Update the correct row within the file
    }


    public void Delete(int id)
    {
        // Delete the correct row within the file
    }
}

The BlogFileGateway implements the IBlogGateway interface. This is where you open, read and parse the file. If you have a generic file format, your "gateway" classes could create an instance of another class responsible for parsing each record in the file.

And just for good measure, the blog factory class:

public class BlogFactory : IBlogFactory
{
    public Blog CreateInstance(IDictionary<string, object> data)
    {
        if (data == null)
            return null;

        Blog instance = new Blog();

        UpdateInstance(data, instance);

        return blog;
    }

    public void UpdateInstance(IDictionary<string, object> data, Blog instance)
    {
        instance.Id = int.Parse(data["id"].ToString());
        instance.Name = data["name"].ToString();
        instance.Description = data["description"].ToString();
    }
}

Why the Repository Pattern Works Best

Since you decouple the data access and mapping from your domain model, you are free to start out with file storage. Moving to database storage would involve writing a new gateway class to access the database, and a new factory class to map the data from the database to your Blog Domain Model. If later on you decide to move to a Service Oriented Architecture, you need only write another gateway to call the web service, and another factory. You can radically change or switch out storage mechanisms without refactoring major parts of your application.

@BartVanIngenSchenau said in his answer:

For other kinds of data access (data exchange with third parties, reading/writing files, etc.) those architectures have the concept of Services to perform the transportation of the data and to transform it between the external format and something that the business layer can work with.

(emphasis mine)

The advantage of using the Repository Pattern is that the code calling a web service doesn't have to live in the generic, and often overused layer of your application called "services". Calling the web service happens in the gateway.

Example Usage

Let's find a blog, change a value and update it:

IBlogRepository repository = new BlogRepository();

Blog blog = repository.Find(3);

blog.Description = "Black and white, and read all over";

repository.Update(blog);

Later on, you need to use a MySQL database so you write the BlogMySqlGateway and BlogMySqlFactory classes, then change one line of code:

IBlogRepository repository = new BlogRepository(new BlogMySqlGateway(), new BlogMySqlFactory());

Blog blog = repository.Find(3);

blog.Description = "Black and white, and read all over";

repository.Update(blog);

Nothing else has to change. Later on, you decide this all needs to live in "The Cloud":

IBlogRepository repository = new BlogRepository(new BlogWebServiceGateway(), new BlogWebServiceFactory());

Blog blog = repository.Find(3);

blog.Description = "Black and white, and read all over";

repository.Update(blog);

Nothing else has to change.

2
  • Nice answer. But I haven't any database. My file does not contains innumerable values. An concrete example: In C# you have a file called assembly info. In this file, they are a version number for the specific project. This I have to parse, change some information and save it. I think in this example, I'm not able to use the repository pattern, correct? – Marcel Hoffmann Oct 9 '15 at 14:51
  • For that limited example you could use the repository pattern, but that would be like hitting a thumbtack with a sledge hammer. – Greg Burghardt Oct 9 '15 at 15:13
0

In parsing data from a file, you can do that in UserServicesLayer, if you are going to manipulate those data that needs any logical condition, it is the time to call the BusinessLayer then do all logical things inside that layer, your DataAccessLayer is like your bridge to access the data from database, therefore it is not advisable to do the extraction of data inside the DataAccesLayer.

Hope this helps.

0

In most multi-layer architectures, the Data Access Layer (DAL) is reserved for transporting data to and from the primary backing storage of the application (usually a database).
For other kinds of data access (data exchange with third parties, reading/writing files, etc.) those architectures have the concept of Services to perform the transportation of the data and to transform it between the external format and something that the business layer can work with.

So, in your architecture, the file reading and parsing would go in a Service, usually named after the kind of file or data it processes. For example a XMLImportService.

2
  • I don't understand the downvote here. What Bart is outlining here is perfectly reasonable and very common. – MetaFight Oct 9 '15 at 9:06
  • @MetaFight: Probably someone disagreeing with my use of "most architectures". – Bart van Ingen Schenau Oct 9 '15 at 9:17
0

I think that this stuff is data access, because the business layer need data. How and where the data come from is not the point. Are your agree?

I'm splitting such things into two layers actually.

Generic file parsing routines (GenericCSVParser for example) go to Infrastructure layer because they have nothing to do with business logic of the application. It's just a file parsing tool which might be a part of the framework itself.

Specialized file parsers (InvoiceCSVFileParser for example) usually compose or inherit from generic ones and they belong to business logic layer.

0

Simply put, you would implement your parsing needs as a layer-independent module or library. You'd use this library in the layer that needs it.

The implementation should accept and work with a device-independent stream (abstract/base or interface), or a string (if the input data is not expected to be too large, otherwise you might run into scalability issues later on).

And about the layer, yes in your case it sounds the functionality would be applied in data access. But that doesn't always have to be the case. It also often happens that you'll need such parsing functionality in the UI layer where user inputs or uploads data, before you're able to populate the business objects and store them in the database in the format that it is required.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.