0

I'm creating a software that's supposed to accept a certain form of data and does something to the data. We decided to create a small dependency beforehand which will be used to make the development of the software smoother. We're planning to accept support for json, csv, yaml, and sql files/url's(consuming rest) and convert them to lists(ArrayList). For now, we have the support for json. We've created a parent Data Reader which contains two getData method contracts, one for file uploads and one for url links - each returns json converted to a list. This is implemented by JsonDataReader. It's working and perfectly usable. However, I have this as the code for the software:

list = JsonDataReader().getData(new File("file"));

I see this as a problem. If my software has this as a code, it means that it will only strictly accept Json. What if we create the YamlDataReader, CsvDataReader, and much more? Is there a better way to design the multiple converter that will be implementable for the software development?

Process/Flow of software

  1. Upload file from upload page
  2. File can now be found in server directory
  3. In data processing page, the file is taken in and converted into the list as the code above shows.

Would it be better if I just create a component that accepts any data, detects the file type, converts it, and returns it as a list?

My colleague suggested this.

What you could do is you have something that stores all of your data readers in a map and the key is the file extension it basically then gets from the Map the appropriate DataReader based on the file type

Is this possible? To store DataReaders as values? DataReaders are the converters.

My colleague also pointed out that if I go with the DataReader that accepts anything and returns a list would be a case of multiple behaviors and not polymorphism - indicating a bad design.

1

Since you have a parent class 'parent Reader' your client code shouldn't use an implementation but rather the parent class. Also you need some method to find the right implementation for a given file. It can be done by a map if you want to store readers between calls. Anyway your client code should look like:

ParentReader reader = ParentReaderFactory.getReader(fileExtension);
List yourList = reader.getData(yourFile);

This way your client doesn't have to know about your direct implementations (json, URL etc.) That's why you created the parent class.

One of the benefits of this pattern that you don't have to deal with extensions and file types in the client code (in the code above). What does happens inside the factory? That's a different business. So you separated two different concern into two different place.

You will use several if statements in your factory? It's your implementation detail (but it is not necessary) so your client don't care about it.

In the factory you have to control the flow of execution someway. It can be if-else statements, switch, using a map or something else.

  • so inside getReader, it tells what the file type is and converts accordingly? Wouldn't that mean more if statements instead of polymorphism? So when implementing a new converter, that would mean modifying the code instead of extending it. I'm not quite sure with where I'm going here. If i'm wrong, please do tell. – Rei Brown Feb 14 '18 at 5:26
0

I worked on a project that had to parse a wide range of file types with very different formats including CSV, Excel, XML, SQLite, Access, proprietary formats, etc. Due to the fact we also had to deal with different versions of proprietary formats we needed a bit more flexibility. We had the concept of a Processor and the output was updating our database.

The interface for our processors looked something like this:

IProcessor
{
    bool CanProcess(FileInfo file);

    Task Process(FileInfo file, IProgress<ParseProgress>);
}

We used a simple list that ordered our processors so that the most specific instances were first and the more generic items were last. The reason for that is that we needed to peak inside the file in some cases. Determining the appropriate processor was as simple as the equivalent:

 var processor = RegisteredProcessors.FirstOrDefault(p => p.CanProcess(file));

 if (processor == null) {
      throw new NotSupportedException("Cannot process file: " + file.FullName);
 }

In other words the first processor that said it knew how to deal with that file. For the generic processors, we only needed to check the extension. For the proprietary formats we needed to inspect the structure and extract the format version information.

NOTE: IProgress<T> is a C# idiom to report status using a struct, since we did have a desktop application to show where we were on the processing. Some of the files were quite large. If you don't have that need, don't worry about it. The Task return type is also a C# idiom to allow the calling code to use async and await keywords. Use a return type that makes sence for your language/application architecture.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.