2 added 79 characters in body
source | link

Here I describe some similar work I did. Perhaps it will work for you. I had 7 different file formats across 2 different files.

What I must assume is that your meta-data is effectively keys. Otherwise how can one possibly know which records to merge?

  • Create a CommonData class

    • A single class to hold any record from any file.
    • A property for each possible meta-field from any incoming file
    • A property to hold the entire record
    • As you clearly illustrate - Where meta-fields are the same, there is only one. I.E. only 1 "Meta1", "Meta2"
    • Populate only the appropriate meta fields for a given file/record format.
    • An enum identifing the record type - what file it comes from essentiallyessentially; or for me, the file format.
    • ImplementHas an IEquatableIEqualityComparer
  • CommonData implements IEquatableCommonDataEqualityComparer

    • Implements IEqualityComparer
    • In my solution your "meta data" were my keys that defined equality for each given record type (file).
    • The IEquatableequality-comparer object wasis passed into the CommonData constructor.
  • An enum to identify each source file or unique file format.

  • The Factory

    • takes the raw record and it's type - enum value
    • Factory passes to appropriate parser based on the enum value
    • Factory returns new CommonData object, with it's record-type-specific IEquatableCommonDataEqualityComparer implementation.
  • CommonDataCollection

    • Between the IEquatableequality-comparer implementations and RecordType property we can find, match, etc. records for each file type.

Here I describe some similar work I did. Perhaps it will work for you. What I must assume is that your meta-data is effectively keys. Otherwise how can one possibly know which records to merge?

  • Create a CommonData class

    • A single class to hold any record from any file.
    • A property for each possible meta-field from any incoming file
    • A property to hold the entire record
    • As you clearly illustrate - Where meta-fields are the same, there is only one. I.E. only 1 "Meta1", "Meta2"
    • Populate only the appropriate meta fields for a given file/record format.
    • An enum identifing the record type - what file it comes from essentially.
    • Implement IEquatable
  • CommonData implements IEquatable

    • In my solution your "meta data" were my keys that defined equality for each given record type (file).
    • The IEquatable object was passed into the CommonData constructor.
  • An enum to identify each source file or unique file format.

  • The Factory

    • takes the raw record and it's type - enum value
    • Factory passes to appropriate parser based on the enum value
    • Factory returns new CommonData object, with it's IEquatable implementation.
  • CommonDataCollection

    • Between the IEquatable implementations and RecordType property we can find, match, etc. records for each file type.

Here I describe some similar work I did. Perhaps it will work for you. I had 7 different file formats across 2 different files.

What I must assume is that your meta-data is effectively keys. Otherwise how can one possibly know which records to merge?

  • Create a CommonData class

    • A single class to hold any record from any file.
    • A property for each possible meta-field from any incoming file
    • A property to hold the entire record
    • As you clearly illustrate - Where meta-fields are the same, there is only one. I.E. only 1 "Meta1", "Meta2"
    • Populate only the appropriate meta fields for a given file/record format.
    • An enum identifing the record type - what file it comes from essentially; or for me, the file format.
    • Has an IEqualityComparer
  • CommonDataEqualityComparer

    • Implements IEqualityComparer
    • In my solution your "meta data" were my keys that defined equality for each given record type (file).
    • The equality-comparer object is passed into the CommonData constructor.
  • An enum to identify each source file or unique file format.

  • The Factory

    • takes the raw record and it's type - enum value
    • Factory passes to appropriate parser based on the enum value
    • Factory returns new CommonData object, with it's record-type-specific CommonDataEqualityComparer implementation.
  • CommonDataCollection

    • Between the equality-comparer implementations and RecordType property we can find, match, etc. records for each file type.
1
source | link

Here I describe some similar work I did. Perhaps it will work for you. What I must assume is that your meta-data is effectively keys. Otherwise how can one possibly know which records to merge?

  • Create a CommonData class

    • A single class to hold any record from any file.
    • A property for each possible meta-field from any incoming file
    • A property to hold the entire record
    • As you clearly illustrate - Where meta-fields are the same, there is only one. I.E. only 1 "Meta1", "Meta2"
    • Populate only the appropriate meta fields for a given file/record format.
    • An enum identifing the record type - what file it comes from essentially.
    • Implement IEquatable
  • CommonData implements IEquatable

    • In my solution your "meta data" were my keys that defined equality for each given record type (file).
    • The IEquatable object was passed into the CommonData constructor.
  • An enum to identify each source file or unique file format.

  • The Factory

    • takes the raw record and it's type - enum value
    • Factory passes to appropriate parser based on the enum value
    • Factory returns new CommonData object, with it's IEquatable implementation.
  • CommonDataCollection

    • Between the IEquatable implementations and RecordType property we can find, match, etc. records for each file type.