0

I'm trying to design a system which can determine whether there is a difference between a field or not given two objects. In my tool, we get multiple versions of the same title, with minor differences in each field. Now based on business rules, we want to determine whether the difference is significant or not. For example: If I have a title object in the following form:

public class Title {
    private String shortDescription;
    private String longDescription; 
    private Images images;
}

public class Images {
    private String id;
    private String checksum;
    private String url;
}

TitleAv1:

{
    "title": {
        "shortDescription": "my name is xyz",
        "longDescription": "my full name is xyz + abc. How about you?",
        "images": {
            "id": "uuid-1233",
            "checksum": "checksum1",
            "url": "abc.com"
        }
    }
}

TitleAv2:

{
    "title": {
        "shortDescription": "xyz is my name ",
        "longDescription": "xyz + abc is my full name. How about you?",
        "images": {
            "id": "uuid-1234",
            "checksum": "checksum1",
            "url": "abc.com"
        }
    }
}

One of the options I was thinking was to store for each field a list of comparators defined for it. For example, for shortDescription, I can have two compartors. One is String equals, and another one which checks cosine similarity. My output for this would be:

{
    "shortDescription": {
        "StringEquals": false,
        "CosineSimilarity": 0.67,
        "LevenshteinDistance": 1.4
    },
    "images": {
        "ObjectEquals": false,
        "idEquals": false,
        "checkSumEquals": true,
        "urlEquals": true
    }
}

Then I can use a rules based engine, I was thinking of something like https://www.npmjs.com/package/json-rules-engine since it can deal with JSON data. I haven't explored the syntax of it, but essentially I can write rules which would mean something of this sort:

//iterate through each field in the json blob
fields_changed_result = {}
for field in output_obj:
    comparators = output_obj[field]
    if field == "shortDescription":
        if comparators["StringEquals"] == true or comparators["CosineSimilarity"] < 0.50 or comparators["LevenshteinDistance"] > 1.0:
            fields_changed_result[field] = false //Rule consideres this as there is no diff for the field
        else:
            fields_changed_result[field] = true // There is a diff for this field
    elif field == "images":
        if comparators["ObjectEquals"] == true or comparators["idEquals"] == true or comparators["checkSumEquals"] == true or   comparators["urlEquals"] == true:
            fields_changed_result[field] = false // No diff for images
        else:
            fields_changed_result[field] = true

"fields_changed_result" : {
    "shortDescription": false,
    "images": false
}

So now based on my business rules I've determined that TitleAv1 and TitleAv2 are the same. This is of the approaches I've thought of. In the ideal world, I would want non SDE's to be able to handle the business rules, so languages like Python, Javascript works best for them rather than Java [Assume that the comparators might be written by SDEs and how the rules are hosted or deployed will also be taken care by SDE]. Is there a better approach to do this? Are there alternate tools I should look into for writing rules. Drools seems like a popular Java based rules engine.

0

Everything here looks good. I would elaborate on how you want to receive input and how you want to send output. I’m thinking business users would like to use this as a web app. They want to select from dropdown menus on a html form and receive some formatted text in response.

Will you ever need to pass your data further down the line? What APIs will need to touch it? It appears to me you are generating reports for human consumption.

  • In terms of business users, a web app would be great, but I would start in a minimal way. Just assume they have an editor in which they can type the code and write some basic tests. I'm more or less looking at a way to get feedback on how should I model by object to write rules irrespective of which rules framework I end up using. – user1692342 Oct 4 at 5:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.