0

Today I had the idea to write a program that compares the prizes between different web-stores. The key question I am asking myself is how to find (with a high probability) the same article on another web-store again. For example:

Three different websites to search on are hard-coded into the code: A, B and C. Every article on each website has a picture, headline and a description.

  1. The program accesses a random article on website A.
  2. Now the task is to go to web-store B and C and try to find the same article there, using the picture, headline and description from website A.

What I am asking now is what possibiliies do I have do identify two articles as the same when I only have two different texts and a picture? What I have thought of so far is:

  1. Obviously, comparing the strings from the headline and look for similarities.
  2. Taking out important key-words and espacially look for them (i.e. manufacturer name, year of creation etc.)
  3. Analizing the pictures

Maybe someone had already made experience with this kind of "pattern-matching". The result should have a high probability of correctness. Of course I am open to new ideas for comparing, which are not in my list.

3
  • 2
    There's a whole field about this called "machine learning". If you read a book about that field I'm sure you'll learn several techniques for creating a program that can be taught (with sample/training data) how to match articles like this. Whether it's possible to recommend one or two specific techniques from that field based solely on the information you've provided I'm not sure.
    – Ixrec
    Feb 6, 2016 at 22:14
  • I have already heard (really only heard) about this, espacially neural networks. I also already thought about this, but creating such a complex structure seemed a little bit too much for this task. Maybe there is a easier, not-so-complex solution for my problem. But still thank you for your comment!
    – Bobface
    Feb 6, 2016 at 22:17
  • Neural networks might actually be the simplest option that would potentially solve this problem. Conceptually they're not complicated at all, they're just really fiddly to get working correctly since the nodes tend to lack any intuitive meaning, especially when you have lots of nodes. The only simpler "AI" algorithms I know of are for solving inherently simpler problems, such as A* for shortest path finding, which clearly won't help you here.
    – Ixrec
    Feb 6, 2016 at 22:25

1 Answer 1

-1

For the text compare part of it, I'd recommend Google's diff-match-patch

https://code.google.com/archive/p/google-diff-match-patch/

It's implemented in a variety of languages

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.