I recently stumbled across a company that has created what appears to be a computer vision technology that is capable of detecting shoplifting automatically and alert its users.


Watching some of the videos and examples provided by the company has left me completely baffled and amazed as to how on earth they may have achieved this functionality.

I understand that no-one here will be able to tell me exactly how this may have been achieved but is anyone aware - and could point me to - research in this field or alternatively perhaps provide details as to how something like this could be implemented or guidance of where one might start?

My understanding was the computer vision algorithms were many years away from being this sophisticated. Is this sort of application really possible? Anyone willing to hazard a guess at how they achieved this?

  • 2
    This doesn't seem so difficult. Games detect collisions all the time between objects; why couldn't you detect collisions between a person and a shelf of items, and then raise an alarm when that person was walking towards the door without paying? – Robert Harvey Dec 11 '12 at 23:41
  • Exactly. It's just object recognition and collision detection. Unless they hook it up to the scanner it is easily over come by moving the objects over the scanner but just slightly above it. The object will have appeared to of collided with the scanner but in fact did not. – Andrew T Finnell Dec 11 '12 at 23:43
  • Anyway, none of the detection mechanisms described at the website (sweethearting, basket-loss and self-checkout) require anything even remotely that sophisticated. They check in a very confined area (the cashier counter), and can cross-check items seen in the basket against what the bar-code scanner is saying was actually scanned. – Robert Harvey Dec 11 '12 at 23:44
  • 1
    I'm sorry I'm confused. Lets take the sweathearting example. I have two items, one a low cost item, one a high. I put the low cost item under the high cost one and scan. At that point we can compare what was scanned in the POS system to what is visible on the camera in the hand of the cashier but that requires the system being able to "understand what is being put in the bag" against hundreds of thousands of potential items through a camera of marginal quality. This seems extremely complicated. What am I missing? – Maxim Gershkovich Dec 11 '12 at 23:48
  • 6
    I think you're assuming too much about how well this works. I bet you there's a decent error rate, and it's likely very easy to game the system. I see this as more of a whistle blower type system, where it just identifies potential places in the video that need human review. As such, inaccuracy is well tolerated. – chris Dec 12 '12 at 0:14

You're misinformed about the state of the art. Several years ago I worked for a company that built such systems for a variety of purposes. One was an extremely successful airport egress-control system, which could easily tell the difference between a person walking the wrong way down the exit hallway and things like balls in motion or people headed the right way. Recognizing objects in a scene in real-time isn't easy, but we were doing it on embedded CPUs, not on supercomputers.

I didn't see anything there that wasn't believable a few years ago.


Actually this company uses a hybrid of computer vision and manual review in India. It is not pure computer vision especially for elements like sweethearting. In fact I know one retailer who has quite a problem with this system not due to the system performance I store but the bandwidth shipping video to India. This manual coding is how they reduce errors and is a typical tech inquest with some vendors now.

Not the answer you're looking for? Browse other questions tagged or ask your own question.