I'm putting together a mobile app that would allow users to search, rate, and upload photos of dishes at restaurants. I have no machine learning experience (just FE/BE/Mobile) so for the time being I was planning on autocompleting search queries based on the 2,700 keyword tags I've individually and appropriately assigned to 10,000 dishes in the db thus far. So for instance, I've given fried chicken dishes the tag, "fried chicken" and so on.

I chose this route for a few reasons: eventually I'd like to implement and train some sort of AI to recognize foods from photos, develop a knowledge graph of what dishes users prefer, and as mentioned above autocomplete search.

With all that being said, something just doesn't feel right with this approach. From my inexperience in this field, it just feels naive. My main obvious priority would be to ship the app but I would definitely appreciate if anyone could point me in the right direction. Is this manual approach a waste of time? How should I be doing this? Let me know if more information is needed. Thanks.


You're building a training set. This is used to teach the AI what you want. The important thing is to be careful that the set doesn't contain false tells like a red and white checkered table cloth every time it's a pasta dish.

We all generalize of course but when humans build training data it's amazingly easy to tip your hand without meaning to.

Why does the AI think this husky is a wolf? The ears? The nose?

enter image description here

It's the snow.

hackernoon : Dogs, Wolves, Data Science, and Why Machines Must Learn Like Humans Do

So as you build your training set be careful about this. But, even if you're careful, be prepared to need to ask your AI why it's classifying the way it is. Or you'll end up classifying every picture of trees as having a tank hiding in it whenever the picture happens to have been taken on a cloudy day1.

  • About how far/long should I continue developing the training set do you think? Based on the numbers given in my post. – Carl Edwards May 2 '18 at 14:58
  • as an additional note, one part of AI training that is often glossed over when thinking about it is correction. If you start seeing false positives a lot, you need to retrain your AI in a targeted way to ensure those false positives don't continue. – Marshall Tigerus May 2 '18 at 14:59
  • @CarlEdwards sadly these numbers are not a good way to tell. You can unknowingly introduce the same false bias with 100 images as you can with 10,000. I'd follow the same advice your financial planner would give you: diversify. Use many different sources of images. Use many different taggers. – candied_orange May 2 '18 at 15:11
  • jefftk.com/p/detecting-tanks -- "It turned out that in the researchers' dataset, photos of camouflaged tanks had been taken on cloudy days, while photos of plain forest had been taken on sunny days. The neural network had learned to distinguish cloudy days from sunny days, instead of distinguishing camouflaged tanks from empty forest." – Robert Harvey May 2 '18 at 19:12
  • It just reminded me of that (possibly apocryphal) anecdote. Was a real eye-opener for me. – Robert Harvey May 2 '18 at 19:17

If you have no other method of generating training data, then manually constructing them is the only way. After all, anything that isn't automatically done by a computer model is by definition "manual".

With more sophisticated algorithms, you can profit from "active learning", i.e. the model has a limited awareness of which examples would be especially useful to add to the training data, and actively asks the user to label specific examples rather than just whatever crosses their path.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.