2

I have decided to start a personal project using Python since I have used it for several years now and I would like to know if the approach that I have considered is good or not.

Description

It's really simple if you know Alexa / Siri / etc. I have a device which will interact with it's owner given a single area: Music. The scenario as I saw it it's like this:

Owner [voice]: "- I want to listen a sad song, please play me one!"

[backend action here]
- The command will be saved as an audio file in a specific folder.
- I'm using watchdog to monitor file system events - in this case, the creation of an audio file in a folder.
- Convert the above audio file to text using Google API Speech service.
- Search in a database for some key words like: "listen", "sad", "song", "play" and create a response [text] based on them (try to narrow down the searches and guess what the owner wants - not sure how to do this, though)
- Convert the above question to speech using gtts. This will save the audio file to another folder. This file will be played by the device.

Device [voice]: "- Would you prefer a specific artist or genre?"

Owner [voice]: "No."

[backend action here again] - this can occur multiple times if needed and if the owner wants more specific results.
- The command is again processed and converted to text.
- If the command would've been "Yes, I want to listen to a song played by Richard Marx" I would've looked again in the database for the new keywords and go to the specific location of songs played by Richard Marx and play one of them.
- When the command is "No.", I have to search for a sad song in the DB, go to the specific location of that song and play it.

This is the main idea (on a larger scale). A device (its knowledge should involve the music field like: artists / songs / genres / etc.) which can interact with its owner so that it can give him exactly what he wants.

Architecture

Now, I know that I'll have to use a large storage where I can keep all the music, and the DB, but let's assume this is already handled.

What I want to know is if my approach / logic is good enough or if there are better / easier ways of doing this. More, a bit of help with the "not sure how to do this, though" part from above would really help me!

Improve this question
6
  • 1
    If I say play a sad song, play a sad song. Don't ask me who it should be by.
    – candied_orange
    Mar 8, 2018 at 20:37
  • 1
    True. Maybe that's the wrong question that I've given as an example. The idea is that I'd like to make the device interact with the owner at a certain level. For example, when the owner is searching for a specific song, and that doesn't exist, it would be nice to give an alternative as a response. Then ask if that alternative is okay or not.
    – Grajdeanu Alex
    Mar 8, 2018 at 20:43
  • Possibly relevant: Natural-language processing
    – Dan Pichelman
    Mar 8, 2018 at 20:47
  • 1
    You could glue together something using AWS Lex, Polly and SoundCloud.
    – S.D.
    Mar 9, 2018 at 6:00
  • 1
    I have no idea why you are storing audio to files at all. You should be able to stream the audio straight to Google, and stream the synthesized response straight to the audio driver. As for understanding the user's intent, that's a hard problem that we haven't solved. There are various natural language processing libraries that can help, but obviously they are far from perfect. You can get good results if you are willing to require some structure from the user. Of course, even when you know you want a "sad song", you'll need to know which songs are "sad".
    – Derek Elkins left SE
    Mar 11, 2018 at 2:08

1 Answer 1

Reset to default
1
+50

Alexa is easy to write commands for because it states the syntax beforehand. That is, it tries to not accept commands in a free form; it waits for certain keywords, and a certain phrase structure after them. If it cannot understand your utterance, it explains what it knows how to recognize.

Let's pretend that "Play me" is such a keyword sequence, and "sad song" is the actual query.

Now you have to recognize "song" as a piece of music with vocals, and "sad" as a certain quality. "A sad piece" might be an purely instrumental piece, and "a sad tune" or "something sad" might be either.

So, the next thing you need is a vocabulary that maps possible user's words to the tags you have in your music DB, with weights ("sad" may have 0.7 correlation with "downtempo", 0.95 correlation with "minor key", and -0.5 correlation with "loud"). Same with genre names, singer names, date ranges ("play me a happy surf tune from fifties"), etc. Certainly, band names should have 1.0 weight for a particular band.

Now you can generate a list of songs with highest weighted sum of tags, randomize it a bit (remembering last N played songs helps), and pick the top match.

I'd also listen to a short command like "Another!" that would repeat the previous search but randomizing the results differently, a command like "Quiet!" to immediately stop, and make e.g. "disregard" as the last word to cancel processing of any command.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.