1

I'm trying to write a program in Python that will take an input of a .wav (sound) file, and determine whether the user is saying "yes" or "no".

The issue is that the sound files are not always the same length.

I'm worried that with a static input dimension (i.e. 5 seconds of audio), I may have a sample that exceeds that dimension.

I recently read this paper written by Google's Deepmind, which uses sound, but I can't tell how they deal with this issue.

Any insights on how to allow my neural network to deal with a variable size input would be appreciated.

3
  • Can you just sample the data? And sample with a standard number of points?
    – Charles
    Commented Mar 27, 2017 at 4:45
  • @Charlie I think that may be a possible option, but would need additional processing in order to deal with things such as pausing before and after the word, or other things that would make two similar words seem very different based on the constant input size
    – Pro Q
    Commented Mar 6, 2018 at 1:40
  • Right now, I'm thinking neural networks with recurrent neurons (things like LSTMs) are probably the best way to deal with this.
    – Pro Q
    Commented Mar 6, 2018 at 1:41

1 Answer 1

Reset to default
2

In general most sound processing works like other natural language processing in that one of the first steps is to slice your data into basic tokens, i.e. words - in human sound processing we split the words based on the silence between them. Accordingly you can pre-process to:

  1. Filter out sound outside of then normal, significant, speech bandwidth, this is what telephone companies do to save bandwidth.
  2. Split each sample into chunks based on the gaps.

This is the equivalent of the visual deep learning systems standardising the size and bit depth of the images.

With some people, who run their words into each other, the software will have some problems but so would most listeners.

3
  • So then my input size becomes the length of the longest word I want it to learn? What happens if I want to expand my model to deal with an even bigger word? Do I then have to retrain everything?
    – Pro Q
    Commented Mar 27, 2017 at 16:46
  • 1
    Your model will have to "learn" to identify the individual words even when they occupy a range of different lengths in fact it is important to train it with various enunciations which will of course have various lengths even for the same word, (some people pronounce No as Nooo), otherwise it is likely to settle on just the length determines the word. The length to train your model will be, and stay, at 1 word, not a number of seconds.
    – Steve Barnes
    Commented Mar 27, 2017 at 17:34
  • But don't I have weights from the input into the first layer? How do I train/have weights there if there's a variable amount of them? For example. the "no" clip may have 5 sound bytes I want the net to take as input, while the "nooo" clip may have 10. If the first layer of my net has 3 neurons, I'll have 5*3=15 weights to train for "no" and 10*3=30 weights to train for "nooo". How does my neural net train neurons that disappear?
    – Pro Q
    Commented Mar 29, 2017 at 4:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.