Note, presently have no experience composing or modifying C++ code, which the code that will need to compose or modify is written in.
window.speechSynthesis.speak() does not currently provide an option to return the audio generated by the function call as a
AudioBuffer at either Chromium or Firefox, where have filed feature request for the same
Issue 733051, Bug 1377893.
The relevant workaround that have composed at SpeechSynthesisRecorder.js is to utilize
MediaRecorder() to record the audio output of
.speak() call from
Monitor of Built-in Audio Analog Stereo, which produces expected result at Chromium, though choppy playback of audio at Firefox. Though, why should we have to use
MediaRecorder() to create a copy of generated audio, instead of providing a parameter to
.speak() which indicates that the generated audio should not be played, but rather, returned as an
AudioBuffer, or other data type?
meSpeak.js library provides a
- rawdata: Do not play, return data only. The type of the returned data is derived from the value (case-insensitive) of 'rawdata':
- 'base64': returns a base64-encoded string.
- 'mime': returns a base64-encoded data-url (including the MIME-header). (synonyms: 'data-url', 'data-uri', 'dataurl', 'datauri')
- 'array': returns a plain Array object with uint 8 bit data.
- default (any other value): returns the generated wav-file as an ArrayBuffer (8-bit unsigned).
There has been previous discussion as to incorporating Web Speech API with Web Audio API, for instance at RE: Interacting with WebRTC, the Web Audio API and other external sources
2) Storing and processing text-to-speech fragments.
Rather than mandating immediate output of the synthesized audio stream, it should be considered to introduce an "outputStream" property on a TextToSpeech object which provides a MediaStream object. This allows the synthesized stream to be played through the element, processed through the Web Audio API or even to be stored locally for caching, in case the user is using a device which is not always connected to the internet (and when no local recognizer is available). Furthermore, this would allow websites to store the synthesized audio to a wave file and save this on the server, allowing it to be re-used for user agents or other clients which do not provide an implementation.
I actually asked about adding this on the Web Speech mailing list, and was basically told "no". To be fair to people on that mailing list, I was unable to think of more than one or two specific use cases when prompted.
So unless they've changed something in the past month or so, it sounds like this isn't a planned feature.
and more recently by this user at MediaStream, ArrayBuffer, Blob audio result from speak() for recording?
1) At this point in the development of this project or pursuit, is it more feasible to attempt to learn C++ and the methodology of submitting a patch to browser maintainers for the described implementation; that is, to make required changes at browser source code myself?
2) Continue to ask for help from authors of Web Speech API at browsers and, or developers at large, towards completion of the described implementation?
3) Both 1) and 2)?