In this tutorial video, we demonstrate how to install and use Mage 2.0 in openFrameworks and Pure Data with Xcode. It shows how to run the available examples and control the various speech parameters on the fly using either Open Sound Control (OSC) signals or mage~, a Pure Data external.
In this video, we demonstrate how to control the spectral acoustic features by manipulating the articulatory features, such as tongue, lips and jaw positions. In this version Mage is supporting the reactive generation of articulatory features. This allows us to interactively and intuitively modify phones synthesized in real time, for example transforming one phone into another, by controlling the configuration of the articulators in a visual display.
In this video, we demonstrate how interactive voice interpolation is implemented. Using as interface a processing application, which allows us to navigate through an interactive map and choosing the voices to be interpolated. Then the chosen models are interpolated on the fly so that at the final output we have the result of the interpolated voice models.
In this video, we demonstrate how interactive voice interpolation is used to control the degree of articulation during synthesis. In total three models are used, a model for hypo articulated speech, a neutral model and a model for hyper articulated speech. While synthesizing, the models used for interpolation as well as their weights are controlled by the application itself or a different application via OSC messages.
In this project we explore the possibility of controlling reactive speech/singing synthesis with an electric guitar. The idea is to use instrumental gestures, i.e guitar playing techniques, as we think they are a good start to obtain refined control over synthetic speech/singing. We want to see how intelligibility, naturalness and speaker identity can be addressed as a guitar performance, involving the player and the audience.
Project completed during the eNTERFACE 2012 Workshop in Supelec, Metz, France, July 2-27, 2012. [enterface12.metz.supelec.fr]
In this video, the beginning of "Alice in Wonderland" is first synthesized with natural prosody. Then, pitch, speed and vocal tract length are altered in real time by moving the pen on the surface of the HandSketch, showing how the voice synthesis properties can be updated as the synthesis is going. We conclude by adding a circular phoneme sequencer to this set-up, which enables us to build a sentence on-the-fly (here, "Alice was beginning to get"), showing how content information can be also sent in real time to Mage.
Here, Mage is integrated as a Pure Data external. As a proof of concept, the example patch is combined with the FaceOSC application (by Kyle Mcdonald) to reactively send random chunks of phonemes and change the prosody.