TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage

simwhi · May 2017

Hi there,

We are looking for someone to develop and document a small plug-in project for our games. The project would be to implement Text To Speech (TTS) plug-in for Android.

If you are interested, please send me a private message so that we can discuss the details further.

Regards

simwhi · May 2017

Hi Gideros maintainers and plug-in authors,

Bump.

We are looking for a developer to help us create a text-to-speech (TTS) plug-in for Android. We are prepared to pay the going rate for this project.

On completion of this project we will make this plug-in available to the whole community so that other would-be plug-in developers can benefit.

We think that a TTS plug-in would be of use in many different kinds of games and applications.

If you are interested, please get in touch with us via private message, so that we can discuss the details in more depth. Alternatively, could anyone suggest where we could find a developer that would be able to complete such as project?

Many thanks,

Simon

hgy29 · May 2017

Hi Simon,

If you find someone who is able to do the android part (java), then I'd be happy to help/do the gideros integration.

simwhi · May 2017

@hgy29
I already have a working example java project in AS that implements basic TTS with Android, but I don't know how to integrate it with Gideros bindings etc.

I'll let you know if we get any offers.

hgy29 · May 2017

Allright, so if you do have the java project and are happy with what it can do, then just give me the API the gideros plugin should expose to lua and I'll write the plugin.

simwhi · May 2017

@hgy29

That sounds great!!

If you would like to discuss payment, please send me a private message.

In the meantime, I'll take another look at my Android project to make sure that I have covered the relevant TTS Android APIs.

Many thanks.

simwhi · May 2017

@hgy29 Now that I have got some other tasks out of the way, I will now start on updating my TTS Android project as well as create a Lua API spec.

simwhi · May 2017

@hgy29 Here's a link to my TTS Android project https://www.dropbox.com/s/lwxxyzjhxlotkgv/TextToSpeechExample.zip?dl=0

Just rebuild the project and everything should work.

I have also attached a simple TTS class that wraps the Lua API function calls. I have kept it fairly simple as I would like to add other features in the future.

Please let me know if you require any further information. I'm really looking forward to getting this plugin working. We will do the same for iOS after.

Many thank in advance.

hgy29 · May 2017

This should do for now, I started writing the plugin.

simwhi · May 2017

@hgy29 WOW!! That's great news! I'll do all the beta testing for... and when I understand how it works I'll write some documentation too...

hgy29 · May 2017

Hi @simwhi,

Here is my first attempt. Requires Gideros 2017.4.1 or better...

Demo code:

require "tts"
 
local tts=TTS.new("UK",1,1)
 
tts:addEventListener(Event.TTS_ERROR,function (e)
	print("TTS Error:"..e.error)
end)
 
tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
	print("TTS Initied")
	tts:speak("Hello world","hello")
end)
 
tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
	print("Utterance complete:"..e.utteranceId)
end)

hgy29 · May 2017

And here is the improved, debugged and polished version.
Sample code:

require "tts"
 
local tts=TTS.new("EN-uk")
 
tts:addEventListener(Event.TTS_ERROR,function (e)
	print("TTS Error:"..e.error)
end)
 
tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
	print("TTS Initied")
	--tts:setLanguage("FR")
	tts:speak("Hello world!","hello")
	--Timer.delayedCall(200,function() tts:stop() end)
end)
 
tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
	print("Utterance:"..e.utteranceId.." state:"..e.state)
end)
 
--tts.speak("Hello world","hello")
--tts.stop()

simwhi · May 2017

@hgy29 Great job!!

simwhi · May 2017

@hgy29

All looks good so far. The only thing I can't do is to change between EN-US and EN-UK (Locale.US and Locale.UK).

hgy29 · May 2017

I checked and it works: try using EN-us and EN-uk instead, and be sure you have UK and US voices samples installed on your phone.

simwhi · May 2017

@hgy29 I tried those combinations, but they have no effect. I know that both en-us and en-uk are installed as the original Android Studio TTS template app works fine. Other languages seem to work fine.

hgy29 · May 2017

Thats weird because I tried that case specifically and did recognize the accent change between uk and us in the spoken sentence..

simwhi · May 2017

@hgy29 That is strange. I have a Samsung A9 Pro (Android 6) and a Huawei (Android 5)

I have Google TTS installed on both.

simwhi · May 2017

Hi @hgy29

The locales "en-us" and "en-uk" should in theory work on my devices, but they don't for some strange reason.

I decided to explicitly set Locale.US and Locale.UK for these cases and everything now works fine. Here is the code:

	public boolean setLanguage(String language) {
		mTtsLanguage = language;
 
        if (isLoaded) {
 
			Locale rl = new Locale(language);
            Log.i("Language", rl.toString());
			if (mTts.isLanguageAvailable(rl) >= 0) {
				String lang = rl.toString();
                Log.i("Language available ", lang);
                if (lang.equals("en-us")) {
                    rl = Locale.US;
                } else if (lang.equals("en-uk")) {
                    rl = Locale.UK;
                }
                mTts.setLanguage(rl);
			}
			else {
				if (android.os.Build.VERSION.SDK_INT >= 21)
					mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
				else
					mTtsLanguage = mTts.getLanguage().getLanguage();
				eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
			}
		}
		return true;
	}

simwhi · May 2017

@hgy29

I created a Gideros project to test setSpeed(), setLanguage(), setPitch() and stop() functions continuously. All work fine.

The stop() function was called using a delayed call. The current speech stopped as expected and the next sentence started (as expected).

I also tested Locale.US and Locale.UK as described in the previous post.

Here's the test code:

require "tts"
 
local tts=TTS.new("en-uk",1,1)
local count = 1
local text = "This is a fairly long sentence that will test the new TTS engine for Android devices." 
 
tts:addEventListener(Event.TTS_ERROR,function (e)
	print("TTS Error:"..e.error)
end)
 
tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
	print("TTS Initialised")
	local sentence = "Test "..count..text	
	tts:speak(sentence,tostring(count))
	count = count + 1
end)
 
tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
	print("Utterance:"..e.utteranceId.." state:"..e.state)
 
	if e.state == "start" then
		print("started")
	elseif e.state == "done" then
		local sentence = "test "..count .. text	
 
		if count == 6 then 
			tts:shutdown()
			tts = nil
		elseif count == 5 then
			tts:setSpeed(math.random(30)/10)
			tts:speak(sentence,tostring(count))
			Timer.delayedCall(3000,function() tts:stop() end)
		elseif (count % 2) == 0 then
			tts:setLanguage("en-us")
			tts:setSpeed(math.random(30)/10)
			tts:speak(sentence,tostring(count))
		else
			tts:setLanguage("en-uk")
			tts:setPitch(math.random(30)/10)
			tts:speak(sentence,tostring(count))		
		end
		count = count + 1
	end
 
end)

hgy29 · May 2017

We need to address this UK/US thing in a more elegant way IMHO. It would be interesting to know what is the difference between Locale.US and Locale.new("EN-us") and the same for UK.
Anyway I am happy you like it, I will make it available as part of gideros distribution if you agree.

simwhi · May 2017

@hgy29 I agree, but I don't know what it is, or how to fix it.

I also agree to making it available to the community, too.

pie · May 2017

Thank you!

it's a great addition to gideros!

>-

NatWobble · May 2017

@hgy29 Thanks, this is great.

I'll polish up my prototype iOS TTS plugin, make it consistent with the Android version, and make it available too.

simwhi · May 2017

@hgy29

I managed to find a solution that is consistent. Here are some test cases:

Locale rl = new Locale("en", "GB");
Locale rl = new Locale("en", "US");
Locale rl = new Locale("en", "AU");
Locale rl = new Locale("fr", "fr");

simwhi · May 2017

@ghy29

I updated this method:

	public boolean setLanguage(String language) {
		mTtsLanguage = language;
 
        //Log.i("Language", mTtsLanguage);
        if (isLoaded) {
			String lang = language.substring(0, 2);
			String country = language.substring(3);
			Log.i("Language", lang);
			Log.i("Country", country);
 
			Locale rl = new Locale(lang, country);
			if (mTts.isLanguageAvailable(rl) >= 0) {
                mTts.setLanguage(rl);
			}
			else {
				if (android.os.Build.VERSION.SDK_INT >= 21)
					mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
				else
					mTtsLanguage = mTts.getLanguage().getLanguage();
				eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
			}
		}
		return true;
	}

hgy29 · May 2017

It looks like en_UK is not a valid locale code: http://stackoverflow.com/questions/7296262/is-en-uk-an-illegal-locale
en_GB is the correct one for UK

simwhi · May 2017

@hgy29 Yes, indeed. Locale.UK is valid, but en-uk isn't. I still couldn't get en-gb to work with language parameter though.

simwhi · May 2017

Here's a more polished version:

	public boolean setLanguage(String language) {
		mTtsLanguage = language;
 
        //Log.i("Language", mTtsLanguage);
        if (isLoaded) {
            String lang;
            String country;
            Locale rl = Locale.US;
 
            if (mTtsLanguage.length() == 5) {
                lang = mTtsLanguage.substring(0, 2);
                country = mTtsLanguage.substring(3);
                Log.i("Language", lang);
                Log.i("Country", country);
 
                rl = new Locale(lang, country);
            }
            else if (mTtsLanguage.length() == 2) {
                rl = new Locale(mTtsLanguage);
            }
 
			if (mTts.isLanguageAvailable(rl) >= 0) {
                mTts.setLanguage(rl);
			}
			else {
				if (android.os.Build.VERSION.SDK_INT >= 21)
					mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
				else
					mTtsLanguage = mTts.getLanguage().getLanguage();
				eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
			}
		}
		return true;
	}

The following Lua test code uses both 2 and 5 character codes ("fr", "en-gb" and "en-us":

require "tts"
 
local tts=TTS.new("fr",1,1)
local count = 1
local text = "This is a fairly long sentence that will test the new TTS engine for Android devices." 
 
tts:addEventListener(Event.TTS_ERROR,function (e)
	print("TTS Error:"..e.error)
end)
 
tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
	print("TTS Initialised")
	local sentence = "Test "..count..text	
	tts:speak(sentence,tostring(count))
	--count = count + 1
end)
 
tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
	print("Utterance:"..e.utteranceId.." state:"..e.state.. " Count: "..count)
 
	if e.state == "start" then
		print("started")
	elseif e.state == "done" then
		count = count + 1
		local sentence = "test "..count .. text	
 
		function stopSpeech()
--			local delay = math.random(2000) + 1000
--			Timer.delayedCall(delay,function() 
--					tts:stop() 
--				end
--			)	
		end
 
		if count > 10 then 
			tts:shutdown()
			tts = nil
		elseif (count % 2) == 0 then
			stopSpeech()
			tts:setLanguage("en-gb")
			--tts:setSpeed(math.random(30)/10)
			tts:speak(sentence,tostring(count))
		else
			stopSpeech()
			tts:setLanguage("en-us")
			--tts:setPitch(math.random(30)/10)
			tts:speak(sentence,tostring(count))		
		end
 
	end
 
end)

hgy29 · May 2017

@simwhi, thanks for your testing and your solution

@NatWobble, yes you you have the iOS part then we can make a dual platform plugin

NatWobble · May 2017

Here's where I am up to with the iOS plugin (plugin attached).

Everything seems to be working consistently with the Android version except for the fact that the iOS plugin initialises instantly, so there isn't a TTS_INIT_COMPLETE event.

I really don't know much about C++/Objective C so if anyone notices anything glaring in the code, please let me know. Specifically, I'm not really sure how to shut it down properly. Currently the shutdown event only stops the current utterance and repeatedly refreshing the player will make it crash. Any ideas would be greatly appreciated.

The plugin requires AVSpeechSynthesis.

I exposed a few more of the ios features:
setVolume,
getVolume
setVoiceIdentifier
getVoicesInstalled
getPitch
getSpeed

Here's the sample lua code:

require "tts"
 
local platform =application:getDeviceInfo()
 
print(platform)
 
local tts=TTS.new("fr",1,1)
local count = 1
local text = "This is a fairly long sentence that will test the new TTS engine for Android devices." 
 
tts:addEventListener(Event.TTS_ERROR,function (e)
	print("TTS Error:"..e.error)
end)
 
local function stopSpeech()
		tts:stop()
 
		local function resumeSpeech()
			tts:speak("It was stopped and has been resumed")
		end
		local timerB=Timer.new(500,1)
		timerB:addEventListener(Event.TIMER,resumeSpeech)
		timerB:start()
end
local timer=Timer.new(9000,1)
timer:addEventListener(Event.TIMER,stopSpeech)
 
 
if platform =="Android" then
	tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
		print("TTS Initialised")
		local sentence = "Test "..count..text	
		tts:speak(sentence,tostring(count))
		timer:start()
	end)
else 
	local sentence = "Test "..count..text	
	tts:speak(sentence,tostring(count))
	timer:start()
 
end
 
tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
 
	print("Utterance:"..e.utteranceId.." state:"..e.state.. " Count: "..count)
 
	if e.state == "start" then
		print("started")
	elseif e.state == "done" then
	    count = count + 1
		local sentence = "test "..count .. text	
 
	    if count == 2 then 
	     tts:setLanguage("en-gb")
	      tts:speak(sentence.. " in British English",tostring(count))
		elseif count == 3 then 
		  tts:setSpeed(0.4)
		  tts:setPitch(2)
		  tts:speak("The pitch and speed have been changed",tostring(count))
		elseif count == 4 and platform == "iOS" then 
		  tts:setSpeed(1)
		  tts:setPitch(1)
		  tts:setVolume(0.2)
		  print("volume: "..tts:getVolume())
		  tts:speak("This is quiet","3")
		  tts:setVolume(1)
		elseif count ==4 then  
		  count=count+1
		  tts:shutdown()		
		elseif count == 5 and platform == "iOS" then
			tts:setVoiceIdentifier("com.apple.ttsbundle.siri_female_en-GB_premium")
			tts:speak("A new voice has been selected by the identifier, but it needs to be installed on the device.",tostring(count))		
			print("pitch: "..tts:getPitch()..", speed: "..tts:getSpeed())
			local voices = tts:getVoicesInstalled()		
			for i=1,#voices do 
				print(voices[i]["language"]..", "..voices[i]["identifier"]..", "..voices[i]["quality"]..", "..voices[i]["name"])
			end		
	    elseif count > 5 then 
		    tts:shutdown()	
		end	
 
	end
 
end)

I'd suggest that we change it so that Event.TTS_UTTERANCE_COMPLETE does't fire at both the start and the end of the utterance and add a new event for the start (Event.TTS_UTTERANCE_STARTED).

Howdy, Stranger!

Categories

In this Discussion

Top Posters

TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage

Comments