Voice Command Adobe Native Extension (ANE) – Control your AIR for Android Apps with Voice Commands

You may have an awesome app that would benefit from hands-free control, and this ANE will let you provide that to your users. No button push is required, there is no Android dialog, and the voice recognition runs in the background similar to a Android service.

You can watch a video demo for Simmer, a hands-free recipe app I’m working on, to see the ANE in action. Also you can download the app and give me some feedback. Check it out y’all!

This ANE has been tested on the Samsung Galaxy Tab 10.1 and the Asus Transformer EeePad TF101, both with Android version 3.2.1 and AIR 3.1.

Here’s how to use the ANE:

  1. Download the Voice Commands ANE and the sample project.
  2. Start a new Flex Mobile Project.
  3. Name your project TestANEProject. Make sure the SDK is Flex 4.6.0.
  4. Under Mobile Settings, leave only Android checked as the Target Platform.
  5. Leave the default settings for Server Settings.
  6. In Build Paths, click the Native Extensions tab. Click Add ANE, browse to the location where you downloaded my ANE, select it and click OK. Click Finish to see your new project.
  7. The first thing you have to do is set the permissions for your new project, or Android won’t let the ANE function. Open TestANEProject-app.xml. Right-click it and open it with a text editor.
  8. The two permissions you need are INTERNET and AUDIO. Scroll down until you see this:
  9. 
            16bit
            
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			    
    			
    			
    		]]>
        
    
  10. Internet permissions are provided by default, so all you have to do is uncomment the RECORD_AUDIO permission. If it isn’t there, just add this line to the rest of the permissions:

    Close TestANEProject-app.xml.

  11. Right-click TestANEProject inside the Project Explorer to open the context menu, and click Properties. Expand Flex Build Packaging and select Google Android.
  12. Click the Native Extensions tab and in the “Package” column make sure that checkbox is selected. If not, select it. It may tell you the ANE is not used in the application, and ask you if you want to include it anyway, say Yes. Click Apply, then OK.
  13. Almost there! Now you just have to add something to your app so you can see the ANE in action :) Replace the contents of TestANEProjectHomeView.mxml with the following:
     
    
    	
    	
    		
    	
    	
    		
    	
    	
    		
    		
    	
    	
    
    
    
  14. Go ahead and run your app on any Android device running AIR 3.1. You will see that after you’ve spoken to the app, the ANE spits a comma-delimited string containing Google’s 5 best guesses as to what was said. Split the string and cycle through the resultant array of strings to evaluate the speech.

    Tada! All done. Enjoy controlling your app with Voice Commands 😀

  15. P.S.! With great power yada yada. Make sure you call stopListening() any time your app loses focus, or when you have no need for voice commands, otherwise Google’s voice recognition will continue in a loop in the background! We don’t want to eat up your precious device’s resources like that so turn the lights off when you’re not using em!

    P.P.S. If you have any problems, just leave em in the comments section here. But do me a favor and use Logcat to see what might be going wrong and try to give me as much information as possible. You can email me the info at ruedaminute at gmail.

  16. Please get in touch on Twitter (@ruedaminute) if you like the ANE, Simmer, or whatever. I’d love to collaborate and create some apps for fun and profit 😀

15 Responses to “Voice Command Adobe Native Extension (ANE) – Control your AIR for Android Apps with Voice Commands”

  1. Ante says:

    Great extension! It’s inluded in my AIR for Android tech demo app – AIRexplorer :)
    https://play.google.com/store/apps/details?id=air.AIRexplorer
    Thank you!

  2. mourada says:

    Bravo for your extension!
    It works remarkably.

    I tested fine with version 3.2 of Android, but it seems it does not work with android 2.2.
    can you confirm?

    thanks.

    Mourada

  3. mourada says:

    Ante,
    very good application, very clean.
    for your example, you use another extension, it seems to you?
    does not solve my problem
    thanks

  4. Erwinus says:

    Hai Ante,

    Working nice, nice done, tried this on a Galaxy S however it uses the language settings of your phone. Speaking words in the phone its language works mostly but in another language the results are mostly unexpectedly. For example when it is using Dutch and you talk English.

    Is it possible to change the language settings anyway or do you can add to support other languages for better results?

  5. murphy says:

    Nice work with the ANE. It works pretty good with Android 3.2.x but on my 2.2 droid I can only get one recognition array back. If I want to make a new request I have to stop listening and start over again.

    You mentioned one could use logcat for debugging. Did I get that right? One can use logcat in FlashBuilder somehow?

  6. Brandon says:

    I tried to run your example on my Acer Iconia a500 but I get a Speech Recognition error 9
    any ideas what this could be or how to fix it?
    I would love to include the voice input in a novelty app I am working on.

    Thanks
    Brandon

  7. Hakan KIRIK says:

    not working for me.
    Android 4.0.3
    Flash CS6
    Mobee 9.7inch
    When I try to publish:
    Error creating files:
    aapt tool failed:main.xml:7:errror:Error:No resource found that matches the given name(at ‘text’ with value ‘@string/hello’).

  8. Bill says:

    Hi Michelle:

    I am using the SpeechService ANE, in Flash CS6 Pro,
    targeting AIR 3.4 on Android 4.1.2

    It works at first … .

    But if you enter a few sentences and wait a while before entering more,
    the “connection” drops, at best.
    That is .. it stops taking input.

    And at worst, the app crashes out completely,
    with the “Unfortunately txt2Speech, has closed”.

    Can I enlist your help to at least trap for the full crash?
    (some of that consulting you mentioned)

    Anyone else, if Michelle is busy?

    I have done a couple little test apps with the SpeechService ANE,
    and all crash out after entering 5 or so sentences — and
    then pausing before entering more.

    Thanks for your help!

    Bill

  9. Hi, I look forward to trying out your ANE. Are you available for consulting work for creating ANEs?

    Thanks,

    Bruce

  10. Bill says:

    It’s working on a month after my post,
    and I still don’t have this ANE working properly.

    Offered the author consulting dollars to help,
    but no response.

    Can anyone else help me to work through this (paid)?
    sam.90001a@gmail.com

    It works fine to start off, entering sentences of text, but flat out dies with a complete crash out of the app, after about 5-6 sentences. No errors are thrown from the ANE back up to Actionscript.

    Are there any other Flash Pro CS6 ready speech ANEs available, other than this and the INOEL one?

    Thanks

  11. Sanjeev says:

    Hi,

    You had deleted my last post. I don’t know why. I just wanted to inform you that ANE is working now with PHP server but now because of the speech service sometimes my app is “not responding”. Can you please share your ANE source code?

    Thnx,
    Sanjeev

  12. Hello and thanks for this awesome ANE! It does what I am looking for. However, I was experiencing the same issue murphy is. I can get 1 to 3 arrays and then I have to restart it. Also, if I don’t provide any input for a few seconds, it will stop listening. I’m looking to make a mobile app for people driving cars and I want to provide voice commands at any point during a long drive. Is there any alternative to constantly turning on and off the listening? Or is this outside the use case this ANE was built for?

    I suppose some alternatives are constantly toggling listening, or trying to access the microphone and look for an activity change before initiating listening, though that could cause some lag in recording.

    Thanks again for an awesome ANE!

  13. Mike says:

    I was hoping you could make a similar tutorial on how this could be accomplished with Flash Pro CS6. I have seen a number of others asking about this and CS6. Thanks and hopefully you will consider it.

  14. Rondel says:

    i tested this one but it seems have a problem at the second line “TestANEProjectHomeView.mxml” because the URL path did not exist

  15. Great extension, really works well.
    I was wondering if it’s possible to restrict the number of words the API is listening out for?
    For example if you only want to hear “left”,”right”,”up”,”down” and ignore everything else. Right now I get a long array of possible values.

    Cheers!

Leave a Reply

Your email address will not be published. Required fields are marked *

This blog is kept spam free by WP-SpamFree.

Get Adobe Flash player