Developing Accessible Apps for Android Devices

A zip file containing the entire project is at AlphabetSample.zip.

Overview

Android accessibility, and indeed accessibility on most platforms, for blind users is accomplished via three main strategies:

Android and the various apps that run on Android devices use all three at times.

Our Project

Our project is going to use a variety of accessibility strategies:

Hopefully our project also demonstrates that creating such an app for Android is not difficult given a basic knowledge of Java programming.

Project Specifications

Our project results in an application that incorporates access through accessibility events, self-voicing of text, and automatic playback of digital audio. The application presents the user with a list of items labelled with a letter of the alphabet and a word. When the item is pressed we use self-voicing to cause the device to speak a list of three words and then play a digital recording of a sentence using all three words. An example might be:

Bbutter, bread, bestButtered bread is the best!

The Tools and Concepts

Some of the tools, features, and concepts discussed include:

Eclipse IDE for Java

Eclipse IDE for Java is a nice, GUI application for developing applications and supports working in a variety of languages. If you want to develop for Android and want it to be easy and intuative then Eclipse is your friend. If you are an experienced developer or just like working from a console (command-line) interface you can do everything we are doing here using a text editor, an android debugger/utility called "adb", and a builder/compiler called "ant". Eclipse is very accessible though complex. I've used it under Windows with both JAWS and NVDA.

Get Java

Eclipse is a Java (tm) application and requires the Java Run-time Environment (JRE). For more advanced development, including Android development, you need the Java Development Kit (JDK).

Get Java from http://www.oracle.com/technetwork/java/javase/downloads/java-se-jre-7-download-432155.html.

Get Eclipse

Download the Eclipse IDE from http://www.eclipse.org/downloads.

There are versions for:

Note: Under Windows you need the Java run-time or Java Development Kit that matches the version of Eclipse, either 32-bit or 64-bit.

Android Tools for Eclipse

Once Eclipse and Java are installed you need the Android development tools for Eclipse:

  1. Start Eclipse and then, from the Help menu, select Install New Software.
  2. Select "Add".
  3. Enter "ADT Plugin" for the Name.
  4. Enter the URL "https://dl-ssl.google.com/android/eclipse/" for the Location.
  5. Select "OK". Note: If you have trouble acquiring the plugin, try using "http" in the Location URL, instead of "https" (https is preferred for security reasons).
  6. In the Available Software dialog, select the checkbox next to Developer Tools and click Next.
  7. In the next window, you'll see a list of the tools to be downloaded. Click Next.
  8. Read and accept the license agreements, then click Finish. Note: If you get a security warning saying that the authenticity or validity of the software can't be established, click OK.
  9. When the installation completes, restart Eclipse.

Retrieved on 2/24/2012 from http://developer.android.com/sdk/eclipse-adt.html#installing.

Activities, Services, Intents, and Events

Almost everything a user works with on an aAndroid device is encapsulated in an Activity. Our project uses one Activity that is a "ListActivity" and is one of the classes of Activities included in the Android framework.

Activities are started in Android with "Intents". An application can use Intents to start other applications or start Activities that are part of other apps. If we wanted to we could provide a way for our Activity to launch the music player or email app. An Intent can start an activity and wait for a result when the activity finishes or the Activity that is launched can become the forground activity, in which case the user interacts with it until they close it. The "back" button for example generally closes one activity and returns users to the one that started it.

"Services" are apps that don't necessarily have an activity. They do things in the background and don't require their own user interface. A service might, for example, sit in the background and watch for text messages and post a notification when one is recieved.

"Events are messages that are sent out from one application and can be recieved by other applications. We are using "AccessibilityEvents" in our project. These are recieved by accessibility services and then acted on.

Android SDK's and Platform Tools

After Eclipse restarts a prompt is presented to install Android SDK's and platform tools. I recommend selecting SDK's for 2.3 and later. If you develop apps for older devices you may need older versions as well. Later you can launch the Android "SDK Manager" to install and update SDK's and tools.

Starting an Android Project

As with most things there is more than one way to get the job done. Here is just one way to start a new Android project in Eclipse:

  1. Bring up the File menu and select "New".
  2. Move down the tree of options and expand "Android".
  3. Select "Android Project".
  4. A dialog is presented asking for the name of your project. We call ours "Alphabet Sample"
  5. Enter the name and select "Next".
  6. You are next asked for the SDK version. Since we are planning to run this on the HTC G2, which uses Android 2.3, choose it and select "Next".
  7. Enter the package name. In this case we are using "com.aph.AlphabetSample".
  8. Leave "Create Activity" checked and keep the default name for the activity.
  9. Select "8, Android 2.3" as the minimum SDK.
  10. We don't have time to cover test projects but if you develop commercial apps it is worthwhile. Leave "Create a test project" unchecked this time.
  11. Select Finish

Activity Main View

Our sample app uses a list activity (ListActivity) as it's main view. For a good tutorial on lists, list views, and list activities see http://www.vogella.de/articles/AndroidListView/article.html by Lars Vogel.

Once we have created our project open the file "AlphabetSampleActivity.java and edit it so that it looks like:
package com.aphAlphabetSample;

import android.app.ListActivity;
import android.os.Bundle;
import android.widget.ArrayAdapter;

public class AlphabetSampleActivity extends ListActivity {
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
		String[] letters = new String[] { "A", "B", "C"};
		ArrayAdapter adapter = new ArrayAdapter(this,
				android.R.layout.simple_list_item_1, letters);
		setListAdapter(adapter);
    }
}

List View Resource

Every Android activity needs a main view. Generally it is a good idea to define these views in an XML resource. Our main view is a list view and so our XML looks like:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
    >

    <ListView
  android:id="@+android:id/list"
        android:layout_width="fill_parent"
        android:layout_height="wrap_content" />

</LinearLayout>

You can locate this file in your project under "res/layout/main.xml". If you are using a screen reader you may want to open it with the Eclipse text editor rather than the default XML editor.

First Run

If we run our app now on a device or emulator with accessibility on we find that it already works and already has some accessibility. It doesn't do much yet. Android has some accessibility features built in to the framework ListActivity class an accessibility event is fired each time one of the list items is selected and our accessibility service, such as Evolve or Talk Back responds to it.

Text-to-speech

Android provides text-to-speech services and an interface that allows any application to use it and thus be a "self-voicing" application. What our applicaion needs is a class to manage the text-to-speech interface and let us "say" a list of words when one of the letters is selected. Sure, we could just put the words in the list and let the accessibility framework take care of it but it wouldn't be as much fun.

  1. Open the "navigator" view in Eclipse and navigate to the directory src/com/aph..
  2. Right-click on the "aph" folder and select "new".
  3. From the dialog select "class" and press return.
  4. The first edit field is for the class name. Enter "SpeechManager".
  5. Everything else in this dialog can remain set to the defaults so select Finish.

The SpeechManager Class

This is what the speech manager class that andles our self-voicing functionality looks like:


public class SpeechManager implements OnInitListener, OnUtteranceCompletedListener {
	public Context context;
	private TextToSpeech mTts ;
	private HashMap uttMap = null;
	int MY_DATA_CHECK_CODE = 0;
	String curUttId = null;
	String name = null;
	SoundManager sound;

    public SpeechManager(Context c) {
    	context = c;
    	mTts = new TextToSpeech( context, this);
    }

    
	public void onInit(int arg0) {
		mTts.setLanguage(Locale.US);
		mTts.setOnUtteranceCompletedListener( this);
    	uttMap = new HashMap();
    	sound = new SoundManager( context);
}


@Override
	public void onUtteranceCompleted(String uttId) {
	if( uttId.equals( "words")) {
		if( name!=null)
			sound.playAsset( name);
	}
		}


	public void say( String str, String id) {
		if( uttMap==null)
			return;
		
		if( str != null) {
			uttMap.put( TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, id);

			curUttId = id;

			mTts.speak( str, TextToSpeech.QUEUE_FLUSH, uttMap);
		}
		}

	
	public void sayFile( String n) {
		name = n;
BufferedReader input = null;
String words = null; 

	    try { 
	        input = new BufferedReader( new InputStreamReader( context.getAssets().open(n + ".txt"))); 
	        words = input.readLine(); 
	    } 
	        catch(IOException e) { 
	            e.printStackTrace(); 
	        } 
	        finally { 
	            try { 
	                input.close(); //stop reading 
	            } 
	            catch(IOException ex) { 
	                ex.printStackTrace(); 
	            } 
	        }
	    if( words!=null)
	    	say( words, "words");
	}
		
    }

How SpeechManager Works

Audio Resources

We created three audio files and copied them to the assets directory of our project:

APH has twelve state-of-the-art recording studios, but I recorded these myself in the closet of my hotel room using an APH Book Port Plus.

The Player Class

The "SoundManager" uses the Android media player to play our sound clips. Create it in much the same way you created the SpeechManager class only call it SoundManager. Edit it so that it looks like:

public class SoundManager {
	
	private Context context;
	
	public SoundManager( Context c) {
		context = c;
	}

	public void playAsset( String name) {
		MediaPlayer 	    	player = new MediaPlayer(); 
	    AssetFileDescriptor afd = null;
		try {
			afd = context.getAssets().openFd(name+".mp3");
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} 
		try {
			player.setDataSource(afd.getFileDescriptor(),afd.getStartOffset(),afd.getLength());
		} catch (IllegalArgumentException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IllegalStateException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} 
	    try {
			player.prepare();
		} catch (IllegalStateException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} 
	    player.start(); 
}

}

References

Android Open Source Project Text-to-speech Example http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/app/TextToSpeechActivity.html
Eclipse Project Download Eclipse IDE for Java http://www.eclipse.org/downloads
Oracle Java 7 download http://www.vogella.de/articles/Android/article.html
Vogel, Lars Android Sounds Tutorial http://www.vogella.de/articles/AndroidMedia/article.html
Vogel, Lars Android ListView and ListActivity Tutorial http://www.vogella.de/articles/AndroidListView/article.html