Pages

Thursday, February 6, 2014

My adventures on playing and looping sounds on android, part 3

Intent

During my adventures I bumped into some definitions. I'll try to explain some of those. 
  • Sample rate
  • Bit depth
  • Channels
  • Bit rate

 

Sample rate

The number of samples required for a second of digital sound. A stream (digital payload) consists of samples. You can imagine that a sample is a unit for the movement of a membrane that vibrates the air in your surrounding area, from your sound amplification device.

 

Bit depth

Depth defines the digital sonic resolution (think: dimensional resolution for screens, but in this case digital sound) or quality and/or fidelity of a single sample.

 

Channels

Channels are the number of tracks on a stream (digital payload) so that each may have different sonic content on them, e.g. track 1 for drums and bass, track 2 for guitars and violins.

 

Bit rate

Bit rate = sample rate x bit depth x channels

For instance the bit rate of a mono channel, 16 byte bit depth, 44.1kHz sample rate, audio file is 88200. Which also means that for each second of digital sound, 88200 bytes are required to be filled with data. Or silence will ensue...

Why is this useful ? You can use this to calculate how much data is required to play a certain length of sound for these specific parameters (sample rate, depth, no. of channels).

To play a sound for 77 milliseconds, in my previous example, it will require 6834 bytes of data, i.e. 88200 x 0.077 = 6834 bytes.

The End

My adventures on playing and looping sounds on android, part 2

AudioTrack

This entry will be about to playing sounds and writing loops with AudioTrack on Android for a single channel, 44.1kHz sample rate (number of samples per second), 16-bit depth per sample (2 bytes per sample, i.e. the fidelity of a sound can be expressed in two bytes per sample), sound file. Let's call this set of parameters, for this specific sound file: S.

I prepared a sample with Audacity, by exporting a wav file (microsoft's uncompressed pcm file) with these parameters S.

Feeding data

There are two ways to feed data to AudioTrack's audio buffer:
  1. Set the data feed type to stream, i.e. feed it in increments:
    private AudioTrack audioTrack;
    
    // ...
    
    InputStream is = getResources().openRawResource(R.raw.click); // res/raw/click.wav
    
    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
      AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT,
      minBufferSize, AudioTrack.MODE_STREAM);
    
    int i = 0;
    byte[] music = null; // feed in increments of 512 bytes
    try{
        music = new byte[512];
    
        audioTrack.play();
        while((i = is.read(music)) != -1)
        {
            audioTrack.write(music, 0, i);
            Log.d(getClass().getName(), "samples: " + Arrays.toString(music));
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
    
  2. Set the data feed type to static, i.e. feed it in one go:
    InputStream is = getResources().openRawResource(R.raw.click);
    
    final int WAV_FILE_BYTE_SIZE = 6878; // command line: wc -c click.wav
     
    assert WAV_FILE_BYTE_SIZE > minBufferSize;
      
    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
     AudioFormat.CHANNEL_CONFIGURATION_MONO,
     AudioFormat.ENCODING_PCM_16BIT, WAV_FILE_BYTE_SIZE,
     AudioTrack.MODE_STATIC);
    
    byte[] music = new byte[WAV_FILE_BYTE_SIZE];
    try {
     is.read(music);
    } catch (IOException e) {
     e.printStackTrace();
    }finally
    {
     try {
      is.close();
     } catch (IOException e) {
      e.printStackTrace();
            }
    }
    
    audioTrack.write(music, 0, music.length);
    audioTrack.play();
    // ...
    

In either case, it would be wise to know the minimum buffer size, for the setting S. The minimum buffer size for a sample like this can be determined like this:
int minBufferSize = AudioTrack.getMinBufferSize(44100,
    AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);

Log.d(getClass().getName(), "minBufferSize: " + minBufferSize); // minBufferSize: 4096


Caveat callbacks

Please bear in mind, that AudioTrack does not have any convenient callbacks (interfaces) to tell you if a sample has been loaded correctly or when a sample has finished playing. If you're doing it wrong, then it will simply blow up in your face.

Although, it has a single very versatile AudioTrack.OnPlaybackPositionUpdateListener callback interface and AudioTrack#setNotificationMarkerPosition/(1 or 2), to do stuff when AudioTrack has reached a certain marker position.

Some examples:
  • mark the end of the sound file to trigger the callback, to set the head to the start then restart itself (i.e. a loop);
  • mark the end of the sound file to clean up resources, e.g. stop, flush, release;
  • feed a few measures of music and mark the time needed for a single beat and use the #onPeriodicNotification/1 to signify the beats, with another sound, i.e. add a metronome function;
  • mark certain points in the sound file to trigger other events, e.g. start other sounds, CRUD new game characters, etc.

WAV files

WAV files always contain a header section of 44 bytes (11 frames), before you get to the actual sonic payload. You can either feed these bytes as if they were sounds or you can choose not to feed them to the sound buffer. These initial bytes will probably sound like noise. In my examples above, I did not skip over these bytes.

This is one way to skip over the header:

InputStream is = getResources().openRawResource(R.raw.click);

// ...

final int WAV_FILE_BYTE_SIZE = 6878;

// ...
final int WAV_HEADER_BYTE_SIZE = 44;
byte[] music = new byte[WAV_FILE_BYTE_SIZE - WAV_HEADER_BYTE_SIZE];
try {
 is.read(music, WAV_HEADER_BYTE_SIZE, WAV_FILE_BYTE_SIZE);
} catch (IOException e) {
// ...


Leak prevention

To prevent any memory leaks, do the following:

  • Release your AudioTrack resources:
     @Override
     protected void onDestroy() {
      if (audioTrack != null) {
       audioTrack.flush();
       audioTrack.release();
      }
      super.onDestroy();
     }
    
  • Close all your streams:
    InputStream is = getResources().openRawResource(R.raw.click);
    
    // ...
     byte[] music = new byte[WAV_FILE_BYTE_SIZE];
     try {
      is.read(music);
     } catch (IOException e) {
      e.printStackTrace();
     }finally
     {
      try {
       is.close();
      } catch (IOException e) {
       e.printStackTrace();
      }
     }
    // ...
    


Loops for fun and profit

There are two general ways to loop with AudioTrack:
  • For incremental feeds, i.e. using the AudioTrack.MODE_STREAM parameter, you can simply achieve a loop by writing to the buffer repeatedly with a while/for loop, while the stream is playing;
  • For a static feed, i.e. one time feed, using the AudioTrack.MODE_STATIC parameter, use #setLoopPoints/3:
    // ...
    
    audioTrack.write(music, 0, music.length);
    final int WAV_FILE_FLOORED_FRAME_SIZE = 1719; // math.floor (6878.0 / 4) == 1719 
    final int WAV_HEADER_FRAME_SIZE = 11;
    final int INFINITE_LOOP = -1;
    audioTrack.setLoopPoints(WAV_HEADER_FRAME_SIZE, WAV_FILE_FLOORED_FRAME_SIZE, INFINITE_LOOP);
    audioTrack.play();
    
    // ...
    
If you're doing something that requires (almost) perfect timing, then use the static approach with #setLoopPoint/3, because it has the least latency issues.

When this starts happening, in the static case:

...:E/AndroidRuntime(3110): Caused by: java.lang.IllegalArgumentException: Invalid audio buffer size
...:E/AndroidRuntime(3110): at android.media.AudioTrack.audioBuffSizeCheck(AudioTrack.java:437)

you probably exceeded the allowed buffer size. That's it, just use your imagination.

Monday, February 3, 2014

My adventures on playing and looping sounds on android, part 1

General wisdom

If you want to do simple looping, use SoundPool and MediaPlayer, they both have a function to do this.

If you want to do low-level manipulations of samples, .e.g. mucking around with byte arrays etc., adding effects, to create your own programmatic sound samples, use AudioTrack.

Use JetPlayer for midi tracks.

I'll provide some code about SoundPool and MediaPlayer to give a general idea.

AudioTrack and JetPlayer both deserve their own blog entry... TODO .

Soundpool

public class MainActivity extends Activity implements OnLoadCompleteListener {
    private final static int INVALID_STREAM_ID = -1;

//...

    private SoundPool soundPool;
    private int streamId = INVALID_STREAM_ID;
    private void setupSound() {
        final int SINGLE_STREAM = 1; // you can use multiple
        soundPool = new SoundPool(SINGLE_STREAM, AudioManager.STREAM_MUSIC, 0);
        soundPool.setOnLoadCompleteListener(this);
        // R.raw.click is located in res/raw/click.ogg
        streamId = soundPool.load(this, R.raw.click, 1); // load the sonic content
    }

//...

    private final static int SP_PLAY_ONCE = 0;
    private final static int SP_PLAY_LOOP = -1;
    /**
     *
     * This part below actually starts playing the sound,
     * otherwise you risk the chance of failing
     * to play because the system hasn't
     * finished loading yet
     */
    @Override
    public void onLoadComplete(SoundPool pool, int id, int status) {
        // to loop, see SP_PLAY_LOOP and replace SP_PLAY_ONCE
        pool.play(id, .5f, .5f, 1, SP_PLAY_ONCE, 1.0f);
    }

    @Override
    protected void onDestroy() {
        if (soundPool != null)
        {
            if (streamId != INVALID_STREAM_ID)
                soundPool.unload(streamId);
            soundPool.release();
        }
        super.onDestroy();
    }

You can also use the onLoadComplete callback to do specific stuff e.g. to start looping stuff under certain conditions, set a delay before playing the sonic content, etc.

MediaPlayer

public class MainActivity extends Activity implements onCompletionListener {

private MediaPlayer mediaPlayer;

// ...

   mediaPlayer = MediaPlayer.create(this, R.raw.click);
   // mediaPlayer.setLooping(true);
   mediaPlayer.start();
  // to reuse the mediaPlayer object, with a different sound
  // you have to #reset, #setDataSource #prepare

// ...

    @Override
    public void onCompletion(MediaPlayer mp) {
        // do stuff...
    }

    @Override
    protected void onDestroy() {
        if (mediaPlayer != null)
        {
            mediaPlayer.reset();
            mediaPlayer.release();
        }
        super.onDestroy();
    }

Getting the media file length in milliseconds

private int getSoundFileLengthInMs(int resId)
{
    MediaPlayer mp = MediaPlayer.create(this, resId);
    int duration = mp.getDuration();
    mp.release();
    return duration;
}

Stuff I learned from trial and error

  1. Do not combine Animation callbacks to do your timing with your sounds if you're doing precision work, in the order of milliseconds. Do not use those callbacks with anything important. Ever. I don't know why I have to relearn that lesson...
  2. Looping sounds using callbacks with either MediaPlayer's OnCompletionListener or some combination of SoundPool and Handler and Runnables, will not give you enough metronome like precision.
  3. Do not use TimerTask, to loop and set delays. The constant allocation of objects will cost you cpu-time and memory. Use a single Handler and single Runnable, to set delays.
    private MediaPlayer mediaPlayer; // initialized and prepared somewhere else
    
    //...    
    
        private Handler handler = new Handler();
        private Runnable runnable = new Runnable() {
            @Override
            public void run() {
               // do stuff beforehand...
               mediaPlayer.start();
            }
        };
    
    //...
    
        handler.postDelayed(this, delayInMs);
    
  4. AudioTrack is the only viable solution if you're planning on building a dynamic metronome application, due to the high degree of control.

The end!

Mass install on android devices via BASH

How to install on all your android devices consecutively

This is the command you copy paste on your bash prompt or script:

for d in $(adb devices | egrep "device$" | sed "s/[[:space:]]device$//");
do
  adb -s $d install -r bin/some.apk;
done

Determine that adb can be accessed from the $PATH variable. That or write out the full path where the adb command is located.

PATH="$ANDROID_HOME/platform-tools:$ANDROID_HOME/tools:$PATH"

Happy coding!

P.S. broken-ass implementations of sed like on mac os x don't acknowledge the existence of

[[:space:]]

so, use something like

\s+

or

[\t ]+

and you might have to use sed like this:

sed -e ...