Pages

Showing posts with label android. Show all posts
Showing posts with label android. Show all posts

Monday, February 16, 2015

Musings on libnodejs (or libiojs) on Android

What can you do with nodejs on android?

Here are some examples:

  • Run your own server in nodejs on your android device, using the same code you would use for a nodejs x86/x86_64 back end! (e.g. udp/tcp/ip, http, websocket, bit/webtorrent server, webrtc client/STUN/TURN/ICE server);
  • Use existing libraries and code for nodejs and run them in your pocket; (which reminds me NPM (not-paid monkey) is on my TODO list)
  • Connect to other android devices running nodejs;
  • Run nodejs on wearables (ehhh... run a server on your watch?);
  • Use the same javascript client code for testing/production on android;
  • Connect to android sensors/instruments/services via unix domain sockets. (e.g. bluetooth);
  • Connect nodejs with android.webkit.WebView and recreate cordoba/phonegap (wretch);
    • Run cordova-cli on your android device directly;
    • Create stunningly fast mockups with (local) back end code in nodejs
I feel that my hobby/job is coming to full circle, I started out coding in C/C++/javascript/java at the uni. Now I'm still coding in the same languages, but with one more in the mix, Xtend(lang).

I'll clean up my code and release a POC (piece of crap) on the android app store.

Friday, February 13, 2015

Progress report: nodejs / iojs on android

I set out to embed node in an android application and I have succeeded in doing so. It's not an earth-shattering accomplishment, but I'm happy all the same.

I'm exploring ways of passing state between an embedded nodejs instance on android and an android zygote process, via unix domain sockets and http sockets. If the unix domain socket method works, then JNI calls to pass state to and fro can be eliminated completely, except for setting up a nodejs instance.

I managed to build shared libraries and executables for arm-android using either libnodejs.so and libiojs.so. I can load it using an activity-bound sticky android.app.Service and load an http-stream repl script.

I documented how I got there here.

Also, see this commit on how to build the shared lib.

I have a working http-stream REPL via an android device's localhost:8000. Any machine can connect to it.

I'm currently working on a way to expose a node CLI/REPL to the user using android.widget.{EditText, TextView} combined with an android domain socket REPL.

After experimenting with node's domain socket server creating abilities on x86-linux and arm-android, I came to the conclusion that android.net.LocalSocket and nodejs domain socket server will not play nice with each other.

After a lot of pacing and some hair-loss, I realized that I have to determine whether bionic is being called properly by libnodejs (on android).

Right now I have two options:
  • Determine that node's C++ code is calling bionic properly to create a domain socket;
  • turn it around: let android create a android.net.LocalServerSocket, let node connect bind/connect with a domain socket client and start the REPL on the client. I'll have to see if this even works on x86-linux.
I also put some effort into stdio redirection, in the hope that the built-in REPL in nodejs would kick in, but that sort of magical thinking usually leads to nowhere, and it didn't.

Update 14-feb-2015:

Next step: fix libuv; I suspect that pipe.c is not properly coded for android; I suspect that android.net.Local bind/connect fails due to a bad addrlen. According to the code in Pro Android with the NDK page 267-268, a socket that is not in the linux abstract namespace and its addrlen must comply to certain rules, the most important rule is the address length.

Update (same day):

Inserted #if defined(__ANDROID__) ... statements to pipe.c and reused libcutils code instead of libuv socket code, that is shared with android.net.Local(Server)Socket's native methods.

Still not connecting.

Update 16-02-2015:

Mission accomplished. My modified libuv can connect with android.net.LocalSocket over a filesystem-based domain unix socket.

In theory my modification supports abstract domain unix sockets, but I don't know if libuv even supports this.

This means that I have IPC over domain sockets on android, between the main thread and the thread running nodejs.

I have tcp/udp/unix domain socket communication, between android and nodejs!

Woohoo!

Thursday, February 6, 2014

My adventures on playing and looping sounds on android, part 2

AudioTrack

This entry will be about to playing sounds and writing loops with AudioTrack on Android for a single channel, 44.1kHz sample rate (number of samples per second), 16-bit depth per sample (2 bytes per sample, i.e. the fidelity of a sound can be expressed in two bytes per sample), sound file. Let's call this set of parameters, for this specific sound file: S.

I prepared a sample with Audacity, by exporting a wav file (microsoft's uncompressed pcm file) with these parameters S.

Feeding data

There are two ways to feed data to AudioTrack's audio buffer:
  1. Set the data feed type to stream, i.e. feed it in increments:
    private AudioTrack audioTrack;
    
    // ...
    
    InputStream is = getResources().openRawResource(R.raw.click); // res/raw/click.wav
    
    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
      AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT,
      minBufferSize, AudioTrack.MODE_STREAM);
    
    int i = 0;
    byte[] music = null; // feed in increments of 512 bytes
    try{
        music = new byte[512];
    
        audioTrack.play();
        while((i = is.read(music)) != -1)
        {
            audioTrack.write(music, 0, i);
            Log.d(getClass().getName(), "samples: " + Arrays.toString(music));
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
    
  2. Set the data feed type to static, i.e. feed it in one go:
    InputStream is = getResources().openRawResource(R.raw.click);
    
    final int WAV_FILE_BYTE_SIZE = 6878; // command line: wc -c click.wav
     
    assert WAV_FILE_BYTE_SIZE > minBufferSize;
      
    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
     AudioFormat.CHANNEL_CONFIGURATION_MONO,
     AudioFormat.ENCODING_PCM_16BIT, WAV_FILE_BYTE_SIZE,
     AudioTrack.MODE_STATIC);
    
    byte[] music = new byte[WAV_FILE_BYTE_SIZE];
    try {
     is.read(music);
    } catch (IOException e) {
     e.printStackTrace();
    }finally
    {
     try {
      is.close();
     } catch (IOException e) {
      e.printStackTrace();
            }
    }
    
    audioTrack.write(music, 0, music.length);
    audioTrack.play();
    // ...
    

In either case, it would be wise to know the minimum buffer size, for the setting S. The minimum buffer size for a sample like this can be determined like this:
int minBufferSize = AudioTrack.getMinBufferSize(44100,
    AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);

Log.d(getClass().getName(), "minBufferSize: " + minBufferSize); // minBufferSize: 4096


Caveat callbacks

Please bear in mind, that AudioTrack does not have any convenient callbacks (interfaces) to tell you if a sample has been loaded correctly or when a sample has finished playing. If you're doing it wrong, then it will simply blow up in your face.

Although, it has a single very versatile AudioTrack.OnPlaybackPositionUpdateListener callback interface and AudioTrack#setNotificationMarkerPosition/(1 or 2), to do stuff when AudioTrack has reached a certain marker position.

Some examples:
  • mark the end of the sound file to trigger the callback, to set the head to the start then restart itself (i.e. a loop);
  • mark the end of the sound file to clean up resources, e.g. stop, flush, release;
  • feed a few measures of music and mark the time needed for a single beat and use the #onPeriodicNotification/1 to signify the beats, with another sound, i.e. add a metronome function;
  • mark certain points in the sound file to trigger other events, e.g. start other sounds, CRUD new game characters, etc.

WAV files

WAV files always contain a header section of 44 bytes (11 frames), before you get to the actual sonic payload. You can either feed these bytes as if they were sounds or you can choose not to feed them to the sound buffer. These initial bytes will probably sound like noise. In my examples above, I did not skip over these bytes.

This is one way to skip over the header:

InputStream is = getResources().openRawResource(R.raw.click);

// ...

final int WAV_FILE_BYTE_SIZE = 6878;

// ...
final int WAV_HEADER_BYTE_SIZE = 44;
byte[] music = new byte[WAV_FILE_BYTE_SIZE - WAV_HEADER_BYTE_SIZE];
try {
 is.read(music, WAV_HEADER_BYTE_SIZE, WAV_FILE_BYTE_SIZE);
} catch (IOException e) {
// ...


Leak prevention

To prevent any memory leaks, do the following:

  • Release your AudioTrack resources:
     @Override
     protected void onDestroy() {
      if (audioTrack != null) {
       audioTrack.flush();
       audioTrack.release();
      }
      super.onDestroy();
     }
    
  • Close all your streams:
    InputStream is = getResources().openRawResource(R.raw.click);
    
    // ...
     byte[] music = new byte[WAV_FILE_BYTE_SIZE];
     try {
      is.read(music);
     } catch (IOException e) {
      e.printStackTrace();
     }finally
     {
      try {
       is.close();
      } catch (IOException e) {
       e.printStackTrace();
      }
     }
    // ...
    


Loops for fun and profit

There are two general ways to loop with AudioTrack:
  • For incremental feeds, i.e. using the AudioTrack.MODE_STREAM parameter, you can simply achieve a loop by writing to the buffer repeatedly with a while/for loop, while the stream is playing;
  • For a static feed, i.e. one time feed, using the AudioTrack.MODE_STATIC parameter, use #setLoopPoints/3:
    // ...
    
    audioTrack.write(music, 0, music.length);
    final int WAV_FILE_FLOORED_FRAME_SIZE = 1719; // math.floor (6878.0 / 4) == 1719 
    final int WAV_HEADER_FRAME_SIZE = 11;
    final int INFINITE_LOOP = -1;
    audioTrack.setLoopPoints(WAV_HEADER_FRAME_SIZE, WAV_FILE_FLOORED_FRAME_SIZE, INFINITE_LOOP);
    audioTrack.play();
    
    // ...
    
If you're doing something that requires (almost) perfect timing, then use the static approach with #setLoopPoint/3, because it has the least latency issues.

When this starts happening, in the static case:

...:E/AndroidRuntime(3110): Caused by: java.lang.IllegalArgumentException: Invalid audio buffer size
...:E/AndroidRuntime(3110): at android.media.AudioTrack.audioBuffSizeCheck(AudioTrack.java:437)

you probably exceeded the allowed buffer size. That's it, just use your imagination.

Monday, February 3, 2014

My adventures on playing and looping sounds on android, part 1

General wisdom

If you want to do simple looping, use SoundPool and MediaPlayer, they both have a function to do this.

If you want to do low-level manipulations of samples, .e.g. mucking around with byte arrays etc., adding effects, to create your own programmatic sound samples, use AudioTrack.

Use JetPlayer for midi tracks.

I'll provide some code about SoundPool and MediaPlayer to give a general idea.

AudioTrack and JetPlayer both deserve their own blog entry... TODO .

Soundpool

public class MainActivity extends Activity implements OnLoadCompleteListener {
    private final static int INVALID_STREAM_ID = -1;

//...

    private SoundPool soundPool;
    private int streamId = INVALID_STREAM_ID;
    private void setupSound() {
        final int SINGLE_STREAM = 1; // you can use multiple
        soundPool = new SoundPool(SINGLE_STREAM, AudioManager.STREAM_MUSIC, 0);
        soundPool.setOnLoadCompleteListener(this);
        // R.raw.click is located in res/raw/click.ogg
        streamId = soundPool.load(this, R.raw.click, 1); // load the sonic content
    }

//...

    private final static int SP_PLAY_ONCE = 0;
    private final static int SP_PLAY_LOOP = -1;
    /**
     *
     * This part below actually starts playing the sound,
     * otherwise you risk the chance of failing
     * to play because the system hasn't
     * finished loading yet
     */
    @Override
    public void onLoadComplete(SoundPool pool, int id, int status) {
        // to loop, see SP_PLAY_LOOP and replace SP_PLAY_ONCE
        pool.play(id, .5f, .5f, 1, SP_PLAY_ONCE, 1.0f);
    }

    @Override
    protected void onDestroy() {
        if (soundPool != null)
        {
            if (streamId != INVALID_STREAM_ID)
                soundPool.unload(streamId);
            soundPool.release();
        }
        super.onDestroy();
    }

You can also use the onLoadComplete callback to do specific stuff e.g. to start looping stuff under certain conditions, set a delay before playing the sonic content, etc.

MediaPlayer

public class MainActivity extends Activity implements onCompletionListener {

private MediaPlayer mediaPlayer;

// ...

   mediaPlayer = MediaPlayer.create(this, R.raw.click);
   // mediaPlayer.setLooping(true);
   mediaPlayer.start();
  // to reuse the mediaPlayer object, with a different sound
  // you have to #reset, #setDataSource #prepare

// ...

    @Override
    public void onCompletion(MediaPlayer mp) {
        // do stuff...
    }

    @Override
    protected void onDestroy() {
        if (mediaPlayer != null)
        {
            mediaPlayer.reset();
            mediaPlayer.release();
        }
        super.onDestroy();
    }

Getting the media file length in milliseconds

private int getSoundFileLengthInMs(int resId)
{
    MediaPlayer mp = MediaPlayer.create(this, resId);
    int duration = mp.getDuration();
    mp.release();
    return duration;
}

Stuff I learned from trial and error

  1. Do not combine Animation callbacks to do your timing with your sounds if you're doing precision work, in the order of milliseconds. Do not use those callbacks with anything important. Ever. I don't know why I have to relearn that lesson...
  2. Looping sounds using callbacks with either MediaPlayer's OnCompletionListener or some combination of SoundPool and Handler and Runnables, will not give you enough metronome like precision.
  3. Do not use TimerTask, to loop and set delays. The constant allocation of objects will cost you cpu-time and memory. Use a single Handler and single Runnable, to set delays.
    private MediaPlayer mediaPlayer; // initialized and prepared somewhere else
    
    //...    
    
        private Handler handler = new Handler();
        private Runnable runnable = new Runnable() {
            @Override
            public void run() {
               // do stuff beforehand...
               mediaPlayer.start();
            }
        };
    
    //...
    
        handler.postDelayed(this, delayInMs);
    
  4. AudioTrack is the only viable solution if you're planning on building a dynamic metronome application, due to the high degree of control.

The end!

Mass install on android devices via BASH

How to install on all your android devices consecutively

This is the command you copy paste on your bash prompt or script:

for d in $(adb devices | egrep "device$" | sed "s/[[:space:]]device$//");
do
  adb -s $d install -r bin/some.apk;
done

Determine that adb can be accessed from the $PATH variable. That or write out the full path where the adb command is located.

PATH="$ANDROID_HOME/platform-tools:$ANDROID_HOME/tools:$PATH"

Happy coding!

P.S. broken-ass implementations of sed like on mac os x don't acknowledge the existence of

[[:space:]]

so, use something like

\s+

or

[\t ]+

and you might have to use sed like this:

sed -e ...

Friday, June 1, 2012

Yet another XMPP Client for Android

First of all, git it from github: https://github.com/Buggaboo/android_xmpp_client

The features

  • GreenDao, database access. Say goodbye to e.g. SQL injection exploits, crap manual data type conversions;
  • Smack library (xmpp), modified for Android, with bugs, but not crashworthy;
  • It vibrates messages to morse code; (3 june 2012: WIP)
  • The messages can be translatable to different languages, just by leveraging the xml resources;
  • Multiple provider support, i.e. connect with google, microsoft, meebo, whatever you want;
  • Licence: GPLv3;
  • Support for android api levels 10 thru 15 (from gingerbread to ice cream sandwich due to compatibility library, 80% of the android  market);
  • Support for android api level 15 (pure ice cream sandwich, less than 10%);
  • Support for fragments (12 june 2012: WIP);
  • Support for android Notifications;


 

 

  The general design

  • Sharing data between Contexts via the database: A context alpha (e.g. Activity or Service) pokes another context beta with an Intent with a Bundle payload containing the record id of a certain database table.
  • To prevent competition for the same database resource, certain services take randomized naps, aka Thread.sleep(Long);
  • A message entity has a buddy entity has a connection entity, i.e. I use foreign keys, to sort and group data etc., GreenDao generates all the code you need;
  • BroadcastReceivers and ArrayAdapters work together to refresh your ListViews.

Known Issues

  1.  There are no bounds on how much data is pulled from the database and rendered as a view, the ListViews are potential memory hogs;
  2. There are no arbitrary limits on data input, which could eat all your memory;
  3. I have not stress-tested how many connections can be kept alive at the same time, before thread-related issues arise;
  4. I suspect that the smack library has its own issues with regard to Presence;
  5. I have not checked  for memory leakages, I use nested classes for my BroadcastReceivers, I haven't checked for cyclical references, I don't understand fully how garbage collection on android works;
  6. The vibrator notification can get annoying, or fun, depending on your mood.