That's A Technical Term

A blog with random tech posts

Learning Android

I have always liked Android. The very idea that you can replace almost anything by installing an app.

I have tried to find online material for learning to program before, but I find that when you want to learn something technical you really need a physical book. I have been able to write very simple programs, but the GUI part has just been a mystery. You need something that tells you why you do something in a certain way.

On Amazon, the highest rated android book is ‘Android Programming – The Big Nerd Ranch Guide’ by Bill Phillips and Brian Hardy. After having read about 22 chapters I have to agree that it is a really good book!

The the book is really well organised. The functionality you add nicely introduces new concepts, and the challenges let you explore them deeper, plus reinforce what you have learnt up until that point.

Unfortunately the book is now quite old, and they aim to teach you how to write apps that run on devices all the way down to android 4.2. They say in the book that Froyo (2.2) and Gingerbread (2.3.x) make up of about 50% of all devices out there. Well according to the latest statistics, that number is now < 10%.

To be honest, since I don’t aim to capitalise I wouldn’t even have bothered with 50%, only that it would work on my phone :)

With the arrival of Lollipop I thought I might as well give it a more serious attempt. No matter what you think about the new design, at least they’ve tried to come up with some relatively clear guidelines!

And this is where I have had a couple of problems when following the book. It looks like a few widgets are broken in Lollipop. One widget (view) that seems to have had a serious overhaul is the DatePicker. The OnChangedDateListener will NOT be triggered when you are in calendar mode. Instead you have to set android:datePickerMode=spinner to get the old style. You have the same problem with TimePicker. You can read more about it here.

Even with these issues, I would still recommend this book if you are interested in learning Android. I would say that it is almost a requirement to know Java though, you will be certainly be struggling otherwise!

Using Jq to Manipulate JSON

Most of our web services at work speak json. Sometimes it is nice to be able to extract select information from the responses. I find jq to be exceptionally well suited for these situations.

The syntax can be perplexing in the beginning, and the tutorial and manual do not (in my opinion) explain in enough detail how you should think when writing your filters, and how you combine them.

The program takes what is called a filter. A filter basically generates output that is usually based on the input.

Basic usage

The simplest case is where you just copy the input to output:

Simple case
1
2
3
4
$ echo '{"key": "value"}' | jq .
{
  "key": "value"
}

Since jq pretty-prints the json by default (can be changed to compact mode by using the -c flag) the output is nicely formatted.

The single dot just tells jq to copy the root node. If you just want the value you can type:

Extract the value of field ‘key’
1
2
3
4
$ echo '{"key": "value"}' | jq .key
"value"
$ echo '{"key": "value"}' | jq -r .key
value

As you can see, -r (raw-output) unwraps the string.

You can use multiple filters on your data. Let us say that you want to count the number of elements in an array. You first want to pick out the array, and then run a filter called length on it:

Extract the value of field ‘key’
1
2
$ echo '{"list": [ "value1", "value2", "value3" ]}' | jq '.list | length'
3

In this example .list picks the array from the list field. This is piped to the length filter which returns the number of elements in the array.

Advanced usage

Now let us consider a slightly more complicated json object. Spotify has an open API where you can search for tracks and albums.

Let us assume that we want to find songs from the Book Of Mormons.

We will use this search: https://api.spotify.com/v1/search?type=track&market=gb&q=The+Book+Of+Mormon

The json has the following format (excluding a lot of to us uninteresting data):

Spotify json formation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
  "tracks": {
    "items": [
      {
        "album": { ... },
        "artists": [
          {
            "name": "artist name 1"
          },
          ...
        ],
        "name": "track name",
        "track_number": 1,
        "duration_ms": 123845
      }
    ]
  }
}

Store the search result as a file:

Save the search result as json
1
$ curl 'https://api.spotify.com/v1/search?type=track&market=gb&q=The+Book+Of+Mormon' > tracks.json

OK now let’s figure out who is performing in these songs:

Extract artist names
1
2
3
4
5
6
7
8
$ jq -r '[.tracks.items[].artists[].name] | unique | .[]' tracks.json
Andrew Rannells
Asmeret Gebremichael
Benjamin Schrader
Brian Sears
Brian Tyree Henry
Clark Johnson
...

Woah, OK that was a big step! Let us break it down.

  • .tracks.items[].artists[].name clearly picks all the artist names
  • this is then wrapped in an array
  • the array is sent to a unique filter which sorts the array and removes any duplicates
  • .[] unwraps the array (we would otherwise see an array being returned by jq)

The second step is hinting at another feature of jq: you can construct json objects, not only pick them out from an existing json object.

Now let us say that we want to construct new json from the track.json file. We want to create a simplifed tracklist for the album.. The tracks should be sorted by track index. We start off by creating the list of tracks.

Track information, sorted by index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ jq -r '
> [
>   .tracks.items[] |
>   {
>     index: .track_number,
>     name,
>     album: .album.name,
>   }] | sort_by(.index)' tracks.json
[
  {
    "index": 1,
    "name": "Hello!",
    "album": "The Book of Mormon"
  },
  {
    "index": 1,
    "name": "The Book of Mormon in 3 Minutes",
    "album": "The Book of Mormon in 3 Minutes"
  },
  {
    "index": 2,
    "name": "Book Of Mormon Stories/The Golden Plates",
    "album": "Sharing Time"
  },
  ...
]

Kind of works. As we don’t specify the source of name, jq will map it to the input name. Unfortunately Spotify doesn’t let you search exact album titles, so will have to do the filtering ourselves:

Track information only for tracks from the specific album, sorted by index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
$ jq -r '
> [
>   .tracks.items[] |
>   select(.album.name == "The Book of Mormon") |
>   {
>     index: .track_number,
>     name,
>     album: .album.name,
>     length: (.duration_ms / 1000 / 60)
>   }
> ] | sort_by(.index)' tracks.json
[
  {
    "index": 1,
    "name": "Hello!",
    "album": "The Book of Mormon",
    "length": 2.8671
  },
  {
    "index": 2,
    "name": "Two By Two",
    "album": "The Book of Mormon",
    "length": 4.5262166666666666
  },
  {
    "index": 3,
    "name": "You And Me (But Mostly Me)",
    "album": "The Book of Mormon",
    "length": 2.720666666666667
  },
  ...
]

Much better! select only picks the elements that match the given criteria. We also added a duration while we were at it. As you can see simple maths can be applied.

Now that we have the data we want, we can generate the final json we wanted:

Album information
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
$ jq -r '
> [
>   .tracks.items[] |
>   select(.album.name == "The Book of Mormon") |
>   {
>     index:.track_number,
>     name,
>     length:(.duration_ms / 1000 / 60)
>   }
> ] | sort_by(.index) |
> {
>   title: "The Book of Mormon",
>   average_length: ([ .[].length ] | add / length),
>   tracks: .
> }' tracks.json
{
  "title": "The Book of Mormon",
  "average_length": 4.2499520833333335,
  "tracks": [
    {
      "index": 1,
      "name": "Hello!",
      "length": 2.8671
    },
    {
      "index": 2,
      "name": "Two By Two",
      "length": 4.5262166666666666
    },
    {
      "index": 3,
      "name": "You And Me (But Mostly Me)",
      "length": 2.720666666666667
    },
    ...
  ]
}

We have the same transformation as before, and then we send it to a new filter:

  • title is a hardcoded string
  • average_length is a calculation of the average length of the songs
    • we extract the length of each song and put them in an array. We then pipe it to both add and length. add will sum up all the values in the array, while length returns the length of the array
  • tracks is simply the json we generated before

So you can see that jq is incredibly powerful when it comes to manipulating json. Don’t worry if you struggle to understand everything. To be honest I keep forgetting how it works, which is the reason I put this post together.

The best way to learn is to simply experiment with it. There is an online tool where you can try: jqplay.org

There are more functions available, but these are most of what I use. You can find them all in the manual. Just make sure you have the latest version of jq!

I-buddy With Raspberry Pi

I had the i-buddy plugged in at work to the Ubuntu box for a long time, but one day I switched to a laptop, which makes it a little bit less convenient to plug in the i-buddy. The poor thing was collecting dust in a drawer for many months, until this summer I decided to plug it in to my Raspberry Pi as a stand-alone physical event notifier.

Since I had had luck with the linux drivers last time I decided to try it again on the Pi. No luck. It just wouldn’t find it! I was probably doing something stupid. Anyways, I decided to try out a python library instead called pybuddy. This was fine by me as most Pi projects are written in Python anyway.

pybuddy

pybuddy can be used in two ways. You can either use pybuddy.lib in your own program, or you can use pybuddy-daemon.py to start a standalone server which takes commands via port 8888 on 127.0.0.1.

Whether you use pybuddy as library or a server, you might have to configure it. It assumes that your i-buddy’s product id is 0x0001. In my case it was 0x0002.

When plugging the i-buddy in, call dmesg:

1
2
3
4
5
6
7
8
pi@raspberrypi ~/hack $ dmesg
 ...

[   23.603277] usb 1-1.2: new low-speed USB device number 5 using dwc_otg
[   23.710086] usb 1-1.2: New USB device found, idVendor=1130, idProduct=0002
[   23.710116] usb 1-1.2: New USB device strings: Mfr=0, Product=2, SerialNumber=0

...

In this case the product id is 0002, so pybuddy has to be updated here:

pybuddylib.py
1
2
3
4
5
6
7
### iBuddy Device Class
class iBuddyDevice:
  USB_VENDOR  = 0x1130
  USB_PRODUCT = int(0x0001) # <-- change this
  BATTERY     = 0
  SETUP       = (0x22, 0x09, 0x00, 0x02, 0x01, 0x00, 0x00, 0x00)
  MESS        = (0x55, 0x53, 0x42, 0x43, 0x00, 0x40, 0x02)

Or equivalent in pybuddy-daemon.py.

To make sure it works, you can run pybuddy-daemon.py (it is self-contained, so remember to update the product id in this file as well!):

1
$ su pybuddy-daemon.py

Then in a separate terminal, type:

1
$ echo "DEMO" | nc -4u -w1 127.0.0.1 8888

and you should see the i-buddy flapping, twisting and flashing. Please note that if your USB power plug isn’t powerful enough your Pi might actually reboot instead! I got it working with my 2A power plug (though I think the Raspberry PI is not supposed to use more than 1A anyway). I use a powered USB hub just to make sure.

I-buddy With Build Scripts

I had been thinking for quite some time that I wanted to build something that I could control from the computer.

Last year it flared up again, when I wanted to get a more obvious indication of how our builds were doing at the office. I then remembered that I’d seen something a few years back online called the i-buddy.

The i-buddy cool computer gadget out there that for some reason has gone unnoticed by most people.

Since it has many different actions (RGB head, flashing heart, flapping wings, twisting) it sounded like just the thing for this purpose.

The i-buddy was originally made solely for use with Live Messenger, but technically inclined people were kind enough to reverse engineer the protocol (at least this guy). We now have a few linux drivers and programming libraries out there that can control it. I had an Ubuntu box at work, and since I’m a bash fan I went with the linux driver route.

Using ibuddy-lkm, I built and installed a kernel driver. The great thing with this driver is that it gives you a number of virtual files with which you control the i-buddy:

/proc/driver/ibuddy/0/heart
/proc/driver/ibuddy/0/twist
/proc/driver/ibuddy/0/wings
/proc/driver/ibuddy/0/red
/proc/driver/ibuddy/0/green
/proc/driver/ibuddy/0/blue

Just write any value to one of the files, and that feature will be toggled!

So all I had to do in my build script was to check the response code, and perform a set of actions depending on that response code:

ibuddy.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/bin/bash

function send_commands() {
    for command in $*; do
        echo 1 > /proc/driver/ibuddy/0/$command
    done
}

function send_multiple_times() {
    for f in $(seq 1 $2); do
        for commands in "$1" "$1"; do
            send_commands $commands
            sleep 0.15
        done
    done
}

"$@"

ERROR="$?"

if [ ! -e "/proc/driver/ibuddy/0" ]; then
    echo "IBUDDY NOT FOUND" >&2
else

( \

send_commands reset

if [ "$ERROR" = "0" ]; then
    send_commands green
    send_multiple_times "green heart" 7
    send_commands reset
else
    send_multiple_times "red wings twist" 15
    send_commands red
fi

) &

fi
 
exit $ERROR

Now I just had to create an alias by adding the following to my .bashrc file:

.bashrc
1
alias mci='$HOME/scripts/ibuddy.sh mvn clean install'

I have yet to build something myself that is triggered on builds :)