Development Cycle

Binary Bits & Pieces

Android Debug Bridge Tricks

Google Android LogoOver the last couple of months I have been doing a lot of work with Cordova and the Ionic Framework creating my field guide of NSW Marine Life for the Android platform.

When I started out working with Android my work flow was pretty simple:

  • Make the required changes to the application code
  • Test in a browser using the simple web server provided with Python package
  • Use Cordova build command to build an apk package for testing on an actual Android device
  • Push the compiled package to the test device using adb to install the package on a device with developer model enable

Over time as my familiarity with the Android tool chain progressed. I found the Android Debug Bridge ( adb ) tool that makes up part of the Android SDK has a lot more to offer. Below are some of handy tips I have found with adb that can make a developers life a little bit easier.
Read the rest of this entry »

leave a comment

December 6th, 2014 at 11:42 am

Posted in Android,Uncategorized

Tagged with , , ,

Simple Iptables Blacklist

firewallAnyone that has run a server for any length of time quickly learns that the Internet is full of spammers, bots and other characters you would prefer didn’t exist. A firewall is key to keeping these guys out and Linux comes with a Kernel level firewall called iptables that can be employed for this purpose. The only downside of iptables is that people who don’t deal with it on a regular basis tend to find its syntax a little bit daunting.

In this post I am going to run you through the process of setting up a basic firewall IP blacklist. Using a bash script to read an blacklist of IP addresses and feeding these to iptables so they can no longer access your server.

First up create decide where you would like the script to live. for this example I am just going to put it in /root/scripts. i.e

mkdir /root/scripts

nano firewall.sh
Read the rest of this entry »

leave a comment

June 16th, 2014 at 7:34 pm

Posted in Linux,Security,Uncategorized

Tagged with ,

Creating a Tor Relay on Debian

tor_logoThe TOR project has proven itself an important tool in the fight to protect the anonymity of people online. The project obviously needs servers to route traffic through for the project to achieve its goals though. So contributors are needed to donate relays nodes ensuring a free Internet for all.

In this post I am going to run you though the process of installing TOR and configuring a relay service for use by the network on a Debian system.

First up login to your machine via a terminal, update the package library and install the Tor server:

apt-get update
apt-get install tor

Then change into the Tor config directory

cd /etc/tor
Read the rest of this entry »

leave a comment

June 12th, 2014 at 6:21 pm

Posted in Linux,Security

Tagged with , , , ,

PHP FPM Errors After Upgrade

PHP LogoRecently I noticed a few of my servers had stopped serving my sites, with  502 errors being to visitors. I took a look at my site logs and noticed that they were full of the “Permission Denied” errors below:

2014/06/09 09:45:17 [crit] 11453#0: *22 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream

A little more investigation and I found that a privilege escalation issue in the PHP default fpm configuration had been found. This had consequently been fixed in the latest releases. But unless a systems administrator is paying attention and knows that changes to the config are required, upgrading the PHP instance leads to a broken web server.
Read the rest of this entry »

leave a comment

June 10th, 2014 at 4:25 pm

Posted in PHP,Security,Sysadmin

Tagged with , , ,

Quick Intro To Memcached Stats

memcached_logoA few days ago I was given the job of changing the caching engine used in a Zend Framework based application from a simple flat file system based caching system to a memcached based object store that could be used by multiple servers.

After extending the existing application bootstrap file to support the new cache mechanism. I started to wonder how I can get information on object store and the items it contains, with a filesystem cache you only have to look in the cache directory to see any cached items. A Memcached instance is not a closed system though stats can be requested on demand, so I hope to cover off some basic methods of interacting with a memcached instance here.

Telnet

The easiest way to interact with your memcached instance is by using telnet. You can simply connect to the port that memcached is running on. This will give you a prompt that can be used to execute commands    to retrieve information about the service.

For example to connect to a local memcached instance running on the default port 11211 you would execute at the command line:

telnet localhost 11211

Telnet Commands

After connecting to the memcached instance over telnet. There are a number of commands which can be executed to find out more about the object cache.

To display general purpose statistic like the uptime, number of connections, version etc of the memcached instance simply type “stats” at the prompt and hit enter.

For more information about the items stored in the cache you can execute the command “stats items”. This will give you things like the item count, age, eviction etc. One entry returned by this command that is wise to watch is evictions. In an ideal world the number of evictions reported for your object cache should be zero. If this is higher at your cache is probably running out of memory to store objects.

For more detailed information on memory usage of the instance you can also use command “stats slabs”.

The whole process of gathering stats can even be automated a little with the use of netcat and watch. To give a kind of “top” like output from memcached with the command:

watch “echo stats | nc 127.0.0.1 11211″

Further Reading:

Memcached project site

Memcached Munin Plugin

leave a comment

May 19th, 2014 at 6:04 pm

PNG Crush: PNG image optimisation script

tinyBuilding a Cordova based mobile application recently at work we were facing an issue with an application size that was literally growing by the day. The project leveraged a lot of PNG templates sent to us by the design department and these template files were contributing about 50mb to the total size of the application.

So the search was on to reduce the size of the project. One of the front end developers on the project ran a few of the PNG templates through the compression service at tinypng.com and was pleased to find the size reduction was around 75%. But with a deadline looming he just didn’t have the time to compress all of the images as there was a few hundred images spread across multiple directories.

I got told about the size improvement and took at the service, to see if I could help out and was pleased to find they had an API available allowing you to optimize up to 500 images per month for free. The documentation and API examples were pretty comprehensive and showed the process to be very straightforward. Simply POST the contents of an image along with your API key to a web service and it will return the location of an optimised copy of the image for you to download.

One of the API examples from the website I took then expanded into a simple PHP shell script. Given a directory to search it will go find all to the PNG files recursively underneath the location. Then post the images to the tinypng.com web service for optimisation and then overwriting the original file with the new copy. The code has been put up on Github so hopefully it can help save others time on their projects in future. So next time you have to optimise the PNG files in your web project feel free to give it a try. With our files we found the service managed to reduces the file sizes on average between 70 – 75%.

Further Reading:

Tinypng.com homepage

PNG Crush github repository

Article on PNG optimisation techniques

leave a comment

March 27th, 2014 at 8:37 pm

Posted in Web,web development

Tagged with , ,

Linux CLI: File Text Search & Replace

magnifying glassOccasionally you need to change the contents of some files easily to save a lot of manual work. In my case it was a server migration, the old IP address of a server was referred to from numerous DNS records. This meant the contents of a couple of hundred Bind zone files would also have to change so that they would reflect the new server address.

This is task that would take way too long to do manually, luckily grep and sed can automate the process. In this post I will show you how to recursively search for and change a string in all files that exist below a directory.

The format for doing a recursive search / replace looks like:

grep -rl ‘search_term’ ./ | xargs sed -i ‘s/search_term/replacement_text/g’
Read the rest of this entry »

leave a comment

December 19th, 2013 at 6:30 pm

Posted in Linux

Tagged with , ,

Tunnelling data over SSH with Putty

An SSH tunnel is used to route unencrypted traffic through an encrypted SSH channel. Its comes in handy for routing traffic over untrusted or hostile networks as the data is encrypted to maintain privacy and prevent eavesdropping. It is also very handy in for bypassing firewall restrictions where the usage of certain services are restricted.

Creating an SSH tunnel with Putty

SSH tunnels may sound complex, but in reality they are very simple to configure on a Windows box using Putty. In this post I am going to run through the process of setting up a SSH tunnel with Putty, and then configuring Firefox to use the SSH tunnel as a SOCKS proxy to securely transfer web traffic over a network.

First open up Putty and enter the connection details for the remote SSH host we are going to tunnel the data too.

  Create SSH connection with Putty

Next expand the SSH option in the side bar on the right under the “Connection” parent node and you should see a section labelled “Tunnels”.

Enter the port you would like to create the tunnel on. This is a port on the local machine which will act as a SOCKS proxy forwarding the data sent to the remote host over via SSH. For the purposes of this demonstration I am going to use port 8080. Also click the radio button labelled “Dynamic”, leaving the destination box above blank.

Creating a dynamic SSH tunnel on port 8080

Now click the “Open” button down the bottom of the application and log in to the remote server via terminal using your new credentials as per usual.

To confirm the tunnel is working you can open a DOS command prompt and execute the command:

netstat -ano

In the results returned you should find a line showing that the local machine is listening for connections on port 8080:

TCP    127.0.0.1:8080         0.0.0.0:0              LISTENING       2680

Now there is an active tunnel, we need to configure a web browser to take advantage of the tunnel. In this example I am going to use Mozilla Firefox, so although the process will be very similar in other browsers, it will vary slightly.

Open up Firefox on your machine, press (ALT + T) to bring up the tools menu and click on the “Options” entry. Click on the entry labelled “Advanced” up the top of the entries page. Then bring up the “Network” tab and click the “Settings” button next to the label “Configure how Firefox connects to the Internet”.

Firefox options page

On this page select the radio button labelled “Manual proxy configuration”. Then next to the “SOCKS Host” heading enter localhost and port 8080 (or the port you setup your tunnel on). After clicking okay to leave this page your browser will now be sending all web requests via the remote host by use of the SSH tunnel.

Configuring Firefox to use local SOCKS proxy

To confirm this is the case search “What is my IP” using Google and it will return the IP address of the remote host you are connected to via SSH.

leave a comment

December 16th, 2013 at 6:30 am

Encypting Diectories With eCryptfs

eCryptfs is an encrypted file system that works at the file system level. In this post I am going to cover the basic usage of eCryptfs and some of the situations you may want to think about using a encrypted file system.

What is an encrypted file system?

File system encryption protects information by converting files and directories that exist on a file system into an unreadable format. To access the files and decipher the information afterward a passphrase is required.

Another related technology is full disk encryption which can be implemented using encfs on Linux. This differs from file system level encryption by encrypting the full disk or partition that the file system containing data resides.

Why would I care about this?

Everyone has some data that they would prefer is kept confidential such electronic receipts, account information, banking information etc. Devices do get lost and stolen at times so encryption so its best confidential data is protected.

Using eCryptfs with Debian

I use Debian on a RaspberryPi powered system at home which is connected to a removable hard drive to which my important information is backed up to using rsync & ssh. So in this post I am going to cover off the process of encrypting the backups with eCryptfs on a system running Debian. The general process is going to be the same in other distributions but the exact steps may vary.

To get started open a terminal and elevate your privileges to root and make sure the package list available to the system is up to date.

sudo su
 
apt-get update

Install the utilities for working with eCryptfs.

apt-get install ecryptfs-utils

installing ecryptfs utils

You then need to ensure the eCryptfs kernel module is loaded with lsmod.

Display loaded kernel modules

Note: On the RaspberryPi I was using the module was not loaded by default so I had to execute first:

cd /lib/modules/$(uname -r)
modprobe ./kernel/fs/ecryptfs

Now to encrypt a directory on the file system for an example. First create a directory that you would like to contain your encrypted data:

mkdir notes

Then we need to mount mount the newly created directory as a eCryptfs so the files are encrypted.

mount -t ecryptfs notes notes

Then you will be run though a few questions about the setup.
 
Select key type to use for newly created files:  
 1) passphrase
 2) tspi
Selection: 1
Passphrase:  
Select cipher:  
 1) aes: blocksize = 16; min keysize = 16; max keysize = 32
 2) blowfish: blocksize = 8; min keysize = 16; max keysize = 56
 3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24
 4) twofish: blocksize = 16; min keysize = 16; max keysize = 32
 5) cast6: blocksize = 16; min keysize = 16; max keysize = 32
 6) cast5: blocksize = 8; min keysize = 5; max keysize = 16
Selection [aes]:  
Select key bytes:  
 1) 16
 2) 32
 3) 24
Selection [16]: 2
Enable plaintext passthrough (y/n) [n]: n
Enable filename encryption (y/n) [n]:  
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_key_bytes=32
  ecryptfs_cipher=aes
  ecryptfs_sig=9b8bcea9e0a5ce7c
WARNING: Based on the contents of [/root/.ecryptfs/sig-cache.txt],
it looks like you have never mounted with this key  
before. This could mean that you have typed your  
passphrase wrong.
 
Would you like to proceed with the mount (yes/no)? : y
Would you like to proceed with the mount (yes/no)? : yes
Would you like to append sig [9b8bcea9e0a5ce71] to
[/root/.ecryptfs/sig-cache.txt]  
in order to avoid this warning in the future (yes/no)? : yes
Successfully appended new sig to user sig cache file
Mounted eCryptfs

Now if the mount command is execute we should the directory we executed against mounted as a ecrypt file system.

Further Reading:

The home of eCryptfs

leave a comment

December 14th, 2013 at 4:15 pm

Posted in Linux

Tagged with , ,

Installing SVN support in Aptana Studio 3

Its been a few years since I have worked anywhere that still uses Subversion as their day to day version control system. But the other day at work I needed to check out some old legacy code which was stored in a svn repository on an old development server. Then it struck me I didn’t even have subversion available on my work machine!

I could have simply installed it from the command line but I use to like have the ability to compare code across multiple revisions directly within my IDE. Here I am going to cover off the required steps to getting support for Subversion installed into Aptana Studio 3.
Read the rest of this entry »

leave a comment

September 5th, 2013 at 10:19 pm

Posted in Programming

Tagged with , ,