Development Cycle

Binary Bits & Pieces

PNG Crush: PNG image optimisation script

tinyBuilding a Cordova based mobile application recently at work we were facing an issue with an application size that was literally growing by the day. The project leveraged a lot of PNG templates sent to us by the design department and these template files were contributing about 50mb to the total size of the application.

So the search was on to reduce the size of the project. One of the front end developers on the project ran a few of the PNG templates through the compression service at tinypng.com and was pleased to find the size reduction was around 75%. But with a deadline looming he just didn’t have the time to compress all of the images as there was a few hundred images spread across multiple directories.

I got told about the size improvement and took at the service, to see if I could help out and was pleased to find they had an API available allowing you to optimize up to 500 images per month for free. The documentation and API examples were pretty comprehensive and showed the process to be very straightforward. Simply POST the contents of an image along with your API key to a web service and it will return the location of an optimised copy of the image for you to download.

One of the API examples from the website I took then expanded into a simple PHP shell script. Given a directory to search it will go find all to the PNG files recursively underneath the location. Then post the images to the tinypng.com web service for optimisation and then overwriting the original file with the new copy. The code has been put up on Github so hopefully it can help save others time on their projects in future. So next time you have to optimise the PNG files in your web project feel free to give it a try. With our files we found the service managed to reduces the file sizes on average between 70 – 75%.

Further Reading:

Tinypng.com homepage

PNG Crush github repository

Article on PNG optimisation techniques

leave a comment

March 27th, 2014 at 8:37 pm

Posted in Web,web development

Tagged with , ,

Linux CLI: File Text Search & Replace

magnifying glassOccasionally you need to change the contents of some files easily to save a lot of manual work. In my case it was a server migration, the old IP address of a server was referred to from numerous DNS records. This meant the contents of a couple of hundred Bind zone files would also have to change so that they would reflect the new server address.

This is task that would take way too long to do manually, luckily grep and sed can automate the process. In this post I will show you how to recursively search for and change a string in all files that exist below a directory.

The format for doing a recursive search / replace looks like:

grep -rl ‘search_term’ ./ | xargs sed -i ‘s/search_term/replacement_text/g’
Read the rest of this entry »

leave a comment

December 19th, 2013 at 6:30 pm

Posted in Linux

Tagged with , ,

Tunnelling data over SSH with Putty

An SSH tunnel is used to route unencrypted traffic through an encrypted SSH channel. Its comes in handy for routing traffic over untrusted or hostile networks as the data is encrypted to maintain privacy and prevent eavesdropping. It is also very handy in for bypassing firewall restrictions where the usage of certain services are restricted.

Creating an SSH tunnel with Putty

SSH tunnels may sound complex, but in reality they are very simple to configure on a Windows box using Putty. In this post I am going to run through the process of setting up a SSH tunnel with Putty, and then configuring Firefox to use the SSH tunnel as a SOCKS proxy to securely transfer web traffic over a network.

First open up Putty and enter the connection details for the remote SSH host we are going to tunnel the data too.

  Create SSH connection with Putty

Next expand the SSH option in the side bar on the right under the “Connection” parent node and you should see a section labelled “Tunnels”.

Enter the port you would like to create the tunnel on. This is a port on the local machine which will act as a SOCKS proxy forwarding the data sent to the remote host over via SSH. For the purposes of this demonstration I am going to use port 8080. Also click the radio button labelled “Dynamic”, leaving the destination box above blank.

Creating a dynamic SSH tunnel on port 8080

Now click the “Open” button down the bottom of the application and log in to the remote server via terminal using your new credentials as per usual.

To confirm the tunnel is working you can open a DOS command prompt and execute the command:

netstat -ano

In the results returned you should find a line showing that the local machine is listening for connections on port 8080:

TCP    127.0.0.1:8080         0.0.0.0:0              LISTENING       2680

Now there is an active tunnel, we need to configure a web browser to take advantage of the tunnel. In this example I am going to use Mozilla Firefox, so although the process will be very similar in other browsers, it will vary slightly.

Open up Firefox on your machine, press (ALT + T) to bring up the tools menu and click on the “Options” entry. Click on the entry labelled “Advanced” up the top of the entries page. Then bring up the “Network” tab and click the “Settings” button next to the label “Configure how Firefox connects to the Internet”.

Firefox options page

On this page select the radio button labelled “Manual proxy configuration”. Then next to the “SOCKS Host” heading enter localhost and port 8080 (or the port you setup your tunnel on). After clicking okay to leave this page your browser will now be sending all web requests via the remote host by use of the SSH tunnel.

Configuring Firefox to use local SOCKS proxy

To confirm this is the case search “What is my IP” using Google and it will return the IP address of the remote host you are connected to via SSH.

leave a comment

December 16th, 2013 at 6:30 am

Encypting Diectories With eCryptfs

eCryptfs is an encrypted file system that works at the file system level. In this post I am going to cover the basic usage of eCryptfs and some of the situations you may want to think about using a encrypted file system.

What is an encrypted file system?

File system encryption protects information by converting files and directories that exist on a file system into an unreadable format. To access the files and decipher the information afterward a passphrase is required.

Another related technology is full disk encryption which can be implemented using encfs on Linux. This differs from file system level encryption by encrypting the full disk or partition that the file system containing data resides.

Why would I care about this?

Everyone has some data that they would prefer is kept confidential such electronic receipts, account information, banking information etc. Devices do get lost and stolen at times so encryption so its best confidential data is protected.

Using eCryptfs with Debian

I use Debian on a RaspberryPi powered system at home which is connected to a removable hard drive to which my important information is backed up to using rsync & ssh. So in this post I am going to cover off the process of encrypting the backups with eCryptfs on a system running Debian. The general process is going to be the same in other distributions but the exact steps may vary.

To get started open a terminal and elevate your privileges to root and make sure the package list available to the system is up to date.

sudo su
 
apt-get update

Install the utilities for working with eCryptfs.

apt-get install ecryptfs-utils

installing ecryptfs utils

You then need to ensure the eCryptfs kernel module is loaded with lsmod.

Display loaded kernel modules

Note: On the RaspberryPi I was using the module was not loaded by default so I had to execute first:

cd /lib/modules/$(uname -r)
modprobe ./kernel/fs/ecryptfs

Now to encrypt a directory on the file system for an example. First create a directory that you would like to contain your encrypted data:

mkdir notes

Then we need to mount mount the newly created directory as a eCryptfs so the files are encrypted.

mount -t ecryptfs notes notes

Then you will be run though a few questions about the setup.
 
Select key type to use for newly created files:  
 1) passphrase
 2) tspi
Selection: 1
Passphrase:  
Select cipher:  
 1) aes: blocksize = 16; min keysize = 16; max keysize = 32
 2) blowfish: blocksize = 8; min keysize = 16; max keysize = 56
 3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24
 4) twofish: blocksize = 16; min keysize = 16; max keysize = 32
 5) cast6: blocksize = 16; min keysize = 16; max keysize = 32
 6) cast5: blocksize = 8; min keysize = 5; max keysize = 16
Selection [aes]:  
Select key bytes:  
 1) 16
 2) 32
 3) 24
Selection [16]: 2
Enable plaintext passthrough (y/n) [n]: n
Enable filename encryption (y/n) [n]:  
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_key_bytes=32
  ecryptfs_cipher=aes
  ecryptfs_sig=9b8bcea9e0a5ce7c
WARNING: Based on the contents of [/root/.ecryptfs/sig-cache.txt],
it looks like you have never mounted with this key  
before. This could mean that you have typed your  
passphrase wrong.
 
Would you like to proceed with the mount (yes/no)? : y
Would you like to proceed with the mount (yes/no)? : yes
Would you like to append sig [9b8bcea9e0a5ce71] to
[/root/.ecryptfs/sig-cache.txt]  
in order to avoid this warning in the future (yes/no)? : yes
Successfully appended new sig to user sig cache file
Mounted eCryptfs

Now if the mount command is execute we should the directory we executed against mounted as a ecrypt file system.

Further Reading:

The home of eCryptfs

leave a comment

December 14th, 2013 at 4:15 pm

Posted in Linux

Tagged with , ,

Installing SVN support in Aptana Studio 3

Its been a few years since I have worked anywhere that still uses Subversion as their day to day version control system. But the other day at work I needed to check out some old legacy code which was stored in a svn repository on an old development server. Then it struck me I didn’t even have subversion available on my work machine!

I could have simply installed it from the command line but I use to like have the ability to compare code across multiple revisions directly within my IDE. Here I am going to cover off the required steps to getting support for Subversion installed into Aptana Studio 3.
Read the rest of this entry »

leave a comment

September 5th, 2013 at 10:19 pm

Posted in Programming

Tagged with , ,

Debugging Web Applications With Fiddler On Linux

Fiddler is by far one of the most popular web debugging proxies available. It allows you to capture, examine, save, edit and replay all HTTP / HTTPS  requests made by a web based application. For many years it was available only on Windows systems. But recently the creator of Fiddler, Eric Lawrence has released a version of Fiddler that runs using the mono runtime allowing it to run successfully on MacOS and Linux platforms.

In this post I am going to run through the basic setup and usage of fiddler under Linux (I am using Linux Mint 15 the process should not vary too for other Debian based variants though).

First up visit the fiddler site and download the alpha mono version of the application. After downloading unzip the package and copy the mono version of fiddler to the directory where you would like it to live on your system (I am simply using a folder named Software in my home directory).

Read the rest of this entry »

leave a comment

August 28th, 2013 at 11:04 pm

Posted in Linux,web development

Tagged with , , ,

Adding MP3 Support To K3b

Being sick of the music selection in my car the other day I decided I needed for fresh blood. But not being motivated enough to replace the car stereo with one that has an auxiliary input. I tried to simply burn a few mp3 files onto a blank CD. Dragging the files into K3b instead resulted in the error below:

k3b erorr

Read the rest of this entry »

leave a comment

August 22nd, 2013 at 11:20 am

Posted in Linux

Tagged with ,

Installing Safari on Linux Mint 15

safari logoCross browser testing your sites is unfortunately a necessary evil. Installing virtual machines with the required browsers installed is one way to test your code. But a lot of the time I don’t want to wait for a VM to boot just to quickly test something.

In this article I will run you through the process of getting the Windows version of Safari running under Wine on Linux Mint 15. Which will hopefully save some time waiting for virtual machines to boot up. To get started simply open a terminal and execute:

sudo apt-get update
sudo apt-get install playonlinux
Read the rest of this entry »

leave a comment

August 13th, 2013 at 10:05 am

Tracing Email Sending PHP Scripts On Servers

PHP LogoIt happens sometimes, a server you are responsible for seems to be sending out spammy emails, and its normally caused by legacy or insecure code. The mail log indicates there are plenty of potential spam messages going out that are originating locally but most Linux servers host a lot of sites. So the question then becomes, how do I find the script thats responsible for sending all of the emails?
Read the rest of this entry »

leave a comment

July 9th, 2013 at 6:25 pm

Posted in Linux,PHP

Tagged with , , ,

A Quick Look At Animate.css

css3I was browsing github today when I came across the very interesting Animate.css repository. Animate.css provides a bunch of cool cross browser CSS3 based  animations under an MIT licence that you can use in your own projects with minimal effort.

First things first you need to get a copy and depending how you want to use it, you can either build a custom package here, that only contains the effects you are after (hover over the text titles to see each effect in action!). Or if size isn’t an issue and you just want to play around simply clone a copy of the complete code base from github repository. From here its pretty easy to get started, just include the Animate.css style sheet into your page and then add the desired classes to your elements as you require.

Below I have created a basic html document to demonstrate a provided “hinge” effect on an element containing some text. The Animate.css style sheet has simply been added in the head of the document. I am also loading jQuery so that when a visitor clicks the hingeElement the animate and hinge classes are then dynamically added to the element:

Read the rest of this entry »

leave a comment

July 3rd, 2013 at 7:00 pm

Posted in CSS,web development

Tagged with ,