Archive for the ‘Linux’ Category

Quick Recursive Code Count

Although the “lines of code” in a project is not the worlds most useful metric. Curiosity sometimes gets the better of us and after many days / weeks of typing you think “gee I wonder exactly how many lines of code are in that project?”.

Wonder no more! By embracing the power of the command line you can easily count the lines in every file recursively. Simply execute the following line in the directory of your project (changing .php to the file extension of the files you would like to count the line of):

find . -name ‘*.php’ | xargs wc -l

Read the rest of this entry »

leave a comment

May 4th, 2012 at 3:18 pm

Posted in Linux,Programming

Tagged with ,

Replace The Default Firefox Google Search In Linux Mint

The recent release of Ubuntu’s Natty Narwhal forced me to reassess my usage of Ubuntu on my work and home machines. As I found the Unity interface drives me crazy and the release on a whole to be very buggy. So I decided to give Linux Mint a go, it has grown in popularity (#2 according to Distrowatch)  over the last few years so I figured they must be doing something right and Linux Mint 11 still uses no frills Gnome 2.32.

The migration went smooth, but by default the Linux Mint Firefox Google search functionality is useless, but luckily it wasn’t too much effort to fix.
Read the rest of this entry »

2 comments

August 11th, 2011 at 9:56 pm

Posted in Linux

Tagged with , ,

Anonymizing CURL Scripts With TOR & Polipo

Last week I received a typical run of the mill phishing email littered with the usual bad Engrish making the instructions far from believable but being bored I decided to take a look at the link they were pushing.

They had created a believable enough looking Paypal form asking clueless punters to enter their credit card details in order to avoid having their account being suspended. After playing around with the form for awhile in Firebug I decided to create a quick script to poison their data by submitting a new fake 16 digit VISA number along with random card holder name, CVV number and expiry date every couple of seconds.

After running this script for half an hour I decided that the script while it did the job would be a lot better if the posted data was coming from an anonymous IP address. Here I will run you through the process of setting up TOR & Polipo on a machine to add an element of anonymity to the data being sent and received by your CURL based script using.
Read the rest of this entry »

leave a comment

August 10th, 2011 at 8:45 pm

Posted in Linux,PHP

Tagged with , , ,

Near Realtime File Replication With Lsyncd

Ever wanted real time backups of  directories on your server? To replicate static media files for a website to a separate box to reduce load? An easy solution lays with lsyncd it allows you to watch a directory structure on your file system, and replicate any changes on a remote system.

How It Works:
inotify is a Linux subsystem available from the 2.6.13 release of the Linux kernel that monitors changes made to a file system and reports these changes to interested applications. lsyncd is an application written in lua that uses the inotify service to inform it of changes made to monitored directories and then when notified of a change uses rsync to replicate the changes on a remote service.

Getting lsyncd
The lsyncd sourcecode can be downloaded from Google Code and compiled but depending on the distribution your server is running, the install process be even easier as lsyncd is included in the repositories of many popular Linux distros.

Installing On Debian / Ubuntu
While lsyncd is included in both the repositories of Debian and Ubuntu but unfortunately its an outdated version so check what version you are going to get beforehand with the command:

apt-cache show lsyncd

If apt-cache tells you that only one of the 1.x versions are available, I would recommend grabbing one of the 2.x versions from the Debian testing repository instead, the amd64 package can be found here  and one for the i386 architecture here. Then install using the dpkg command i.e

cd /root
wget wget http://ftp.au.debian.org/debian/pool/main/l/lsyncd/lsyncd_2.0.4-1_amd64.deb
apt-get install lua5.1 rsync
dpkg -i lsyncd_2.0.4-1_amd64.deb

Read the rest of this entry »

2 comments

July 21st, 2011 at 6:28 pm

Posted in Linux

Tagged with , , , , , ,

Tracking Bandwidth Usage With vnStat

Outside of internal networks bandwidth is paid often for by the gigabyte and as such it becomes important to be able to see the bandwidth consumption of machines and plan future quotas accordingly as usage grows. There are plenty of packages available that fill this void like darkstat, ntop etc, but they tend to be overkill in a lot of situations were you don’t need pretty graphs you just want to know how much data is entering and leaving an interface.  Enter vnStat a simple console based tool that allows you to monitor traffic flow to and from interfaces on your servers.
Read the rest of this entry »

leave a comment

July 5th, 2011 at 4:06 pm

Posted in Linux

Tagged with , ,

Ubuntu Natty Narwhal Invisible Scroll Bars In Eclipse

Since its release on April 28th Natty Narwhal (11.04) seems to have gained it fair share of haters. The Unity interface is a huge change to the UI and in most peoples experience its buggy and doesn’t seem ready for the prime time.

No issue I thought after I upgraded I will just use the “Ubuntu Classic” option at login until the new Unity interface matures, unfortunately whilst coding in Eclipse earlier today I noticed using the classic interface or not some issues are still there. I was looking at a wide piece of code in Eclipse and started to become alarmed, what the hell has happened to my editors scroll bars?

I didn’t really pick up on the missing vertical scroll straight away as the scroll wheel on the mouse still got me where I was headed without issue, but I was left without any easy way to browse to the far right if the code was wider than the display area.

No Horizontal Scrollbar in Eclipse

I am told the scroll bars are supposed to appear when the mouse hovers in their vicintiy although this doesn’t seem to happen when using Eclipse at present. Luckily it seems pretty easy to fix, I just wrote a 3 line bash file that gives me back the classic scroll bars I have come to love.

#!/bin/sh
LIBOVERLAY_SCROLLBAR=0 eclipse
/usr/bin/eclipse

Simply saved the file made it executable with:

chmod +x start_eclipse.sh

Updated my menu and desktop shortcuts to point to my script rather than directly to the Eclipse executable, restarted the application and thankfully problem solved!

3 comments

May 4th, 2011 at 10:44 pm

Posted in Linux

Tagged with , , , ,

Easily Rip Youtube Videos On Ubuntu Maverick

Downloading youtube videos to your local machine for offline veiwing is easily on Ubuntu 10.10 with the youtube-dl utility.

Simply open a terminal window and execute:

apt-get update
apt-get install youtube-dl

Read the rest of this entry »

one comment

March 20th, 2011 at 11:50 am

Posted in Linux

Tagged with , ,

Disabling CPU Scaling With Ubuntu 10.04

If your like me it annoys you that your desktop keeps messing with your CPU speeds when all you wan’t is performance. Fortunately its easy enough to fix.

Open a terminal windows and execute:

sudo apt-get update

sudo apt-get install rcconf

(Chances are you may already have the rcconf installed.)

sudo rcconf

Scroll down the list of services till you find the service labelled “Ondemand”, this is the service that controls the CPU scaling. Unselect the option and hit Ok to exit, now when you reboot your system should run at full speed all the time.

leave a comment

May 6th, 2010 at 4:19 pm

Posted in Linux

Tagged with , ,

Ubuntu Lucid Lynx A Quick Look

The newest Long Term Support (LTS) version of the Ubuntu distro labelled Lucid Lynx was released earlier today. This release is packed with new packages and enhancements, making me keen to try it out. Not being brave enough to gamble with my current desktop machine running Karmic Koala I decided to download the .iso and play with it in a virtual machine first to give me a feel of what to expect.

Read the rest of this entry »

leave a comment

April 30th, 2010 at 6:43 pm

Posted in Linux

Tagged with , ,

Setting Up PowerDNS With A MySQL Backend On Ubuntu 9.10

Bind9 has treated us well over the years, but its lack of support for database back ends in a default install is very annoying, especially when an application needs to control DNS services dynamically. PowerDNS is a powerful alternative it supports a wide variety of back ends and can also be configured to serve different records dependant on the clients Geographical location making it a valuable asset in these coming days of private clouds gaining traction.

This article will walk you through the process of setting up a PowerDNS server with a replicated MySQL back end on two Ubuntu 9.10 systems.

Step 1 – Configuring the Master Nameserver

This nameserver is going to be your primary server, any changes that need to be made to the DNS records are made on this server and they will be replicated on the the slave. To get started log into your primary name server via SSH and execute:


apt-get update
apt-get install apt-get install pdns-server pdns-backend-mysql mysql-server

When prompted by the installer create a password for the MySQL root user and then configure the MySQL service itself:

pico /etc/mysql/my.cnf

Go down to line 53, it should look like the line below comment it out.

bind-address = 127.0.0.1 <- becomes -> # bind-address = 127.0.0.1

This is needed so the slave server can talk to the master and be aware of any changes made to its records. Save the file, then restart the MySQL service to make your changes active.

/etc/init.d/mysql restart

Okay now its time to sort out the database side of things by creating a MySQL user for PowerDNS to use and the database for the storage of its DNS records:

mysql -u root -p

Whilst in the MySQL client execute the following commands:

CREATE DATABASE dns_server;

GRANT ALL ON dns_server.* TO ‘pdns’@'localhost’ IDENTIFIED BY ‘pdns_pass’;

NOTE: Be sure to change the password for the pdns user to something a tad stronger.

GRANT ALL ON dns_server.* TO ‘pdns’@'localhost.localdomain’ IDENTIFIED BY ‘pdns_pass’;

FLUSH PRIVILEGES;

USE dns_server;

CREATE TABLE domains (
id INT auto_increment,
name VARCHAR(255) NOT NULL,
master VARCHAR(128) DEFAULT NULL,
last_check INT DEFAULT NULL,
type VARCHAR(6) NOT NULL,
notified_serial INT DEFAULT NULL,
account VARCHAR(40) DEFAULT NULL,
primary key (id)
);

CREATE UNIQUE INDEX name_index ON domains(name);

CREATE TABLE records (
id INT auto_increment,
domain_id INT DEFAULT NULL,
name VARCHAR(255) DEFAULT NULL,
type VARCHAR(6) DEFAULT NULL,
content VARCHAR(255) DEFAULT NULL,
ttl INT DEFAULT NULL,
prio INT DEFAULT NULL,
change_date INT DEFAULT NULL,
primary key(id)
);

CREATE INDEX rec_name_index ON records(name);
CREATE INDEX nametype_index ON records(name,type);
CREATE INDEX domain_id ON records(domain_id);

CREATE TABLE supermasters (
ip VARCHAR(25) NOT NULL,
nameserver VARCHAR(255) NOT NULL,
account VARCHAR(40) DEFAULT NULL
);

Then exit the MySQL client with the command:

exit;

Alternatively if you don’t feel that comfortable using the command line MySQL client you could install phpmyadmin and use that to create the database and user within web based environment.

Now that the database has been setup we need to configure PowerDNS to use our newly created MySQL database and user for any lookups it needs to make.

pico /etc/powerdns/pdns.conf

Go down to line 82 or thereabouts and find the line:

# launch=

Uncomment this line and modify so that it reads:

launch=gmysql

Edit the file /etc/powerdns/pdns.d/pdns.local

:

pico  /etc/powerdns/pdns.d/pdns.local

Add the following lines so that PowerDNS can connect with the MySQL service.

gmysql-host=127.0.0.1
gmysql-user=pdns
gmysql-password=pdns_pass
gmysql-dbname=dns_server

Save the file and its show time, just restart the PowerDNS service for your changes to take effect and the DNS side of your server should be good to go!

/etc/init.d/pdns restart

Step 2 – Setting up PowerAdmin

Okay its working but not very useful at the present, as it has no DNS records to serve. So lets install PowerAdmin so we can manage our DNS records using a web based interface. I am assuming the server you are working on already has Apache2, PHP and its modules installed already. If this is not the case you will need to install these before continuing as they are required for the operation PowerAdmin.

PowerAdmin also requires the mdb2 module to work properly, which i doubt many people will have installed already. So:

apt-get install php-mdb2-driver-mysql

Or on some older versions of Ubuntu it wont be in the repository so make sure you have PEAR installed and

apt-get install php-pear
pear install MDB2-2.5.0b2
pear install MDB2_Driver_mysql-1.5.0b2

Then continue the poweradmin install with:

cd /tmp
wget https://www.poweradmin.org/download/poweradmin-2.1.3.tgz
tar zxvf poweradmin-2.1.3.tgz
mv poweradmin-2.1.3 /var/www/poweradmin
cd /var/www/poweradmin

Now the code base is in place open your browser and point it at the Poweradmin location to continue the install process e.g http://ns1.mydomain.com/poweradmin/install

Follow the install instructions and when you reach step 6 of the install process copy the PHP code displayed on screen, go back to your terminal window create the file:

pico /var/www/poweradmin/inc/config.inc.php

Then paste the code from the browser window into the new file and save.

NOTE: The browser gives you the code with white space between some of the lines make sure these have been removed before saving the file. Otherwise after logging in you will be greeted by a “Warning: Cannot modify header information” PHP error.

Step 7 is the last step of the install process and simply asks you to delete the install directory. If you fail to complete this step, the program will not let you login!

rm -rf /var/www/poweradmin/install

The operation of PowerAdmin is pretty straight forward although if you have only used Bind in the past you may find some of the terms a little confusing. A quick look at the PowerDNS documentation should point you in the right direction though.

Step 3 – Replicating Changes on the Slave Server

Log in to the server you will be using as your MySQL slave. Update the repositories and then install the MySQL server application:

apt-get update
apt-get install mysql-server

On your master server edit the /etc/mysql/my.cnf file go down to around line 92. Uncomment and modify the lines that regard replication so they resemble:

server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
binlog_do_db = dns_server

Save your changes to the file and then open up the MySQL client to create a new user for the replication process:

CREATE USER ‘replication_user’@’192.168.0.2′ IDENTIFIED BY ‘mypassword’;

GRANT REPLICATION SLAVE ON * . * TO ‘replication_user’@’192.168.0.2′ IDENTIFIED BY ‘mypassword’;

NOTE: replace the IP address ‘192′.168.0.2‘ with the IP of your slave server, and change the password to something a bit stronger than ‘mypassword

Now edit the /etc/mysql/my.cnf file of your slave server and append the following lines after the [mysqld] declaration so that the slave can connect to its master:

server-id=2
master-host = 192.168.0.1
master-user = replication_user
master-password = mypassword
master-port = 3306

Be sure to change the ip address of the master server and the users password to match your setup. Save the file and restart the MySQL service on both the Master and Slave nodes so that your changes will take effect.

On the slave now use the mysql command line tool and execute the following commands:

start slave;
show slave status\G;

You you should see something like:


*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.0.1
Master_User: replication_user
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysqld-bin.000001
Read_Master_Log_Pos: 98
Relay_Log_File: mysqld-relay-bin.000002
Relay_Log_Pos: 236
Relay_Master_Log_File: mysqld-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 98
Relay_Log_Space: 236
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
1 row in set (0.00 sec)

Now test it by creating a record on the master and with any luck you will also be able to see it on the slave. That said i stuffed around a bit when setting mine up and the slave service wouldn’t start as it was looking for the wrong bin file on the master. I was able to follow these instructions though to point the slave in the right direction to get it working again.

one comment

December 1st, 2009 at 12:21 pm

Posted in Linux,Sysadmin

Tagged with , , ,