Wednesday, August 20, 2008

LAMP

I had been looking at setting up a LAMP development environment on Ubuntu as i use Wampserver on windows. Setting up lamp on the desktop version turned out to be more complicated than i imagined as there is no 'lamp' package. The server version has lamp running but getting the GUI installed was also more complicated than anticipated.

I did get xampp running on the Ubuntu desktop but then decided to rethink this a bit. I'm teaching a course in the winter and want to use LAMP but the IT department has made it difficult, so i've been researching wamp stacks that can run without being installed. Although the number of mac and linux students has gone up most students are using windows and the lab computers run only windows.

The other problem is i need to do some maintenance on an old application written for php4 and mysql4 that does not run on php5 so i need a lamp stack that can be 'downgraded' (sort of like vista). Xampp does this but the windows version has an installer. I do like the stuff it bundles such as webalizer but the security settings on the computers will not allow students to install. Hmnn - the Zip version does not seem to require an install - you just unzip and run a batch file. Let's try this out.

We go to the XAMPP home page at :
http://www.apachefriends.org/en/xampp-windows.html#641
and find the windows zip download which is about 88 mb.

Basically it looks like you unzip the files to a folder of your choice. This means it can be on an external drive, internal hd or a usb stick. If this works then our windows and linux development environments would be similar.

Once downloaded we extract the XAMPP folder from the archive to our E:\ drive which happens to be an external HD. We also copy the XAMPP folder to a USB stick.

The next step is to change to the XAMPP folder and run the setup_xampp.bat script which updates the configuration files with the path of the current location.

For the external hd that would be e:\xampp - the question is what about the USB stick? Since windows uses drive letters as mount points and assigns them based on the hardware in the machine the USB stick is mounted as drive G: in my configuration but will be drive E: or F: depending on which computer lab the students are in. We will have to see if this causes a problem - they may have to run setup_xampp.bat for each location change.

To test we will run the setup on G: and then move the stick to another machine and test if out there.

For now we open a terminal session by going to Start,Run and typing:

cmd
cd \xampp
dir *.bat

and we see the setup_xampp.bat file so we type the filename to run it. It generates a couple of messages and we are done.

Testing it Out

Back in the windows GUI we first run the port checking utility called xampp-portcheck.exe which shows if your port are free and we see the http ports 80,81 and 443 are free along with the mysql 3306 and ftp at 21 and the email ports of 25,110 and 143.

Next we run the xampp-control.exe utility which is the XAMPP Control Panel. This lists the current directory as c:\xampp but warns that there is a directory mismatch with the install directory. Not sure - will investigate later.

We click START to get Apache running. Green text indicates it is up. Next we START mysql. We do not start the filezilla FTP server or Mercury email.

Runing Firefox we goto http://localhost first and get the XAMPP splash screen. We click on English and the home page is displayed.

The Status menu item shows what is running and the standard phpinfo() entry shows that PHP version 5.2.6 is running. We have to figure out how to 'downgrade' to version 4.

One of the neat things about XAMPP vs other standalone stacks is that it ships with a number of demo applications such as a CD Collection app which can output the record list as a PDF document.

The next step is to bring over the files from our application which raises the question of where it should go. Looks like all content goes into XAMPP\htdocs so we will copy our folder to there.

We then stop and restart Apache just in clase, although we probably could have just entered the http://localhost/folder where folder is the one we just copied over. We run update.php and the application is displayed - of course most of the objects don't work as the database tables are missing. So our next step is to export the mysql tables and data from the old app and re-create it here. This gives us an idea. We are big fans of Navicat (http://www.navicat.com/) as a gui tool for mysql so we fire it up and create a connection using localhost as the IP address with a username of root and no password as this is the default insecure XAMPP setup.

The connect runs and we are able to see the databasees and tables. We create a new database as the target for our sql data transfer. One of the neat things about mysql and other sql databases is that you can replicate them with ease.

First we open a VPN connection to the server. This is not required by mysql but our IT folks don't allow any connection from the big, bad internet. Now we open a mysql connection to the mysql server and start to transfer some databases.

In Navicat this is as easy a creating a blank target database on your machine and then selecting Tools, Data Transfer from the menu and choosing what tables to bring over.

Unfortunately the data transfers fail. Why? We have done this a hundred times but not with the new VPN. Let's try another server that is not behind a security blanket. Of course it works like a charm. Is the server the problem or is it that fact the one server is running the latest version of mysql (5.x) while the other is running 4.x? and i am running an older version of the navicat tool? WE could test this out by downing a trial version of the latest client and trying a data transfer.

We do see a timeout on the server behind the firewall - we get a cannot connect message (nothing to do with a torrent of traffic...:) - in fact our transfers of the wordpress and wiki databases go really quickly so it is not a bandwidth issue. Error message is

"2003 - can't connect to mysql server on host (10060)"

We goto the navicat home page and download a 30 day trial of the new client. And that solves the problem - the move to mysql 5 breaks some older sql tools. We were able to transfer the databases with ease, even the large AOL queries database.

The server is 5.1.22 while the working one is verions 4.0.20. At least we were able to get the tables and data transferred over to the laptop. Now we have to set up the user for the database so the pages can connect.

We then use the updated client to data transfer the databases to the older machine and we are ready to get to work.

Ok we added the username for this database along with a password and restricted it to localhost and tested the app and it can read the database. So now it's maintenance time!

Just discovered something neat about WAMPserver - you can select PHP or MYSQL while it is running and choose Version, Get More and download older versions such as php4 or mysql4 and then choose then when you start up the service. Of course, you also have to 'downgrade' apache as the latest version does not work with php4.

Excellent! I can do the changes to the appz without having to figure out where they need to be updated for 5!

As usual it is a little more complicated than that as the versions i selected did not work together so i decided to replicate the existing environment as closely as possible using:

Apache 2.0.49
PHP 4.1.2
mySQL 4.1.20

Ok and we run wampserver again and the services start this time but the page gives errors - probably the modifications i was doing. Let's undo and see.

Nope - its the change in mysql - all the databases are not there in the other version. So we either have to replicate them between versions or....let's restore ver 5 and see what happens.

Cool - we closed the sql connection to localhost and changed to ver 5 in the wamp control panel and re-opened the mysql connection and all the databases are there. This lets us run a quick test.... the login page returns database errors:

Warning: mysql_connect() [function.mysql-connect]: Client does not support authentication protocol requested by server; consider upgrading MySQL client in C:\wamp\www\perl\login.php on line 17

Warning: mysql_select_db() [function.mysql-select-db]: Access denied for user 'ODBC'@'localhost' (using password: NO) in C:\wamp\www\perl\login.php on line 21

Warning: mysql_select_db() [function.mysql-select-db]: A link to the server could not be established in C:\wamp\www\perl\login.php on line 21

So lets try another test by changing the php version back to 5 and reload the page. Now it loads the page correctly but the data is not read and displayed from the database. However, that was with mysql 5 and php 5 so now we test by backing off mysql to ver 4.1.22 which results in the same errors.

So we need to change mysql version to 4.1.20 and those error messages go away. So let's replicate the database.

The problem was the Navicat tool we purchased back in 2004 cannot do a data transfer between a mysql 5 database server and a mysql 4 database server. The solution was to goto to the mysql 5 server and to dump each table to an SQL file over the network to the mysql 4 server and then to execute the .sql file to recreate the tables and load the data. This is basically the same method you can use to do a backup or a data transfer. SQL is so flexible and since the files are ASCII text you can even edit them or send them by email or whatever. It looks like this:

/*
MySQL Backup
Source Host: localhost
Source Server Version: 5.0.51b-community
Source Database: perl
Date: 2008/08/21 14:15:22
*/

use perl;
SET FOREIGN_KEY_CHECKS=0;
#----------------------------
# Table structure for authors
#----------------------------
CREATE TABLE `authors` (
`Author_ID` int(11) NOT NULL auto_increment,
`Fname` varchar(20) NOT NULL default '',
`Lname` varchar(40) NOT NULL default '',
PRIMARY KEY (`Author_ID`)
) ENGINE=MyISAM AUTO_INCREMENT=20 DEFAULT CHARSET=latin1;
#----------------------------
# Records for table authors
#----------------------------

insert into authors values
...

Ok - let's see how it works - no joy here. No errors - we connect to the database ok but the data is not read and displayed from the table. Why?

We decide this is the long way around so we find the old windows server which used to run things and plug it in. RODAN comes up fine for an ancient dual PII/512mb/scsi box. The old username and password gets us into the database but of course the version is too old for a direct data transfer from Navicat. We try to run the batch .sql files we used earlier but they also contain newer sql so we edit them to delete the charset statement. We then run the sql files and to create the database tables and insert the records. We now have the database and we create the user and copy over the files.

Now to test - we open Firefox and goto Rodan and run the application and viola! - finally some joy - the application gets it data from the database and displays it. WE figured it would work as this is the machine we developed the application on originally. The working versions are:

Apache 2.0.52
mySQL 4.0.18
PHP 4.3.9

These are not far off what we were trying to do on the desktop with Apache 2.0.49, PHP 4.1.2 and mySQL 4.1.20 so why the database does not work there remains a mystery.

At least a matching dev environment is up and running so the maintenance can begin...but after lunch.

Monday, August 4, 2008

Backup II

We had looked at backup options some months ago but nothing really seemed to fit. What we wanted was something similar to time machine for OSX - a backup that would not lose anything and would run to an external drive that could be restored in case of hard drive failure. We also have a backup motherboard and backup cpu's in case of real disaster. What we don't want to lose is the countless hours getting our system just right.

This has always been a problem on windows machines - you just get the machine set up right but when you upgrade or buy a new machine you have to spend so much time reconfiguring and re-installing.

Well last week we read about a program called flyback that seems to do the same things as time machine. The project web site is at: http://flyback-project.org/

It is a google code project and there is more info at http://code.google.com/p/flyback/ including some install instructions. It is not in the ubuntu repositories so you have to get it. Off to the terminal we go.

$ sudo apt-get install python python-glade2 python-gnome2 python-sqlite3 python-gconf rsync

which works. We already had most of the packages installed except for python sqlite. Now we do:

wget http://flyback.googlecode.com/files/flyback_0.4.0.tar.gz

Resolving flyback.googlecode.com... 209.85.173.82
Connecting to flyback.googlecode.com|209.85.173.82|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25,616 (25K) [application/x-gzip]

100%[====================================>] 25,616 135.79K/s

14:54:03 (135.06 KB/s) - `flyback_0.4.0.tar.gz' saved [25616/25616]

Now we do:

tar -zxvf flyback_0.4.0.tar.gz

to extact the compressed files from the archive. It extracts the following:

flyback/
flyback/flyback.py
flyback/backup_backend.py
flyback/viewer.glade
flyback/GPL.txt
flyback/config_backend.py
flyback/opengl_viewer.py
flyback/desktop.py
flyback/help_data.py

Now we change to the flyback directory and fire it up as a regular user (not as root) with:

python flyback.py

and the result is an error when the GUI runs:

The folder contents could not be displayed
error accessing 'file:///external_storage_location': File not found

Ok so now we do some digging. BTE it would be useful if the gui installed a menu entry under system. Looks like the error is not a problem you just have to change the default configuration under preferences. It suggests an external drive so we pull an 80gig usb drive out for a test run.

The drive mounts but it is a mac HFS+ partition that was used as a temp backup for the macbook when replacing the drive. So we should probably reformat this as an ext partition. On windows this can be done with partition magic. On linux the similar program is qparted which is installed. So we run it. The mac drive shows up as dev/sdc2

So we unmount the volume. We reformat the partition as ext3 and commit the changes. so we now an external vol called backup.

Returning to the flyback gui and in the Storage Location tab set the backup location to the external hard drive. Then in the Included/Excluded directories we select the filesystem which puts a / in the window pane. We could select our home folder if all we wanted was a document backup but we are interested in a complete backup. We leave the excludes along, assuming it is a reasonable setting.

Finally, in the schedule tab we tick the automatic box and click OK. In the flyback main window we click on backup to start the process. Backup is now running....


We'll come back later to see how it is going.

A good article by Falko shows how to do all this. See:
http://www.howtoforge.com/creating-snapshot-backups-with-flyback-ubuntu-7.10