Internet Broadcasting Phase II

Broadcasting Redux

Last term we had fiddled around with broadcasting from a USB stick using icecast and a hacked winamp but it was not entirely successful. So we decided to try a different tack. We purchased a G4 mac mini (1 gig ram/80 gig hd) and installed osx server 10.5 on it. The idea is to see how well the mac stands up as a broadcast platform. We can use our trusty dual G5 (2 gig, 1.5 terabyte) as the client that produces the content. It is hooked up to a sony miniDV via firewire and has iMovie, Garageband, Final Cut, Soundtrack etc installed. It is running 10.4 because of the need for the occasional classic environment session. Quicktime player will be used as the client and the scheduled should be available on a web page. We also want to experiment with iTunes as a client.

The basic idea is to:

- have users create either audio or video shows with their own content, or to weave together acquired content

- the shows would either be theme based or maybe even revolve around an 'on-air personality'

- the content would be edited and encoded for internet streaming and moved to a server

- the server would broadcast like a radio station or tv station

We are not so much interested in live events as in creating playlists of pre-recorded and encoded content, in much the same way as most radio and tv stations.

So let's begin the adventure.

Getting Started

Mini is up and running. So the first step is figuring out the quicktime streaming server (QTSS) and the quicktime broadcaster.

Quicktime Broadcaster is used for encoding a live source and delivering it to the server so that everyone sees the same point in the video at the same time. Actually, it does not have to be live, the same effect can be achieved by broadcasting content via playlists. This requires a faster broadband connection which is fine as the clients will be in the same building. For our development environment we will run quicktime broadcaster and quicktime pro on the dual G5.

The website should also have links to shows on the schedule and since these are basically self-contained video they can start playing as soon as the movie has the data it needs. This means the movie starts before it is fully downloaded and can be watched in a web browser. The movie is actually downloaded to the viewer's computer.  Quicktime 6 or higher recommended. This can be used for viewing at home. The web page will specify what content to send to the client.

We will run the quicktime streaming server on the mac mini and the clients will be the G5 iMac, an XP machine and a Ubuntu box.

Note: When we get to the IMC lab we will have to see if they quicktime broadcaster software on the clients, if not it can be downloaded for free from the apple web site.

Quicktime Streaming Server can operate in multicast or unicast mode. In multicast mode each client connects to the stream sort of like a FM radio tuning in a station. This is the more efficient model. Unicast requires each connection to have its own stream putting a heavy load on the network. Since the number of client connections is likely to be small, this may be ok.

Note: Find out if the network is multicast-enabled. It probably is as there is a windows media server on it.

Setup

Damn we downloaded the 10.4 documentation. Off to google to find the QTSS_and_Broadcasting_Admin_V10.5 pdf file. Goto page 25

1. Goto Applications, Server, Server Admin

2. Log in as administrator

A warning message appears:

The server gnickers is a standard configuration of Mac OS X Server, which is best administered using Server Preferences. You can use Server Admin to customize or add services, but your changes could have unintended effects. Before making changes, carefully note current settings in case you need to revert to them.

Alternatively, you can convert this server to an advanced configuration.

3. We select Convert to Advanced and a wizard appears noting that if you do this you will not be able to use server preferences to administer the server anymore but have to use Server Admin and Workgroup Manager. These were the tools in 10.3 and 10.4, is apple doing away with them in favor of a simplified preferences? Now it comes back to me - when i installed it asks what is the role of the server and i choose standard. This allows you to do some stuff and uses the server preferences as its control panel. However, the standard and workgroup roles do NOT allow you to run podcast producer or quicktime broadcasting which is why we are here! 

4. We select Convert and away it goes we are now in the Server Admin control panel

It is obvious the documentation is out of  date, neither the printed 10.4 or greater nor the 10.5 version we just downloaded has text that matches the screens. Ok - let's figure it out.

1. From the menu select Server, Add Service to display the list of services. We decide to turn off some things that are running like mail, ichat, ical, open directory, vpn and turn on Podcast Producer and QuickTime Streaming. We will leave mySQL off for now but later we may need it if we take a look at the open source Jinzora streaming software.

2. Click SAVE

3. Select the QuickTime Streaming service and click the General tab to edit the default

Max connections: change from 1000 to 50

Maximum throughput: change from 100 mbit/s to 50 (will see how it goes)

Media Directory: the default is /Library/QuickTimeStreaming/Movies

We will leave the default directory alone for now but wonder if it is shared so that clients can save directly to it. If not, which seems likely, we will change it later to a public folder that users place content in as part of their production workflow. The manual notes Make sure the owner of the new directory is the system user “qtss”. The owner can be changed from Workgroup Manager,

The other option is when we add the users they will have home directories and we may want to stream from there. The manual notes this can done in the Access tab by selecting 'Enable home directory streaming'. Users then put their hinted quicktime movies into the folder that was created in the their home directory (/username/Sites/Streaming). The downside of this approach is to stream a live broadcast from a directory other than the default media directory you must create a qtaccess file. These files are like the htaccess files used with the Apache web server. It contains information about users and groups who are authorized to view media in that directory.

The Access tab has a mp3 broadcast password box. We leave this empty

The IP Bindings tab has two options - we enable streaming on all IP addresses

4. After Saving the changes we click the Start Quicktime Streaming button.

Testing it Out

There are some sample movies included with QTSS in the default folder. You use a quicktime player to  view these movies. 

1. Went to the G5 mac and ran quicktime player

2. Select Preferences, Streaming, Speed and set it to LAN with Enable Instant On

3. Select File, Open URL and put in rtsp://gnickers.fims.uwo.ca/sample_300kbit.mov

4. Went to the XP machine and it played no problem there too

Can't for ubuntu as there is no quicktime player. We then copied over a bunch of old quicktime movies when Liam was 2 and tried to view them. We get 'unsupported media type' from the windows player and 'live broadcast paused' from the mac. Oh, these movies are probably not hinted so they will have to be converted to streaming format. Tomorrow.

Conclusion

We now have a streaming media server. However there are a number of things to improve.

• The use of the quicktime player with rtsp:// is clumsy and not suitable for end users so we need to set up  some basic web pages to link to content. These pages should really be dynamic so as to reflect new content files. Will start with test pages and then maybe brush up on php 
• Existing content files in quicktime format will have to be converted so we need to know which is the best or easiest way of doing this and it has to work with the software on the IMC lab client machines

To Do:

• Install quicktime broadcaster on Dual G5 - done!
• Figure out how it works and set up a workflow between G5 and the media server
• Test it out with live camera broadcase
• Look at getting a newer mac laptop

Xsane in the OSX brain

Using a Scanner with OSX

My brother is getting married in July so i thought i'd scan all the old photos and create a movie. There are 2 scanners - the Canon usb one is hooked to //godzilla, a Ubuntu 8.04 box and the other is a big HP scanjet scsi model hooked up to //rodan, a windows 2003 server that is due to retire sometime soon.

Rather than scan and move files around i decided to hook a scanner to //mothra, a dual G5 running osx 10.4 and pulled out the almost identical canon canonscan N650U which is a usv scanner. I once tried to hook it up to windows but it complained about drivers etc. so it has been sitting around collecting dust.

We plug it into the G5 and it does show up in the 'About this mac' but there is no way to operate it. So off to the canon site where there is some downloads, apparently a driver download and the canonscan software. We download and try to install. No joy - the canon software cannot find the canon scanner. Reading around the net we discover this situation is still very common with osx and scanners. What's the solution?

Well on the Ubuntu box we plugged in the canon scanner and it picked it up and we scanned stuff with Xsane. No install, no problem = it just works. Hey wasn't that supposed to be windows big thing? Well it worked in ubuntu and not in windows. So, can we use the open source SANE software on osx?

We check the list of supported sane devices at: http://www.sane-project.org/sane-supported-devices.html and the canoscan n650u is listed as 

Model	Interface	USB id	Status	Comment	Backend	Manpage

Cano Scan N650U N656U

USB

0x04a9/0x2206

Complete

NA

plustek  (0.52)

sane-plustek

We find osx binaries at http://www.ellert.se/twain-sane/ and download them. First we extract the pkg files from the zipped tarballs. We now have the following:

• gettext
  
• libusb
  
• sane-backends
  
• twain-sane interface
  
• sane-preference pane
  

And we install them in that order. We now check the system preferences and there is SANE in the other section. Clicking brings up a list of drivers for many models - we uncheck all but the canon and the plustek. We do not activate scanner sharing.

Off to applications to run image capture which says 'no image capture device connected'. We select Devices, Browse Devices, TWAIN devices from the menu. SANE is shown as an option so we select it. Do we tick the Use TWAIN software box? Ok we will tick it and see. Now we click CONNECT. Put a photo on the plate and click SCAN. Now we have some options. We select 14 bit color at 1200 dpi. Nice to see the CanoScan as the image source! We click SCAN but get an error during device I/O.

We should test this with a known working scanner. Off we go to get the other canon usb scanner. Ok the scanner we were using may have had a problem. Let's try the canon something - it doesn't seem to have a model number but is almost exactly the same.

Back to image capture and we try Devices, Sane, Overview to see if we can get a preview. The scanner makes a little noise and it takes a minute but then it starts scanning. But not scan.jpg is found in the pictures folder and no preview. Try again. The SCAN button is still grayed out.

Ok the scanner is in fact a canon 630U and now the SCAN button is there - i had to reboot to get everything working. I wasted another hour trying the canon software and driver which did not work. The SANE twain is a bit slow to make the scan - not its fault i think, probably due to the weird way twain is implemented in osx. 

Summary and Conclusions

In this test of installing a 'plug n play' USB device - namely a common scanner on Windows, OSX and Linux, the easiest OS was...Linux.  Ubuntu 8.04 picked up the scanner without installing drivers and i was able to scan immediately using XSANE. When plugged into windows the scanner/camera wizard could not find the scanner. I went to the canon site but and there was software there for XP but since i don't want to use windows anymore i decided not to download and install. I next tried osx and did download and install the manufacter's software which did not work. This is not to say it would never work - just that it was not an easy or obvious thing. Finally, i tried the open source SANE and it worked. Chalk one up for open source.

Linux Server

Installing SME Server

I used this some years ago for class when it was called e-smith and it was a very nice server setup that was administered from a web browser.

The test machine is a dell SFF gx270 with P4 2.6ghz, 1 gig DDR ram, 320 gig IDE hd. There is no cd so i opened the top and plugged in a cd using a long ide cable. The machine is very compact, about the size of a large 3 ring binder with 6 usb ports, low profile agp, pci etc and it has a SATA controller. We debated putting in one of the sata drives lying around but this old IDE drive should be fine. It also has an Nvidia 6600 256mb video card which is totally wasted in a server. We decided to leave the sata cable and the video card in the machine if we later need to use it as a ubuntu desktop for class.

The version we booted from cd is 7.4 and the main page for the distro is at http://wiki.contribs.org/Main_Page. This version is based on Centos.

The installation menu uses the old text config that will be familiar to anyone who grew up on red hat. You choose a language, a time zone and reformat the hard drive. It then starts installing the packages about 1.2 gig worth - 522 packages to be exact which takes about 4 minutes as the text installer flies. This gives us time to read the FAQ's.

Interesting, if you change the boot loader from grub to lilo you can run it on a mac mini (intel only). Unfortunately our mac mini is G4 so no joy there - however it is running osx server which is a very nice distro.

Done installing so we reboot. We will leave the CD connected for now in case we need it but we take out the cd-rom. The os starts up and mounts the ext3 fle system and then starts creating quota files which take a very long time. I also thought it was hung and was ready to swith to the other console.

Now some questions. We choose an administrator password. Next is the server name. Next we choose an IP address. This is a fixed one - not a dhcp. I assume we can change that later.

Lets give it 192.168.1.37 with a subnet mask of 255.255.255.0 and we choose it not to act as a router but to be just a server. I recall now that i used this as a router (server/gateway) many years ago when it first came out as it was less trouble than fiddling with red hat all the time. The machine was an old 486 with 2 network cards, one cat 3 for the cable modem and 1 coax for the 10 base T internal network. Since the server supported both samba and appletalk it provide a whopping 2 gig of storage for my coax network of a 586, a 486, a 386 for Doom fragfests and a mac.

Once the configuration questions are finished the settings are written and the machine finishes booting to the login prompt:

We login as root with our admin password. Success. Now from the machine we are writing this one we open a new tab in Firefox and goto http://192.168.1.37 and success - a web page with a message 'This web site under construction' is displayed. Ok we need the admin interface.

Best read the adminstration manual on the wiki. Damn - the wiki is down. I seem to remember that in the old e-smith version you did not log in as root but as admin. Let's reboot the system and see.

Success- logging in at the # prompt as admin with the administrator password brings up a text configuration menu. There is a new option - back up to a USB device. Let's try the server manager option. There is the info - use a web browser and goto /server-manager or use the text mode browser (lynx) from the console. Back to our machine here and firefox.

Succes - except that firefox complains the security certificate from sff is invalid so we have to create an exception. Once that is done we get a login box. We don't have any users so lets try logging in as admin. Success - the admin page is displayed.


Ok - let's play

First we change the windows workgroup name from mitel to workgroup. We set up a user and it will not allow a weak user password but it did allow us to create a weak root password! We will have to look into overidding the security policies tomorrow.

Pics of the new machine

Soul of the New Machine

Fast, fast, fast. Been busing installing lots of goodies, got to fill 1.5 terabytes somehow. Of course there is a 2nd SATA controller and space for another 1.5 t hard drive...

Found a neat application called aptoncd which allows you to copy all the appz you downloaded and installed to a optical disc so they can be installed on another machine without hogging all your bandwidth with downloads.

This pic shows Xara Extreme in one workspace (impressive program) and a movie running in another workspace. On the old 7.10 machine i could not play movies and run the compiz fusion effects at the same time. Looks like that has been fixed.

Only problem i have encountered so far is that when 2-3 cores get near max the variable fan at the front spins like crazy. From post by others it looks like this is a known problem with the Dell Precision 470. I'll have to look at replacing the stock fan.

This pic shows the other 2 workspaces, one is doing some 3D modelling and the other is playing some music. Best get back to work...more fun later.

New Ubuntu Baby

Daddy's got a New Computer

The original machine from last summer's ubuntu project was such a nice success (typing on it now) that we decided to upgrade to a better machine. There is nothing wrong with this computer, it runs 24 hours a day without a crash - in fact has never crashed, or hung or had a virus, or got malware or anything. It does everything from manage my ipod, run the color and bw postscript laser printers, burn movies and cd's etc.

The original pc was a cobbled together Compaq EVO W6000 bought for $64 from vfxweb.com (my favourite store). I upped the ram to 2 gig, added a DVD burner and a Nvidia 7600 GT 512 mb video card and a USB 2.0 card. The whole thing was < 200 bucks. It's now the main machine at chaos basement. The only problem is hard drive space - it has a 73 gig scsi hard drive for the os and the home folder a 18gig scsi scratch disk for temp junk and a USB plug in drive for backup - now 80 gigs and to be replaced tomorrow with a 250gig one.

A good machine but time to move on up. So we went back to vfxweb (my favorite store) and purchased 2 matching 3.20 Ghz hyperthreaded Xeon processors (1 mb L2, 800 mhz bus) and then off to ebay for a bare bones dell machine with 6 USB 2.0 ports, gigabit ethernet, 2 sata controllers (raid), room for 2 SATA hard disks and 2 IDE optical drives. I prefer compaq and HP to dell, they use metal and their machines just say 'quality'' while dell uses cheap plastic. But the price was right.

Added 3 gig of DDR2 ECC ram and a SATA 1.5 terabyte hard drive ($149 at tiger) and an LG DVD burner. Briefly considered putting in 1 gig sticks for a 6 gig machine but that would be excessive. Even my dual G5 mac only has 3 gig.

Next was an Nvidia 8600 GT 512mb pci express video card. Rummaging around the parts bin we added a 2 port firewire pci card and a 64 bit U320 scsi card to fill out the slots.

This project cost > 200 mainly due to the new parts. Probably $400 by the time we were though. Right now it is using a ps/2 kb and mouse and hooked to a 20" sony monitor via 5 coax to vga - later we will replace that with a larger monitor and usb kb/mouse.

Popped in the Ubuntu 8.10 CD and away we go! While we wrote this it has partitioned the hard disk and installed the files. Then it went off to the mirrors. While we let apt do its thing, one nice bit about 8.10 over earlier releases - the liveCD picked up the sony monitor a lot better. Although the monitor is capable of 1600x1200 rez at high refresh the older distros ran at 800x600 until you tweaked the configuration. This time it is running at a much higher screen resolution.

While the last bit is being done i was reading some reviews of windows 7 (really Vista SP2) and was thinking - why would i want this? Yes it has some better security (if stupid nagging is security) and a more modern mac/linux-like interface and.....ummm - can't think of anything it has that i don't have now. But it's user-friendly - or more accutately, user-annoying.

Switching from windows to ubuntu had some pain but now with a smooth running system there is no reason i can think of why i would want to return to windows world. The browser is worse, the system is annoying, it is insecure and it costs money. So far there has been nothing i did in windows that i have not been able to do in Ubuntu.

so why switch?

Joomla on XAMPP

Joomla Installation on a USB stick

These are some quick notes before i forget what i've done.

Goto:http://www.joomla.org/
Download the English 1.5.8 Full Package and save it
Double-click the downloaded Joomla_1.5.8-Stable-Full_Package.zip file
Extract the files to your USB stick in the xampp\htdocs folder into a folder called joomla

If the folder does not exist then create it (about 24mb of space is needed). When finished you should have a file \xampp\htdocs\joomla\index.php on your USB stick

Run the portableapps Xampp control panel and start the mysql and apache servers
Open a web browser and goto http://localhost
Under Tools click on phpMyAdmin
Select Databases
In the Create a database text box type joomla
Select utf8_unicode_ci from the pulldown menu
Click Create

Select the Privileges tab
Click Open new phpMyAdmin window
Click Add a New User
Select Use Text field in the User name: drop down menu
In the User name: text box type in joomla
Select Local in the Host: drop down menu
Select Use text Field in the Password drop down menu
Type joomla in the Password: text box
Type joomla in the Re-type: text box
Scroll down to the Database-specific priviledges
Select joomla From the Add Privileges on the Following database pulldown menu
Add tickmarks to all the boxes in the database specific privileges
Click GO
Close the extra phpmyadmin window
Goto http://localhost/joomla

It takes a minute to load the setup program and then the Joomla Installation page is displayed. The pre-installation check is run. All items should be in GREEN. The default XAMPP install has PHP Display Errors set to ON. This tells PHP to send any errors to the browser so you can see them. For a production site, this option should be off since an error message could give information about your system, a potential security risk. For development purposes, however, this option is useful. You have to edit php.ini and change the line with display_errors to On: (line 292?) to be:

display_errors = Off

Make sure to shutdown and then restart the servers when making this change. It may also be possible to override this on a directory basis with a .htaccess file using the directive:

php_value display_errors on

But i will have to test this sometime later. Anyway, we continue with the install as the error is not fatal.

Click NEXT
Read the license and click Next
The Database Configuration screen is shown
Select mySQL as the database type
hostname: localhost
username: joomla
password: joomla
database name: joomla
click NEXT

The FTP configuration page is shown. Since we will be copying files directly to the USB stick this is not needed.

Click NEXT
Enter your uwo username as the Site Name (ie gnickers)
Add your email address
Enter joomla as the admin password
Enter joomla as the Confirm admin password
Click INSTALL SAMPLE DATA
click NEXT

After setup is complete, you are prompted to delete the installation folder (for security reasons); in fact, Joomla itself won’t run until it detects the installation folder is gone.

Goto your USB stick and delete the xampp\htdocs\joomla\installation folder
Return to the web browser and click on the Site icon
If successful the Joomla home page is displayed
Scroll down to the LOGIN section and login
username admin
password joomla

And the joomla home page is displayed. Now we have to figure out how to create custom sites.

Joomla Administrator

We go to http://localhost/joomla/administrator and the login page appears. We enter admin as the username and joomla as the password and the administration screen appears. There are 3 types of administrative users:

• super admins - access to everything (the admin user we just logged in as)
  
• admins - can access the backend but cannot install extensions or modify super admin accounts
• managers - can access the backend but are limited to media, content, components and the main menu
  

Users are created from the control panel using Site, User Manager. Let's create a user with content priviledges as this will allow us to see how a regular user might use the system.

Select Site, User Manager
Click the New icon
Fill in the details for the user
Set the language to english
Set the user editor to TinyMCE
Set the time zone to Eastern Time
Select the Author, Editor or Publisher group - we decide to select Editor
Click the Apply icon

TinyMCE is a platform independent open source web based Javascript HTML WYSIWYG editor that can easily be integrated into content management systems.
See: http://tinymce.moxiecode.com/

The 3 groups of contributors have different priviledges.

• An author can contribute content to locations defined by the administrator and modify their account information
• An editor has all the priviledges of an author and is able to edit content written by an author
• A publisher has editor priviledges and is able to publish content to the site if you have restricted publishing to publishers only
  

Odd - when we hit apply it kicked up back to the login screen and when we logged back in and looked for the new gnickers user it was not there. Did the session time-out because we took too long to add the user?

We add it again and when we go to save we see the problem. It cannot save the gnickers user because we gave it our email address which is the same as the administrator. We either need to add a different email or set the user not to receive any email - which might be the better option.

I problably should have made the gnicker user a member of the super admin group and created an editor with a different name with no email.

New - solved problem with installing extensions

When i went to install an extension i got the following error message:

* JFolder::create: Infinite loop detected
* Warning! Failed to move file.

The problem is caused not being able to find the tmp and log folders which are hard-coded to a drive letter. The problem is when you move your USB stick from machine to machine the drive letter changes.

1. Select Site, Global Configurations
2. Click SERVER
3. I see the path to temp-folder is hard-coded to J:\xampp\htdocs\joomla\tmp
4. Click SYSTEM
5. I see the path to log folder is also hard-coded J:\xampp\htdocs\joomla\logs

I remove the J: from both places and click SAVE. Now when i load an extension the error message is gone. However, their is some debate on the forums as to the desirability of making other changes to configuration files.

The first extension i tried to install timed out when this error:

Fatal error: Maximum execution time of 60 seconds exceeded
in F:\xampp\htdocs\joomla\libraries\joomla\filesystem\folder.php on line 395

We open folder.php and see that the section being referenced in the one that reads the source directory:

$handle = opendir($path);
while (($file = readdir($handle)) !== false)
{
$dir = $path.DS.$file;
$isDir = is_dir($dir);
if (($file != '.') && ($file != '..')
&& (!in_array($file, $exclude))) {


The offending line looks to be the isDir function - if the directory name is not correct then the program is probably spinning it's wheels and then times out. We decide to change the line to be:

$isDir = is_dir($dir.'/' );

and save the file, exit Joomla and restart Joomla. Now we try to load the extension again.

1. Select Extensions, Install/unistall
2. Click Browse
3. Select com_joomlaexploter_1.6.3.zip
4. Click Upload File & Install

result - no joy so i went back to the file and discovered the same code segment so i changed line 457 and save the file.

Try again - same result - the spinner spins for a while and then the screen goes white. More research is needed.

We check the file structure on the USB stick. First the joomla global configuration settings:

xampp\htdocs\joomla\tmp

Folder exists and has files! In fact the .zip extension we tried to install is in this folder.
There are folders for each install appempt.
Each install attempt folder contains 2 folders, scripts and style
The .zip extenstion archive has 6 folders

xampp\htdocs\joomla\logs

Folder exists but only has a blank index.html file with an old creation date

Next we check the php configuration by going to Help, System Information, Php settings. We find:

upload_tmp_dir \xampp\tmp \xampp\tmp
open_basedir no value no value

Could this be the problem? We check and xampp\tmp - exists but it contains a bunch of files that were just created but with 0 bytes and are empty. Is there a conflict between the joomla global configuration and the php.ini settings?

We search php.ini for open_basedir and find:

; open_basedir, if set, limits all file operations to the defined directory
; and below. This directive makes most sense if used in a per-directory
; or per-virtualhost web server configuration file. This directive is
; *NOT* affected by whether Safe Mode is turned On or Off.
;open_basedir =

Looks like it is not set which confirms the global settings. Joomla has set this to:

\xampp\htdocs\joomla\tmp

But that path is inthe format of a local path. The php.ini setting would be a relative path for the web server i would think. Maybe this local path should be changed to a relative path?

The path of the web server root is \xampp\htdocs so the tmp folder should be \xampp\htdocs\tmp but there is no such folder. Perhaps \xampp\htdocs\joomla\tmp is correct but that is exactly what we have in the global configuration. I think the php basedir is more likely, can we do something like:

open_basedir = /xampp/htdocs/joomla/tmp

The change is made and we restart. Same result - no joy.
I must say the Joomla wiki entry on this VERY common problem is vague and little help.

Ghosting Ubuntu to a different Hard Drive

Cloning a Hard Disk

When we originally set up Godzilla, the dual Xeon Evo 6000 it had plenty of disk space, a 73 gig boot drive and an 18 gig drive for storing junk. However, over the year it has had vmware installed to do virtual machine, transmission for getting torrents and a raft of video conversion tools. The plus is the machine is very useful, the downside is that it is filling up. A temporary solution was to plug in an old external USB 80 gig drive. But now that the term is over we need a longer term solution.

The first goal of the project is to get more disk space without re-installing the OS. With a simple machine consisting of a single IDE drive this can be easily accomplished by cloning or 'ghosting' the smaller drive to the larger. We have used Norton Ghost to do this successfully for many years. Put in the new larger drive on the 2nd IDE controller, boot to a ghost floppy and start the clone. Remove the first drive and put the new drive on the primary controller and boot away.

However, the situation gets more complicated if you use multiple drive types such as IDE, SCSI and SATA. Windows has rules about the order in which drive letters are assigned- it is possible to clone a SCSI to an IDE or vice-versa but a system with multiple scsi and ide drives is more difficult. Linux has a similar problem - a simple clone from ide to ide can be done but what if your needs are more complex?

In our case the system has 2 SCSI drives and 1 external IDE mounted in a USB case. We want to clone to scsi boot disk to a old 320 gig IDE disk and maybe later clone that to a 1 terabyte SATA disk.

The first step is to find out exactly what the disk are labelled. We open a terminal window get the following:

gnickers@godzilla:~$ sudo fdisk -l
[sudo] password for gnickers:

Disk /dev/sda: 18.2 GB, 18209792000 bytes
255 heads, 63 sectors/track, 2213 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00006694

Device Boot Start End Blocks Id System
/dev/sda1 * 1 2213 17775891 b W95 FAT32

Disk /dev/sdb: 73.4 GB, 73407868928 bytes
255 heads, 63 sectors/track, 8924 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00045a17

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 8555 68718006 83 Linux
/dev/sdb2 8556 8924 2963992+ 5 Extended
/dev/sdb5 8556 8924 2963961 82 Linux swap / Solaris

WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'!
The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdc: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdc1 1 9730 78150743+ ee EFI GPT

We decide to use a couple of other methods to check our results.

gnickers@godzilla:~$ sudo cfdisk -P s /dev/sda
[sudo] password for gnickers:
Partition Table for /dev/sda

First Last
# Type Sector Sector Offset Length Filesystem Type (ID) Flag
1 Primary 0 35551844 63 35551845 W95 FAT32(0B) Boot

and

gnickers@godzilla:~$ sudo tune2fs -l /dev/sda1 | grep UUID
tune2fs: Bad magic number in super-block while trying to open /dev/sda1
Couldn't find valid filesystem superblock.

Which gives an error message. I'm thinking we could simplify matters by disconnecting the external USB disk and the 18 gig internal scsi drive. We probably should reboot the system anyway as it has been up all term and probably needs to fsck the filesystem anyway.

Reboot - nothing. It does not come up to sdb1. Check connections and ensure adaptec scsi controller is set to boot the hd with id = 1. Reboot. no joy. So let's put the 18gig scsi back in and reboot again. The GRUB boot loader appears but lists Error 21 which means it cannot find the disk. We go into the scsi bios and see the 18gig is ID = 0 but the 73 gig has disappeared. A loose power connector. Fix and restart. no boot. So i undo the only other changes and set the scsi bios to boot to id0 which is the 18 gig and it works.

We need to take a closer look at this.