Friday, October 22, 2010

Nintendo DS Redux






A long time ago i was playing with Liam's new DS and thought it was a clever little platform with a lot of potential. It has 802.11b wireless, a touch pad, sound etc in a compact package. A few months ago i picked up an R4 card for his DS but never got around to doing much with it.

Today i downloaded and installed Moonshell 2 to the card on a new 1 gig micro SD card. Tried an 8 gig card but it must be too big. Will have to try a 4 or 2 gig card because 1 gig is a bit small. The moonshell install can take about 30-60mb, depending on functions. See: http://www.ds-xtra.com/MoonShell2

Anyway, the goal was to make the DS into a nice travel machine with photos, music, movies and games - a poor man's iTouch. I had played with the Sony PSP for a while as that also has wireless and a nicer screen. The DS seems to have a more active developer community.

Once Moonshell was installed i ran the LanguageSelect.exe program to set the language to English. The program is written in Japanese and some of the translations are a bit rough but Liam was able to navigate ver 1 ok and version 2.10 is a real improvement. It also contains a bunch of utilities, such as a file converter for movies. Since most movies Liam watches are on Youtube (we don't have TV) I used http://keepvid.com/ to capture Youtube videos and downloaded them in .FLV format and to use the dpgenc DPG video converter included with Moonshell in the dpgtools folder. Now if netflix would write an nds app...

I plugged the microSD card into my pc and created movies, photos and music folders and copied over some files. The next step was to get the shell to run automatically when the DS is booted up - otherwise you have to select the NDS file.

When 'moonshl2.nds' is renamed into' DEFAULT.NDS ', MoonShell2 is started automatically but there is no audio when playing music or video files.

The moonshl2_AltLoader_DKPr27.nds is an alternative loader and renaming this to DEFAULT.NDS solved the problem. Now movies and music play - excellent!

To add applications see: http://dl.qj.net/nintendo-ds/homebrew-applications.html which lists a lot of NDS downloads. You can also get roms ripped from games if you want. Of more use is being able to make backup roms of the games you bought and being able to put a bunch of them on a single card for travel.

Nintendo actively discourages the homebrew developers by changing things - for example this won't work on his DS XL he got for xmas. Oddly enought, despite the bigger and brighter screens, he prefers the convenience of having multiple games so he uses the older DS much more! If Nintendo wanted to sell to the adult market they should open up the platform by:

- moving from 802.11 b to at least G
- creating an app store for downloading of apps, movies, music directly to the ds
- allow third party developers to write apps for the app store

When liam gets older he will probably give up the DS for an iphone/itouch. If the R4 card worked on the DS XL i would trade my netbook to Liam for it and use it at work. It writes memos, keeps track of appointments, takes photos, surf's the web and can play the odd game. There is even a ebook reader app for the DS. I would actually prefer the DS to my iTouch as a portable device because of the card slot and the stylus. My fingers are just too big to use the iTouch/iPhone keypad so it sits unused and unloved.

Photo 1 - DS Organizer - Calendar



Photo 2 - Music



Photo 3 - Photos




Photo 4 - Movies


Wednesday, October 13, 2010

Greenstone and Z39.50

The local windows install of Greenstone for Windows contains a Z39.50 client but the documentation on how to use it is out of date and there seems to be some problems in saving the file in shared computer labs. The following was tested on Greenstone 2.83 on Windows XP and describes how to do a title search for books on SQL against the Library of Congress z39.50 server database.

1. Click on the Download tab and select Z39.50
2. Enter the following parameters:

Host: lx2.loc.gov
Port: 210
Database: LCDB
Find: @attr 1=4 "SQL"

3. Click Download

You can set the Max Records if you want. A max of 500 records is returned for any query. The file of MARC records is saved to wherever folder was specified when Greenstone was installed. If installed on your own computer running windows XP this is most likely the C: drive in Documents and Settings\yourusername\Application Data\Greenstone\GLI\lx2.loc.gov with the filename set to the search string with an extension of .marc, in this example the file name was LCDB_@attr 1=4 SQL_500.marc.

When we ran this in the GU and GRC computer labs the file was not written even though the Greenstone log said it was. The problem seems to be that user application data is not saved locally but to a network share.

The file itself is a plain text ASCII file containing the 500 marc records. It looks like this:

Records: 500
[LCDB]Record type: USmarc

001 15430621
005 20080908175110.0
008 080827s2008 caua 001 0 eng d
906 $a 7 $b cbc $c copycat $d 2 $e ncip $f 20 $g y-gencatlg
925 0 $a acquire $b 2 shelf copies $x policy default
955 $a ps04 2008-08-27 z-processor 2 copies to ASCD $i jx09 2008-09-08 $e jx09 2008-09-08 c. 1-2 to BCCD
010 $a 2008297695
020 $a 9781590599693 (pbk.)
020 $a 1590599691 (pbk.)
035 $a (OCoLC)ocn179801564
040 $a BTCTA $c BTCTA $d BAKER $d YDXCP $d OCO $d CDX $d BWX $d OCLCQ $d DLC
042 $a lccopycat
082 04 $a 005.7565 $2 22
050 00 $a QA76.9.D3 $b A284 2008

For information on the Library of Congress Z39.50 server see:
http://www.loc.gov/z3950/lcserver.html and for information on the syntax of Z39.50 queries using Yaz see: http://www.indexdata.com/zebra/doc/querymodel-rpn.html

The next step is to import the MARC records into Greenstone.

  1. Click the Gather tab
  2. Expand the Local Filespace and drag LCDB_@attr 1=4 SQL_500.marc into the Collection window
  3. When asked click Add Plugin to add the MARCplug import program

MARCplugin uses a file called marctodc.txt located in the /gsdl/etc folder to map MARC field numbers to Dublin Core metadata based on (http://lcweb.loc.gov/marc/dccross.html). It is also possible to use the RFC 1807 Bibliographic records metadata set for the following exercise but we will use Dublin Core as that is the metadata scheme most commonly used for online digital collections. You could also use the qualified Dublin Core metadata set. To use the qualified dublin core or the RFC 1807 metadata set clikc on Enrich, Manage Metadata Sets, select a set and click Add. With those additional sets added to Greenstone you can choose them in the next sequence. However, you cannot have both Dubin Core 1.1 and the qualifed Dublin Core at the same time; you must choose one or the other.

The next step is to extract or 'explode' the individual MARC records from the file.

Select
LCDB_@attr 1=4 SQL_500.marc in the Collection Window and right-click
Select Explode Metadata Database from the menu
Place a tick mark in the metadata_set option
Select Dublin Core from the metadata_set pulldown menu and click Explode
Click the Enrich tab



Because Greenstone assigns metadata to files, each MARC record has been assigned to a file with a .nul extension (to indicate the files are really null). Select 00000006.nul to view the metadata.

The next step is to process these .nul files using the NULplugin. First we have to remove the MARCplugin so it does not try and process the records.

  1. Click the Design tab
  2. Select MARCplugin
  3. Click Remove Plugin
  4. Click Create
  5. Click Build Collection
  6. Click Preview Collection
The record index will look something like this:


View the document Beginning Microsoft SQL server 2008 administration / Chris Leiter ... [et al.].
(00000010.nul)
View the document Best damn Exchange, SQL and IIS book period / Conrad H. Agramont, Jr. ...[et al.]
(00000011.nul)
View the document Data transformation with dts: sql server 7 and 2000 / James Samuelson ... [et al.] ; [edited by] Gina Brown, Karen Wachs, Laura Loveall.
(00000012.nul)
View the document Database benchmarking : practical methods for Oracle & SQL server / Bert Scalzo ... [et al.].
(00000013.nul)


If you select a Title and click on the icon to get the text of the record, nothing is shown. This is because there are no files with text, we only have metadata. The Document icon should either be removed or the DocumentText instructions changed to display meaningful metadata and a cover image would be nice.

1.



You can now proceed to create some useful indexes using dc elements and to format the display of your indexes (and to remove the full-text search function) to create a useful bibliographic collection.

Monday, October 11, 2010

Digitization of old CD's


Since i haven't played a CD in some years, i finally got around to getting rid of the dusty cd cabinet in the upstairs 'media room' sometimes called living room. As i boxed them up i realized there are some songs i don't have digital copies of so i will have to either spend time searching and downloading or digitizing them myself. Most of the stuff i have is eclectic so it is unlikely to be in an online music store.

Now that we are using Connect360 on the mac pro for all the existing MP3 files and Tversity on the windows server for all the FLAC's and movie files, there is no need to keep audio CD from long term storage. We will not be converting any audio cd's to MP3 but rather to the higher resolution FLAC as digital masters and then downconverting to MP3 as needed.

The problem is that i want to use the mac for ripping the CD's but i don't want to use iTunes for that. I want to rip to high resolution FLAC's and then convert to MP3's if needed. First we take a look at Max (http://sbooth.org/Max/) for OSX. While this is still beta (less than version 1.0) it has a lot of nice features. We set the output as FLAC to a folder on the external drive and insert and an audio CD. It gets the metadata from the net, rips and converts the files and ejects the disc, all in 3 minutes. Excellent!

Wonder if it supports multiple drives, i have a couple of Firewire external cd-rom drives, a plextor and a lacie i could hook up. All you would have to do is to feed it a CD every few minutes.

Thursday, October 7, 2010

Greenstone Z39.50 lookup to Library of Congress

In Greenstone 2.83 for Windows click the Download tab and select Z3950. Type in the following settings:

HOST: z3950.loc.gov
PORT: 7090
DATABASE: voyager
FIND: @attr 1=4 "SQL"

This sets up a connection to the Library of Congress Z39.50 server and does a search for books with SQL in the title.

Tick the max records checkbox and set the number to 5.
Click Download to start the retrieval

The search log indicates it completes successfully but no file of marc records is found in Application Data folder:

from Yaz log:

Opening connection to z3950.loc.gov:7090
Access database: "voyager"
Searching for keyword: "@attr 1=4 SQL "
<>
Yaz is Gathering records: 1 - 50
<>
Saving records to "\files3\Faculty$\users\gnickers\Application Data\Greenstone\GLI\cache\z3950.loc.gov\voyager_@attr 1=4 SQL _50.marc"
Closing connection...

The log file was written in:

U:\Application Data\Greenstone\GLI\log\download-Z3950994507293.log

but no .marc file. I tested with 500 records and 50 records and 5 records. I created z3950 folder manually and re-ran query. No joy. The Greenstone documentation notes:

"You can view the downloaded MARC files on the Gather panel. On the left-hand side of the panel, double click the Downloaded Files folder to expand its content. The subfolders are named by the Z39.50 server url. The MARC files are named as the combination of database name, query, and max_records if max_records is specified. These MARC files are physically stored in a temporary cache directory. "

This could be a Greenstone bug, further investigation reveals 3 PERL errors. The c:\program files\greenstone\perlib\downloaders\Z3950Download.pm file at:

Line 146
Line 156
Line 159

Next we replaced the voyager database label with
LCDB

Which i got from the loc page. In fact i replace the greenstone settings, which seem quite old with:

HOST: lx2.loc.gov
PORT: 210
DATABASE: LCDB
FIND: @attr1=4 "SQL"
MAX RECORDS: 5

which retrieved 5 records but none were saved. I changed the MODE setting from Librarian to Expert and re-ran the query. No joy. We then removed the max records setting and got 500 hits. Nothing saved.

I also tried the OAI harvester function, although this is not supposed to work on the local version, only on the server install.

Select Download, OAI and enter the following settings:

SOURCE URL: http://memory.loc.gov/cgi-bin/oai2_0
RESTRICT to SET: cwp
MAX RECORDS: 5

Click DOWNLOAD and see what happens. 0 of 5 files are retrieved and the log shows:

Parsing OAI identifiers.....
<>
Gathering OAI record with ID oai:lcoa1.loc.gov:loc.pnp/cph.3a00275.....
Unable to save oai metadata record: No such file or directory

Looks like it kind of worked except there was no place to save it or no file to save. An examination of the perl code might help.

I also tried the SRU function using the information supplied in the Greenstone documentation but we could not get a connection. The log shows:

Opening connection to http://z3950.loc.gov :7090
Cannot connect to http://z3950.loc.gov :7090

We tried the LOC SRW server at:
http://lx2.loc.gov:210/LCDB? 
But Greenstone gave error messages in trying to connect.
For details on the LOC server see: http://www.loc.gov/z3950/lcserver.html

Would be very interested in seeing an example or hearing from someone who has this working!