DLConsulting recently received a grant from the Foundation for Research Science & Technology to further improve Greenstone’s performance when scaled up to very large collections. Of particular interest are scalability issues caused by large collections of uncorrected OCRed text (e.g. digitized newspaper collections). As part of this research we are testing and benchmarking the performance of a number of different search engine and metadata database options, as well as improving Greenstone’s ability to distribute a collection across multiple servers. The research grant runs until April 2008.

We’ve been gradually updating our website over the past few weeks, in an attempt to make it much clearer how DL Consulting relates to Greenstone and open source.

After an email request, we’ve developed a new plugin, MetadataXMPPlug, to extract eXtensible Metadata Platform information from PDFs. The plugin is very basic – but should read most XMP information as long as it adheres to the RDF standard.

The neatest part of this project it that it makes use of Greenstone’s (relatively) new multipass import functionality. The MetadataXMPPlug is used during the metadata_read pass to extract metadata from PDF files, while the files themselves are handled by PDFPlug during the later import pass.

I finally remembered to add my paper on The Adoption of Open Source technologies in New Zealand to the main DLC site.  This was the final report for the first module of the Post Graduate Certificate in Professional Development (Electronics and ICT) that I’m currently undertaking.

While the report could probably do with some more polish, it does provide an interesting background to open source issues (such as intellectual property and copyright/left issues) and provides statistics and references which could be used as evidence in future contracts. Earned me an A too!