National federated searching Arthur Sale Emeritus Professor of Computer Science University of Tasmania [email protected] .

National federated searching

Arthur Sale

Emeritus Professor of Computer ScienceUniversity of Tasmania

[email protected]://eprints.utas.edu.au/8694

Australian national options

1. Australasian Digital Theses Program

2. ARROW Discovery Service

3. DIY: AuseSearch

4. DIY: eSearch-Africa

ADT

The Australasian Digital Theses Program (ADT) harvests data from 41 universities in Australia and New Zealand, providing a two-nation gateway to PhD theses.

Searching on individual universities is also possible.

ADT provides the thesis metadata and local search – the actual text is always held in the university’s repository.

Let’s look at PhDs

ADT is the oldest gateway to Australian research, pilot operating in 1999, went live July 2000.

Please look at http://adt.caul.edu.au

ADS

The ARROW Discovery Service (ADS) was created under an initial government grant to ARROW (Australian Research Repositories Online to the World).

It is operated by the National Library of Australia on an ongoing basis.

ADS harvests metadata from all Australian universities through their OAI interfaces and provides a search facility.

Federated again

As with ADT, ADS is a gateway to a federation of repositories. ADS holds copies of all the metadata relating to research papers, but the full texts (‘Versions of Record’) are always held in the various university repositories.

Please look at http://search.arrow.edu.au

Digression to BASE

BASE is a European repository search engine with global coverage, which is developing the capability of country search. See http://base.ub.uni-bielefeld.de/en/about_sources_map.php?menu=2&submenu=1. At present only European countries are searched, but an extension to anywhere is easy. Put pressure on BASE (Bielefeld University) and help them.

BASE by country

Do-it-yourself

Google provides the capability of creating Custom Search Engines: search sites that use the Google database, but are limited to a defined set of sites or characteristics.

I created such a search facility for Australia. Later I added New Zealand, but viewers can separate the two countries.

Universities can add this search to their Library pages. Scripts are available.

AuseSearch

AuseSearch does NOT harvest metadata. Instead it uses Googlebot’s crawl data to provide access to the html/pdf pages held in repositories as well as metadata. This means its results are differently ranked from ADS or ADT, since the PageRank algorithm is used.

See: http://www.google.com/coop/cse?cx=012189697858739272261:yyyqychcumo

South Africa

At this point, I should say that the University of Pretoria has created such a custom search engine for South Africa. I have no knowledge of what repositories are included, so talk to the UP people here.

See http://www.google.com/coop/cse?cx=013518019117943970829%3Atlw8-sayn_q

So what about Africa?

So, I decided to create a custom search engine for all of Africa, accessing every repository I could find via eIFL, OpenDOAR and ROAR (25).

Time spent: maybe 1 hour, with an additional hour or two in adding new repositories and classifying their countries. Low cost.

Result: an Africa-only search, capable of being refined to a country.

eSearch-Africa

You won’t be surprised to look at this site too:

http://www.google.com/coop/cse?cx=012189697858739272261:nppegeei_q4

This could easily be maintained in Africa say by UP, with additional nations as refinements, or when it gets too large turned into national CSEs. All UP needs to do is add country refinements.

Summary I

Let’s start summarizing:1 National (or larger) gateway

aggregations are not hard to do, and may be relatively inexpensive.

2 The issue is always having content to share – in other words repositories at the level of the university or research centre (such as CSIR).

Summary II

3 Lobby national government to provide money to establish repositories for all higher education research. By university is best if you can afford it, otherwise shared consortial arrangements are very acceptable.

4 Register existing repositories into global networks!

5 Establish cheap national searches.

Summary III

6 Start pressuring national government to make global exposure of national research a policy priority.

7 Suggest a national mandate, law or regulation that all publicly funded research in universities and research institutes must be made open access as soon as possible. This implies repositories to put them in.

Questions?

© Copyright 2009 Arthur Sale

All rights reserved

Arthur Sale asserts the right to be recognized as author of this work

Contact: [email protected]

eSearch-Africa on your web page?

<form action="http://www.google.com/cse" id="cse-search-box">

<div> <input type="hidden" name="cx"

value="012189697858739272261:nppegeei_q4" /> <input type="hidden" name="ie" value="UTF-8" /> <input type="text" name="q" size="31" /> <input type="submit" name="sa" value="Search" /> </div></form>

<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form=cse-search-box&lang=en"></script>