Mandala 8: A Database System that Can Start Out Small and End Up Big Gail E. Kampmeier University of Illinois Institute for Natural Resource Sustainability.

Mandala 8: A Database System that Can Start Out Small and End Up Big

Gail E. KampmeierUniversity of IllinoisInstitute for Natural Resource SustainabilityDivision of Illinois Natural History [email protected]

Supplemental materials to live demo 27 October 2008 of Mandala Supplemental materials to live demo 27 October 2008 of Mandala database systemdatabase system

at Biodiversity Information Standards (TDWG) meeting, Fremantle, at Biodiversity Information Standards (TDWG) meeting, Fremantle, AustraliaAustralia

Table of Contents

What is Mandala?HistoryWhy use it?

Where to beginSpecimensTaxaLocalities & Collecting EventsPeopleBulk samples

Loans, DepositionMuseumsImagesLiteratureJournalsLiterature Mining

…more

• Navigate via buttons in upper right of each screen, home returns you here.

• Click "more" above to see p.2 of Contents

• To quit from slide show, hit escape (esc)

Table of ContentsResources

HelpOnline services

GazetteersTaxonomic Names

TroubleshootingElectronic sticky notesDocument structural changesChanging key fieldsVerifying dataBuilding a thesaurus of people names

The Payoff: Reports

PhenologyTaxon illustrationsImage comparisonsSynonymiesSpecimens examined & Faunal listsLoan managementMapping

What is Mandala?Cross-platform database system supporting data acquisition & management for systematics & biodiversity studies

Specimens

LiteratureTaxonomicNames

Illustrations

Interrelated & Interconnected

Viewed this Viewed this way, it can way, it can seem seem intimidating, intimidating, but realistically, but realistically, most functions most functions are accessed are accessed from only 3 or 4 from only 3 or 4 tables dealing tables dealing with with TAXA, TAXA, SPECIMENs, SPECIMENs, ILLUS (images), ILLUS (images), & LITERATURE& LITERATURE

Highlights in Mandala's History

Begun in 1995 for therevid(fly) NSF PEET* projectUses FileMaker® Pro 8.x/9

Relational Works on Macintosh & Windows operating systems

Extensible, adaptable, & evolvingFully operational demo available by request

*U.S. National Science Foundation Partnerships for Enhancing Expertise in Taxonomy DEB 95-21925 & 99-77958

So Why Use Mandala?

Completely open architecture with cross-platform, well-supported, off-the-shelf database engine (FileMaker Pro)Multiuser support through FileMaker Pro Server or Server Advanced. Access your databases remotely from FileMaker & a good internet connectionWeb serving options using FMP Server (for PHP) or Server Advanced

So Why Use Mandala?Extensive data entry options

Overwhelmed by the options & possibilities? News flash:

Use only the features you need in MandalaBreak up data entry tasks by putting in verbatim label data & basic specimen info on single layout in SPECIMEN or a basic taxon entry in TAXA Return later to georeference, interpret collection data in LOCALITY and COLLEVENT; add nomenclatural history to TAXA, and construct a classification hierarchyCustomize layouts & pick (value) lists to your needs

Extensive built-in reporting optionsIntegrated context sensitive HELPIntegrated problem & resolution tracking

Users Enter through NAVIGATION…

Whether from your desktop or via a network, open the NAVIGATION file first, sign in with an account name & password (1), then your user name (2), & navigate (3) to Mandala.

11

22

33

Data Entry & Viewing Options

Buttons allow you to navigate from a central point for data entry.Customize* this screen to rename, bring to front, or remove various navigational features Quit Mandala & FileMaker from here.

*Customization should be done by the database *Customization should be done by the database administrator.administrator.

Illustrated Features…

11 22

7799

33

44

88

ResourcResource filese files

55

66

Enter Specimen Data

Enter basic information about a specimen:

Unique IDVerbatim label(s)Life historyCuratorial info

Link to locality, collecting event & taxon IDNavigate via tabbed interface for additional data entry options

11

Minimum Taxon Name Entry

22

RankSpecific epithet, or if in combinationStatus (Valid, Invalid, Unregulated)

Add to the minimum…22

Common nameValid nameParent taxonBuild classificationList author(s) from PEOPLESource of name informationCommentsVerification

And history of the type…22

And name changes, homonyms, & other conflicts…

22

Resource: PeopleThe PEOPLE table is used to reference, via the PeopleID, all the information stored about a person, permits the flexibility of constructing that person’s name in various ways, & to determine of the various ways a name is presented, which is the senior synonym.This table (part of TAXA) is referenced for authors of literature; authorities of taxa; illustrators & copyright holders; and determiners, borrowers, lenders, & collectors of specimens.

Interpreting Collection Labels

3-43-4

In prospective data entry of information for specimens or bulk samples, it is more likely that labels will be generated from the information entered & that it will not need much additional interpretation, e.g., latitude, longitude, elevation, & more complete locality & collecting data will be available.

In retrospective capture of specimen data, often from museum specimens that may be over 100 years old & with incomplete or difficult to interpret data, it is more efficient to split the labor of recording the verbatim label(s) in SPECIMEN, which does not require much training, from their interpretation in the LOCALITY & COLLEVENT tables.

Serial Locality

Serial LocalityIDs (layout=1) are automatically generated with new records.

Locality records parse the political description as well as that of named geographic features that may cross political boundaries.

33aa

Custom LocalityCustom locality identifiers do not rely on the automatically generated serialID (layout=2)

Configure a 3-part unique identifier that may be meaningful to the collector.Ecological Community is added (can be used with serial localities).No displacement calculator appears for georeferencing here, but can be done on Additional Details tab.

3b3b

Choosing Locality Type

Generally you would choose a single locality entry style (choose serial unless having meaningful identifiers is important)Three types exist for scripting purposes, layout=

1=Serial locality2=Custom locality3=Bioblitz locality

Recording Serial Localities

Because serial localities are most often used with retrospective data capture with greater interpretation needs, features were added to aid in this task:

Translating elevation in feet to meters

Calculating displacement from a known lat/long by distance in km & one of 16 compass directions.

Translating miles to kilometers

33

From Locality to Collecting Event

Each locality may have one or more collecting events attached to it. To see what collecting events are already associated with a locality, click on the small magnifier to see these records.The LocalityID is represented by the left side of the underscore character.It is easiest to find or create a locality first, then find or create a new collecting event to associate with it.You can mix/match serial & custom identifiers, but if you don't have to, stick to serial localities & collecting events

33

Serial Collecting EventCollecting events include the collecting method, collectors, collection date range, abiotic conditions at the time of collection, & a localityID.Serial collecting events (layout=1) could be unique references to both the locality & event, but because custom collecting eventIDs may NOT be unique, a combination of the LocCollEventID is used as the key.

44

Custom Collecting Event

Both Custom (non-serial) and serial collecting events (CEVs) are displayed on the same tab, but layout=2.But, custom CEVs are composed from the method code, trap/site#, & custom CollEventID, all of which are designated by the person doing data entry.These codes may make more sense to collectorsCustom CEVs may be combined with custom or serial localities. They are only guaranteed to be unique in combination with the localityID as LocCollEventID.

44

44Create Records in SPECIMEN

Whether from serial or non-serial collecting event recordings, you can create two kinds of new records in SPECIMEN:

Single specimen records with a unique identifier for the specimenBulk sample record that reflects the collection of many specimens that may or may not have been sorted, but whose tracking is of importance.

You can see specimens or a bulk sample already attached to this LocalityCollEventID.

Bulk Sample TrackingBulk samples, or samples representing more >1 specimen for which you are recording a locality & collecting event are recorded as a special type of specimen.Its CollectionCode, if generated from CEV, is composed of the localityID and first half of the CEV.The CatalogNumber is the last number of the LocCollEventID.The BulkSampleID = the LocCollEventID in this case.

55

Bulk Sample Tracking in SPECIMEN

Each bulk sample may be sorted into groups that may be sent to specialists for further identification. Each taxon is

Identified to group & sexedCounted or not, including any specimens removed from the sample & not sent to the specialistGiven a loan number specific to the borrower for each shipment (not each bulk sample)The loan is recorded using the pencil icon next to the loanID. If there are multiple loans to the same person in the same bulk sample, this is recorded only once.Create a taxon list of the sample

55

Track Bulk Sample Loans

When you click on the pencil icon, you create DEPOSIT record based on the sampleID, but also reflecting taxa in a loan that may be accumulated by loan.Print out the loan form after filling out shipping details

66

Basic Loan & Shipping

Basic loan & shipping details are common to all loans & appear on the top tab.For loans based on samples, loan forms are accessed on their respective tabs

66

Sample Loan Form

Note that this loan form lists taxa from one or more Sample IDs that have accumulated under a single loan number.Shown here in print preview, you see this form as it will print. Print this form, or return to the data entry screen using the icons seen when you are in browse mode (not print preview as these do not print & would not function in this mode)

66

Museums & Collections

The 3 or 4 letter coden (Museum Code) represents a museum or collection & is used to represent loans to & from collections & in ILLUS for physical storage of images & documents.The populated table provided is based on Arnett, et al.’s 1993 Insect & Spider Collections of the World, which is kept current by the Bishop Museum’s codens-R-us site*

* http://hbs.bishopmuseum.org/codens/ also accessible from NAV’s resources help layout

Digital Archiving of Images

Store images or other documents or links to these to the ILLUS table.Give detailed description of method, medium, background, copyright, view, stage, etc.Detail archiving of both digital & physical images.Link to literature citation.

77

Enter Literature Citations

Enter all types of literatureLink to journals, URLTranslate titles from non-target languageGive location & level of curation of the literatureSample citations prebuilt by citation typeNew bibliographic export feature

88

Journal Titles & Abbreviations

Journal titles & their abbreviations may be stabilized for data entry & flexibility of output in literature citations.Publisher, City, & State/Country information is also sometimes required for some citations.

Dissect the Literature

Not likely the first task you will undertake in Mandala, mining the literature & linking (via an organizing principle) it to a taxon or specimen are activities associated with maturing databases with a specific taxonomic or geographic focus.

99

Eyes Glazed Over Yet?

OK, so you have that sick feeling in your stomach that this is WAY more database than you bargained for…too complex, too many features, too hard to understand.STOP! DON’T PANICThere are help & troubleshooting resources built into Mandala or are accessible online, which are only a click or two away…

Help ResourcesA variety of help resources are built into Mandala, from general, file, and field context-sensitive help, to explanations of the icons & terms used in Mandala, to links to online gazetteers, taxonomic name servers, & an index to codens used for many museums & collections.

11

22

3344

Glossary of Icons & Terms

Find out what all of those icons meanLearn the language of databasing & conventions used throughout Mandala

11

Integrated Help

File/Table & field context sensitive Help. Click ? icon in table or while in field for which you want information.

22

Access Online Gazetteers

33

Access Taxonomic Names Servers (TNS)

44

Tools for Troubleshooting

Track & resolve problems encountered by usersDocument changes to database structureOpen all files or verify integrity after a power outage or crashSee the ramifications of making changes to or deleting unique identifiers & prevent orphaned records

11

22

33

44

Electronic Sticky Notes

Track user problems from any file even if you never see the userTrack resolutions to problemsKeep track of records deleted & the reason for deletion

11

Document Structural Changes

Tracks changes by table & types of changesLists changes made & changes neededInclude screen captures of changes madeDocuments version concerned & environment (server, web, demo, etc.)For database administrators

22

Implications of ID Changes

Specimens can be tied to at least 13 other tables related through the SpecimenID.Change the related records first before changing the SpecimenID

33

Verifying Data

Many levels of verificationLook for the obvious

Duplicate entriesInformation that doesn’t agreeOutliers & ocean dwellers for terrestrial organisms

Nothing beats proofing from originalExamination by expert who will see errors on labels: misspellings, inconsistencies

Synonymies Among People Names

Because people may be listed as authors, taxon authorities, illustrators, determiners, lenders, borrowers, collectors, etc. their names may be presented in different forms.By designating synonymic status & senior synonym ID, you can choose the format to display in various output scenarios (labels, specimens examined, literature), even if a junior synonym is used.

44

So What’s the REAL Payofffor All This Work??

Why do we make databases to organize our data? It’s not as though we’d rather be doing this than be out in the field!!We record our data so that we can examine it in novel ways that couldn’t otherwise be seen, to identify:

Changes through time (phenologies)Changes & relationships through space (distributions; mapping)Holes in our dataStrategic dates & places in which to collectCorrelations of our data with datasets of othersFacilitate publishing our data

So on to the built-in reporting features!!

Reporting Features

Report examples 1 Phenology2 Illustrations of taxon3 Image comparisons4 Synonymies5 Specimens examined6 Faunal list7 Loan management8 Export for mapping

Reports: Phenologies

You can search for all of a taxonID in a state or province to find the range of collecting dates that this species has been found in the this geographic range.The search can also specify a range of sequential dates spanning more than 1 year & a calculator is supplied for this purpose. In find mode, use “…” between two numbers to find the range.

11

Julian vs. Sequential Date

Julian date is a day from 1-365 (366 in leap years) that represents a day of any year. Use this to map phenologies for the same days regardless of year.Sequential date specifies a range of days linked to specified years and will not automatically accumulate data for the same day across a year.Date information is pulled from the initial collecting date information entered for a collecting event (in COLLEVENT)

11

Reports: Export Phenology

Print, export, or view only for outliers (unexpected collecting dates such as January in Illinois!), the phenology of one or more taxa.Export to plot in your own charting software such as Microsoft Excel.

11

Plot Exported Results in Excel11

Reports: Illustration of Taxon

Some “reports” are collated automatically because of a relationship specified such as that between a taxonID & illustrations associated with that taxon.Click on magnifier icon to see details about these illustrations.

22

Compare Images

Image comparisons now appear beside the entry form for all images.This image comparison is filtered automatically by the field “Illustration of,” allowing you to view comparisons with other illustrations in the database.Other comparisons could be made by creating other relationships (see your database administrator)

33

Specialized Search: Synonymies

Script only finds valid species or genera. You can export synonymies from here.Or use the back-curved yellow arrow to view the synonymy on the tab for the taxon (see next).

44

Reports: Mining Portals

Although the portal giving the full synonymic list for this species is dynamically generated, the indented list at the right, which is often used in catalogs, is not. This list needs to be regenerated by running the script associated with the single pencil icon on individual records or by clicking on the multiple pencil icon to update all found records.

44

Specimens Examined & Faunal Lists

Complex finds may be facilitated here to generate faunal lists for a locality or specimens examined lists for a single or group of taxa.Start the find process with the scripted button “click to start FIND”, add criteria with + button, & execute find with blue find button in status bar.

55

66

Specimens Examined...

Although you may do all of your work in SPECIMEN to assemble your specimens examined list, behind the scenes, Mandala is actually pulling your data from many different sources!

PEOPLE -> PEO_JOIN ->TAXA (taxon authorities)PEOPLE -> PEO_JOIN -> COLLEVENT (collectors)LOCALITY -> COLLEVENTMUSEUMS -> DEPOSIT (lending, borrowing, original lending, redistribute to institutions)

55

View, Print, or Export Specimens Examined List

Or, after finding the records from “Illinois,” for example, click a button at the bottom of the layout on the previous slide to sort your records for specimens examined for a single taxon or multiple taxa. Note here, the taxa are output as listed, without interpolation to a valid taxon name. You could change this on your layout.Choose to print, export or simply view your results.

55

View, Print, or Export Faunal List

After you’ve found the records you want (we searched for the state of “Illinois”), you can choose a button at the bottom of the layout on the previous slide to sort your records for faunal lists by the taxon listed or by the valid name.Choose to print, export or simply view your results.

66

Reports: Managing Loans

Mandala now manages loans from several different perspectives, so you must choose in which you are interested.

Find this Find this button on button on

layout to lead layout to lead you hereyou here

77

Loan Management...

……depends on depends on perspective & perspective & contextcontext and is and is

handled in handled in DEPOSIT.DEPOSIT.

77

Context = Context = taxataxa, sent by , sent by or received by you from or received by you from museums or museums or collections; bypasses collections; bypasses specimens & lists specimens & lists groups of taxa in a loan groups of taxa in a loan in BULKSAMPLE.in BULKSAMPLE.

Context = Context = specimensspecimens, owned by , owned by

you, or sent by or you, or sent by or received from received from

museums or museums or collections. Recorded collections. Recorded

on tab in SPECIMEN on tab in SPECIMEN table. May be the table. May be the

partitioning of partitioning of context = context = taxataxa Context = Context = borrower & loanIDborrower & loanID, ,

where where subsamplessubsamples (usually (usually taxonomically based) of one taxonomically based) of one or many larger samples are or many larger samples are tagged to send to a specialist tagged to send to a specialist (=borrower) under a single (=borrower) under a single loan (=shipment). Uses loan (=shipment). Uses Sample entry of SPECIMEN, Sample entry of SPECIMEN, BULKSAMPLE, & DEPOSIT BULKSAMPLE, & DEPOSIT tables.tables.

Sort & View, Export, or Print

Once found, choose option for sorting & viewing results. Print or export if desired.

77

Finding Data to Map

Narrow to place, make sure georeferenced data exists by using the buttons to initiate & extend finds, &/or narrow to taxonOnce found, export data & use your favorite mapping program or map your data with DiscoverLife or GBIF.

88

Map in Desired Program

Collecting sites (black dots) in Madagascar where therevid fly specimens have been collected.iMap* was used here to map latitude/longitude in decimal degrees. Other mapping programs may be used by user preference.

88

**http://www.biovolution.com/imap

OnlineMapping…

Map One or More Taxa with DiscoverLife.orgThese data were exported from the Therevid PHP database If others had contributed therevid fly data, their data would also appear here unless databases were limited.Can map species against potential plant hosts or associates.

Zoom in Progressively

Shows taxa remaining on the mapClick on map to zoom inClick on point to see data or on links in list below map

Point Data Served in DL

These are data exported from Mandala to a tab-delimited text file for DiscoverLife.Links to the taxonomic name & back to the originating PHP record output (by RecID)

PHP Searches

Drill back & forth between DL & PHP versions of MandalaThe database version queried by PHP is not the production database but an optimized version of data exported from Mandala for display on the web (not pretty but functional!)

Contribute to GBIF

Data served on DiscoverLife (DL) can now be contributed to GBIF (Global Biodiversity Information Facility)GBIF datasets will be able to be mapped in DL

Newest additions…Support for

LSIDs (life science identifiers) for taxa, specimens, people)DOIs (digital object identifiers) for literatureUnique resource identifiers for illustrations (may be DOIs)BCI GUIDs (Biodiversity Collections Index Global Unique Identifiers) for collections

Track the progress of molecular studies based on

Extractions from a specimenPCR products using specified primersSequences derived & deposition into Genbank

Coming in Early 2009

Seminal reference for MandalaKampmeier, G.E. & Irwin, M.E. 2009. Meeting the interrelated challenges of tracking specimen, nomenclature, and literature data in Mandala. Chapter 15 in T. Pape, D. Bickel, & R. Meier [eds.], Diptera Diversity: Status, Challenges and Tools. Koninklijke Brill NV.

AcknowledgementsMichael E. Irwin

F. Chris Thompson

Neal Evenhuis

Chris Lambkin

Don Webb

Mark Metz

Martin Hauser

Kevin Holston

Steve Gaimari

J. Marie Metz

Amanda Buck

Kristin Algmin

Andy Bennett

John Pickering

Shelah Morita

FMWebschool

University of Illinois

DiscoverLife.org

Schlinger Foundation

Hatch ILLU 875-380

NSF Therevid PEET DEB 95-21925 & 99-77958

NSF BSI for Fiji Bioinventory of Arthropods DEB 0425790

Illinois Natural History Survey

NSF AToL for Diptera EF-0334948

NSF Tabanid PEET DEB 07-31528