NIMAC and the AAP: Shared Accomplishments & Current Challenges Julia Myers Nicole Gaines June 2008.

NIMAC and the AAP:

Shared Accomplishments & Current Challenges

Julia MyersNicole GainesJune 2008

What is the NIMAC?

NIMAC is a central repository for NIMAS file sets. NIMAS file sets are source files that can be used to produce accessible, student-ready specialized formats, such as Braille and audio, for students in K-12 with qualifying disabilities.

What does the NIMAC do?

• The NIMAC receives, reviews, and catalogs publishers' electronic files of print instructional materials in the NIMAS format.

• We also sign up registered users, provide the searchable web interface, provide feedback to publishers and vendors, and answer questions from NIMAC stakeholders and others.

• OverDrive, Inc. is the vendor who is providing the database software and off-site storage for the NIMAC’s file sets.

NIMAC: From IDEA to Reality• In December 2004 …

– There was no repository, no plan for repository.

– IDEA 2004 mandates establishment of NIMAC.

– The NIMAS standard was still being tweaked.– Final regs not published until July 2006.– Target date to begin operations: December 4,

2006!

Planning Phase: ExpectationsThe NIMAC will …

• Catalog 2,400 NIMAS file sets per year (Projection provided by AAP and basis for OSEP- approved NIMAC plan/grant request)

• Register authorized users from 50 states• Provide searchable web interface• Provide ability for registered users to download

files• Protect copyright

Key Accomplishments

• December 3, 2006: Opened on schedule—less than six months after final regs published!

– Searchable web interface– SC, AU, AMP and publisher accounts and registration– Publisher portal and inventory management – File download functionality for AUs and AMPs– NIMAC staff process for review and certification of

files– Began accepting files for certification on Dec. 3, 2006!

Key Accomplishments

• Cataloged and made available for download 8,435 NIMAS file sets in less than 19 months.

• This is more than double the projected number of NIMAS file sets NIMAC was built to handle in that time period.

Key Accomplishments

• Coordinated with 52 states and outlying areas.

• NJ, PA and TX currently in process of coordinating.

• Registered 86 Authorized Users and 74 Accessible Media Producers.

• Established 67 publisher accounts.

Key Accomplishments

• Copyright protection provided via multiple methods, including:

• Coordination and Limitation of Use Agreements

• Digital Rights Management (i.e., digital fingerprinting and watermarking of files)

• Secure environment • Two tiered repository access (AUs and

AMPs).

All Figures Correct as of June 17, 2008

Key Accomplishments

NIMAC Download Statistics:

– Unique Downloads by AMPs: 153

– Unique Downloads by AUs: 458

– Total Unique Downloads: 611

Key Accomplishments

NIMAC Outreach

• Outreach was an important part of NIMAC’s work in 2007 and continues to be a key aspect of our work in 2008.

• NIMAC routinely offers training webcasts and does further outreach through conference calls and participation in conferences and workshops.

Challenges: Logistical Issues

• Compressed time frame for receipt of significant percentage of files— intense “busy season”—March through May, 2008 (mostly unexpected!)

• Tight contractual deadlines from customers for publishers to create and deposit files

• Tight timeframes for OverDrive to validate and load and for NIMAC to review and certify NIMAS files.

• High percentage of file errors exacerbates problem.

Challenges: Technical Issues• Number of files received several times greater

than planned

• Size of files being received versus those provided for NIMAC in development

• File size driven by images• File size affects uploading and processing

from publishers as well as downloading by NIMAC users

Challenges: File Quality Issues

Nearly 2/3 of files submitted must initially be rejected due to critical errors, such as:

• Poorly formatted, incorrect or incomplete metadata in batch load spreadsheets

• Poorly formatted, incorrect or incomplete metadata in OPF files

• Submission of corrupt zip files• Submission of files that fail validation

Challenges: File Sizes

• File sizes larger than planned—Why?

– Few files received in development phase and those averaged 30 MB

– Largest file received in development phase was 200 MB

– Reality is that files of 7 GB have been received and 10 GB to 12 GB have been forecast by publishers

NIMAC: File Storage Statistics

• NIMAC files currently mass over 1 TB.

• During spring 2008, the NIMAC was receiving 35-60 GB of file sets per week.

• NIMAC could add as much as 2 more TB this year.

NIMAC: File Storage Statistics

• To put this in perspective:

– 2 Terabytes = An academic research library.

– 10 Terabytes = The print collections of the U.S. Library of Congress.

(From "How Much Information", 2003. University of California at Berkeley: Lyman, Peter and Hal R. Varian. Retrieved from http://www.sims.berkeley.edu/how-much-info-2003 on April 15, 2008.

Challenges: File Numbers

Thousands more NIMAS file sets received than planned, per estimate by AAP—Why?

– “Published after” effective date versus “sold after:” thousands of file sets are pre-2006 (about 25% of total file sets in NIMAC inventory are pre-2006.)

Challenges: File Numbers

– Unanticipated materials: receiving flash cards, black line masters, sections of Teacher editions, etc. (Only about 20% of certified file sets to date are textbooks.)

– Supplementary Readers: 67% of certified files are readers

– State editions

Challenges: Project Capacity

Number of file sets is a capacity issue

– NIMAC funded and staffed to handle 2,400 a year.

– NIMAC certified 6,570 NIMAS file sets in only 16 months.

– OverDrive was contracted to handle 2,400 NIMAS file sets a year, with an anticipated NIMAS file set size much less than what we actually are receiving.

Challenges: Project Capacity

• For full 5-year grant, NIMAC was prepared to handle 12,000 NIMAS file sets.

• In less than 16 months, NIMAC received 12,000 NIMAS file sets.

• In first third of 2008, NIMAC certified 2,590 NIMAS file sets.

• NIMAC received 4,000 file sets in March and April 2008.

Challenges: Metadata Quality• The impact on NIMAC workload is substantial as

extensive feedback must be provided to vendors and 2/3 of files reviewed a second time.

• Vendors sometimes take weeks or months to correct rejected file sets.

• This significantly delays the availability of NIMAS files.

• Resubmission also increases costs of NIMAS file creation for publishers.

Challenges: Images/File Size

• Images--their number and size--drive NIMAS File size.

• NIMAS File sets received can be 6 GB and may reach 12 GB in size--or more.

• NIMAS Textbook Files contain thousands of images.

• File size can mean slow downloading, even for those with fast connections. (T-1 is dial-up slow at these sizes.)

Challenges: Images/File Size

• PNGs are 5-10 times larger than JPEG.

• NIMAS File size in context: typically a 90-minute movie is less than 1 GB. (This is due to compression method used for MPEG.)

• Impact: Huge file sizes are a considerable obstacle for end users attempting to download files as well as for publishers submitting files for batch load.

NIMAC Seeking Solutions: File Sizes

• Decrease file sizes, retain image quality

– Best practices for compression and creation of images (e.g. not from layout, not cropped, techniques for recurring images)

– Consultant will be engaged to develop these best practices

– Share best practices with publishers

NIMAC Seeking Solutions: Increasing Capacity

• Currently, NIMAC is expanding capacity by using overtime, contract staff and staff loaned from other APH depts.

• OverDrive has added staff, servers, multiple times to handle volume.• OverDrive is loading files continuously--24/7.• Long-term, the solution is to increase capacity to match reality.

Increasing capacity has been discussed with OSEP.

NIMAC Seeking Solutions: Metadata Errors

• Reconsider value of metadata in OPF; consider not requiring corrections for metadata in OPF or corrections only for select fields. (Data in NIMAC searchable database, XML and PDF would still be required to be correct.)

• Explore development of a conversion tool to create valid OPF metadata from the OverDrive batch load Excel sheet.

NIMAC Seeking Solutions: Metadata Errors

• Continue to offer one-on-one support and training to vendors by email and telephone

• Continue to offer training through webcasts and conference calls

• Continue to offer pre-submission review of sample OPFs and PDFs from new vendors

• Meet with publishers in first AAP/NIMAC conference in June to provide training and focus on improvements!

NIMAC System Development: • System development on hold for several months

due to need to shift all OverDrive resources to handle processing of high volume of NIMAS file submissions.

• Key remaining development includes:– Publisher portal improvements – Search interface improvements– Reports functionality – Admin portal workflow management functionality– Mac compatibility

NIMAC System Development:

• APH is pleased to announce that we have contracted with Firebrand Technologies to move Louis to a new system that will support federated search via OAI.

• Louis/NIMAC federated search a high priority for APH and NIMAC.

Collaboration with AAP

• Invaluable support from AAP representatives to NIMAC Advisory Council

• AAP members’ support and participation in the NIMAC User Group

• Quarterly Publisher conference calls to provide procedural/policy updates and receive feedback from publishers

• Today’s meeting!

Collaboration with AAP

Future possibility:

• Cooperate in bringing specialized format producers and publishers’ editorial/production staff together to share information and insight on how print book editorial decisions affect later conversion into specialized formats.

Challenges: Policy Concerns• Blind and visually impaired students who may

not qualify for services from NIMAS NIMAC• Pressure to change NIMAC policies and

procedures requirements regarding who can become an AU. (Currently must be part of educational structure of state.)

• Directing those who do not qualify for NIMAC/NIMAS to best resources

• Getting feedback for NIMAS file problems from states to publishers

How to Reach Us

• http://www.nimac.us

• 502-899-2230

• 1-877-526-4622 (1-877-52-NIMAC)

• [email protected]