Transcript
7/30/2019 Data Loader User Guide (2010-2011)
1/46
Data Loader
User Guide
2010-2011 School Year
7/30/2019 Data Loader User Guide (2010-2011)
2/46
7/30/2019 Data Loader User Guide (2010-2011)
3/46
iii | P a g e
Introduction
Data Driven Decisions for Academic Achievement
Data Driven Decisions for Academic Achievement (D3A2) is a long-term initiative focused on
developing the capacity of educators while improving instruction and student achievement.
D3A2 provides systemic access to timely data and educational resources aligned to Ohios
Academic Content Standards.
Data Warehouse & Data Tool
The D3A2 Data Warehouse and Data Tool provide the capacity for school districts in Ohio to
store, access, and analyze student course listings and performance data over multiple years.
District demographics data are supplied to the warehouse in the form of extracts fromthe eSIS and DASL student information systems or the EMIS reporting system. They in-
clude student demographic data, HQT teacher qualifications, and student-teacher-
course enrollment data.
Student performance data are provided in the form of results files distributed by Ohiostesting vendors following each administration of an Ohio Achievement Assessment
(OAA) or Ohio Graduation Test (OGT).
D3A2 Data Scrubber & Loader Utility
The D3A2 Data Scrubber and Loader is a web-based application that members of the State
Software Development Team (SSDT), Information Technology Centers (ITC), and school district
personnel use to prepare and load district demographics and student testing data into the D3A2
Data Warehouse. It combines features of the Data Scrubber and Loader applications, which used
to be stand-alone applications, under a single framework.
Implementation
The process of preparing and transferring data for upload to the D3A2 Data Warehouse is a
shared responsibility between ITCs and school districts.
ITCs are generally responsible for the following tasks associated with loading data into D3A2:
Extract and transfer demographics data from eSIS and DASL to their respective D3A2Drop Boxes in Oracle Collaboration Suite (OCS).
Monitor the extraction, transformation, and loading of data into the Data Warehouse. Work with the SSDT to identify and resolve issues which prevent files from loading into
D3A2.
Correct problems in raw demographics data to enable it to load successfully when appro-priate, or help district data entry staff correct errors in their SIS entries to correct faulty
demographics data.
http://www.d3a2.org/scrubber/http://www.d3a2.org/scrubber/http://www.d3a2.org/scrubber/7/30/2019 Data Loader User Guide (2010-2011)
4/46
Data Driven Decisions for Academic Achievement
iv | P a g e
Create and manage ITC-administrator and district-user accounts in the D3A2 Scrubberand Loader.
Train district staff in the use of the Data Scrubber to prepare test results for loading. Validate scrubbing results and forward scrubbed testing data to the Loader. Report possible application bugs to the SSDT.
Districts are generally responsible for the following tasks associated with loading data into D3A2:
Maintain accurate data in eSIS and DASL. Correct data issues in eSIS or DASL that result in failed file loads. Upload and scrub raw vendor data in the D3A2 Data Scrubber (unless their ITC elects to
perform these activities on their behalf).
Verify the integrity of demographics and testing data in the D3A2 Data Tool applicationfollowing file uploads.
Report possible application bugs to their ITC.Loader ITC User Guide
The purpose of this guide is to document the use of the D3A2 Data Loader application and pro-vide details to support its use. It is made available to all Ohio school districts and Information
Technology Centers (ITC) by means of downloads from the project website.
Companion Documents
Additional documentation is available to support D3A2 data implementers.
D3A2 Data Scrubber User Guide D3A2 ITC Implementation Guide
These documents are both available for download from theSupport Materialspage on the
project website.
Project Website
More information about the D3A2 project, including the district implementation process, may be
discovered by visitingwww.d3a2.org.
Student Privacy
The names and course enrollments of students displayed in this guide have been fictionalized. No
actual students or district data is displayed in this document.
http://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http://www.d3a2.org/support/http://www.d3a2.org/support/http://c/Users/bell/Documents/D3A2/Documentation/Data%20Scrubber%20Manual/www.d3a2.orghttp://c/Users/bell/Documents/D3A2/Documentation/Data%20Scrubber%20Manual/www.d3a2.orghttp://c/Users/bell/Documents/D3A2/Documentation/Data%20Scrubber%20Manual/www.d3a2.orghttp://c/Users/bell/Documents/D3A2/Documentation/Data%20Scrubber%20Manual/www.d3a2.orghttp://www.d3a2.org/support/http://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdf7/30/2019 Data Loader User Guide (2010-2011)
5/46
7/30/2019 Data Loader User Guide (2010-2011)
6/46
Data Driven Decisions for Academic Achievement
vi | P a g e
Appendix A: User Permissions
User Permissions .......................................................................................... 31
Appendix B: Notifications
Load Errors................................................................................................... 35
Prerequisite Errors ........................................................................................ 35
Purge Notification ........................................................................................ 35
Post Data Load ............................................................................................. 36
Threshold Exceeded ...................................................................................... 36
7/30/2019 Data Loader User Guide (2010-2011)
7/46
vii | P a g e
This page has intentionally been left blank.
7/30/2019 Data Loader User Guide (2010-2011)
8/46
7/30/2019 Data Loader User Guide (2010-2011)
9/46
1 | P a g e
Data Loader Basics
Features
The D3A2 Data Scrubber & Loader applications have been combined into a single framework to
provide Ohios Information Technology Centers (ITC) and school districts access to D3A2s com-
plete ETL process.
Testing File Scrubber Application
The D3A2 Data Scrubber provides a mechanism for Ohio School Districts to prepare their testing
results files for upload to the D3A2 Data Warehouse.
Refer to theD3A2 Data Scrubber User Guidefor complete instruction on how to access and usethe Data Scrubber application. It is available for download from theSupport Materials1 area of
the Project Website.
Loader Application
The D3A2 Data Loader provides an application to upload district demographics data and student
testing results into D3A2s longitudinal data system.
District Demographics Data
Student demographics, teacher HQT qualifications, enrollment data, and student-teacher-course
assignments are provided to the D3A2 system in the form of extract files created from eSIS,
DASL and Ohios EMIS reporting system.
These extracts consist of seven files for each school year: ID_SSID, DIST_INFO, BUILD_INFO,
STAFF_INFO, STU_INFO, STU_ENROLL, and STU_TEACH_COURSE. ITC personnel create
these extracts by running the correct procedure against the appropriate source.
1http://www.d3a2.org/support
Source of Demographics Data
1. 2010-2011 School Year: extracts from the eSIS and DASL student information systems.2. 2009-2010: School Year: extracts from the eSIS and DASL student information systems.
Note: eSIS extracts are not able to be created for historical reporting periods
3. 2008-2009 (and Prior) SchoolYear: extracts created from the EMIS database.
http://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/supporthttp://www.d3a2.org/supporthttp://www.d3a2.org/supporthttp://www.d3a2.org/supporthttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdf7/30/2019 Data Loader User Guide (2010-2011)
10/46
Data Driven Decisions for Academic Achievement
2 | P a g e
Note: district demographics files must be loaded into the D3A2 Data Warehouse in chronologi-
cal order according to school year, sometimes referred to as reporting period.
More Info: districts should decide how far back they intend to scrub testing results and request
their ITC to create and load district demographics for all school years for each of these years.
District demographics data should be sent to the D3A2 Loader for processing once for each prior
school year and at regular periods throughout the current school year. Providing regular uploads
of current year demographics data ensures that D3A2s re-rostering features are kept up-to-date.
More Info: ITCs are advised to submit current year district demographics to the Loader as soon
as participating districts initial class lists are available in theirstudent information systems and
then at least once per quarter thereafter.
Note: the D3A2 extract procedure for eSIS and DASL may be set up to run at scheduled inter-
vals. ITCs are asked not to send data to the D3A2 Loader more frequently than once per monthunless there has been significant student movement or data entry corrections within a districts SIS.
General instruction on how to create district demographic file extracts from the EMIS database
are provided in theD3A2ITC Implementation Guide. eSIS and DASL users should refer to their
SIS documentation for additional information.
Oracle Collaboration Suite
Oracle Collaboration Suite (OCS) is used by ITCs to securely transfer district demographics files
to the Loader staging area. Each ITC has its own username and D3A2 Drop Box in OCS into
which district demographics files are deposited for automatic upload and processing.
More information about OCS is provided in later sections of this Guide, as well as in the D3A2 ITC
Implementation Guide.
Note: the automatic loading of files deposited into the D3A2 Drop Boxes is a new feature of the
D3A2 implementation process beginning with the 2010-2011 school year.
http://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdfhttp://www.d3a2.org/support/http:/www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20ITC%20Implementation%20Guide%202010-2011.pdf7/30/2019 Data Loader User Guide (2010-2011)
11/46
Loader Basics
3 | P a g e
Testing Vendor Files
Vendors of the Ohio Achievement and Graduation tests provide districts with electronic copies of
results files following each test administration. These files may be prepared for upload to the
D3A2 Data Warehouse and consist of thegeneral (student scores) and item analysis files.
These files must be processed using the D3A2 Data Scrubber prior to upload to the Data Ware-house. Detailed instruction in how to access and process these files is provided in theD3A2 Data
Scrubber User Guide.
More Info: unlike district demographics files which are automatically picked up and processed
from the ITCs OCS Drop Boxes, testing files must be uploaded to the D3A2 Data Scrubber & Loader
application manually. Information about this process is provided in detail in the D3A2 Data Scrub-
ber User Guide.
http://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdfhttp://www.d3a2.org/uploads/docs/Support%20Materials/D3A2%20Data%20Scrubber%20User%20Guide%20(2010-2011).pdf7/30/2019 Data Loader User Guide (2010-2011)
12/46
7/30/2019 Data Loader User Guide (2010-2011)
13/46
1 | P a g e
Accessing the Data Scrubber & Loader
Login Accounts
The Loader login screen is located at
http://Loader.d3a2.org.
Login accounts are required to use the D3A2
Scrubber & Loader Application. Each ITC is
assigned an initial userID and password and then
begins creating and managing accounts for their
own organization and school districts.
Who Should Use the Scrubber & Loader?Depending upon policies established by each ITC, districts are required to participate to a greater
or lesser extent in the scrubbing process. The policy of most ITCs is that, once trained, districts
are required to log in to the Data Scrubber to scrub their test return files independently. The ITC
provides support as needed to help districts through any difficulties they experience with the
scrubbing process.
The district staff member(s) assigned to the task of scrubbing testing vendor data need to have
knowledge of and administrative access to the districts student information system and EMIS
data. They must be comfortable using data systems. These staff often serve in the role of EMIScoordinators, technology coordinators, or curriculum directors within their districts.
User Types
The Scrubber supports three types of user accounts: District, ITC, and Administrator. Each type
of account provides differing levels of access to features within the Loader as described in Ap-
pendix A of this Guide.
Figure 1: Scrubber & Loader Login Screen
http://www.d3a2.og/scrubberhttp://www.d3a2.og/scrubberhttp://www.d3a2.og/scrubber7/30/2019 Data Loader User Guide (2010-2011)
14/46
Data Driven Decisions for Academic Achievement
2 | P a g e
Creating User Accounts (Step by Step)
The D3 A2 Operations Group assigns each ITC an initial username and password for the Loader
application. ITCs are able to use this account to create additional ITC and district-level accounts.
1.) Log in to the application using an ITC-levelaccount
2.) Select User Maintenance from the main
menu.
3.) Click .
4.) Enter the users information.
Note: ITC staffmay only select District
or ITCin the User Role selection field.
Note: when creating ITC-level accounts,
be sure to select a school districtin the Dis-
trict field, rather than the ITC.
More Info: the District selection field only
dislays districts that are registered to the ITC
users organization. If a district is not showing
as it should, the ITC should issue a heldesk
ticket to the SSDT.D3A2.SUPPORT request
area asking that the district be reassigned.
Caution:assigning an ITC-level account to an ITC rather than a school district in the District
selection field will result in the user not being able to view district-user accounts or data as they
should. This can only be corrected by submitting a help desk ticket to the SSDT.D3A2.SUPPORT
request area in CA-Unicenter.
5.) Click .
Figure 2: Create New User
Figure 3: User Details Screen
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
15/46
Loader Basics
3 | P a g e
Managing User Accounts (Step-By-Step)
Recover Forgotten Passwords
ITC and district-level account holders are able to
reset forgotten passwords from the Loader login
page.
1.) Click on Forgot Your Password? Click here to
resetfrom the Login screen.
2.) Enter valid email address to which the account
is registered. After you click Reset Password the
Scrubber will send a temporary password to your
email account.
Figure 4: Forgot Password
Figure 5: Reset Password
7/30/2019 Data Loader User Guide (2010-2011)
16/46
Data Driven Decisions for Academic Achievement
4 | P a g e
Changing Passwords
All users of the Loader should personalize their password the first time they log into the Scrubber
& Loader application.
Caution: the Scrubber exposes sensitive student information. Always select a complex pass-
word and do not share your password with anyone else.
3.) Click on the Password maintenance link shown in the page header.
4.) Enter and confirm a new password,
then click .
5.) Verify that the Password changedmessage is displayed.
Figure 8: Password Change Confirmation
More Info: Scrubber passwords should be at least 6 characters in length and include at least one
number.
Figure 6: Change Password Link
Figure 7: Password Page
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
17/46
Loader Basics
5 | P a g e
Edit User Accounts
ITC-level Loader accounts are able to view and manage accounts from their local user base.
6.) Log in to the application using an ITC-levelaccount
7.) Select User Maintenance from the menu.
8.) Browse for an account to edit, or search, and
click the edit ( ) icon to access the User
Details page.
More Info: ITC-level account holders only
see users belonging to their organizations dis-
tricts on the User Maintenance page.
Note: if a user who is known to have an
existing account is not appearing on the User
Maintenance page it is likely that they have been
assigned to another ITCs district. Submit a
helpdesk ticket to the SSDT.D3A2.SUPPORTrequest area in Unicenter asking to have them
reassigned to a different district and/or have their
district assigned to a different ITC.
9.) Edit the users details as necessary.
10.) Click .
Figure 9: Edit User Details Link
Figure 10: Edit User Details Page
7/30/2019 Data Loader User Guide (2010-2011)
18/46
Data Driven Decisions for Academic Achievement
6 | P a g e
Main Application Pages
The various tasks associated with loading data into the D3A2 Data Warehouse are accomplished
using the Loaders three main pages: Loader Activity, Status Details, and Unmet Data Prerequi-
sites pages.
Loader Activity Page
The Loader Activity page provides a snapshot
of recent activity in the D3A2 Loader and
provides access to score cards for demograph-
ic and testing file loads.
Status Details PageThe Status Details page provides detailed
information about what has been loaded for a
given school district and/or what is currently
being processed by the Loader. This page
allows users to:
research which files have been loaded andany errors associated with them;
identify files that can not be processed dueto critical errors;
release testing files that have been for-warded to them for review by district
scrubber personnel; and
observe loading progress for files that havebeen enqueued in the Loader.
Unmet Load Prerequisites Page
The Unmet Load Prerequisites page provides
an area to view unmet prerequisites which are
preventing files to be acted upon by the Loader
after they have made their way into the
processing area.
Fi ure 11: Loader Activit Pa e
Figure 12: Status Details Page
Figure 13: Unmet Load Prerequisites Page
7/30/2019 Data Loader User Guide (2010-2011)
19/46
Loader Basics
7 | P a g e
Navigating the Loader Pages
Navigating between the Loader pages is accom-
plished by hovering over the Navigation Menu
and making a selection from the drop down.
Using the Scoping Tool (Step-By-Step)
ITCs are initially presented with loading activities and status for all of their member districts when
they log into the Loader.
The Scoping Tool allows ITC staff to filter the display to show data for just one district.
1.) Click on or
to open the Scoping Tool.
2.) Select a School District , Year , or both from
the prompts.
Note: ITC staff are only presented districts
that are assigned to their own organizations when
they use the Scoping Tool.
3.) Click to register the selection.
Figure 14: Data Loader Navigation Menu
Figure 15: Access the Scoping Tool
Figure 17: Verify Scoping Selection
Figure 16: District & Year Selection
7/30/2019 Data Loader User Guide (2010-2011)
20/46
7/30/2019 Data Loader User Guide (2010-2011)
21/46
9 | P a g e
Loading District Demographics Files
ITC support staff participate in the loading of test results files by performing the following tasks:
Extracting demographic files from an EMIS database or the student information system; Uploading demographics files to their organizations D3A2 Drop Box in Oracle Collabora-
tion Suite (OCS);
Monitoring the progress of demographic file loads using the Status Details page; Evaluating the results of successful loads using the Status Details page; and Responding when errors in the raw extracts data prevent the normal processing of data.
Uploading District Demographics Files to
OCS (Step-By-Step)1.) Extract district demographics data from an
appropriate source.
2.) Use the ITCs username and password to
access OCS:.
a.) use Internet Explorer 7 to log in to
https://wildfire.nwoca.org/content/dav; or
b.) map a network drive in MS Windows My
Computer tohttps://wildfire.nwoca.org/content/.
More Info: operational support for OCS
may be requested by issuing a helpdesk to the
SSDT.D3A2.SUPPORT request area in CA-
Unicenter.
2.) Upload district demographics data to the
ITCs D3A2 Drop Box in OCS to initiate the
loading.
More Info: do not copy the ID_SSID infor-
mation file to the Drop Box in OCS.
Figure 18: OCS D3A2 Drop Boxes
Figure 19: Uploaded Files
https://wildfire.nwoca.org/content/davhttps://wildfire.nwoca.org/content/davhttps://wildfire.nwoca.org/content/dav7/30/2019 Data Loader User Guide (2010-2011)
22/46
Data Driven Decisions for Academic Achievement
10 | P a g e
3.) The Loader automatically picks up and processes the new district demographics files into its
Processing Files list on the Status Details page.
Note: it is possible to send individual files for load in order to recover from a prior critical error.
For example, a Student Enrollment File may be dropped into the Drop Box for processing. However,
this can result in unexpected results. ITC staff should discuss specific situations with the D3A2
Operations Group that may require processing files individually before attempting this procedure.
More Info: file extracts from prior years should be loaded once into the Data Warehouse, whe-
reas extracts from the current school year should be loaded at least quarterly and/or whenever
significant changes have been made in the student information system.
Caution:district demographics loads must be loaded into the Data Warehouse in chronological
order according to school year. ITC staff must first determine how far back their districts intend to
scrub testing data before sending demographics files to the Loader. This will help ensure that ex-
tracts from all necessary school years are processed through the Loader in the proper order.
District Demographics Files
D3A2 demographics loads consist of six files:
Load Type Filename File Type
1 DIST_INFO_XXXXXX_xxL.csv District Information
2 BUILD_INFO_XXXXXX_XXL.csv Building Information
3 STU_INFO_XXXXXX_XXL.csv Student Information
4 STAFF_INFO_XXXXXX_XXL.csv Staff Information
5 STU_ENROLL_XXXXXX_XXL.csv Student Enrollment
6 STU_TEACH_COURSE_INFO Student Teacher Course Information
The Loader processes each of these files sequentially in order of type.
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
23/46
Loading District Demographics Files
11 | P a g e
Monitoring Demographic File Loads (Step- By-Step)
The Status Details page is used to monitor the progress of district demographic file loads.
1.) Log into the Data Loader application using an ITC-level account.
2.) Navigate to the Status Details page.
3.) Use the Scoping Tool to narrow the view
to a single district.
4.) Check to make sure that there are no critical errors reported by clicking on the Status Details
page Files in Error list.
5.) Select the Released File list by clicking on and observe that all six district demo-
graphic files have been transferred from the ITCs Drop Box to the Loader.
Figure 20: Navigate to Status Details
Figure 21: Verify Scoping Selection
7/30/2019 Data Loader User Guide (2010-2011)
24/46
Data Driven Decisions for Academic Achievement
12 | P a g e
6.) The Loader will process all files sequentially in order of load type as indicated by the
Processing or Waiting to be Processed progress indicators .
7.) Observe that the testing files are eventually processed out of the list into
.
More Info: ITC and district users both receive notification by email when files have successfully
been loaded into the data warehouse. Notification occurs by email following the next weekend run
of the D3A2 post data load procedure. The email message includes a directly link to a scorecard for
the load.
More Info: both the ITC and district users receive notifications whenever (1) the minimum
threshold of rows in error is exceeding during loading of a file (typically this 25% or more), or (b) a
critical error is encountered with a file that stalls processing.
Note: Appendix B of this Guide provides a summary of Loader email notifications.
Load Prerequisites
Rule Sequence Rule Description
10 District must be authorized to be included in D3A2 processing. 20 District must have no unresolved load errors.
30 District must have no other loads in progress.
40 Sequential load types for district must be completed for demographics
files.
50 District must have no prior load types pending.
60 Multiple instantiations for a load type for a given district must be
processed in the order in which they were received.
70 District must have no loads from prior school years pending.
80 Processing for prior school year demographics must be completed for
the given district.
7/30/2019 Data Loader User Guide (2010-2011)
25/46
Loading District Demographics Files
13 | P a g e
Evaluating Results of Successful Demographics Loads (Step-By-Step)
The Status DetailsCompleted Files list is used to evaluate the relative completeness of successful
file loads.
More Info: in this Guide, loads are said to be successfulwhen they have been processed
through to the Data Warehouse. This terminology does not consider how may rows were excluded
from loading due to failed validations.
1.) Open the Status DetailsCompleted Files list by clicking .
Note: each of the demographic file loads will show multiple times since they are typically
processed through the Loader more than once per school year. Scroll to the Start Time and End
Time columns to find the most recently loaded version of the file you are interested in researching.
2.) Check to see that an appropriately high percentage of rows were either inserted or updatedversus being in error.
Figure 22: Completed Loads List
Caution:if a very large percentage of rows are in error, then the data that is stored in the Data
Warehouse is not accurate
Note: create a CA-Unicenter ticket assigned to the SSDT.D3A2.SUPPORT request area asking
that the D3A2 Operations Group evaluate and purge a file from the Data Warehouse which has not
loaded properly or displays a large proportion of failed to successful rows.
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
26/46
Data Driven Decisions for Academic Achievement
14 | P a g e
3.) If necessary, click on the links under the Rows in Errorto display a row-by-row account of
records that were not uploaded to the Data Warehouse and the reasons they failed.
Figure 23: Failed Validation (Rows in Error) Report
Note: the Failed Validation Report is valuable in diagnosing the cause for failed records, as well
as researching the cause of missing data in the D3A2 Data Tool.
More Info: clicking on the links under the Rows Inserted column displays a Load Rule
Processing Report which the D3A2 Operations Team uses to evaluate failed loads.
Correcting Errors in the Extract Files
Errors found in extract files typically originate from inaccurate entries in the districts student
information systems. Severe problems in extract files will trigger fatal errors in the Loader which
prevents them from being processed. Such errors are typically resolved by correcting data in the
SIS, re-extracting the D3A2 demographics files, and re-processing through the Loader.
The D3A2 Operations Staff can assist in diagnosing the cause of load errors. Occasionally errors
can be repaired directly in an extract file to allow it to pass the Loaders validation rules.
7/30/2019 Data Loader User Guide (2010-2011)
27/46
7/30/2019 Data Loader User Guide (2010-2011)
28/46
Data Driven Decisions for Academic Achievement
16 | P a g e
5.) Use the Scoping Tool to narrow the view to
the correct district.
6.) Navigate to the file list and ob-
serve the high-level scrubbing results for the
test file(s).
Note: hovering over the testing file status
indicator instantly provides a display of the
numbers of unmatched teacher crosswalk
records, unmatched student crosswalk records,
and null teacher IDs in the testing data.
7.) If the number of unresolved IDs warrants
closer inspection, navigate to the Student
Scores Data page and thoroughly review the
testing data in the Scrubber.
8.) If additional scrubbing is necessary, click
from the Student Scores Data or
Item Analysis Data page and then Contin-
ue to back the file out of the Loaders Files
on Holdlist or .
Figure 27: Testing Files on "Hold" Status
Figure 28: Reviewing Scrubbing Results
Figure 26: Verify Scoping Selection
Figure 29: Confirm Testing File Recall
7/30/2019 Data Loader User Guide (2010-2011)
29/46
Loading Testing Files
17 | P a g e
9.) Release testing file(s) for processing by
check marking them and clicking
while viewing Status
DetailsFiles on Holdlist.
More Info: two files are forwarded to the Files on Hold list when districts finalize and forward
student scores filesTest Scores andTest Part Scores.
10.) The Loader will move the files from the list to the list.
11.) Check to make sure that there are no critical errors reported on the Status Details pages
Files in Errorlist .
Caution:the Loader will not process any additional files for a district when a previously loaded
file has been held up in the Files in Error list. ITC staff must clear any critical errors before loading of
additional files can proceed.
Issues Affecting how Unresolved Testing Data is Processed
Unresolved testing records are handled differently during loading depending upon the type
of problem:
Records with null teacher IDs are resolved to the unknown teacher. Records with null student IDs are excluded from loading into the warehouse. Records with unmatched student or teacher crosswalk entries are excluded from loading
into the warehouse
Note: this could represent a significant amount of data if the unmatched teacher taught a
large number of students.
Records with unknown teacher IDs, i.e. IDs that are not represented in the districts EMISor SIS data, are resolved to the unknown teacher.
Figure 30: Release Testing Files for Processing
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
30/46
Data Driven Decisions for Academic Achievement
18 | P a g e
Monitoring Testing File Loads (Step- By-Step)
The Status Details page is used to monitor the progress of testing file loads.
1.) Log into the Data Loader application using an ITC-level account.
2.) Navigate to the Status Details page.
3.) Use the Scoping Tool to narrow the view
down to a single district.
4.) Check to make sure that there are no critical errors reported on the Status Details pages Files
in Errorlist .
5.) Click on and observe that
the testing files have transferred from the
Student Scores Dataor Item Analysis
Datapage to the Loader.
6.) The Loader automatically begins processing all files sequentially by order of load type as
indicated by the Processing or Waiting to be Processed progress indicators. Processing will
occur automatically unless unmet prerequisites or critical errors prevent files from being acted
upon.
Figure 31: Navigate to Status Details
Figure 32: Verify Scoping Selection
Figure 33: Monitor Progress Using Status Details
7/30/2019 Data Loader User Guide (2010-2011)
31/46
Loading Testing Files
19 | P a g e
7.) Observe that the testing files are eventually processed out of the list into
.
More Info: ITC and district users both receive notification by email when files have successfully
been loaded into the data warehouse. Notification occurs following the next weekly run of the D3A2
post data load procedure and provides a link to a scorecard for the load.
More Info: both the ITC and district users receive notifications whenever (1) the minimum
threshold of rows in error is exceeding during loading of a file (typically 25% or more), or (b) a critical
error is encountered with the file that stalls processing.
Load Prerequisites
Rule Sequence Rule Description
10 District must be authorized to be included in D3A2 processing.
20 District must have no unresolved load errors.
30 District must have no other loads in progress.
40 Sequential load types for district must be completed for demographics
files.
50 District must have no prior load types pending.
60 Multiple instantiations for a load type for a given district must be
processed in the order in which they were received.
70 District must have no loads from prior school years pending.
80 Processing for prior school year demographics must be completed for
the given district.
7/30/2019 Data Loader User Guide (2010-2011)
32/46
Data Driven Decisions for Academic Achievement
20 | P a g e
Evaluating Results of Successful Testing File Loads (Step-By-Step)
The Status DetailsCompleted Files list is used to evaluate the relative completeness of successful
file loads.
1.) Open the Status DetailsCompleted Files list by clicking .
2.) Check to see that an appropriately high proportion of rows were either inserted or updated
versus being in error.
Figure 34: Completed Loads List
Caution:if a very large percentage of rows are in error, then the data contained in the Data
Warehouse is not accurate
Note: create a CA-Unicenter ticket assigned to the SSDT.D3A2.SUPPORT request area asking
that the D3A2 Operations Group evaluate and purge a file from the Data Warehouse which has not
loaded properly or displays a large proportion of failed to successful rows.
http://upload.wikimedia.org/wikipedia/commons/thumb/2/24/Warning_icon.svg/420px-Warning_icon.svg.png7/30/2019 Data Loader User Guide (2010-2011)
33/46
Loading Testing Files
21 | P a g e
3.) If necessary, click on the links under the Rows in Errorto display a row-by-row account of
records that were not uploaded to the Data Warehouse and the reasons they failed.
Figure 35: Failed Validation (Rows in Error) Report
Note: the Failed Validation Report is valuable in diagnosing the cause for failed records, as well
as for determining exactly which records were excluded from the load.
More Info: clicking on the links under the Rows Inserted column displays a Load Rule
Processing Report which the D3A2 Operations Team uses to evaluate failed loads.
Correcting Errors in Testing File Loads
Errors found in testing files typically originate from the Scrubber. Such errors must be reported
to and resolved by the D3A2 Operations Staff.
More Info: submit a CA-Unicenter ticket to the SSDT.D3A2.SUPPORT request area to report
critical errors preventing the loading of testing files.
7/30/2019 Data Loader User Guide (2010-2011)
34/46
7/30/2019 Data Loader User Guide (2010-2011)
35/46
23 | P a g e
Using the Loader Activity and
Unmet Load Prerequisites Pages
The Loader Activity and Unmet Load Prerequisitespages provide at a glance information to
ITCs about Loader activity and status.
The Loader Activity page shows a listing of districts most recent loading activity. It also
provides access to a Score Cardreport for each load. The Score Carddetails the number of rows
in the raw data, the number not loaded, and the number loaded in each file.
The Unmet Prerequisites report provides information about files that have queued for loading
but cannot be acted upon due to unmet prerequisites.
Use the Loader Activity Page to Review Load Score Sheets (Step-By-Step)
The Status Details page is used to monitor the progress of district demographic file loads and
view load score cards.
1.) Log into the Data Loader application using an ITC-level account.
2.) Navigate to the Status Details page.
3.) If necessary, use the Scoping Tool to
narrow the view down to a specific district.
Figure 36: Navigate to Loader Activity
Figure 37: Verify Scoping Selection
7/30/2019 Data Loader User Guide (2010-2011)
36/46
Data Driven Decisions for Academic Achievement
24 | P a g e
4.) Review all recent Loading activity for your
districts.
5.) Click on a file to open its Score Sheet.
6.) Review the score sheet from the latest demo-
graphics or testing file load or ...
Figure 39: Click on Row to Access Score Sheet
Figure 40: Review Latest Score Card Information
Figure 38: Review Recent Loading Results
7/30/2019 Data Loader User Guide (2010-2011)
37/46
Loading Testing Files
25 | P a g e
7. Click on a blue demographic file link
(when available) to expose complete
loading history for the selected file
during any academic reporting period.
Note: score card results for multiple demographics loads performed during a given academicperiod are able to be listed in chronological order on this report.
Note: links are not provided if a demographic file has only been loaded once for the academic
term.
More Info: the Added for Groups column only applies to general (student scores) testing files.
It indicates how many scores were duplicated to create standards reporting groups in the subjects of
Science and Social Studies.
8.) Click on a number link under the Not
Loadedcolumn to display the Failed Valida-
tion report for the load.
Figure 41: Analyze All Score Card Information
Figure 42: Failed Validation Report
7/30/2019 Data Loader User Guide (2010-2011)
38/46
7/30/2019 Data Loader User Guide (2010-2011)
39/46
Appendix A: User Permissions
7/30/2019 Data Loader User Guide (2010-2011)
40/46
7/30/2019 Data Loader User Guide (2010-2011)
41/46
User Permissions
The following table shows all permissions based upon user role.
Menu Page Subpage Function User Role
District ITC Administrator
Password Change Password Yes Yes Yes
Administration Access Data User Maintenance Create New User Yes YesAdministration Access Data User Details Delete User Yes Yes
Administration Access Data User Details Edit User Yes Yes
Administration Access Data District Maintenance Create New District Yes
Administration Access Data District Details Edit District Yes
Administration Access Data District Details Delete District Yes
Administration Loader Control Load Rule Maint Create Load Rule Yes
Administration Loader Control Load Rule Maint Edit Load Rule Yes
Administration Loader Control Load Rule Maint Disable Load Rule Yes
Administration Loader Control Prerequ Rule Maint Create Prerequisite Yes
Administration Loader Control Prerequ Rule Maint Edit Prerequisite Yes
Administration Loader Control Prerequ Rule Maint Delete Prerequisite Yes
Administration Loader Control Prerequ Rule Maint Disable Prerequisite Yes
Administration Loader Control Load Type Maint Edit Load Type Yes
Administration System Control Disable Sched Job Yes
Administration System Control Update Base URL Yes
Administration User Statistics View Activity Stats Yes
Administration Data Mapping Test Part Mapping Create New Mappings Yes
Administration Data Mapping Test Part Mapping Edit Mappings Yes
Administration Data Mapping Test Part Mapping Delete Mappings Yes
Administration Data Mapping Vendor Layout Map View Layout Maps Yes
Data Scrubber Vendor File Upload Upload Test File Yes Yes Yes
Data Scrubber Vendor File Upload Start Scrubbing Yes Yes Yes
Data Scrubber Stud Score Data Finalize & Forward Yes Yes Yes
Data Scrubber Stud Score Data Recall Yes Yes Yes
Data Scrubber Stud Score Data View Verification Yes Yes Yes
Data Scrubber Stud Score Data Teach X-Walk Match Edit Crosswalk Record Yes Yes Yes
Data Scrubber Stud Score Data Teach X-Walk Match Regenerate X-Walk Yes Yes Yes
Data Scrubber Stud Score Data Teacher X-Walk Mat Run BMW Yes Yes Yes
Data Scrubber Stud Score Data Edit Stud Score Dtail Edit Score Record Yes Yes Yes
Data Scrubber Stud Score Data Edit Stud Score Dtail Delete Score Record Yes Yes Yes
Data Scrubber Stud Score Data Advance Rpt Filter Mass Update Records Yes Yes Yes
Data Scrubber Stud Score Data Stud X-Walk Match Edit Crosswalk Record Yes Yes Yes
Data Scrubber Stud Score Data Stud X-Walk Match Regenerate X-Walk Yes Yes Yes
Data Scrubber Stud Score Data Stud X-Walk Match Upload ID_SSID File Yes Yes Yes
Data Scrubber Item Analysis Data Teach X-Walk Match Edit Crosswalk Record Yes Yes Yes
Data Scrubber Item Analysis Data Teach X-Walk Match Regenerate X-Walk Yes Yes Yes
Data Scrubber Item Analysis Data Teacher X-Walk Mat Run BMW Yes Yes Yes
Data Scrubber Item Analysis Data Edit Stud Score Dtail Edit Score Record Yes Yes Yes
Data Scrubber Item Analysis Data Edit Stud Score Dtail Delete Score Record Yes Yes Yes
Data Scrubber Item Analysis Data Edit Item Detail Edit IA Record Yes Yes Yes
Data Scrubber Item Analysis Data Edit Item Detail Delete IA Record Yes Yes Yes
Data Loader Loader Activity Score Sheet View Score Sheet Yes Yes Yes
Data Loader Loader Activity Score Sheet View Fail Valid Rpt Yes Yes Yes
Data Loader Status Details Completed Purge Files Yes
Data Loader Status Details Completed View Fail Valid Rpt Yes Yes Yes
Data Loader Status Details Completed View Fail Process Rpt Yes Yes Yes
Data Loader Status Details Errors Purge Files Yes
Data Loader Status Details Errors Reprocess Files Yes Yes
Data Loader Status Details Hold Release Files Yes Yes
Data Loader Status Details Released View Unmet Prreqs Yes Yes Yes
7/30/2019 Data Loader User Guide (2010-2011)
42/46
7/30/2019 Data Loader User Guide (2010-2011)
43/46
Appendix B: Notifications
7/30/2019 Data Loader User Guide (2010-2011)
44/46
7/30/2019 Data Loader User Guide (2010-2011)
45/46
Load Errors
Recipients:mail_staff_d3a2@nwoca.org,mail_staff_d3a2@jamesgrp.com, ITC Users
Subject: D3A2 Loader Error IRN
An error was encountered while loading for IRN , school year .
The error reported was :
The file ID is :
Prerequisite Errors
Recipients:mail_staff_d3a2@nwoca.org,mail_staff_d3a2@jamesgrp.com,
ITC Users, District Users
Subject: D3A2 Prerequisite Notice IRN
could not be loaded for IRN , school
year
A critical prerequisite failed. The reported prerequisite was :' ;
-
A hold has been placed on the file.
The file ID is :
Purge Notification
Recipients:mail_staff_d3a2@nwoca.org,mail_staff_d3a2@jamesgrp.com,
ITC Users, District Users
Subject: D3A2 Purge Notice IRN
for IRN , school year
has been purged from the Loader and, if applicable, the D3A2
warehouse.
mailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.org7/30/2019 Data Loader User Guide (2010-2011)
46/46
Data Driven Decisions for Academic Achievement
Post Data Load
Recipients:mail_staff_d3a2@nwoca.org,mail_staff_d3a2@jamesgrp.com,
ITC Users, District Users
Subject: Score Sheet Update
You have new data in D3A2. Please review the activity report and therelated scoresheets.
http://
Threshold Exceeded
Recipients:mail_staff_d3a2@nwoca.org,mail_staff_d3a2@jamesgrp.com,
ITC Users, District Users
Subject: D3A2 Loader Error IRN
An error was encountered while loading Student Information for IRN 044875, schoolyear 10L.
The error reported was :
Oracle Exception encountered
Error code
Error text ORA-12899:
".""."" (, )
The file ID is :
mailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.orgmailto:mail_staff_d3a2@jamesgrp.commailto:mail_staff_d3a2@nwoca.org
top related