This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1. Databases and Database Management Systems (Based on Chapters
1-2) 1
2. DBMS concepts and architecture ER model Relational Databases
Relational Algebra Query Languages (SQL) Database Design :
Normalization and Functional Dependencies Storage and Indexing
Emerging Trends in Database Technology Web Data management, XML,
Web mining, 2
3. 1. Basic DefinitionsDatabase: A collection of related
data.Data: Known facts that can be recorded and have an implicit
meaning.Mini-world: Some part of the real world about which data is
stored in a database. For example, consider student names, student
grades and transcripts at a university. 3
4. Approximation The data stored in a database often tries to
capture some aspect of the real-world in an approximate manner
Often not all real-world information is (or can be) captured
Generally, only the information, which is estimated to be needed
for decision-making, is recorded in the database 4
5. Summarization Sometimes, data could be stored in a database
in a summarized form Example1: In a student database, the grades
column provides a summary It does not record explicitly what the
student scored in mid-sems, end-sems, labs etc Example 2: In a
database containing sales info, the number of units (of a product)
sold may be shown as a single aggregated number That is, it may not
record explicitly how many units were sold during each hour, but
only records the total sales for the whole day 5
6. Summarization, materiality and usefulness The extent to
which you summarize the data largely depends on the intended use of
the database Example: If you just want to know a students grades,
recording the summarized grade is adequate But if you also want to
know whether the student is good in theory or in lab work, you
would need to record theory and lab grades separately Materiality
criteria: Is recording the data at a certain level of detail
relevant to decision-making (i.e., intended use of the data)? Too
much detail may not be required, and could even be
counter-productive e.g., clouding decision-making 6
7. Semantics of the data Semantics of the data: What does the
data really mean? Sometimes, by looking at a column name e.g.,
student name, you will know that this column contains student names
But assume a column name called student grades This does not really
tell you how the grades were calculated, and a grade of B
corresponds to what absolute score etc 7
8. Semantics of the data Now assume a column called Total sales
in a sales database This does not really tell you whether the
information is about total sales for 1 day, 1 month or 1 year Are
the total sales figures about the sales done only within India, or
this is about worldwide sales? These total sales figures pertain to
which month? Does this include/exclude the effect of refunds
(returned products e.g., due to defects)? Do these sales figures
correspond to some marketing campaign or sales strategy of the
company? (Example: If you offer 50% discount, you are highly likely
to sell more than if you offer 2% discount) What percentage of the
sales were done in bulk e.g., to suppliers instead of individual
consumers? 8
9. Semantics of the data Bottomline: To be able to query and
use the data, you need to understand clearly what the data means,
the context surrounding the data and the assumptions associated
with the data Without understanding the semantics of the data, you
cannot figure out and interpret the meaning of the query results
that the database will provide to you 9
10. Database Management System (DBMS): A software package/
system to facilitate the creation and maintenance of a computerized
database.It defines (data types, structures, constraints) construct
(storing data on some storage mediumcontrolled by DBMS) manipulate
(querying, update, report generation) databases for various
applications.Database System: The DBMS software together with the
data itself. Sometimes, the applications are also included. 10
11. 2. Example of a Database (Conceptual Data Model)Mini-world
for the example: Part of a UNIVERSITY environment. Some mini-world
entities (Data elements):- STUDENTs- COURSEs- SECTIONs (of
COURSEs)- (academic) DEPARTMENTs- INSTRUCTORs Some mini-world
relationships:- SECTIONs are of specific COURSEs- STUDENTs take
SECTIONs- COURSEs have prerequisite COURSEs- INSTRUCTORs teach
SECTIONs- COURSEs are offered by DEPARTMENTs- STUDENTs major in
DEPARTMENTs 11
12. Figure 1.1: A simplified database system
environment,illustrating the concepts and terminology discussed in
Section 1.1 12
13. Figure 1.2: An example of a database that stores student
records and their grades. 13
14. File Processing and DBMSFile Systems : Store data over long
periods of time Store large amount of dataHowever : No guarantee
that data is not lost if not backed up No support to query
languages No efficient access to data items unless the location is
known Application depends on the data definitions (structures)
Change to data definition will affect the application programs
Single view of the data Separate files for each application Limited
control to multiple accesses - Data viewed as physically stored
14
15. 3. Main Characteristics of Database Technology-
Self-contained nature of a database system: A DBMS catalog stores
the description (structure, type, storage format of each entities)
of the database. The description is called meta-data). This allows
the DBMS software to work with different databases.- To understand
meta-data, think of a web-page which contains certain tags-
Example: A webpage containing a UNIX tutorial could have tags
related to some of the UNIX commands 15
16. Meta-data- Meta-data is essentially data about the data -
Often meta-data is a set of keywords which try to approximately
describe the data- Meta-data is often used for facilitating search
16
17. Main characteristics (Cont.) Insulation between programs
and data: Called program-data independence. Allows changing data
storage structures and operations without having to change the DBMS
access programs. You will see this concept of independence
throughout CS in general The whole idea of creating layers is to
ensure that whatever you change in one layer will have no effect on
the other layer 17
18. Main characteristics (Cont.) Data Abstraction: A data model
is used to hide storage details and present the users with a
conceptual view of the database; does not include how data is
stored and how the operations are implemented. Now can anyone tell
me why data abstraction is important? Answer is on the next slide
18
19. Why is data abstraction important? We do not want the user
to be burdened with all the details of how the data is being
actually stored, which data structures are being used etc If we did
not have any data abstraction, databases would not be very
usable/consumable because we cannot expect every user to go through
this learning curve of understanding data structures, storage,
indexing etc before they can even start to use a database Users of
databases come from diverse backgrounds i.e., they are not
necessarily only CS majors 19
20. Support of multiple views of the data: Each user may see a
different view of the database, which describes only the data of
interest to that user. This could sometimes relate to access
control Example: The professor is allowed to see all student
grades, but each student can only see his own grade from the
database Sharing of Data and Multiple users 20
21. Figure 1.3: Internal storage format for a STUDENT RECORD
21
22. Figure 1.4:Two views derived from the example database
shown in Figure 1.2 (a) The student transcript view. (b) The course
prerequisite view. 22
23. DBA Database Administrator- Responsible for authorizing
access to the database, coordinating, monitoring its use, acquiring
hardware, software needed.Database designers- Responsible for
identifying the data to be stored, storage structure to represent
and store data. This is done by a team of professionals in
consultation with users, and applications needed. 23
24. Type of data stored: Temporal angle- The type of data to be
stored often changes over time- For example: The amount of data
stored now per customer interaction is much more than the amount of
data that used to be stored 10 years ago- Storing more data about a
given user allows for more personalized services (some privacy
constraints exist)- In practice, as more personalization becomes
desirable, you need to store data at a more detailed level 24
25. 4. Additional Benefits of Database Technology- Controlling
redundancy in data storage and in development and maintenance
efforts.- Sharing of data among multiple users.- Restricting
unauthorized access to data.- Providing multiple interfaces to
different classes of users.- Representing complex relationships
among data.- Enforcing integrity constraints on the database.-
Providing backup and recovery services.- Potential for enforcing
standards.- Flexibility to change data structures.- Reduced
application development time.- Availability of up-to-date
information. Economies of scale. 25
26. Figure 1.5: The redundant storage of Data items. (a)
ControlledRedundancy: Including StudentName and CourseNumber in the
grade_report file. (b) Uncontrolled redundancy: A GRADE_REPORT
record that is inconsistent with the STUDENT records in Figure 1.2,
because the Name of student number 17 is Smith, not Brown. 26
27. 5 When not to use a DBMS Main inhibitors (costs) of using a
DBMS:- High initial investment and possible need for additional
hardware.- Overhead for providing generality, security, recovery,
integrity, and concurrency control. When a DBMS may be
unnecessary:- If the database and applications are simple, well
defined, and not expected to change.- If there are stringent
real-time requirements that may not be met because of DBMS
overhead.- If access to data by multiple users is not required.
When no DBMS may suffice: - If the database system is not able to
handle the complexity of data because of modeling limitations- If
the database users need special operations not supported by the
DBMS. 27
28. 6. Data ModelsData Model: A set of concepts to describe the
structure (data types, relationships) of a database, and certain
constraints that the database should obey.Data Model Operations:
Operations for specifying database retrievals and updates by
referring to the concepts of the data model. 28
29. Categories of data models:- Conceptual (high-level,
semantic) data models: Provide concepts that are close to the way
many users perceive data. (Also called entity-based or object-based
data models.) - Physical (low-level, internal) data models: Provide
concepts that describe details of how data is stored in the
computer.- Implementation (record-oriented) data models: Provide
concepts that fall between the above two, balancing user views with
some computer storage details. 29
30. 6A. HISTORY OF DATA MODELS Relational Model: proposed in
1970 by E.F. Codd (IBM), first commercial system in 1981-82. Now in
several commercial products (ORACLE, SYBASE, INFORMIX, INGRES).
Network Model: the first one to be implemented by Honeywell in
1964-65 (IDS System). Adopted heavily due to the support by CODASYL
(CODASYL - DBTG report of 1971). Later implemented in a large
variety of systems - IDMS (Cullinet - now CA), DMS 1100 (Unisys),
IMAGE (H.P.), VAX -DBMS (Digital). Hierarchical Data Model :
implemented in a joint effort by IBM and North AmericanRockwell
around 1965. Resulted in the IMS family of systems. The most
popular model.Other system based on this model: System 2k (SAS
inc.) 30
31. Object-oriented Data Model(s) : several models have been
proposed for implementing in a database system. One set comprises
models of persistent O-O Programming Languages such as C++ (e.g.,
in OBJECTSTORE or VERSANT), and Smalltalk (e.g., in GEMSTONE).
Additionally, systems like O2, ORION (at MCC - then ITASCA), IRIS
(at H.P.- used in Open OODB). Object-Relational Models : Most
Recent Trend. Exemplified in ILLUSTRA and UNiSQL systems. 31
32. Figure 2.1: Schema diagram for the database of Figure 1.2
32
33. 7. Schemas versus InstancesDatabase Schema: The description
of a database. Includes descriptions of the database structure and
the constraints that should hold on the database.Schema Diagram: A
diagrammatic display of (some aspects of) a database
Schema.Database Instance: The actual data stored in a database at a
particular moment in time . Also called database state (or
occurrence).The database schema changes very infrequently . The
database state changes every time the database is updated . Schema
is also called intension, whereas state is called extension.
33
34. 8. Three-Schema ArchitectureProposed to support DBMS
characteristics of:- Program-data independence.- Support of
multiple views of the data.Defines DBMS schemas at three levels :-
Internal schema at the internal level to describe datastorage
structures and access paths. Typically uses a physicaldata model.-
Conceptual schema at the conceptual level to describe thestructure
and constraints for the whole database. Uses aconceptual or an
implementation data model.- 34
35. External schemas at the external level to describe the
various user views. Usually uses the same data model as the
conceptual level.Mappings among schema levels are also needed.
Programs refer to an external schema, and are mapped by the DBMS to
the internal schema for execution. 35
36. Figure 2.2: Illustrating the three-schema architecture
36
37. 9 Data IndependenceLogical Data Independence: The capacity
to change the conceptual schema without having to change the
external schemas and their application programs.Physical Data
Independence: The capacity to change the internal schema without
having to change the conceptual schema.When a schema at a lower
level is changed, only the mappings between this schema and
higher-level schemas need to be changed in a DBMS that fully
supports data independence. The higher- level schemas themselves
are unchanged.Hence, the application programs need not be changed
since they refer to the external schemas. 37
38. 10. DBMS LanguagesData Definition Language (DDL): Used by
the DBA and database designers to specify the conceptual schema of
a database.In many DBMSs, the DDL is also used to define internal
and external schemas (views). In some DBMSs, separate storage
definition language (SDL) and view definition language (VDL) are
used to define internal and external schemas.Data Manipulation
Language (DML): Used to specify database retrievals and updates. -
DML commands (data sublanguage) can be embedded in a
general-purpose programming language (host language), such as
COBOL, PL/1 or PASCAL. - Alternatively, stand-alone DML commands
can be applied directly (query language). 38
39. High Level or non-Procedural DML Describes what data to be
retrieved rather than how to retrieve. - Process many records at a
time - SQL Low Level or Procedural DML It needs constructs for
both, what to retrieve and what to retrieve - Uses looping etc.
like programming languages Only access one record at a time 39
40. 11. DBMS Interfaces-Stand-alone query language interfaces.-
Programmer interfaces for embedding DML in programminglanguages:-
Pre-compiler Approach- Procedure (Subroutine) Call Approach-
User-friendly interfaces:- Menu-based- Graphics-based (Point and
Click, Drag and Drop etc.)- Forms-based- Natural language-
Combinations of the above- Speech as Input (?) and Output- Web
Browser as an interface- 40
41. Parametric interfaces using function keys. - Report
generation languages. - Interfaces for the DBA: - Creating
accounts, granting authorizations - Setting system parameters -
Changing schemas or access path 41
42. Figure 2.3: Typical component modules of a DBMS. Dotted
linesshow accesses that are under the control of the stored data
manager. 42
43. 13. Database System Utilities To perform certain functions
such as: - Loading data stored in files into a database. - Backing
up the database periodically on tape. - Reorganizing database file
structures. - Report generation utilities. - Performance monitoring
utilities. - Other functions, such as sorting , user monitoring ,
data compression , etc. Data dictionary / repository: - Used to
store schema descriptions and other information such as design
decisions, application program descriptions, user information,
usage standards, etc. - Active data dictionary is accessed by DBMS
software and users/DBA. 43- Passive data dictionary is accessed by
users/DBA only.
44. 14. Classification of DBMSs Based on the data model used: -
Traditional: Relational, Network, Hierarchical. - Emerging:
Object-oriented, Object-relational. Other classifications: -
Single-user (typically used with micro- computers) vs. multi-user
(most DBMSs). - Centralized (uses a single computer with one
database) vs. distributed (uses multiple computers, multiple
databases)Distributed Database Systems have now come to be known as
client server based database systems because they do not support a
totally distributed environment, but rather a set of database
servers supporting a set of clients. 44
45. Weekly assignment Go through the lecture slides thoroughly
Your TA will send you a link to inform you where to download the
lecture slides Reading assignment: Read Chapters 1-2 of your
textbook If you have any doubts/questions, pls let me know 45
46. Weekly assignment Remember that in this course, you will
often need to understand the concepts of Chapter N before you can
deal with Chapter N+1 Do not fall behind the class 46
47. Weekly assignment (Cont.) Refer to the following:
http://www2.sims.berkeley.edu/research/projects/how-much-info-
2003/ This website provides information about how much data exists
in the world, and will broaden your perspective on data management
in general Reading through the entire website is not required for
this course, but I want all of you to read through the Summary of
Findings
http://www2.sims.berkeley.edu/research/projects/how-much-info-
2003/execsum.htm While reading, pay particular attention to Section
5, where the authors discuss issues concerning assumptions and
estimates Questions about the exact numbers will not be asked in
your exams, but you should be prepared to answer questions about
the general material in the Summary of Findings e.g., 47
assumptions, estimates etc
48. Weekly assignment After reading through the Summary of
Findings, prepare a 1-2 page report to indicate what you learned by
reading through the material Submit the report to your course TA in
hard-copy format latest by Jan 22, 5 pm IST No submissions will be
accepted after the deadline 48
49. Weekly assignment You can choose to format the report in
any way you like Focus on the content of the report, not on minor
details such as font size, what font to use etc The report will be
graded based on content, hence mention only the key points You will
get full grade in this assignment as long as you mention some key
points that you learned Printouts are preferable, but handwritten
reports are also ok as long as your handwriting is understandable
49