Hadoop Edit by Cassell Hsu 2013.04.19
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
HadoopEdit by Cassell Hsu
2013.04.19
Hadoop
Master
Slave
Slave
Slave
Hadoop•Master•NameNode•JobTracker•SecondaryNameNode
NameNode•Where is NameNode?•Master•HDFS(Hadoop Distributed File
System)•What is NameNode?•資料之位置資訊 •資料之屬性
？？
NameNode
•位置資訊 ?•所有資料皆存放在 – DataNode
DataNode•What is DataNode•存放資料
•Where is DataNode•HDFS•Slaves (and Master)
UserUser
DataNodeDataNode
NameNodeNameNode
DataNodeDataNode
DataNodeDataNode
128Mb
B64Mb
A64Mb
Check hdfs-site.xml
B64Mb
A64Mb
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
DataNodeDataNode
DataNodeDataNode
Slaves
Hadoop•Master•NameNode•JobTracker•SecondaryNameNode
JobTracker•What is JobTracker?•排程工作
•Where is JobTracker?•Master
誰來工作？
JobTracker & TaskTrackerJobTracker TaskTracker
Where Master Slaves
What 排程工作 執行工作
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
Slaves
JobTrackerJobTracker
TaskTrackerTaskTracker
Hadoop•Master•NameNode•JobTracker•SecondaryNameNode
SecondaryNameNode
•What is SecondaryNameNode?•NameNode發生錯誤時補救
•Where is SecondaryNameNode?•Master
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
Slaves
JobTrackerJobTracker
TaskTrackerTaskTracker
SecondaryNodeSecondaryNode
MapReduce
User Master
Slave2
Slave1
A
A1
A2Task
NameNode
Task
Result1Result2
ReduceFinal
ResultHDFS
MapReduce
•檔案切割•Hadoop上區塊切割•程式指定
Related Documents
