- 1. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Hadoop Setting Eunsil Yoon
2. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Linux Setting (version_2012.12) 1. Linux - USB CentOS 2. Hadoop - hadoop-1.0.4.tar - : /usr/local/hadoop-1.0.4 - Hadoop conf - /usr/local/hadoop-1.0.4/conf/hadoop-env.sh - screenshot 1 typing 3. Java - jdk-7u10-linux-x64.rpm - : /usr/java/jdk1.7.0_10 - Java path - /etc/profile - export JAVA_HOME=/usr/java/jdk1.7.0_10 - export PATH=$PATH:$JAVA_HONE/bin - export CLASS_PATH=". . Screenshot 2 3. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Screenshot 1 4. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Screenshot 2 5. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon /etc/hosts 6. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Network setting pc 7. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon /usr/local/hadoop-1.0.4/conf/core-site.xml 8. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon /usr/local/hadoop-1.0.4/conf/hdfs-site.xml 9. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon /usr/local/hadoop-1.0.4/conf/mapred-site.xml 10. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon (wordcount) jar Wordcount (jar) hadoop-env.sh hadoop-env.sh HDFS (screenshot 3 ) ./bin/hadoop fs put conf/hadoop-env.sh conf/hadoop-env.sh jar (screenshot 3 ) conf/hadoop-env.sh, output .bin/hadoop jar hadoop-examples-*.jar wordcount conf/hadoop-env.sh wordcount_output , HDFS (fs cat ) ./bin/hadoop fs cat wordcount_output/part-r-00000 11. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Screenshot 3 12. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Screenshot 4 13. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Linux Setting (version 2013.1) Ubuntu 12.04 LTS Java jdk6u38 , 1.6.0_38 Ubuntu root , . (next page) 2013.01.15 14. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Memory setting /boot 500MB Swap 16384 /usr 15360 / 10240 Var 5120 Tmp 5120 Home 2013.01.15 15. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Network setting pc 2013.01.15 16. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Root 2013.01.15 17. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon ./bin/hadoop namenode -format 2013.01.15 18. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Wordcount 2013.01.15 19. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon wordcount_output/part-r-00000 2013.01.15 20. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon fs(filesystem) lsr 2013.01.15 21. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon yum 2013.01.15 22. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon nabi 2013.01.15 23. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon openssh-server 2013.01.15 24. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Localhost:50070 NameNode 2013.01.15 25. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon NameNode Live 2013.01.15 26. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Eclipse 27. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon eclipse Eclipse , Java project Project name libraries external jar . jar , lib Finish Build.xml , jar Build.xml ant jar 2013.01.15 28. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon New file build.xml 2013.01.15 29. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Build.xml run as 2013.01.15 30. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 81p_HDFS API Blrunner.com , .java . package .java 2013.01.16 31. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Build.xml jar jar java 2013.01.16 32. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon jar workspace , hadoop 2013.01.16 33. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Jar 2013.01.16 34. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Fs ls HDFS 2013.01.16 35. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon -cat input.txt 2013.01.16 36. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Example1_20130119 37. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 1. txt hadoop 38. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 2. Hdfs 39. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 3. Mapreduce 40. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 3-1. mapreduce 41. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 3-2. mapreduce 42. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 4. Mapreduce output 43. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 5. Output Text count Output: _user_medinfo_path_output_part-r- 00000 44. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon WordCountMapper.java package wikibooks.hadoop.chapter04; import java.io.IOException; import java.util.StringTokenizer; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class WordCountMapper extends Mapper { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { StringTokenizer itr = new StringTokenizer(value.toString()); while (itr.hasMoreTokens()) { word.set(itr.nextToken()); context.write(word, one ); } } } 45. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon WordCountReducer.java 46. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon WordCount.java 47. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 48. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Example2. 116p. 130129 49. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon ASA(American Standards Association: ) 2009 http://stat- computing.org/dataexpo/ 2009 1987-2008 50. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon Download the data 51. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 1987.csv (29 ) * 52. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon 22 11.4GB 53. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon HDFS ( 6) 54. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon ./bin/hadoop fs ls input 55. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon DepartureDelayCount (19) 56. Medical Informatics Laboratory Department of Biomedical engineering College of Medicine , Seoul National Univ. Eunsil Yoon (117 ) 30, (1 ) 18 30 . Map input records = 117,161,410 Map output & Reduce input records = 47,765,560 Reduce output records = 245
