成功搭建Hadoop集群后简单实验
2013-02-22 16:07:30 来源:WEB开发网 闂傚倸鍊搁崐鎼佸磹閹间礁纾归柟闂寸绾惧綊鏌熼梻瀵割槮缁炬儳缍婇弻鐔兼⒒鐎靛壊妲紒鐐劤缂嶅﹪寮婚悢鍏尖拻閻庨潧澹婂Σ顔剧磼閻愵剙鍔ょ紓宥咃躬瀵鎮㈤崗灏栨嫽闁诲酣娼ф竟濠偽i鍓х<闁诡垎鍐f寖闂佺娅曢幑鍥灳閺冨牆绀冩い蹇庣娴滈箖鏌ㄥ┑鍡欏嚬缂併劎绮妵鍕箳鐎n亞浠鹃梺闈涙搐鐎氫即鐛崶顒夋晬婵絾瀵ч幑鍥蓟閻斿摜鐟归柛顭戝枛椤牆顪冮妶搴′簼缂侇喗鎸搁悾鐑藉础閻愬秵妫冮崺鈧い鎺戝瀹撲礁鈹戦悩鎻掝伀缁惧彞绮欓弻娑氫沪閹规劕顥濋梺閫炲苯澧伴柟铏崌閿濈偛鈹戠€n€晠鏌嶆潪鎷屽厡闁汇倕鎳愮槐鎾存媴閸撴彃鍓卞銈嗗灦閻熲晛鐣烽妷褉鍋撻敐搴℃灍闁绘挻娲橀妵鍕箛闂堟稐绨肩紓浣藉煐濮樸劎妲愰幘璇茬闁冲搫鍊婚ˇ鏉库攽椤旂》宸ユい顓炲槻閻g兘骞掗幋鏃€鐎婚梺瑙勬儗閸樺€熲叺婵犵數濮烽弫鍛婃叏椤撱垹纾婚柟鍓х帛閳锋垶銇勯幒鍡椾壕缂備礁顦遍弫濠氱嵁閸℃稒鍊烽柛婵嗗椤旀劕鈹戦悜鍥╃У闁告挻鐟︽穱濠囨嚃閳哄啰锛滈梺褰掑亰閸欏骸鈻撳⿰鍫熺厸閻忕偟纭堕崑鎾诲箛娴e憡鍊梺纭呭亹鐞涖儵鍩€椤掑啫鐨洪柡浣圭墪閳规垿鎮欓弶鎴犱桓闂佸湱枪閹芥粎鍒掗弮鍫熷仺缂佸顕抽敃鍌涚厱闁哄洢鍔岄悘鐘绘煕閹般劌浜惧┑锛勫亼閸婃牠宕濋敃鈧…鍧楀焵椤掍胶绠剧€光偓婵犱線鍋楀┑顔硷龚濞咃絿妲愰幒鎳崇喓鎷犻懠鑸垫毐闂傚倷鑳舵灙婵炲鍏樺顐ゆ嫚瀹割喖娈ㄦ繝鐢靛У绾板秹寮查幓鎺濈唵閻犺櫣灏ㄥ銉р偓瑙勬尭濡繂顫忛搹鍦<婵☆垰鎼~宥囩磽娴i鍔嶉柟绋垮暱閻g兘骞嬮敃鈧粻濠氭偣閸パ冪骇鐎规挸绉撮—鍐Χ閸℃ê闉嶇紓浣割儐閸ㄥ墎绮嬪澶嬪€锋い鎺嶇瀵灝鈹戦埥鍡楃仯闁告鍕洸濡わ絽鍟崐鍨叏濡厧浜鹃悗姘炬嫹

1> 检查对HDFS的访问(在namenode上操作)
#./hadoop-0.20.2/bin/hadoop fs -ls
2> hadoop中的文件系统操作
#./hadoop-0.20.2/bin/hadoop fs -mkdir test
#./hadoop-0.20.2/bin/hadoop fs -ls test
#./hadoop-0.20.2/bin/hadoop fs -rmr test
3> 测试hadoop
a> 创建子目录
#./hadoop-0.20.2/bin/hadoop fs -mkdir input
b> 把本地文件移动到hdfs
#./hadoop-0.20.2/bin/hadoop fs -put /usr/share/doc/pam-1.1.1/rfc86.0.txt input
#./hadoop-0.20.2/bin/hadoop fs -put /usr/share/doc/pam-1.1.1/Linux-PAM_SAG.txt input
c> 查看文件是否存在
#./hadoop-0.20.2/bin/hadoop fs -ls input
d> 执行计算单词频率的MapReduce作业
#./hadoop-0.20.2/bin/hadoop jar /root/hadoop-0.20.2/hadoop-0.20.2-examples.jar wordcount input output
Found 2 items
drwxr-xr-x - root supergroup 0 2012-11-26 04:25 /user/root/output/_logs
-rw-r--r-- 2 root supergroup 8315 2012-11-26 04:25 /user/root/output/part-r-00000
e> 查看输出结果
#./hadoop-0.20.2/bin/hadoop fs -ls output
#./hadoop-0.20.2/bin/hadoop fs -cat output/part-r-00000 | head -n 13
f> 从HDFS提取输出
#./hadoop-0.20.2/bin/hadoop fs -get output/part-r-00000 output.txt
更多精彩
赞助商链接