大數(shù)據(jù)培訓(xùn)之測(cè)試
1 Jar包準(zhǔn)備
1) 將MySql驅(qū)動(dòng)包放入Flume的lib目錄下
[atguigu@hadoop102 flume]$ cp \
/opt/sorfware/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar \
/opt/module/flume/lib/
2) 打包項(xiàng)目并將Jar包放入Flume的lib目錄下
2 配置文件準(zhǔn)備
1）創(chuàng)建配置文件并打開(kāi)
[atguigu@hadoop102 job]$ touch mysql.conf
[atguigu@hadoop102 job]$ vim mysql.conf
2）添加如下內(nèi)容
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = com.atguigu.source.SQLSource?
a1.sources.r1.connection.url = jdbc:mysql://192.168.9.102:3306/mysqlsource
a1.sources.r1.connection.user = root?
a1.sources.r1.connection.password = 000000?
a1.sources.r1.table = student?
a1.sources.r1.columns.to.select = *?
#a1.sources.r1.incremental.column.name = id?
#a1.sources.r1.incremental.value = 0
a1.sources.r1.run.query.delay=5000
# Describe the sink
a1.sinks.k1.type = logger
# Describe the channel
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
3 MySql表準(zhǔn)備
1) 創(chuàng)建MySqlSource數(shù)據(jù)庫(kù)
CREATE DATABASE mysqlsource；
2) 在MySqlSource數(shù)據(jù)庫(kù)下創(chuàng)建數(shù)據(jù)表Student和元數(shù)據(jù)表Flume_meta
CREATE TABLE `student` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `flume_meta` (
`source_tab` varchar(255) NOT NULL,
`currentIndex` varchar(255) NOT NULL,
PRIMARY KEY (`source_tab`)
);
- 向數(shù)據(jù)表中添加數(shù)據(jù)
1 zhangsan
2 lisi
3 wangwu
4 zhaoliu
4? 測(cè)試并查看結(jié)果
- 任務(wù)執(zhí)行
[atguigu@hadoop102 flume]$ bin/flume-ng agent --conf conf/ --name a1 \
--conf-file job/mysql.conf -Dflume.root.logger=INFO,console
- 結(jié)果展示，如圖6-2所示：
想要了解跟多關(guān)于大數(shù)據(jù)培訓(xùn)機(jī)構(gòu)課程內(nèi)容歡迎關(guān)注尚硅谷大數(shù)據(jù)培訓(xùn)機(jī)構(gòu)，尚硅谷除了這些技術(shù)文章外還有免費(fèi)的高質(zhì)量大數(shù)據(jù)培訓(xùn)機(jī)構(gòu)課程視頻供廣大學(xué)員下載學(xué)習(xí)
