Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
I
ihooyah-flume
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
黄杰
ihooyah-flume
Commits
c9538723
Commit
c9538723
authored
May 14, 2021
by
吴凯波
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
readme更新
parent
9cea6875
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
29 additions
and
29 deletions
+29
-29
README.md
README.md
+29
-29
No files found.
README.md
View file @
c9538723
...
...
@@ -116,44 +116,44 @@
启动命令flume-ng agent --conf-file ../conf/ftp.conf --name agent -property flume.root.logger=INFO,console
<br>
#####flume实战案例二:从ftp环境读取文件存到本地服务器路径
conf文件示例
conf文件示例
<br>
# flume监控ftp文件功能的配置如下
agent.sources = ftp1
agent.sinks = k1
agent.channels = ch1
agent.sources.ftp1.type = org.keedio.flume.source.ftp.source.Source
agent.sources.ftp1.client.source = ftp
agent.sources = ftp1
<br>
agent.sinks = k1
<br>
agent.channels = ch1
<br>
agent.sources.ftp1.type = org.keedio.flume.source.ftp.source.Source
<br>
agent.sources.ftp1.client.source = ftp
<br>
#ftp服务地址
agent.sources.ftp1.name.server = 61.159.214.198
agent.sources.ftp1.name.server = 61.159.214.198
<br>
#端口
agent.sources.ftp1.port = 50021
agent.sources.ftp1.port = 50021
<br>
#ftp用户名
agent.sources.ftp1.user = flume
agent.sources.ftp1.user = flume
<br>
#密码
agent.sources.ftp1.password = sSt23dDmcikTNiw8
agent.sources.ftp1.password = sSt23dDmcikTNiw8
<br>
#文件所在目录 (相对于根目录)
agent.sources.ftp1.working.directory = /
agent.sources.ftp1.working.directory = /
<br>
#文件的名称匹配 (java正则表达式)
agent.sources.ftp1.filter.pattern = .+
agent.sources.ftp1.filter.pattern = .+
<br>
#ftp的根目录
agent.sources.ftp1.folder = /flume
agent.sources.ftp1.run.discover.delay=5000
agent.sources.ftp1.flushlines = false
agent.sources.ftp1.search.recursive = true
agent.sources.ftp1.processInUse = false
agent.sources.ftp1.processInUseTimeout = 30
agent.sources.ftp1.channels = ch1
agent.sources.ftp1.chunk.size = 102400
agent.channels.ch1.type = memory
agent.channels.ch1.capacity = 100000000
agent.channels.ch1.transactionCapacity = 100000000
agent.sinks.k1.type = com.ihooyah.flume.sink.LocalUploadSink
agent.sinks.k1.uploadDir = D:
\\
agent.sinks.k1.batchSize = 10000
agent.sinks.k1.channel = ch1
<br>
agent.sources.ftp1.folder = /flume
<br>
agent.sources.ftp1.run.discover.delay=5000
<br>
agent.sources.ftp1.flushlines = false
<br>
agent.sources.ftp1.search.recursive = true
<br>
agent.sources.ftp1.processInUse = false
<br>
agent.sources.ftp1.processInUseTimeout = 30
<br>
agent.sources.ftp1.channels = ch1
<br>
agent.sources.ftp1.chunk.size = 102400
<br>
agent.channels.ch1.type = memory
<br>
agent.channels.ch1.capacity = 100000000
<br>
agent.channels.ch1.transactionCapacity = 100000000
<br>
agent.sinks.k1.type = com.ihooyah.flume.sink.LocalUploadSink
<br>
agent.sinks.k1.uploadDir = D:
\\
<br>
agent.sinks.k1.batchSize = 10000
<br>
agent.sinks.k1.channel = ch1
<br>
<br>
与实战一区别的配置
<br>
agent.sources.ftp1.flushlines = false 图片需要按照块去读不能按行读取否则会出现字节丢失导致图片损坏打不开
<br>
agent.sinks.k1.type = com.ihooyah.flume.sink.LocalUploadSink 重新写的一个sink
<br>
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment