有两个办法
第一, 使用linux的split命令,分割文件,然后在逐个读取文件
读取的时候,可以使用如下方法
第一, 使用linux的split命令,分割文件,然后在逐个读取文件
#first cut file into 50k byte (51200) slice, then send each slice using resume
#
# we accept file max size 500M now. should we support larger file, change -a
def split
prefix = "chunk#{@session_id}_"
cmd = "split -b #{@slice_block} -d -a 4 #{@file_path} #{prefix}"
puts "cmd is #{cmd}"
`#{cmd}` #shell to split files
raise Exception, "fail to split file, command as split -b #{@slice_block} -d -a 4 #{@file_file} #{prefix}" if $?.to_i != 0
end
读取的时候,可以使用如下方法
file_chunks = `ls chunk#{@session_id}_*`.split("\n")
file_chunks.each { |chunk|
... ...
}
第二, 扩展Ruby的File, 增加each_chunk方法
class File
MEGABYTE = 300 * 1024
def each_chunk(chunk_size=MEGABYTE)
yield read(chunk_size) until eof?
end
end