在android开发过程中,文件上传非常常见。但是文件的断点续传就很少见了。因为android都是通过http协议请求服务器的,http本身不支持文件的断点上传。但是http支持文件的断点下载,可以通过http文件断点下载的原理来实现文件的断点上传,文件的断点下载比较简单,主要步骤如下
(1)开启服务,通过服务后台下载文件
(2)conn.getContentLength();获取要下载文件的长度,建立对应大小的文件
(3) 通过range字段来设置我们要下载的文件的开始字节和结束字节。http.setRequestProperty(“Range”, “bytes=” +
startPos + “-” + endPos) //注意格式中间加“-”
(4)通过sqllite保存我们下载的进度,以便在重新开始任务的时候断点下载
(5)通过广播将下载的进度回调到主线程,更新进度条
(6)利用RandomAccessFile的seek方法,跳过已下载的字节数来将下载的字节写入文件
通过这六步就可以实现简单的断点续传,当然单线程下载性能比较差。在最后的优化过程中还要加入多线程下载来提高我们的下载效率。ok文件的断线下载就说到这里,这不是我们要谈论的重点,接下来才是核心内容。
文件的上传对于android来说是非常耗时的操作,因此不能再主线程中进行。还是要利用服务来完成。但是上传的时候并没有range字段来传入我们要上传的开始字节和结束字节。这应该怎样做呢,这就要求对于文件上传,我们的服务器要做出相应的改变了。在上传文件的时候,将文件的大小的名称和大小作为参数上传到服务器,这样服务器就记录了要上传的文件大小。在上传过程中服务端要对已上传的字节数进行记录。在断点时,移动端不用记录文件的已上传的大小,而是首先去请求服务器得到已上传的字节数,移动端做的只是跳过相应的字节数来读取文件即可。在这里有必要说一下android通过http请求时如何与服务器交互的。移动端得到与服务器响应的链接后,通过流来进行数据的传递,在移动端读取文件的时候,并没有全部缓存到流中,而是在android端做了缓存,而android的内存有限,在上传大文件时就会内存溢出。在往流里面写入数据的时候,服务器也不能及时得到流内的数据,而是当移动端提交以后服务器才能做出response,移动端就是依据response判断文件上传是否成功。这样就会出现一个问题,android要实时提交文件,负责内存溢出导致程勋崩溃。在上传文件时必须先对文件校验,以免文件重复上传。对于重复上传的文件,服务端只要将以上传的文件与用户关联就可以了。文件的校验当然是md5校验了,md5校验也是耗时操作,因此也要多线程处理。在进度条的更新上还是使用广播来更新就可以了,当然也可以用handler来更新进度条。这个全凭个人喜好。这样文件的断点上传思路就有了。
(1)开启服务,通过服务后台检验文件和上传文件
(2)在检验文件的response中得到文件的已上传字节数
(3) 利用RandomAccessFile的seek方法,跳过已上传的字节数来读取文件
(4)多文件上传时通过sqllite保存文件的状态,对于已上传的文件将状态值设置为已上传或者在数据库删除
(5)通过广播将下载的进度回调到主线程,更新进度条
下面是文件断点上传的主要代码
@Override
public void onClick(View v) {
switch (v.getId()) {
case R.id.tv_shangchuan:
//开始上传时将选中的文件路径通过intent传给服务
list_filePath.clear();
//将文件的路径传入service
int file_count = map_check_file.size();
// L.i("-----count"+file_count);
if(file_count==0){
T.showShort(this,"空文件夹不能上传");
return;
}
loading_dialog1.show();
for(Map.Entry<String, File> entry: map_check_file.entrySet()){
File value = entry.getValue();
list_filePath.add(value.getAbsolutePath());
}
Intent intent_service = new Intent();
intent_service.setAction(MainActivity.ACTION_START);
intent_service.setPackage(getPackageName());
intent_service.putExtra("file_list", (Serializable) list_filePath);
getApplication().startService(intent_service);
break;
case R.id.tv_lixian:
if (map_check_file.size() != 0) {
for (Map.Entry<String, File> entry : map_check_file.entrySet()) {
//将未上传的文件加入数据库
fileDaoImp.insertData(entry.getValue().getAbsolutePath());
}
Fragment fragmentById = manager.findFragmentById(R.id.fl_director_activity);
FileFragment fileFragment = (FileFragment) fragmentById;
if (fileFragment != null) {
sendMessageToFragment(fileFragment);
}
T.showShort(this, "已加入离线");
rl_director_bottom.setVisibility(View.GONE);
} else {
T.showShort(this,"空文件夹不能加入离线");
}
break;
}
}
因为在项目中有离线功能,就一并贴出来吧,离线的实现也较为简单。就是当用户点击离线的时候,将离线文件的路径进入数据库,然后通过广播判断该网络状态,当wifi条件下,读取数据库中未下载文件然后开启服务下载。
服务的代码如下:
public class DownLoadService extends Service implements APICallBack {
private Handler handler = new Handler() {
@Override
public void handleMessage(Message msg) {
switch (msg.what) {
case 1:
downLoadTask = new DownLoadTask(DownLoadService.this);
downLoadTask.download();
break;
}
}
};
public static final String RECEIVI = "UPDATEPROGRESS";
//下载文件的线程
private DownLoadTask downLoadTask = null;
//文件断点上传的数据库管理类
FileDaoImp fileDaoImp = new FileDaoImp(DownLoadService.this);
boolean isFirst = true;
List<String> list_file_path = new ArrayList<>();
@Nullable
@Override
public IBinder onBind(Intent intent) {
return null;
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
if (intent != null) {
if (MainActivity.ACTION_START.equals(intent.getAction())) {
downLoadTask.isPause = false;
String loading_shangchuan = intent.getStringExtra("loading_shangchuan");
if (loading_shangchuan != null && loading_shangchuan.equals("loading_shangchuan")) {
isFirst = false;
new InitThread().start();
return super.onStartCommand(intent, flags, startId);
}
list_file_path = (List<String>) intent.getSerializableExtra("file_list");
isFirst = true;
Log.i("main", "--------list---Service--------------" + list_file_path.size());
//初始化线程
new InitThread().start();
} else if (MainActivity.ACTION_STOP.equals(intent.getAction())) {
if (downLoadTask != null) {
downLoadTask.isPause = true;
downLoadTask = null;
}
} else if (MainActivity.ACTION_CANCEL.equals(intent.getAction())) {
downLoadTask.isPause = true;
downLoadTask = null;
fileDaoImp.deletDateFileTask();
fileDaoImp.deleteFileUrl();
}
}
// START_NO_STICKY
// START_STICKY 默认调用
return super.onStartCommand(intent, flags, startId);
}
//初始话文件线程
class InitThread extends Thread {
@Override
public void run() {
if (isFirst) {
for (int i = 0; i < list_file_path.size(); i++) {
File file = new File(list_file_path.get(i));
// L.i("-------file-------------" + file.length());
FileInfo fileInfo2 = null;
try {
if (!file.isDirectory()) {
//将选中的文件存入数据库
fileInfo2 = new FileInfo(2, file.getAbsolutePath(), file.getName(), file.length(), 0, MD5Util.getFileMD5String(file));
fileDaoImp.insertFileUrl(fileInfo2.getUrl(), fileInfo2.getLength(), fileInfo2.getMd5(), fileInfo2.getFileName());
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
handler.obtainMessage(1).sendToTarget();
}
}
}
//文件上传线程,将文件按照5M分片上传。下面也给出了android如何不再本地缓存的方法。
/**
* Created by zhoukai on 2016/5/3.
* // int fixedLength = (int) fStream.getChannel().size();
* // 已知输出流的长度用setFixedLengthStreamingMode()
* // 位置输出流的长度用setChunkedStreamingMode()
* // con.setChunkedStreamingMode(块的大小);
* // 如果没有用到以上两种方式,则会在本地缓存后一次输出,那么当向输出流写入超过40M的大文件时会导致OutOfMemory
* //设置固定流的大小
* // con.setFixedLengthStreamingMode(fixedLength);
* // con.setFixedLengthStreamingMode(1024 * 1024*20);
*/
public class DownLoadTask {
private Context context;
private FileDaoImp fileDaoImp;
public static boolean isPause = false;
private long file_sum = 0;
String isExistUrl = "http://123.56.15.30:8080/upload/isExistFile";
String actionUrl = "http://123.56.15.30:8080/upload/uploadFile";
private int finishedLength;
public DownLoadTask(Context context) {
this.context = context;
fileDaoImp = new FileDaoImp(context);
}
public void download() {
new DownThread().start();
}
class DownThread extends Thread {
private double load_lenth = 0;
String end = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
@Override
public void run() {
//未上传的文件
List<FileInfo> list = fileDaoImp.queryFileByState();
Log.i("main", "--------list--数据库---------------" + list.size());
int sum_filelength = (int) fileDaoImp.getLengthByState(0);
if (list.size() == 0) {
return;
}
Intent intent = new Intent();
intent.setAction(DownLoadService.RECEIVI);
int nSplitter_length = 1024 * 1024 * 5;
for (int i = 0; i < list.size(); i++) {
int file_length = (int) list.get(i).getLength();
int count = file_length / nSplitter_length + 1;
// L.i("-------------------md5------------" + list.get(i).getMd5());
// L.i("------------------fileName------------" + list.get(i).getFileName());
//---------------------验证文件--------------------------------------------------
URL realurl = null;
InputStream in = null;
HttpURLConnection conn = null;
try {
realurl = new URL(isExistUrl);
conn = (HttpURLConnection) realurl.openConnection();
conn.setRequestProperty("accept", "*/*");
conn.setRequestProperty("connection", "Keep-Alive");
conn.setRequestProperty("user-agent",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;SV1)");
conn.setRequestMethod("POST");
conn.setChunkedStreamingMode(1024 * 1024 * 10);
//无穷大超时
conn.setReadTimeout(0);
conn.setConnectTimeout(0);
conn.setDoInput(true);
conn.setDoOutput(true);
PrintWriter pw = new PrintWriter(conn.getOutputStream());
pw.print("userId=" + AppUtils.getUserName(context) + "&md5=" + list.get(i).getMd5()
+ "&did=" + getDid() + "&name=" + list.get(i).getFileName() + "&size=" + list.get(i).getLength());
Log.i("main", "-------------userId---------" + AppUtils.getUserName(context));
// Log.i("main", "-------------md5---------" + list.get(i).getMd5());
// Log.i("main", "-------------did---------" + getDid());
// Log.i("main", "-------------name---------" + list.get(i).getFileName());
// Log.i("main","-------------size---------"+list.get(i).getLength());
pw.flush();
pw.close();
/* 取得Response内容 */
in = conn.getInputStream();
int ch;
StringBuffer stringBuffer = new StringBuffer();
while ((ch = in.read()) != -1) {
stringBuffer.append((char) ch);
}
String json = stringBuffer.toString();
JSONObject jsonObject = new JSONObject(json);
boolean isSuccess = jsonObject.optBoolean("success");
if (isSuccess) {
int lengths = jsonObject.optJSONObject("info").optJSONObject("file").optInt("length");
finishedLength = lengths;
if (finishedLength == list.get(i).getLength()) {
fileDaoImp.deleteFilebyMd5(list.get(i).getMd5());
fileDaoImp.deleteFilebyPath(list.get(i).getUrl());
if (i == list.size() - 1) {
intent.putExtra("progress", (load_lenth * 100 / ((double) sum_filelength)));
intent.putExtra("state", "success");
context.sendBroadcast(intent);
}
continue;
}
Log.i("main", "-----length_finished------" + finishedLength);
}
} catch (Exception eio) {
Log.i("main", "-----Exception------" + eio.toString());
}
//---------------------上传文件--------------------------------------------------
for (int j = 0; j < count; j++) {
try {
File file = new File(list.get(i).getUrl());
URL url = new URL(actionUrl);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setChunkedStreamingMode(1024 * 1024 * 10);
//无穷大超时
con.setReadTimeout(0);
con.setConnectTimeout(0);
/* 允许Input、Output,不使用Cache */
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
/* 设置传送的method=POST */
con.setRequestMethod("POST");
/* setRequestProperty */
con.setRequestProperty("Connection", "Keep-Alive");//建立长连接
con.setRequestProperty("Charset", "UTF-8"); //编码格式
con.setRequestProperty("Content-Type",
"multipart/form-data;boundary=" + boundary);//表单提交文件
DataOutputStream ds = new DataOutputStream(con.getOutputStream());
//添加参数
StringBuffer sb = new StringBuffer();
Map<String, String> params_map = new HashMap<>();
params_map.put("nSplitter", "3");
params_map.put("md5", list.get(i).getMd5());
params_map.put("dId", getDid());
params_map.put("userId", AppUtils.getUserName(context));
params_map.put("name", file.getName());
params_map.put("from", finishedLength + "");
Log.i("main", "-------------userId----上传-----" + AppUtils.getUserName(context));
if (finishedLength + nSplitter_length > file_length) {
params_map.put("to", file_length + "");
} else {
params_map.put("to", (finishedLength + nSplitter_length) + "");
}
params_map.put("size", list.get(i).getLength() + "");
//添加参数
for (Map.Entry<String, String> entries : params_map.entrySet()) {
sb.append(twoHyphens).append(boundary).append(end);//分界符
sb.append("Content-Disposition: form-data; name=" + entries.getKey() + end);
sb.append("Content-Type: text/plain; charset=UTF-8" + end);
sb.append("Content-Transfer-Encoding: 8bit" + end);
sb.append(end);
sb.append(entries.getValue());
Log.i("main", "-----------params----------" + entries.getValue());
sb.append(end);//换行!
}
ds.writeBytes(sb.toString());
//添加文件
ds.writeBytes(twoHyphens + boundary + end);
ds.writeBytes("Content-Disposition: form-data; "
+ "name=\"file" + "\";filename=\"" + file.getName() + "\"" + end);
ds.writeBytes(end);
/* 设置每次写入1024bytes */
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int length = -1;
long time = System.currentTimeMillis();
/* 从文件读取数据至缓冲区 */
file_sum = file.length();
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r");
randomAccessFile.seek(finishedLength);
load_lenth = finishedLength;
double current_lenth = load_lenth;
while ((length = randomAccessFile.read(buffer)) != -1) {
/* 将资料写入DataOutputStream中 */
ds.write(buffer, 0, length);
load_lenth += length;
if (load_lenth - current_lenth > nSplitter_length) {
current_lenth = load_lenth;
break;
}
if (System.currentTimeMillis() - time > 500) {
time = System.currentTimeMillis();
//使用广播发送上传百分比
// intent.putExtra("progress", (load_lenth * 100 / ((double) file_sum)));
intent.putExtra("progress", (load_lenth * 100 / ((double) sum_filelength)));
context.sendBroadcast(intent);
}
if (isPause) {
//将文件的进度修改
ds.writeBytes(end);
randomAccessFile.close();
ds.flush();
ds.writeBytes(twoHyphens + boundary + twoHyphens + end);
/* 取得Response内容 */
InputStream is = con.getInputStream();
int ch;
StringBuffer b = new StringBuffer();
while ((ch = is.read()) != -1) {
b.append((char) ch);
}
String json = b.toString();
JSONObject jsonObject = new JSONObject(json);
boolean isSuccess = jsonObject.optBoolean("success");
if (isSuccess) {
int lengths = jsonObject.optJSONObject("info").optJSONObject("file0").optInt("length");
if (lengths == list.get(i).getLength()) {
Log.i("main", "----文件上传-lengths------" + lengths);
}
//更新进度
fileDaoImp.upDateProgress(list.get(i).getMd5(), lengths);
}
ds.close();
con.disconnect();
return;
}
}
ds.writeBytes(end);
randomAccessFile.close();
ds.flush();
ds.writeBytes(twoHyphens + boundary + twoHyphens + end);
/* 取得Response内容 */
InputStream is = con.getInputStream();
int ch;
StringBuffer b = new StringBuffer();
while ((ch = is.read()) != -1) {
b.append((char) ch);
}
String json = b.toString();
JSONObject jsonObject = new JSONObject(json);
boolean isSuccess = jsonObject.optBoolean("success");
Log.i("main", "----文件分片------" + json);
if (isSuccess) {
int lengths = jsonObject.optJSONObject("info").optJSONObject("file0").optInt("length");
finishedLength = lengths; //更新跳过的字节数
if (lengths == list.get(i).getLength()) {
boolean b1 = fileDaoImp.deleteFilebyMd5(list.get(i).getMd5());
//删除离线文件
boolean b2 = fileDaoImp.deleteFilebyPath(list.get(i).getUrl());
Log.i("main", "----文件上传-成功------" + lengths);
//当最后一个文件
if (i == list.size() - 1) {
intent.putExtra("progress", (load_lenth * 100 / ((double) sum_filelength)));
intent.putExtra("state", "success");
context.sendBroadcast(intent);
}
break;
}
}
ds.close();
con.disconnect();
Log.i("main", "--------end----------");
} catch (Exception e) {
Log.i("main", "---------e------------" + e.toString());
}
}
}
}
}
}
//数据库管理代码:
public class FileDaoImp implements FileDao{
private DbHelper dbHelper = null;
public FileDaoImp(Context context){
dbHelper = new DbHelper(context);
}
@Override
public void insertFileUrl(String url,long length,String md5,String file_name) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
//将文件的状态存入到数据库,默认为0表示未完成上传
db.execSQL("insert into file_info(fileUrl,file_state,file_length,file_md5," +
"file_progress,file_name)values(?,?,?,?,?,?)",new Object[]{url,0,length,md5,0,file_name
});
db.close();
}
//根据文件的md5值去除已经上传的文件
@Override
public boolean deleteFilebyMd5(String md5) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
// db.execSQL("delete from file_info where file_md5 = ?",new String[]{md5});
int file_info = db.delete("file_info", "file_md5=?", new String[]{md5});
db.close();
if(file_info>0){
return true;
}
return false;
}
@Override
public void deleteFileUrl() {
SQLiteDatabase db = dbHelper.getReadableDatabase();
db.execSQL("delete from file_info");
db.close();
}
//删除数据
public boolean deletDateFileTask( ) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
//int shebei_info = db.delete("shebei_info", "_id= ?", new String[]{""+id});
int shebei_info = db.delete("fileTask", null, null); //全部删除数据
//sql中含有自增序列时,会自动建立一个名为sqlite_sequence的表,其中包含name与seq
//name记录自增所在的表,seq记录当前的序号。删除数据后想要将自增id置为0只需upadtaseq即可
db.execSQL(" update sqlite_sequence set seq=0 where name='fileTask'");
return false;
}
@Override
public List<String> queryFileUrl() {
List<String> list = new ArrayList<>();
SQLiteDatabase db = dbHelper.getWritableDatabase();
Cursor cursor = db.rawQuery("select * from file_info", null);
while (cursor.moveToNext()){
String url = cursor.getString(cursor.getColumnIndex("fileUrl"));
list.add(url);
}
cursor.close();
db.close();
return list;
}
@Override
public List<FileInfo> queryFileByState() {
List<FileInfo> list_fileInfo = new ArrayList<>();
SQLiteDatabase db = dbHelper.getWritableDatabase();
Cursor cursor = db.rawQuery("select * from file_info where file_state = 0", null);
while (cursor.moveToNext()){
String url = cursor.getString(cursor.getColumnIndex("fileUrl"));
int file_length = cursor.getInt(cursor.getColumnIndex("file_length"));
String file_md5 = cursor.getString(cursor.getColumnIndex("file_md5"));
String file_name = cursor.getString(cursor.getColumnIndex("file_name"));
FileInfo fileInfo = new FileInfo();
fileInfo.setUrl(url);
fileInfo.setMd5(file_md5);
fileInfo.setLength(file_length);
fileInfo.setFileName(file_name);
list_fileInfo.add(fileInfo);
}
cursor.close();
db.close();
return list_fileInfo;
}
//根据文件的url来更新文件的状态,将文件状态更新为已经上传
@Override
public void updateFile(String md5) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
db.execSQL("update file_info set file_state = ? where file_md5 = ?",new Object[]{
1,md5
});
db.close();
}
//得到文件的总长度
@Override
public long getLengthByState(int state) {
long length = 0;
SQLiteDatabase db = dbHelper.getReadableDatabase();
Cursor cursor = db.rawQuery("select * from file_info where file_state = ?", new String[]{state+""});
while (cursor.moveToNext()){
String url = cursor.getString(cursor.getColumnIndex("fileUrl"));
length+=new File(url).length();
}
cursor.close();
db.close();
L.i("----------length--------"+length);
return length;
}
@Override
public void upDateProgress(String md5,int progress) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
db.execSQL("update file_info set file_progress = ? where file_md5 = ?",new Object[]{
progress,md5
});
db.close();
}
@Override
public int getFileFinishedProgress(String md5) {
int length = 0;
SQLiteDatabase db = dbHelper.getReadableDatabase();
Cursor cursor = db.rawQuery("select * from file_info where file_md5 =?", new String[]{md5});
while (cursor.moveToNext()){
length = cursor.getInt(cursor.getColumnIndex("file_progress"));
}
cursor.close();
db.close();
Log.i("main","------------fileProgress-------"+length);
return length;
}
@Override
public int getFileSumLength(String file_length) {
int sum = 0;
// SQLiteDatabase db = dbHelper.getReadableDatabase();
// Cursor cursor = db.rawQuery("select sum(file_length) from file_info ", null);
// cursor.close();
// db.close();
// Log.i("main","------------fileProgress-------"+sum);
return sum;
}
//-------------------------------离线---------------
//增加数据
public boolean insertData( String values) {
/* ContentValues values = new ContentValues();
values.put("title", title);
values.put("content", content);*/
SQLiteDatabase db = dbHelper.getReadableDatabase();
ContentValues contentValues = new ContentValues();
contentValues.put("filepath",values);
long shebei_info = db.insert("fileTask", null, contentValues);
contentValues.clear();
if (shebei_info > 0) {
Log.i("main","---------shebei_info-----------------");
return true;
}
return false;
}
//删除数据
public boolean deletelian_Date( ) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
//int shebei_info = db.delete("shebei_info", "_id= ?", new String[]{""+id});
int shebei_info = db.delete("fileTask", null, null); //全部删除数据
//sql中含有自增序列时,会自动建立一个名为sqlite_sequence的表,其中包含name与seq
//name记录自增所在的表,seq记录当前的序号。删除数据后想要将自增id置为0只需upadtaseq即可
db.execSQL(" update sqlite_sequence set seq=0 where name='fileTask'");
return false;
}
public List<String> query_lianxian_data( ) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
List<String> list_db = new ArrayList<>();
Cursor cursor = db.rawQuery("select * from fileTask", null);
if (cursor != null) {
String columns[] = cursor.getColumnNames(); //得到对应的字段
while (cursor.moveToNext()) {
String file_path = cursor.getString(cursor.getColumnIndex(columns[1]));
list_db.add(file_path);
}
cursor.close();
}
return list_db;
}
public boolean deleteFilebyPath(String path) {
SQLiteDatabase db = dbHelper.getReadableDatabase();
// db.execSQL("delete from file_info where file_md5 = ?",new String[]{md5});
int file_info = db.delete("fileTask", "filepath=?", new String[]{path});
db.close();
if(file_info>0){
return true;
}
return false;
}
}
通过这几步就可以实现文件的断点上传了,但是由于时间的原因,项目中并没有加入多线程上传。其实多线程上传的原理也很简单,类似文件的分片上传。在这里就不说了。文件的断点上传是实现了,但是性能上面还是有很多欠缺。只能慢慢改善了。