java网络编程：RandomAccessFile, URLConnection和多线程机制实现了Http下载

设计思路:

1. 首先读取文件的长度, 并判断网站是否支持分段下载

2. 如果支持分段下载则创建多个线程同时下载该文件,否则使用单线程下载

3. 在各个线程中,分别使用RandomAccessFile对象写入对应的文件位置

配置文件：

    package  com.ross.httpdownload.util;  
    public   class  SysValue  
    {  
        //constant values   
        //maximum number of the download thread   
        public   static   int  MAX_Num_Of_Thread =  3 ;  
        //download file store path   
        public   static  String Save_As_Path =  "E:/myspace/download/" ;  
        //buffer size   
        public   static   int  Buffer_Size =  1024 ;      
    }  

将Http相关的公共操作放在这个类：

    package  com.ross.httpdownload.util;  
    import  java.io.IOException;  
    import  java.net.*;   
    public   class  HttpProcess  
    {  
        /**  
         * Author: Jiangtao He; Email: ross.jiangtao.he@gmail.com  
         * Description: get an object of opened URL connection  
         * @param sURL: completed URL string  
         * @return oHttpURLCon: an object of opened URL connection  
         * @throws IOException  
         */   
        public  URLConnection  getURLConnection(String sURL)  throws  IOException  
        {  
            URLConnection  oURLCon = null ;  
            // replace space with '+' in the URL string   
            if  ( null  != sURL)  
            {  
                sURL = sURL.replaceAll("\\s" ,  "+" );  
                // generate URL   
                URL oURL = new  URL(sURL);  
                oURLCon = (URLConnection) oURL.openConnection();  
            }  
            return  oURLCon;  
        }  
    }  

a. 首先看下如何判断网站资源是否支持分段下载：

 public   boolean  isSupportMultipleThreadDown(String sURL)  throws  IOException  
{  
    boolean  bRet =  true ;  
   // get URL connection   
    URLConnection oURLCon = oHttpProcess.getURLConnection(sURL);  
   // set the resource range   
    oURLCon.setRequestProperty("Range" ,  "bytes="  +  0  +  "-"  +  1023 );  
  
    if  ( 1024  < oURLCon.getContentLength())  
    {  
        bRet = false ;  
    }  
    return  bRet;  
}   

b. 然后我们看下如何,将文件分段下载：

1) 首先是预处理

先是做了个简单的判断,防止线程过多或者小于1; 然后判断网络资源是否支持分段下载,如果不支持则将线程数重置为1; 在获取整个文件长度后, 使用总长度除以线程数得出每个线程负责下载的字节范围.

2) 使用线程池管理并启动多线程下载

DownloadTread就是我们的线程下载实现类, 后面会单独说明. 众所周知java下标是从0开始的,所以各个线程分配字节范围的时候,要考虑进去. 将对应分配的字节范围,URL,文件名信息设置给DownloadTread对象后,使用ExecutorService启动线程.

3)当然不要忘记在finally里面调用线程池的关闭方法.

 public   void  download(String sURL,  int  iNumOfThread, String sSaveAsName)  
{  
    long  lFileLen =  0 ;  
    long  lFileSegementLen =  0 ;  
    long  iLeft =  0 ;  
    URLConnection oURLCon = null ;  
    String sFullFileName = SysValue.Save_As_Path + sSaveAsName;  
  
    if  (iNumOfThread > SysValue.MAX_Num_Of_Thread)  
    {  
        iNumOfThread = SysValue.MAX_Num_Of_Thread;  
    }  
    if  (iNumOfThread <  1 )  
    {  
        iNumOfThread = 1 ;  
    }  
  
    // build the thread pool to manage the thread   
    ExecutorService oExecService = Executors  
            .newFixedThreadPool(iNumOfThread);  
    try   
    {  
        if  (!isSupportMultipleThreadDown(sURL))  
        {  
            System.out  
                    .println("This site do not support multiple thread download, reset the thread to 1" );  
            iNumOfThread = 1 ;  
        }  
        // get URL connection   
        oURLCon = oHttpProcess.getURLConnection(sURL);  
        // get the length of the file   
        lFileLen = oURLCon.getContentLength();  
        // split the length of the file according to the number of thread.   
        lFileSegementLen = lFileLen / iNumOfThread;  
        iLeft = lFileLen % iNumOfThread;  
  
        for  ( int  i =  0 ; i < iNumOfThread; i++)  
        {  
            // initial the thread data   
            DownloadThread oDownloadThread = new  DownloadThread();  
            oDownloadThread.setIThreadId(i);  
            // Start index of the current thread   
            oDownloadThread.setLStartIndex(i * lFileSegementLen);  
  
            // last thread will take all the rest content   
            if  (i == (iNumOfThread -  1 ))  
            {  
                oDownloadThread.setLEndIndex((i + 1 ) * lFileSegementLen  
                        + iLeft - 1 );  
            }  
            else   
            {  
                oDownloadThread  
                        .setLEndIndex((i + 1 ) * lFileSegementLen -  1 );  
            }  
            // set save file name with path   
            oDownloadThread.setSFullFileName(sFullFileName);  
            // set URL   
            oDownloadThread.setSFileURL(sURL);  
            // execute the download thread   
            oExecService.execute(oDownloadThread);  
        }  
        System.out.println("the file is stored as \""  + sFullFileName  
                + "\"." );  
        System.out.println("the file length is "  + lFileLen +  "." );  
    }  
    catch  (IOException e)  
    {  
        System.out.println("download the file failed, IO Exception"   
                + e.getMessage());  
        e.printStackTrace();  
    }  
    finally   
    {  
        oExecService.shutdown();  
    }  
}   

c.最后就是我们的核心下载线程实现类了：

1) DownloadTread类实现了Runnalbe抽象接口, 实现了其run()方法.

2) 两个关键的地方一是通过使用URLConnection 的setRequestProperty方法设置本线程读取的流的范围, 另外一个是通过RandomAccessFile 的seek方法或者当前线程写入文件的位置, 一个字节都不能错哦,错了你的文件就有差错了, 还是要记住java下标是从0开始的.

3)然后通过循环从URLConnection的InputStream里面通过buffer一组一组的读出来, 通过RandomAccessFile 对象一组一组的写入到目标文件里面(代码里面判断实际读取长度部分是不需要的, 可以去掉, 是我探索不支持分段下载网络资源的多线程下载方式时留下的).

4)需要提一下的是run()方法是没有输入参数的,所以我们通过增加类字段的方式将参数传入该方法.

 public   class  DownloadThread  implements  Runnable  
{  
    private  String sFileURL;  
    private   long  lStartIndex;  
    private   long  lEndIndex;  
    private   int  iThreadId;  
    private  String sFullFileName;  
    private  HttpProcess oHttpProcess;  
    /**  
     * Description: default constructor, it is used for initializing the fields.  
     */   
    public  DownloadThread()  
    {  
        oHttpProcess = new  HttpProcess();  
    }    
    public   void  run()  
    {  
        System.out.println("The download thread "   
                + this .iThreadId  
                + " is started at "   
                + (new  SimpleDateFormat( "yyyy-MM-dd HH:mm:ss" ))  
                        .format(new  Date()));  
        System.out.println("The download index of "  +  this .iThreadId +  " is "   
                + this .lStartIndex +  " to "  +  this .lEndIndex);  
  
        // content length   
        long  lContentlen =  0 ;  
        // used to store the new url connection   
        URLConnection oURLCon = null ;  
        // used to store the input stream of the response   
        BufferedInputStream oBIn = null ;  
        // used to store the output writer   
        RandomAccessFile oRAFile = null ;  
        // create the buffer   
        byte [] bBuffer =  new   byte [SysValue.Buffer_Size];  
  
        // get URL connection   
        try   
        {  
            // create a link for each thread   
            oURLCon = oHttpProcess.getURLConnection(this .sFileURL);  
            // allow the user interaction, for example: a verify pop window   
            oURLCon.setAllowUserInteraction(true );  
            // set the resource range   
            oURLCon.setRequestProperty("Range" ,  "bytes="  +  this .lStartIndex  
                    + "-"  +  this .lEndIndex);  
  
            // get the url connection input stream   
            oBIn = new  BufferedInputStream(oURLCon.getInputStream());  
   
            // initialize the random access file object   
            oRAFile = new  RandomAccessFile( this .sFullFileName,  "rw" );   
            oRAFile.seek(this .lStartIndex);  
  
            // read the stream from http connection   
            int  iLen =  0 ;  
            int  iActualLen =  0 ;  
            while  (iActualLen < ( this .lEndIndex -  this .lStartIndex +  1 ))  
            {  
                iLen = oBIn.read(bBuffer, 0 , SysValue.Buffer_Size);  
                if  (- 1  == iLen)  
                {  
                    break ;  
                }  
  
                // write the read data to file   
                oRAFile.write(bBuffer, 0 , iLen);  
  
                // move the position mark   
                iActualLen = iActualLen + iLen;  
            }  
            System.out.println("Thread "  +  this .iThreadId  
                    + " download is finished, totaly: "  + iActualLen);  
        }  
        catch  (IOException e)  
        {  
            System.out.println("download the file failed, IO Exception"   
                    + e.getMessage());  
            e.printStackTrace();  
        }  
        finally   
        {  
            if  ( null  != oRAFile)  
            {  
                try   
                {  
                    oRAFile.close();  
                }  
                catch  (IOException e)  
                {  
                    System.out   .println("close object of RandomAccessFile failed, IO Exception"   + e.getStackTrace());  
                }  
            }  
  
            if  ( null  != oBIn)  
            {  
                try   
                {  
                    oBIn.close();  
                }  
                catch  (IOException e)  
                {  
                    System.out.println("close input stream of http connection failed, IO Exception"+ e.getStackTrace());  
                }  
            }  
        }  
        System.out.println("The download thread " + this .iThreadId  + " is ended at "   + (new  SimpleDateFormat( "yyyy-MM-dd HH:mm:ss" ))  .format(new  Date()));  
    }  
   //省略了set/get方法     
}   

来测试下我们的实现:

看下我们的main函数, 我们就通过3个例子来测试我们的实现: 一个可以多线程下载的zip文件,一个可以多线程下载的图片,一个不能多线程下载图片.

    package  com.ross.httpdownload;  
    import  com.ross.httpdownload.util.*;  
    /**  
     * Author: Jiangtao He; Email: ross.jiangtao.he@gmail.com  
     * Date: 2012-1-19  
     * Since: MyJavaExpert v1.0  
     * Description: Test the download tool class  
     */   
    public   class  MyMain  
    {  
        public   static   void  main(String[] args)  
        {  
            // set system configuration, in real software, it can be configured   
            // through GUI, file,etc.   
            SysValue.MAX_Num_Of_Thread = 3 ;  
            SysValue.Save_As_Path = "E:/myspace/download/" ;  
            SysValue.Buffer_Size = 1024 ;  
            //file web address   
            //String sFileURL = "http://xinsheng-image.huawei.com/cn/forumimage/showimage-991441-20990f56c5b11261fa63e969d8d5580c-self.jpg";   
            //String sFileURL = "http://lh5.googleusercontent.com/-F7a0loCDyoQ/AAAAAAAAAAI/AAAAAAAAAFY/TrDxSDdQuhQ/s512-c/photo.jpg";   
            //String sFileURL = "http://xinsheng-image.huawei.com/cn/forumimage/showimage-816607-f61c8fe14fc359d49b044376a0956acd-self.jpg";   
            //String sFileURL = "http://3.bp.blogspot.com/-oVO24VW8F6M/TxJ9O0opP3I/AAAAAAAAANY/anpu8S4FAC4/s640/100_9223.JPG";   
            String sFileURL = "http://hi.csdn.net/attachment/201201/1/0_1325433530Bg5e.gif" ;  
            sFileURL = "http://west263.newhua.com:82/down/jia-audio-converter.zip" ;  
            //file save as name   
            String sFileName = sFileURL.substring(sFileURL.lastIndexOf("/" ) +  1 );  
             
            // create a object of download tool   
            HttpDownloadTool oHttpDownloadTool = new  HttpDownloadTool();  
            // download the file   
            oHttpDownloadTool.download(sFileURL, 3 , sFileName);  
        }  
    }