快捷導(dǎo)航

關(guān)于SpringBoot大文件RestTemplate下載解決方案

更新時(shí)間：2021年10月25日 08:42:58 作者：人無名，則可專心練劍

這篇文章主要介紹了SpringBoot大文件RestTemplate下載解決方案,最近結(jié)合網(wǎng)上案例及自己總結(jié)，寫了一個(gè)分片下載tuling/fileServer項(xiàng)目，需要的朋友可以參考下

近期基于項(xiàng)目上使用到的RestTemplate下載文件流，遇到1G以上的大文件，下載需要3-4分鐘，因?yàn)檎{(diào)用API接口沒有做分片與多線程，文件流全部采用同步方式加載，性能很慢。最近結(jié)合網(wǎng)上案例及自己總結(jié)，寫了一個(gè)分片下載tuling/fileServer項(xiàng)目： 1.包含同步下載文件流在瀏覽器加載輸出相關(guān)代碼； 2.包含分片多線程下載分片文件及合并文件相關(guān)代碼；

另外在DownloadThread項(xiàng)目中使用代碼完成了一個(gè)遠(yuǎn)程RestUrl請(qǐng)求去獲取一個(gè)遠(yuǎn)端資源大文件進(jìn)行多線程分片下載到本地的一個(gè)案例，可以下載一些諸如.mp4/.avi等視頻類大文件。相關(guān)代碼也一并打包上傳。

同步下載，支持分片下載Range主要代碼：

@Controller
public class DownLoadController {
    private static final String UTF8 = "UTF-8";
    @RequestMapping("/download")
    public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
        File file = new File("D:\\DevTools\\ideaIU-2021.1.3.exe");
        response.setCharacterEncoding(UTF8);
        InputStream is = null;
        OutputStream os = null;
        try {
            // 分片下載 Range表示方式 bytes=100-1000  100-
            long fSize = file.length();
            response.setContentType("application/x-download");
            String fileName = URLEncoder.encode(file.getName(), UTF8);
            response.addHeader("Content-Disposition", "attachment;filename=" + fileName);
            // 支持分片下載
            response.setHeader("Accept-Range", "bytes");
            response.setHeader("fSize", String.valueOf(fSize));
            response.setHeader("fName", fileName);

            long pos = 0, last = fSize - 1, sum = 0;
            if (null != request.getHeader("Range")) {
                response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
                String numberRange = request.getHeader("Range").replaceAll("bytes=", "");
                String[] strRange = numberRange.split("-");
                if (strRange.length == 2) {
                    pos = Long.parseLong(strRange[0].trim());
                    last = Long.parseLong(strRange[1].trim());
                    if (last > fSize-1) {
                        last = fSize - 1;
                    }
                } else {
                    pos = Long.parseLong(numberRange.replaceAll("-", "").trim());
                }
            }
            long rangeLength = last - pos + 1;
            String contentRange = new StringBuffer("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
            response.setHeader("Content-Range", contentRange);
            response.setHeader("Content-Length", String.valueOf(rangeLength));

            os = new BufferedOutputStream(response.getOutputStream());
            is = new BufferedInputStream(new FileInputStream(file));
            is.skip(pos);
            byte[] buffer = new byte[1024];
            int length = 0;
            while (sum < rangeLength) {
                int readLength = (int) (rangeLength - sum);
                length = is.read(buffer, 0, (rangeLength - sum) <= buffer.length ? readLength : buffer.length);
                sum += length;
                os.write(buffer,0, length);
            }
            System.out.println("下載完成");
        }finally {
            if (is != null){
                is.close();
            }
            if (os != null){
                os.close();
            }
        }
    }
}

多線程分片下載分片文件，下載完成之后合并分片主要代碼：

@RestController
public class DownloadClient {
    private static final Logger LOGGER = LoggerFactory.getLogger(DownloadClient.class);
    private final static long PER_PAGE = 1024L * 1024L * 50L;
    private final static String DOWN_PATH = "F:\\fileItem";
    ExecutorService taskExecutor = Executors.newFixedThreadPool(10);

    @RequestMapping("/downloadFile")
    public String downloadFile() {
        // 探測(cè)下載
        FileInfo fileInfo = download(0, 10, -1, null);
        if (fileInfo != null) {
            long pages =  fileInfo.fSize / PER_PAGE;
            for (long i = 0; i <= pages; i++) {
                Future<FileInfo> future = taskExecutor.submit(new DownloadThread(i * PER_PAGE, (i + 1) * PER_PAGE - 1, i, fileInfo.fName));
                if (!future.isCancelled()) {
                    try {
                        fileInfo = future.get();
                    } catch (InterruptedException | ExecutionException e) {
                        e.printStackTrace();
                    }
                }
            }
            return System.getProperty("user.home") + "\\Downloads\\" + fileInfo.fName;
        }
        return null;
    }

    class FileInfo {
        long fSize;
        String fName;

        public FileInfo(long fSize, String fName) {
            this.fSize = fSize;
            this.fName = fName;
        }
    }

    /**
     * 根據(jù)開始位置/結(jié)束位置
     * 分片下載文件，臨時(shí)存儲(chǔ)文件分片
     * 文件大小=結(jié)束位置-開始位置
     *
     * @return
     */
    private FileInfo download(long start, long end, long page, String fName) {
        File dir = new File(DOWN_PATH);
        if (!dir.exists()) {
            dir.mkdirs();
        }
        // 斷點(diǎn)下載
        File file = new File(DOWN_PATH, page + "-" + fName);
        if (file.exists() && page != -1 && file.length() == PER_PAGE) {
            return null;
        }
        try {
            HttpClient client = HttpClients.createDefault();
            HttpGet httpGet = new HttpGet("http://127.0.0.1:8080/download");
            httpGet.setHeader("Range", "bytes=" + start + "-" + end);
            HttpResponse response = client.execute(httpGet);
            String fSize = response.getFirstHeader("fSize").getValue();
            fName = URLDecoder.decode(response.getFirstHeader("fName").getValue(), "UTF-8");
            HttpEntity entity = response.getEntity();
            InputStream is = entity.getContent();
            FileOutputStream fos = new FileOutputStream(file);
            byte[] buffer = new byte[1024];
            int ch;
            while ((ch = is.read(buffer)) != -1) {
                fos.write(buffer, 0, ch);
            }
            is.close();
            fos.flush();
            fos.close();
            // 最后一個(gè)分片
            if (end - Long.parseLong(fSize) > 0) {
                // 開始合并文件
                mergeFile(fName, page);
            }

            return new FileInfo(Long.parseLong(fSize), fName);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }

    private void mergeFile(String fName, long page) {
        File file = new File(DOWN_PATH, fName);
        try {
            BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file));
            for (long i = 0; i <= page; i++) {
                File tempFile = new File(DOWN_PATH, i + "-" + fName);
                while (!file.exists() || (i != page && tempFile.length() < PER_PAGE)) {
                    try {
                        Thread.sleep(100);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
                byte[] bytes = FileUtils.readFileToByteArray(tempFile);
                os.write(bytes);
                os.flush();
                tempFile.delete();
            }
            File testFile = new File(DOWN_PATH, -1 + "-null");
            testFile.delete();
            os.flush();
            os.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    /**
     * 獲取遠(yuǎn)程文件尺寸
     */
    private long getRemoteFileSize(String remoteFileUrl) throws IOException {
        long fileSize = 0;
        HttpURLConnection httpConnection = (HttpURLConnection) new URL(remoteFileUrl).openConnection();
        //使用HEAD方法
        httpConnection.setRequestMethod("HEAD");
        int responseCode = httpConnection.getResponseCode();
        if (responseCode >= 400) {
            LOGGER.debug("Web服務(wù)器響應(yīng)錯(cuò)誤!");
            return 0;
        }
        String sHeader;
        for (int i = 1;; i++) {
            sHeader = httpConnection.getHeaderFieldKey(i);
            if (sHeader != null && sHeader.equals("Content-Length")) {
                LOGGER.debug("文件大小ContentLength:" + httpConnection.getContentLength());
                fileSize = Long.parseLong(httpConnection.getHeaderField(sHeader));
                break;
            }
        }
        return fileSize;
    }

    class DownloadThread implements Callable<FileInfo> {
        long start;
        long end;
        long page;
        String fName;

        public DownloadThread(long start, long end, long page, String fName) {
            this.start = start;
            this.end = end;
            this.page = page;
            this.fName = fName;
        }

        @Override
        public FileInfo call() {
            return download(start, end, page, fName);
        }
    }
}

代碼都在本地親測(cè)（已修復(fù)Bug）可用，目前比較欠缺的是沒有實(shí)現(xiàn)在分片下載時(shí)對(duì)應(yīng)瀏覽器進(jìn)行下載展示，需要暫存在本地磁盤目錄。目前將代碼開源，希望能有更好解決方案的Coder Fork支持！也歡迎Star捧場(chǎng)。

博文參考了圖靈學(xué)院相關(guān)的分片下載案例教程，并修改了部分代碼實(shí)現(xiàn)：

WebUploader--基于SpringBoot搭建，Java文件上傳下載高階實(shí)戰(zhàn)

本文代碼已上傳至GitHub：

BurstDownload

到此這篇關(guān)于SpringBoot大文件RestTemplate下載解決方案的文章就介紹到這了,更多相關(guān)SpringBoot RestTemplate下載內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: