提示：以下是本篇文章正文内容，下面案例可供参考

一、简略的分片上传

针对第一个问题，假如文件过大，上传到一半断开了，若重新开始上传的话，会很消耗时间，并且你也并不知道距离上次断开时，已经上传到哪一部分了。因而我们应该先对大文件进行分片处理，防止上面说到的问题。

前端代码：

<!DOCTYPE html>
<html>
<head>
    <title>文件上传示例</title>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
</head>
<body>
<form>
    <input type="file" id="fileInput" multiple>
    <button type="button" onclick="upload()" >上传</button>
</form>
<script>
    function upload() {
        var fileInput = document.getElementById('fileInput');
        var fileName = document.getElementById("fileInput").files[0].name;
        var files = fileInput.files;
        var chunkSize = 1024 * 10; // 每个块的巨细为10KB
        var totalChunks = Math.ceil(files[0].size / chunkSize); // 文件总块数
        var currentChunk = 0; // 当时块数
        // 分片上传文件
        function uploadChunk() {
            var xhr = new XMLHttpRequest();
            var formData = new FormData();
            // 将当时块数和总块数添加到formData中
            formData.append('currentChunk', currentChunk);
            formData.append('totalChunks', totalChunks);
            formData.append('fileName',fileName);
            // 核算当时块在文件中的偏移量和长度
            var start = currentChunk * chunkSize;
            var end = Math.min(files[0].size, start + chunkSize);
            var chunk = files[0].slice(start, end);
            // 添加当时块到formData中
            formData.append('chunk', chunk);
            // 发送分片到后端
            xhr.open('POST', '/file/upload');
            xhr.send(formData);
            xhr.onload = function() {
                // 更新当时块数
                currentChunk++;
                // 假如还有未上传的块，则持续上传
                if (currentChunk < totalChunks) {
                    uploadChunk();
                } else {
                    // 一切块都上传完毕，进行文件兼并
                    mergeChunks(fileName);
                }
            }
        }
        // 兼并一切分片
        function mergeChunks() {
            var xhr = new XMLHttpRequest();
            xhr.open("POST", "/file/merge", true);
            xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
            xhr.onreadystatechange = function() {
                if (xhr.readyState === 4) {
                    if (xhr.status === 200) {
                        console.log("文件上传完结：", xhr.responseText);
                    } else {
                        console.error(xhr.responseText);
                    }
                }
            };
            xhr.send("fileName=" + fileName);
        }
        // 开始上传
        uploadChunk();
    }
</script>
</body>
</html>

ps：以上代码运用了html+js完结，请求是运用了xhr来发送请求。其中xhr.open的地址为自己本地的接口地址。因为平时测验并不需求真正上传大型文件，所以每个分片的巨细界说为10KB，以此模仿大文件上传。

后端代码：

@RestController
@RequestMapping("/file")
public class FileController {
    @Autowired
    private ResourceLoader resourceLoader;
    @Value("${my.config.savePath}")
    private String uploadPath;
    private Map<String, List<File>> chunksMap = new ConcurrentHashMap<>();
    @PostMapping("/upload")
    public void upload(@RequestParam int currentChunk, @RequestParam int totalChunks,
                       @RequestParam MultipartFile chunk,@RequestParam String fileName) throws IOException {
        // 将分片保存到临时文件夹中
        String chunkName = chunk.getOriginalFilename() + "." + currentChunk;
        File chunkFile = new File(uploadPath, chunkName);
        chunk.transferTo(chunkFile);
        // 记录分片上传状况
        List<File> chunkList = chunksMap.get(fileName);
        if (chunkList == null) {
            chunkList = new ArrayList<>(totalChunks);
            chunksMap.put(fileName, chunkList);
        }
        chunkList.add(chunkFile);
    }
    @PostMapping("/merge")
    public String merge(@RequestParam String fileName) throws IOException {
        // 获取一切分片，并依照分片的顺序将它们兼并成一个文件
        List<File> chunkList = chunksMap.get(fileName);
        if (chunkList == null || chunkList.size() == 0) {
            throw new RuntimeException("分片不存在");
        }
        File outputFile = new File(uploadPath, fileName);
        try (FileChannel outChannel = new FileOutputStream(outputFile).getChannel()) {
            for (int i = 0; i < chunkList.size(); i++) {
                try (FileChannel inChannel = new FileInputStream(chunkList.get(i)).getChannel()) {
                    inChannel.transferTo(0, inChannel.size(), outChannel);
                }
                chunkList.get(i).delete(); // 删去分片
            }
        }
        chunksMap.remove(fileName); // 删去记录
        // 获取文件的访问URL
        Resource resource = 
        			resourceLoader.getResource("file:" + uploadPath + fileName); //因为是本地文件，所以最初是"file"，假如是服务器，请改成自己服务器前缀
        return resource.getURI().toString();
    }
}

ps: 运用一个map记录上传了哪些分片，这里将分片存在了本地的文件夹，等到分片都上传完结后兼并并删去分片。用ConcurrentHashMap代替HashMap是因为它在多线程下是安全的。

以上仅仅一个简略的文件上传代码，可是只要在这上面另做修改就能够处理上面说到的问题。

二、处理问题

1. 怎样防止很多的硬盘读写

上面代码有一个坏处，就是将分片的内容存在了本地的文件夹里。而且在兼并的时候判别上传是否完全也是从文件夹读取文件的。对磁盘的很多读写操作不只速度慢，还会导致服务器崩溃，因而下面代码运用了redis来存储分片信息，防止对磁盘过多读写。（你也能够运用mysql或许其他中间件来存储信息，因为读写尽量不要在mysql，所以我运用了redis）。

2.目标文件过大，假如在上传过程中止开了怎样办

运用redis来存储分片内容，当断开后，文件信息还是存储在redis中，用户再次上传时，检测redis是否有该分片的内容，假如有则越过。

3. 前端页面上传的文件数据与原文件数据不一致该怎么发现

前端在调用上传接口时，先核算文件的校验和，然后将文件和校验和同时传给后端，后端对文件再核算一次校验和，两个校验和进行比照，假如持平，则阐明数据一致，假如不一致则报错，让前端重新上传该片段。
js核算校验和代码：

    // 核算文件的 SHA-256 校验和
    function calculateHash(fileChunk) {
        return new Promise((resolve, reject) => {
            const blob = new Blob([fileChunk]);
            const reader = new FileReader();
            reader.readAsArrayBuffer(blob);
            reader.onload = () => {
                const arrayBuffer = reader.result;
                const crypto = window.crypto || window.msCrypto;
                const digest = crypto.subtle.digest("SHA-256", arrayBuffer);
                digest.then(hash => {
                    const hashArray = Array.from(new Uint8Array(hash));
                    const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('');
                    resolve(hashHex);
                });
            };
            reader.onerror = () => {
                reject(new Error('Failed to calculate hash'));
            };
        });
    }

    public static String calculateHash(byte[] fileChunk) throws Exception {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        md.update(fileChunk);
        byte[] hash = md.digest();
        ByteBuffer byteBuffer = ByteBuffer.wrap(hash);
        StringBuilder hexString = new StringBuilder();
        while (byteBuffer.hasRemaining()) {
            hexString.append(String.format("%02x", byteBuffer.get()));
        }
        return hexString.toString();
    }

注意点:

这里前端和后端核算校验和的算法一定要是一致的，否则得不到相同的成果。
在前端中运用了crypto对文件进行核算,需求引进相关的js。
你能够运用script引进也能够直接下载js

<script src="https://cdn.bootcss.com/crypto-js/3.1.9-1/crypto-js.min.js"></script>

crypto的下载地址假如github打不开，或许需求运用npm下载了

4. 上传过程中假如断开了应该怎么判别哪些分片没有上传

对redis检测哪个分片的下标不存在，若不存在则存入list，最后将list回来给前端

	boolean allChunksUploaded = true;
	List<Integer> missingChunkIndexes = new ArrayList<>();
	 for (int i = 0; i < hashMap.size(); i++) {
	     if (!hashMap.containsKey(String.valueOf(i))) {
	         allChunksUploaded = false;
	         missingChunkIndexes.add(i);
	     }
	 }
	 if (!allChunksUploaded) {
	     return ResponseEntity.status(HttpStatus.BAD_REQUEST).body(missingChunkIndexes);
	 }

三、完好代码

1、引进依靠

 <dependency>
  	<groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
    <version>6.1.4.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

lettuce是一个Redis客户端，你也能够不引进，直接运用redisTemplat就行了

2、前端代码

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>File Upload Demo</title>
</head>
<body>
<input type="file" id="fileInput" multiple>
<button type="button" onclick="uploadFile()" >上传</button>
<div id="progressBar"></div>
<script src="https://cdn.bootcss.com/crypto-js/3.1.9-1/crypto-js.min.js"></script>
<script>
    var fileId = "";
    var fileName = null;
    var file;
    const chunkSize = 1024 * 10; // 每个分片的巨细10KB
    async function uploadFile() {
        var fileInput = document.getElementById('fileInput');
        file = fileInput.files[0];
        fileName = document.getElementById("fileInput").files[0].name;
        // 分片上传文件
        const chunks = Math.ceil(file.size / chunkSize);
        for (let i = 0; i < chunks; i++) {
            try {
                await uploadChunk(file, i);
            } catch (error) {
                console.error('Failed to upload chunk', i, error);
                // 假如上传失败，则测验康复上传
                try {
                    await uploadChunk(file, i);
                } catch (error) {
                    console.error('Failed to resume upload', i, error);
                    return;
                }
            }
        }
        // 兼并文件
        try {
            const fileUrl = await mergeFile();
            console.log('File URL:', fileUrl);
        } catch (error) {
            console.error('Failed to merge file', error);
        }
    }
    function uploadChunk(file, chunkIndex) {
        return new Promise((resolve, reject) => {
            let fileTemp = file.slice(chunkIndex * chunkSize, (chunkIndex + 1) * chunkSize);
            var myPromise = calculateHash(fileTemp);
            myPromise.then(result =>{
                const formData = new FormData();
                formData.append('chunk',fileTemp);
                formData.append('chunkIndex', chunkIndex);
                formData.append('chunkChecksum', result);
                formData.append('chunkSize', chunkSize);
                formData.append('fileId',fileId);
                const xhr = new XMLHttpRequest();
                xhr.open('POST', '/hospital/file2/upload', true);
                xhr.onload = () => {
                    if (xhr.status === 200) {
                        resolve(xhr.response);
                        fileId = xhr.responseText
                    } else {
                        reject(xhr.statusText);
                    }
                };
                xhr.onerror = () => {
                    reject(xhr.statusText);
                };
                xhr.send(formData);
            })
        });
    }
    function mergeFile() {
        return new Promise((resolve, reject) => {
            const xhr = new XMLHttpRequest();
            const formData = new FormData();
            formData.append('fileId',fileId);
            formData.append('fileName',fileName);
            xhr.open('POST', '/hospital/file2/merge', true);
            xhr.onload = () => {
                if (xhr.status === 200) {
                    resolve(xhr.response);
                } else {
                    reject(xhr.statusText);
                    resume(xhr.response.replace(/\[|]/g,'').split(','));
                }
            };
            xhr.onerror = () => {
                reject(xhr.statusText);
            };
            xhr.send(formData);
        });
    }
    async function resume(list){
        for (let i = 0; i < list.length; i++) {
            try {
                await uploadChunk(file, i);
            } catch (error) {
                console.error('Failed to upload chunk', i, error);
                // 假如上传失败，则测验康复上传
                try {
                    await uploadChunk(file, i);
                } catch (error) {
                    console.error('Failed to resume upload', i, error);
                    return;
                }
            }
        }
        // 兼并文件
        try {
            const fileUrl = await mergeFile();
            console.log('File URL:', fileUrl);
        } catch (error) {
            console.error('Failed to merge file', error);
        }
    }
    // 核算文件的 SHA-256 校验和
    function calculateHash(fileChunk) {
        return new Promise((resolve, reject) => {
            const blob = new Blob([fileChunk]);
            const reader = new FileReader();
            reader.readAsArrayBuffer(blob);
            reader.onload = () => {
                const arrayBuffer = reader.result;
                const crypto = window.crypto || window.msCrypto;
                const digest = crypto.subtle.digest("SHA-256", arrayBuffer);
                digest.then(hash => {
                    const hashArray = Array.from(new Uint8Array(hash));
                    const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('');
                    resolve(hashHex);
                });
            };
            reader.onerror = () => {
                reject(new Error('Failed to calculate hash'));
            };
        });
    }
</script>
</body>
</html>

3、后端接口代码

@RestController
@RequestMapping("/file2")
public class File2Controller {
    private static final String FILE_UPLOAD_PREFIX = "file_upload:";
    @Autowired
    private ResourceLoader resourceLoader;
    @Value("${my.config.savePath}")
    private String uploadPath;
    @Autowired
    private ThreadLocal<RedisConnection> redisConnectionThreadLocal;
//    @Autowired
//    private RedisTemplate redisTemplate;
    @PostMapping("/upload")
    public ResponseEntity<?> uploadFile(@RequestParam("chunk") MultipartFile chunk,
                                        @RequestParam("chunkIndex") Integer chunkIndex,
                                        @RequestParam("chunkSize") Integer chunkSize,
                                        @RequestParam("chunkChecksum") String chunkChecksum,
                                        @RequestParam("fileId") String fileId) throws Exception {
        if (StringUtils.isBlank(fileId) || StringUtils.isEmpty(fileId)) {
            fileId = UUID.randomUUID().toString();
        }
        String key = FILE_UPLOAD_PREFIX + fileId;
        byte[] chunkBytes = chunk.getBytes();
        String actualChecksum = calculateHash(chunkBytes);
        if (!chunkChecksum.equals(actualChecksum)) {
            return ResponseEntity.status(HttpStatus.BAD_REQUEST).body("Chunk checksum does not match");
        }
//        if(!redisTemplate.opsForHash().hasKey(key,String.valueOf(chunkIndex))) {
//            redisTemplate.opsForHash().put(key, String.valueOf(chunkIndex), chunkBytes);
//        }
        RedisConnection connection = redisConnectionThreadLocal.get();
        Boolean flag = connection.hExists(key.getBytes(), String.valueOf(chunkIndex).getBytes());
        if (flag==null || flag == false) {
            connection.hSet(key.getBytes(), String.valueOf(chunkIndex).getBytes(), chunkBytes);
        }
        return ResponseEntity.ok(fileId);
    }
    public static String calculateHash(byte[] fileChunk) throws Exception {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        md.update(fileChunk);
        byte[] hash = md.digest();
        ByteBuffer byteBuffer = ByteBuffer.wrap(hash);
        StringBuilder hexString = new StringBuilder();
        while (byteBuffer.hasRemaining()) {
            hexString.append(String.format("%02x", byteBuffer.get()));
        }
        return hexString.toString();
    }
    @PostMapping("/merge")
    public ResponseEntity<?> mergeFile(@RequestParam("fileId") String fileId, @RequestParam("fileName") String fileName) throws IOException {
        String key = FILE_UPLOAD_PREFIX + fileId;
        RedisConnection connection = redisConnectionThreadLocal.get();
        try {
            Map<byte[], byte[]> chunkMap = connection.hGetAll(key.getBytes());
//            Map chunkMap = redisTemplate.opsForHash().entries(key);
            if (chunkMap.isEmpty()) {
                return ResponseEntity.status(HttpStatus.NOT_FOUND).body("File not found");
            }
            Map<String,byte[]> hashMap = new HashMap<>();
            for(Map.Entry<byte[],byte[]> entry :chunkMap.entrySet()){
                hashMap.put((new String(entry.getKey())),entry.getValue());
            }
            // 检测是否一切分片都上传了
            boolean allChunksUploaded = true;
            List<Integer> missingChunkIndexes = new ArrayList<>();
            for (int i = 0; i < hashMap.size(); i++) {
                if (!hashMap.containsKey(String.valueOf(i))) {
                    allChunksUploaded = false;
                    missingChunkIndexes.add(i);
                }
            }
            if (!allChunksUploaded) {
                return ResponseEntity.status(HttpStatus.BAD_REQUEST).body(missingChunkIndexes);
            }
            File outputFile = new File(uploadPath, fileName);
            boolean flag = mergeChunks(hashMap, outputFile);
            Resource resource = resourceLoader.getResource("file:" + uploadPath + fileName);
            if (flag == true) {
                connection.del(key.getBytes());
//                redisTemplate.delete(key);
                return ResponseEntity.ok().body(resource.getURI().toString());
            } else {
                return ResponseEntity.status(555).build();
            }
        } catch (Exception e) {
            e.printStackTrace();
            return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(e.getMessage());
        }
    }
    private boolean mergeChunks(Map<String, byte[]> chunkMap, File destFile) {
        try (FileOutputStream outputStream = new FileOutputStream(destFile)) {
            // 将分片依照顺序兼并
            for (int i = 0; i < chunkMap.size(); i++) {
                byte[] chunkBytes = chunkMap.get(String.valueOf(i));
                outputStream.write(chunkBytes);
            }
            return true;
        } catch (IOException e) {
            e.printStackTrace();
            return false;
        }
    }
}

4、redis装备

@Configuration
public class RedisConfig {
    @Value("${spring.redis.host}")
    private String host;
    @Value("${spring.redis.port}")
    private int port;
    @Value("${spring.redis.password}")
    private String password;
    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        RedisStandaloneConfiguration config = new RedisStandaloneConfiguration();
        config.setHostName(host);
        config.setPort(port);
        config.setPassword(RedisPassword.of(password));
        return new LettuceConnectionFactory(config);
    }
    @Bean
    public ThreadLocal<RedisConnection> redisConnectionThreadLocal(RedisConnectionFactory redisConnectionFactory) {
        return ThreadLocal.withInitial(() -> redisConnectionFactory.getConnection());
    }
}

运用 redisConnectionThreadLocal 是为了防止屡次建立衔接，很耗时间

总结

以上就是该功能的完好代码。运用代码记住修改uploadPath，防止代码找不到目录途径。在代码最后，能够运用mysql对整个文件核算校验和，将校验和成果和文件名、文件巨细、文件类型存入数据库中，鄙人次大文件上传前先判别是否存在。若存在就不要上传防止占用空间。

java实现文件分片上传并且断点续传

一、简略的分片上传

前端代码：

后端代码：

二、处理问题

1. 怎样防止很多的硬盘读写

2.目标文件过大，假如在上传过程中止开了怎样办

3. 前端页面上传的文件数据与原文件数据不一致该怎么发现

4. 上传过程中假如断开了应该怎么判别哪些分片没有上传

三、完好代码

1、引进依靠

2、前端代码

3、后端接口代码

4、redis装备

总结

作者信息

java实现文件分片上传并且断点续传

一、简略的分片上传

前端代码：

后端代码：

二、处理问题

1. 怎样防止很多的硬盘读写

2.目标文件过大，假如在上传过程中止开了怎样办

3. 前端页面上传的文件数据与原文件数据不一致该怎么发现

4. 上传过程中假如断开了应该怎么判别哪些分片没有上传

三、完好代码

1、引进依靠

2、前端代码

3、后端接口代码

4、redis装备

总结

相关文章

面试整理-okHttp源码分析

基于Vue 3 + TypeScript + Vite + TSX + Pinia 开发的一个音乐播放器

从源代码构建TensorFlow流程记录

Java8 lambda 表达式 forEach 如何提前终止？

作者信息