一个项目中文件保存方式有很多种:保存本地、ftp保存、nars盘、云存储、redis、数据库...... 但在一个分布式集群环境下如何选择呢?
1.本地保存
缺点:1.占用服务器资源,当文件量大时不推荐。2.在集群环境下实现不了资源共享
优点:本地存取效率高
2.ftp保存(推荐)----有ftp与sftp方式
缺点:需确保ftp服务器正常,若一台ftp服务器挂了,影响文件存取
优点:1.ftp与sftp方式唯一区别是,sftp方式传输时确保了文件安全。2.集群环境下也可使用
3.nars盘存储(推荐)
云存储、redis、数据库不推荐,项目中应用价值不高,云存储需要购买云存储服务器,没有必要为了文件存储去花费,且通过接口传输文件有风险;redis一般用来缓存常用数据,若大量文件放入redis,占用缓存资源,极为影响性能,甚者会造成项目崩溃;数据库存储是最为原始的方法,但对数据库内存要求很高,所以有其他选择还是别选数据库存储吧,而且数据库存取性能也不高。
以下介绍ftp文件传输 ftp 与sftp方式 开发过程
一、Ftp方式
@Component
public class FtpUtil {
private static Logger logger = LoggerFactory.getLogger(FtpUtil.class);
private static SysConfigService sysConfigService;
@Autowired
public void setSysConfigService(SysConfigService sysConfigService) {
FtpUtil.sysConfigService = sysConfigService;
}
private static SystemSerialService systemSerialService;
@Autowired
public void setSystemSerialService(SystemSerialService systemSerialService) {
FtpUtil.systemSerialService = systemSerialService;
}
/**
* 获取FTPClient对象
*
* @param ftpHost FTP主机服务器
* @param ftpPassword FTP 登录密码
* @param ftpUserName FTP登录用户名
* @param ftpPort FTP端口 默认为21
* @return
*/
private static FTPClient getFTPClient() {
FTPClient ftpClient = new FTPClient();
String ftpHost = sysConfigService.getValueBynType(SysConfigType.SYN_FTP_IP);
int ftpPort = CommonFun.strToInt(sysConfigService.getValueBynType(SysConfigType.SYN_FTP_PORT));
try {
ftpClient = new FTPClient();
ftpClient.connect(ftpHost, ftpPort);// 连接FTP服务器
ftpClient.login(sysConfigService.getValueBynType(SysConfigType.SYN_FTP_USERNAME), SercurityUtil.decrypt(sysConfigService.getValueBynType(SysConfigType.SYN_FTP_PASSWORD)));// 登陆FTP服务器
if (!FTPReply.isPositiveCompletion(ftpClient.getReplyCode())) {
logger.error("未连接到FTP,用户名或密码错误。FTPIP:{};FTPPORT:{}", ftpHost, ftpPort);
ftpClient.disconnect();
} else {
logger.info("FTP连接成功。FTPIP:{};FTPPORT:{}", ftpHost, ftpPort);
}
} catch (SocketException e) {
logger.error("FTP的IP地址可能错误,请正确配置。FTPIP:{};FTPPORT:{}", ftpHost, ftpPort, e);
} catch (IOException e) {
logger.error("FTP的端口错误,请正确配置。FTPIP:{};FTPPORT:{}", ftpHost, ftpPort, e);
}
return ftpClient;
}
/**
* 从FTP服务器下载文件
* @param ftpPath FTP服务器中文件所在路径 格式: ftptest/aa
* @param localPath 下载到本地的位置 格式:H:/download
* @param fileName 文件名称
* @return 成功返回true,否则返回false
* @since 2.1
*/
public static boolean downloadFtpFileOld(String ftpPath, String localPath, String fileName) {
boolean success = false;
FTPClient ftpClient = null;
try {
ftpClient = getFTPClient();
ftpClient.setControlEncoding("UTF-8"); // 中文支持
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
ftpClient.changeWorkingDirectory(ftpPath);
File localFile = new File(localPath + File.separatorChar + fileName);
OutputStream os = new FileOutputStream(localFile);
success = ftpClient.retrieveFile(fileName, os);
os.close();
ftpClient.logout();
logger.info("文件:{}下载至:{}结果:{}", ftpPath + File.separatorChar +fileName, localPath, success);
} catch (FileNotFoundException e) {
logger.error("没有找到{}文件", localPath + File.separatorChar + fileName, e);
} catch (SocketException e) {
logger.error("连接FTP失败。", e);
} catch (IOException e) {
logger.error("文件读取错误。", e);
}finally {
if (ftpClient.isConnected()) {
try {
ftpClient.disconnect();
} catch (IOException ioe) {
}
}
}
return success;
}
/**
* Description: 向FTP服务器上传文件
* @param ftpPath FTP服务器中文件所在路径 格式: ftptest/aa
* @param fileName ftp文件名称
* @param localPath 本地文件路径 格式:D:\\ee\\11测试.txt
* @return 成功返回true,否则返回false
* @throws UnsupportedEncodingException
*/
public static boolean uploadFile(String ftpPath, String fileName,String localPath) {
boolean success = false;
FTPClient ftpClient = null;
try {
int reply;
ftpClient = getFTPClient();
reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect();
return success;
}
ftpClient.setControlEncoding("UTF-8"); // 中文支持
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
ftpClient.changeWorkingDirectory(ftpPath);
FileInputStream input=new FileInputStream(new File(localPath));
success = ftpClient.storeFile(fileName, input);
input.close();
ftpClient.logout();
logger.info("文件:{}上传至{}。结果:{}", localPath, ftpPath, success);
} catch (IOException e) {
e.printStackTrace();
} finally {
if (ftpClient.isConnected()) {
try {
ftpClient.disconnect();
} catch (IOException ioe) {
}
}
}
return success;
}
/**
* Description: 向FTP服务器上传文件
* @param ftpPath FTP服务器中文件所在路径 格式: ftptest/aa
* @param fileName ftp文件名称
* @param localPath 本地文件路径 格式:D:\\44\\11测试.txt
* @param canCover 是否能被覆盖 true直接覆盖,false生成新的文件名并返回
* @return FileResult 成功返回result=true,否则返回result=false
* @throws UnsupportedEncodingException
*/
public static FileResult uploadFileStreamFtp(String ftpPath, String fileName, InputStream input, boolean canCover) {
FileResult fileResult = new FileResult(false);
FTPClient ftpClient = null;
try {
int reply;
ftpClient = getFTPClient();
reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect();
return fileResult;
}
ftpClient.setControlEncoding("UTF-8"); // 中文支持
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
ftpClient.changeWorkingDirectory(ftpPath);
if(!canCover){
String[] fileNameInfos = null;
if(fileName.contains(".")){
fileNameInfos = fileName.split("\\.");
}else{
logger.info("文件:{}上传至{}失败:文件名不符合要求", fileName, ftpPath);
return fileResult;
}
fileName = systemSerialService.getSequence("09","4") + "." + fileNameInfos[1];
}
boolean success = ftpClient.storeFile(fileName, input);
input.close();
ftpClient.logout();
fileResult.setResult(success);
fileResult.setFileName(fileName);
logger.info("文件:{}上传至{}。结果:{}", fileName, ftpPath, success);
} catch (IOException e) {
logger.info("文件:{}上传至{}异常:{}", fileName, ftpPath, e.getMessage(), e);
} finally {
if (ftpClient.isConnected()) {
try {
ftpClient.disconnect();
} catch (IOException ioe) {
}
}
}
return fileResult;
}
/**
* 删除FTP服务器上的文件
* @param ftpPath 文件路径 格式: ftptest/aa
* @param fileName 文件名
* @return 成功返回true,否则返回false
* @since 2.1
*/
public static boolean deleteFtpFile(String ftpPath, String fileName){
boolean success = false;
FTPClient ftpClient = null;
try {
int reply;
ftpClient = getFTPClient();
reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect();
return success;
}
ftpClient.setControlEncoding("UTF-8"); // 中文支持
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
ftpClient.changeWorkingDirectory(ftpPath);
success = ftpClient.deleteFile(ftpPath + File.separatorChar + fileName);
ftpClient.logout();
logger.info("FTP文件:{}删除。结果:{}", ftpPath + File.separatorChar + fileName, success);
} catch (IOException e) {
e.printStackTrace();
} finally {
if (ftpClient.isConnected()) {
try {
ftpClient.disconnect();
} catch (IOException ioe) {
}
}
}
return success;
}
// //测试
// public static void main(String[] args) throws IOException {
// String ftpPath = "/";
// String localPath = "D:\\";
// String fileName = "11测试.txt";
//
//上传一个文件
boolean test = FtpUtil.uploadFile(ftpPath, fileName,localPath);
System.out.println(test);
//
//下载一个文件
FtpUtil.downloadFtpFile(ftpPath, localPath, fileName);
//
// //删除一个文件
FtpUtil.deleteFtpFile(ftpPath, fileName);
//
// InputStream input = new ByteArrayInputStream("test ftp jyf".getBytes("UTF-8"));
// FileResult flag = FtpUtil.uploadFileStream(ftpPath, fileName,input, false);;
// System.out.println(flag.isResult() + ";" + flag.getFileName());
// }
}
二、SFTP方式
<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.54</version>
</dependency>
@Component
public class FtpUtil {
private static Logger logger = LoggerFactory.getLogger(FtpUtil.class);
private static SysConfigService sysConfigService;
@Autowired
public void setSysConfigService(SysConfigService sysConfigService) {
FtpUtil.sysConfigService = sysConfigService;
}
private static SystemSerialService systemSerialService;
@Autowired
public void setSystemSerialService(SystemSerialService systemSerialService) {
FtpUtil.systemSerialService = systemSerialService;
}
private static ChannelSftp sftp = null;
private static Session sshSession = null;
/**
* 获得SFTP Channel
* @return
* @since 2.1
*/
public static ChannelSftp getChannel(){
String ftpHost = sysConfigService.getValueBynType(SysConfigType.SYN_FTP_IP);
int ftpPort = CommonFun.strToInt(sysConfigService.getValueBynType(SysConfigType.SYN_FTP_PORT));
String ftpUserName = sysConfigService.getValueBynType(SysConfigType.SYN_FTP_USERNAME);
logger.info("FtpUtil-->getChannel--ftp连接开始>>>>>>host=" + ftpHost + ">>>port" + ftpPort + ">>>username=" + ftpUserName);
// 创建JSch对象
JSch jsch = new JSch();
try {
// 根据用户名,主机ip,端口获取一个Session对象
sshSession = jsch.getSession(ftpUserName, ftpHost, ftpPort);
logger.info("Session created.");
String ftpPassword = SercurityUtil.decrypt(sysConfigService.getValueBynType(SysConfigType.SYN_FTP_PASSWORD));
if (ftpPassword != null) {
// 设置密码
sshSession.setPassword(ftpPassword);
}
Properties configTemp = new Properties();
configTemp.put("StrictHostKeyChecking", "no");
// 为Session对象设置properties
sshSession.setConfig(configTemp);
// 设置timeout时间
// sshSession.setTimeout(5000);
sshSession.connect();
logger.info("ftp---Session connected.");
// 通过Session建立链接
Channel channel = sshSession.openChannel("sftp");
channel.connect();
logger.info("Opening Channel.");
// 打开SFTP通道
sftp = (ChannelSftp) channel;
logger.info("Connected successfully to ftpHost = {},as ftpUserName = {}, returning: ", ftpHost, ftpUserName, sftp);
} catch (JSchException e) {
logger.error("FtpUtil-->connect异常|ftpHost = {},as ftpUserName = {}|", ftpHost, ftpUserName, e.getMessage(), e);
}
return (ChannelSftp) sftp;
}
/**
* 断开SFTP Channel、Session连接
* @throws Exception
*/
public static void closeChannel(){
if (sftp != null) {
sftp.disconnect();
}
if (sshSession != null) {
sshSession.disconnect();
}
logger.info("disconnected SFTP successfully!");
}
/**
* Description: 向FTP服务器上传文件(sftp方式)
* @param directory FTP服务器中文件所在路径 格式: ftptest/aa
* @param fileName ftp文件名称
* @param localPath 本地文件路径 格式:D:\\03ff\\11测试.txt
* @param canCover 是否能被覆盖 true直接覆盖,false生成新的文件名并返回
* @return FileResult 成功返回result=true,否则返回result=false
*/
public static FileResult uploadFileStream(String directory, String fileName, InputStream input, boolean canCover){
logger.info(">>>>>>>>FtpUtil-->uploadFileStream--ftp上传文件(sft方式)开始>>>>>>>>>>>>>");
FileResult fileResult = new FileResult(false);
FileInputStream in = null;
getChannel();
try {
sftp.cd(directory);
}catch (SftpException e) {
try {
sftp.mkdir(directory);
sftp.cd(directory);
}catch (SftpException e1) {
logger.error("ftp创建文件路径失败,路径为:{}", directory);
return fileResult;
}
}
try {
if(!canCover){
String[] fileNameInfos = null;
if(fileName.contains(".")){
fileNameInfos = fileName.split("\\.");
}else{
logger.info("文件[{}]上传至[{}]失败:文件名不符合要求", fileName, directory);
return fileResult;
}
fileName = systemSerialService.getSequence("09","4") + "." + fileNameInfos[1];
}
sftp.put(input, fileName);
fileResult.setResult(true);
fileResult.setFileName(fileName);
}catch (SftpException e) {
logger.error("文件:{}上传到ftp:{}异常-->{}", fileName, directory, e.getMessage(), e);
}finally {
if (in != null){
try {
in.close();
}catch (IOException e) {
logger.error("Close stream error.{}", e.getMessage(), e);
}
}
closeChannel();
}
logger.info(">>>>>>>>FtpUtil-->uploadFile--ftp上传文件(sft方式)成功>>>>>>>>>>>>>");
return fileResult;
}
/**
* 从FTP服务器下载文件(sft方式)
* @param ftpPath FTP服务器中文件所在路径 格式: ftptest/aa
* @param localPath 下载到本地的位置 格式:H:/download
* @param fileName 文件名称
* @return 成功返回true,否则返回false
* @since 2.1
*/
public static boolean downloadFtpFile(String ftpPath, String localPath, String fileName) {
boolean success = false;
logger.info(">>>>>>>>FtpUtil-->downloadFile--ftp[{}]下载文件(sft方式)[{}]开始>>>>>>>>>>>>>", ftpPath, fileName);
getChannel();
File localFile = null;
OutputStream output = null;
try {
localFile = new File(localPath + File.separatorChar + fileName);
if (localFile.exists()){
localFile.delete();
}
localFile.createNewFile();
sftp.cd(ftpPath);
output = new FileOutputStream(localFile);
sftp.get(fileName, output);
success = true;
logger.info("===DownloadFile:{} success from sftp.", localPath + File.separatorChar + fileName);
}catch (SftpException e) {
logger.error("ftp目录或者文件异常,检查ftp目录[{}]和文件[{}].{}", ftpPath ,fileName, e.getMessage(), e);
}catch (FileNotFoundException e) {
logger.error("没有找到{}文件", localPath + File.separatorChar + fileName, e);
}catch (IOException e) {
logger.error("创建本地文件失败{}文件", localPath + File.separatorChar + fileName, e);
}finally {
if (output != null) {
try {
output.close();
}catch (IOException e) {
logger.error("Close stream error.{}", e.getMessage(), e);
}
}
closeChannel();
}
logger.info(">>>>>>>>FtpUtil-->downloadFile--ftp[{}]下载文件[{}](sftp方式)结束>>>>>>>>>>>>>", ftpPath, fileName);
return success;
}
}