zoukankan      html  css  js  c++  java
  • :Apache FTPClient操作“卡死”问题的分析和解决

        最近在和一个第三方的合作中不得已需要使用FTP文件接口。由于FTP Server由对方提供,而且双方背后各自的网络环境都很不单纯等等原因,造成测试环境无法模拟实际情况。测试环境中程序一切正常,但是在部署到生产环境之后发现FTP操作不规律性出现“卡死”现象:程序捕获不到任何异常一直卡着,导致轮巡无法正常工作(由于担心在轮巡时间间隔内处理不能完成,我没有采用类似quartz或者crontab的定时任务,而是采用while-true然后sleep的方式)。

        为了解决这个问题,我首先考虑的是对于FTPClient的使用上没有设置超时时间,于是设置了ConnectTimeout、DataTimeout、DefaultTimeout后在生产环境上继续观察,但是问题依旧没有解决。后来我有些怀疑FTPClient api本身是不是有什么问题,想实在不行自己实现一个超时机制吧,不过还是不甘心,还是想从FTPClient api本身去解决问题。又经过一翻研究之后发现:需要使用被动模式,以下摘抄别人的一段简单描述:
    在项目中使用commons-net-3.0.1.jar实现FTP文件的下载,在windows xp上运行正常,但是放到linux上,却出现问题,程序运行到 FTPClient.listFiles()或者FTPClient.retrieveFile()方法时,就停止在那里,什么反应都没有,出现假死状态。google一把,发现很多人也出现了此类问题,最终在一个帖子里找到了解决办法。在调用这两个方法之前,调用FTPClient.enterLocalPassiveMode();这个方法的意思就是每次数据连接之前,ftp client告诉ftp server开通一个端口来传输数据。为什么要这样做呢,因为ftp server可能每次开启不同的端口来传输数据,但是在linux上,由于安全限制,可能某些端口没有开启,所以就出现阻塞。OK,问题解决。

        于是我回滚了之前的修改,改为被动模式(关于FTP主动/被动模式的解释,这里我不多说了,关注的朋友可以自己查阅)。但是问题依旧。于是能想到的就是最有的绝招:实在不行自己实现一个超时机制吧。经过一翻研究最简单的方式就是使用:Future解决:

    复制代码
     1 public static void main(String[] args) throws InterruptedException, ExecutionException {
     2         final ExecutorService exec = Executors.newFixedThreadPool(1);
     3 
     4         Callable<String> call = new Callable<String>() {
     5             public String call() throws Exception {
     6                 Thread.sleep(1000 * 5);
     7                 return "线程执行完成.";
     8             }
     9         };
    10 
    11         try {
    12             Future<String> future = exec.submit(call);
    13             String obj = future.get(4 * 1000, TimeUnit.MILLISECONDS); // 任务处理超时时间设置
    14             System.out.println("任务成功返回:" + obj);
    15         } catch (TimeoutException ex) {
    16             System.out.println("处理超时啦....");
    17             ex.printStackTrace();
    18         } catch (Exception e) {
    19             System.out.println("处理失败.");
    20             e.printStackTrace();
    21         }
    22         // 关闭线程池
    23         exec.shutdown();
    24         
    25         System.out.println("完毕");
    26     }
    复制代码

    当然了还有很多其他方式:
    http://tech.sina.com.cn/s/2008-07-04/1051720260.shtml
    http://itindex.net/blog/2010/08/11/1281486125717.html
    http://darkmasky.iteye.com/blog/1115047
    http://www.cnblogs.com/wasp520/archive/2012/07/06/2580101.html
    http://coolxing.iteye.com/blog/1476289
    http://www.cnblogs.com/chenying99/archive/2012/10/24/2737924.html

        虽然找到了终极的“必杀技”,但是此时我还是不甘心,还是想从FTPClient api本身去解决问题,但此时看来也别无它他法。只能试试:即设置被动模式又设置超时时间。经过实际测试,发现问题得以解决。下面把我的FTP工具类贴给大家分享,希望能帮到遇到同样问题的人。

    复制代码
      1 import org.apache.commons.net.ftp.FTP;
      2 import org.apache.commons.net.ftp.FTPClient;
      3 import org.apache.commons.net.ftp.FTPFile;
      4 import org.apache.commons.net.ftp.FTPReply;
      5 
      6 import java.io.BufferedInputStream;
      7 import java.io.BufferedOutputStream;
      8 import java.io.File;
      9 import java.io.FileInputStream;
     10 import java.io.FileNotFoundException;
     11 import java.io.FileOutputStream;
     12 import java.io.IOException;
     13 import java.io.InputStream;
     14 import java.io.OutputStream;
     15 import java.net.UnknownHostException;
     16 import java.util.ArrayList;
     17 import java.util.List;
     18 
     19 public class FtpUtil {
     20     public static final String ANONYMOUS_LOGIN = "anonymous";
     21     private FTPClient ftp;
     22     private boolean is_connected;
     23 
     24     public FtpUtil() {
     25         ftp = new FTPClient();
     26         is_connected = false;
     27     }
     28     
     29     public FtpUtil(int defaultTimeoutSecond, int connectTimeoutSecond, int dataTimeoutSecond){
     30         ftp = new FTPClient();
     31         is_connected = false;
     32         
     33         ftp.setDefaultTimeout(defaultTimeoutSecond * 1000);
     34         ftp.setConnectTimeout(connectTimeoutSecond * 1000);
     35         ftp.setDataTimeout(dataTimeoutSecond * 1000);
     36     }
     37 
     38     /**
     39      * Connects to FTP server.
     40      * 
     41      * @param host
     42      *            FTP server address or name
     43      * @param port
     44      *            FTP server port
     45      * @param user
     46      *            user name
     47      * @param password
     48      *            user password
     49      * @param isTextMode
     50      *            text / binary mode switch
     51      * @throws IOException
     52      *             on I/O errors
     53      */
     54     public void connect(String host, int port, String user, String password, boolean isTextMode) throws IOException {
     55         // Connect to server.
     56         try {
     57             ftp.connect(host, port);
     58         } catch (UnknownHostException ex) {
     59             throw new IOException("Can't find FTP server '" + host + "'");
     60         }
     61 
     62         // Check rsponse after connection attempt.
     63         int reply = ftp.getReplyCode();
     64         if (!FTPReply.isPositiveCompletion(reply)) {
     65             disconnect();
     66             throw new IOException("Can't connect to server '" + host + "'");
     67         }
     68 
     69         if (user == "") {
     70             user = ANONYMOUS_LOGIN;
     71         }
     72 
     73         // Login.
     74         if (!ftp.login(user, password)) {
     75             is_connected = false;
     76             disconnect();
     77             throw new IOException("Can't login to server '" + host + "'");
     78         } else {
     79             is_connected = true;
     80         }
     81 
     82         // Set data transfer mode.
     83         if (isTextMode) {
     84             ftp.setFileType(FTP.ASCII_FILE_TYPE);
     85         } else {
     86             ftp.setFileType(FTP.BINARY_FILE_TYPE);
     87         }
     88     }
     89 
     90     /**
     91      * Uploads the file to the FTP server.
     92      * 
     93      * @param ftpFileName
     94      *            server file name (with absolute path)
     95      * @param localFile
     96      *            local file to upload
     97      * @throws IOException
     98      *             on I/O errors
     99      */
    100     public void upload(String ftpFileName, File localFile) throws IOException {
    101         // File check.
    102         if (!localFile.exists()) {
    103             throw new IOException("Can't upload '" + localFile.getAbsolutePath() + "'. This file doesn't exist.");
    104         }
    105 
    106         // Upload.
    107         InputStream in = null;
    108         try {
    109 
    110             // Use passive mode to pass firewalls.
    111             ftp.enterLocalPassiveMode();
    112 
    113             in = new BufferedInputStream(new FileInputStream(localFile));
    114             if (!ftp.storeFile(ftpFileName, in)) {
    115                 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
    116             }
    117 
    118         } finally {
    119             try {
    120                 in.close();
    121             } catch (IOException ex) {
    122             }
    123         }
    124     }
    125 
    126     /**
    127      * Downloads the file from the FTP server.
    128      * 
    129      * @param ftpFileName
    130      *            server file name (with absolute path)
    131      * @param localFile
    132      *            local file to download into
    133      * @throws IOException
    134      *             on I/O errors
    135      */
    136     public void download(String ftpFileName, File localFile) throws IOException {
    137         // Download.
    138         OutputStream out = null;
    139         try {
    140             // Use passive mode to pass firewalls.
    141             ftp.enterLocalPassiveMode();
    142 
    143             // Get file info.
    144             FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
    145             if (fileInfoArray == null) {
    146                 throw new FileNotFoundException("File " + ftpFileName + " was not found on FTP server.");
    147             }
    148 
    149             // Check file size.
    150             FTPFile fileInfo = fileInfoArray[0];
    151             long size = fileInfo.getSize();
    152             if (size > Integer.MAX_VALUE) {
    153                 throw new IOException("File " + ftpFileName + " is too large.");
    154             }
    155 
    156             // Download file.
    157             out = new BufferedOutputStream(new FileOutputStream(localFile));
    158             if (!ftp.retrieveFile(ftpFileName, out)) {
    159                 throw new IOException("Error loading file " + ftpFileName + " from FTP server. Check FTP permissions and path.");
    160             }
    161 
    162             out.flush();
    163         } finally {
    164             if (out != null) {
    165                 try {
    166                     out.close();
    167                 } catch (IOException ex) {
    168                 }
    169             }
    170         }
    171     }
    172 
    173     /**
    174      * Removes the file from the FTP server.
    175      * 
    176      * @param ftpFileName
    177      *            server file name (with absolute path)
    178      * @throws IOException
    179      *             on I/O errors
    180      */
    181     public void remove(String ftpFileName) throws IOException {
    182         if (!ftp.deleteFile(ftpFileName)) {
    183             throw new IOException("Can't remove file '" + ftpFileName + "' from FTP server.");
    184         }
    185     }
    186 
    187     /**
    188      * Lists the files in the given FTP directory.
    189      * 
    190      * @param filePath
    191      *            absolute path on the server
    192      * @return files relative names list
    193      * @throws IOException
    194      *             on I/O errors
    195      */
    196     public List<String> list(String filePath) throws IOException {
    197         List<String> fileList = new ArrayList<String>();
    198         
    199         // Use passive mode to pass firewalls.
    200         ftp.enterLocalPassiveMode();
    201         
    202         FTPFile[] ftpFiles = ftp.listFiles(filePath);
    203         int size = (ftpFiles == null) ? 0 : ftpFiles.length;
    204         for (int i = 0; i < size; i++) {
    205             FTPFile ftpFile = ftpFiles[i];
    206             if (ftpFile.isFile()) {
    207                 fileList.add(ftpFile.getName());
    208             }
    209         }
    210         
    211         return fileList;
    212     }
    213 
    214     /**
    215      * Sends an FTP Server site specific command
    216      * 
    217      * @param args
    218      *            site command arguments
    219      * @throws IOException
    220      *             on I/O errors
    221      */
    222     public void sendSiteCommand(String args) throws IOException {
    223         if (ftp.isConnected()) {
    224             try {
    225                 ftp.sendSiteCommand(args);
    226             } catch (IOException ex) {
    227             }
    228         }
    229     }
    230 
    231     /**
    232      * Disconnects from the FTP server
    233      * 
    234      * @throws IOException
    235      *             on I/O errors
    236      */
    237     public void disconnect() throws IOException {
    238 
    239         if (ftp.isConnected()) {
    240             try {
    241                 ftp.logout();
    242                 ftp.disconnect();
    243                 is_connected = false;
    244             } catch (IOException ex) {
    245             }
    246         }
    247     }
    248 
    249     /**
    250      * Makes the full name of the file on the FTP server by joining its path and
    251      * the local file name.
    252      * 
    253      * @param ftpPath
    254      *            file path on the server
    255      * @param localFile
    256      *            local file
    257      * @return full name of the file on the FTP server
    258      */
    259     public String makeFTPFileName(String ftpPath, File localFile) {
    260         if (ftpPath == "") {
    261             return localFile.getName();
    262         } else {
    263             String path = ftpPath.trim();
    264             if (path.charAt(path.length() - 1) != '/') {
    265                 path = path + "/";
    266             }
    267 
    268             return path + localFile.getName();
    269         }
    270     }
    271 
    272     /**
    273      * Test coonection to ftp server
    274      * 
    275      * @return true, if connected
    276      */
    277     public boolean isConnected() {
    278         return is_connected;
    279     }
    280 
    281     /**
    282      * Get current directory on ftp server
    283      * 
    284      * @return current directory
    285      */
    286     public String getWorkingDirectory() {
    287         if (!is_connected) {
    288             return "";
    289         }
    290 
    291         try {
    292             return ftp.printWorkingDirectory();
    293         } catch (IOException e) {
    294         }
    295 
    296         return "";
    297     }
    298 
    299     /**
    300      * Set working directory on ftp server
    301      * 
    302      * @param dir
    303      *            new working directory
    304      * @return true, if working directory changed
    305      */
    306     public boolean setWorkingDirectory(String dir) {
    307         if (!is_connected) {
    308             return false;
    309         }
    310 
    311         try {
    312             return ftp.changeWorkingDirectory(dir);
    313         } catch (IOException e) {
    314         }
    315 
    316         return false;
    317     }
    318 
    319     /**
    320      * Change working directory on ftp server to parent directory
    321      * 
    322      * @return true, if working directory changed
    323      */
    324     public boolean setParentDirectory() {
    325         if (!is_connected) {
    326             return false;
    327         }
    328 
    329         try {
    330             return ftp.changeToParentDirectory();
    331         } catch (IOException e) {
    332         }
    333 
    334         return false;
    335     }
    336 
    337     /**
    338      * Get parent directory name on ftp server
    339      * 
    340      * @return parent directory
    341      */
    342     public String getParentDirectory() {
    343         if (!is_connected) {
    344             return "";
    345         }
    346 
    347         String w = getWorkingDirectory();
    348         setParentDirectory();
    349         String p = getWorkingDirectory();
    350         setWorkingDirectory(w);
    351 
    352         return p;
    353     }
    354 
    355     /**
    356      * Get directory contents on ftp server
    357      * 
    358      * @param filePath
    359      *            directory
    360      * @return list of FTPFileInfo structures
    361      * @throws IOException
    362      */
    363     public List<FfpFileInfo> listFiles(String filePath) throws IOException {
    364         List<FfpFileInfo> fileList = new ArrayList<FfpFileInfo>();
    365         
    366         // Use passive mode to pass firewalls.
    367         ftp.enterLocalPassiveMode();
    368         FTPFile[] ftpFiles = ftp.listFiles(filePath);
    369         int size = (ftpFiles == null) ? 0 : ftpFiles.length;
    370         for (int i = 0; i < size; i++) {
    371             FTPFile ftpFile = ftpFiles[i];
    372             FfpFileInfo fi = new FfpFileInfo();
    373             fi.setName(ftpFile.getName());
    374             fi.setSize(ftpFile.getSize());
    375             fi.setTimestamp(ftpFile.getTimestamp());
    376             fi.setType(ftpFile.isDirectory());
    377             fileList.add(fi);
    378         }
    379 
    380         return fileList;
    381     }
    382 
    383     /**
    384      * Get file from ftp server into given output stream
    385      * 
    386      * @param ftpFileName
    387      *            file name on ftp server
    388      * @param out
    389      *            OutputStream
    390      * @throws IOException
    391      */
    392     public void getFile(String ftpFileName, OutputStream out) throws IOException {
    393         try {
    394             // Use passive mode to pass firewalls.
    395             ftp.enterLocalPassiveMode();
    396             
    397             // Get file info.
    398             FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
    399             if (fileInfoArray == null) {
    400                 throw new FileNotFoundException("File '" + ftpFileName + "' was not found on FTP server.");
    401             }
    402 
    403             // Check file size.
    404             FTPFile fileInfo = fileInfoArray[0];
    405             long size = fileInfo.getSize();
    406             if (size > Integer.MAX_VALUE) {
    407                 throw new IOException("File '" + ftpFileName + "' is too large.");
    408             }
    409 
    410             // Download file.
    411             if (!ftp.retrieveFile(ftpFileName, out)) {
    412                 throw new IOException("Error loading file '" + ftpFileName + "' from FTP server. Check FTP permissions and path.");
    413             }
    414 
    415             out.flush();
    416 
    417         } finally {
    418             if (out != null) {
    419                 try {
    420                     out.close();
    421                 } catch (IOException ex) {
    422                 }
    423             }
    424         }
    425     }
    426 
    427     /**
    428      * Put file on ftp server from given input stream
    429      * 
    430      * @param ftpFileName
    431      *            file name on ftp server
    432      * @param in
    433      *            InputStream
    434      * @throws IOException
    435      */
    436     public void putFile(String ftpFileName, InputStream in) throws IOException {
    437         try {
    438             // Use passive mode to pass firewalls.
    439             ftp.enterLocalPassiveMode();
    440             
    441             if (!ftp.storeFile(ftpFileName, in)) {
    442                 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
    443             }
    444         } finally {
    445             try {
    446                 in.close();
    447             } catch (IOException ex) {
    448             }
    449         }
    450     }
    451 }
    复制代码
  • 相关阅读:
    leetcode108 Convert Sorted Array to Binary Search Tree
    leetcode98 Validate Binary Search Tree
    leetcode103 Binary Tree Zigzag Level Order Traversal
    leetcode116 Populating Next Right Pointers in Each Node
    Python全栈之路Day15
    Python全栈之路Day11
    集群监控
    Python全栈之路Day10
    自动部署反向代理、web、nfs
    5.Scss的插值
  • 原文地址:https://www.cnblogs.com/hzcya1995/p/13318174.html
Copyright © 2011-2022 走看看