Uncatchable error upon error-no-response

[cobasajaxinpho]

Hi @theophilusx ,

Just wonder. have you encounter this issues?
https://repost.aws/questions/QUsUpFCoYHRMuLd_yfi7tOEQ/lambda-ssh2-sftp-client-throw-uncatchable-error-no-response-from-server-when-run-in-parallel

I have been trying to figure out too. as try catch won't catch the error, will lead to crash the application

here's some snapshot of the code

public uploadFile = async (file: Express.Multer.File): Promise<boolean> => {
    const client = new Client();
    try {
      await client.connect(this.config);

      logger.info('[uploadFile]Read stream: ' + file.path);
      logger.info('[uploadFile]Write stream: ' + file.filename);

      const rs: Readable = client.createReadStream(file.path, {
        autoClose: true,
      });

      const ws: Writable = client.createWriteStream(file.filename, {
        autoClose: true,
      });
      rs.pipe(ws, { end: true });

      // on success of output file being saved
      ws.on('close', async () => {
        logger.error('ssh2 - WebSocket closed with code:');

        try {
          logger.error('ssh2 - Attempting to close SSH client...');
          setTimeout(async () => {
            await client.end();
          }, 1000);
        } catch (e) {
          logger.error('ssh2 - Error closing SSH client:', e);
        }

        logger.info('[uploadFile] Successfully saved file');
        return true;
      });

      ws.on('error', async (err) => {
        logger.error('ssh2 - error - uploadFile');
        try {
          logger.error('ssh2 - error - uploadFile1');
          setTimeout(async () => {
            await client.end();
          }, 1000);
        } catch (e) {
          logger.error('ssh2 - error - uploadFile2', e);
        }
        logger.error('[uploadFile] Failed to save file: ' + err);
        return false;
      });
    } catch (e) {
      logger.error('ssh2 - error - uploadFile2XXX', e);
    } finally {
      setTimeout(async () => {
        await client.end();
      }, 1000);
    }
    return true;
  };

Somehow need to

setTimeout(async () => {
            await client.end();
          }, 1000);

to prevent it from crashing the application

[theophilusx]

You should either be using sftp.get() or sftp.downloadFiles() to get the files. The sftp.createReadStream() is a low level function provided as a special escape hatch and should only be used in special situations. You have to take a lot more precautions and do a lot more work to make it perform correctly (have a look at the code for sftp.downloadFiles() to see what fi mean). In general, trying to download many files in parallel never works well with sftp. It typically doesn't improve performance as much as you might expect and just makes things more complicated. One of the additional complications is how sftp servers handle many requests and the variolus rate limiting techniques they use to protect against DDoS etc.

…

On Fri, 24 Jan 2025 at 18:05, cobasajaxinpho ***@***.***> wrote: Hi @theophilusx <https://github.com/theophilusx> , Just wonder. have you encounter this issues? https://repost.aws/questions/QUsUpFCoYHRMuLd_yfi7tOEQ/lambda-ssh2-sftp-client-throw-uncatchable-error-no-response-from-server-when-run-in-parallel I have been trying to figure out too. as try catch won't catch the error, will lead to crash the application — Reply to this email directly, view it on GitHub <#566>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFIKIX4AIP2AX4RMGJ5WL2MHQ4ZAVCNFSM6AAAAABVZAYYWWVHI2DSMVQWIX3LMV43ASLTON2WKOZSHAYDQNRXGQ3DQMA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- regards, Tim

-- Tim Cross

[cobasajaxinpho]

Ok. Thanks for clarifying.

public uploadFile = async (file: Express.Multer.File): Promise<boolean> => {
    const client = new Client();
    try {
      await client.connect(this.config);
      logger.info('[uploadFile] Uploading file: ' + file.filename);
      await client.put(file.path, file.filename);
      logger.info('[uploadFile] Successfully saved file: ' + file.filename);
      return true;
    } catch (error) {
      logger.error('ssh2 - error - uploadFile', error);
      return false;
    } finally {
      try {
        await client.end();
        logger.info('[uploadFile] SSH client closed');
      } catch (e) {
        logger.error('ssh2 - error closing SSH client:', e);
      }
    }
  };

got error
[error] ssh2 - error - uploadFile put: Bad path: documents/files/data/ not exist. eventhought i can see the documents uploaded.

any idea why the error happen?

tried "documents/files/data" also same error.

[theophilusx]

The error is telling you that the file path you are suppling does not match on the remote server. There are quite a few reasons why the path may not be what you expect. - The sftp server may be operating within a chroot or container environment which presents a different view of the file system from what you expect. - The sftp servfer might be logging you in at a point in the file system different to what you expect. - The sftp server may have different permission settings. The file is there, but the server doesn't allow you to read it. The ftp protocol (which is what sftp uses and is different from just ssh), has many optiolns and sftp servers can be set up in many different ways. For example, a 'traditional' sftp server configuration has the server setup within a chroot environment and for anonymous ftp, it allow uploading of files but does not allow listing or retrieving of files (even ones you have uploaded). It is unlikely the error you are seeing is due to an issue with ssh2-sftp-client. Most likely the issue is that your using the wrong/bad file path or you don't have the righyt permissions. I would suggest using the sftp.realPath() and sftp.list() methods to verify your assumptions regarding file paths and verify relative and absolute paths. I would also use something like the openSSH CLI sftp client to test/verify.

[cobasajaxinpho]

i have modified my code to use realPath , also result in the same error

public uploadFile = async (file: Express.Multer.File): Promise<boolean> => {
    const client = new Client();
    try {
      await client.connect(this.config);

      const realPath = await client.realPath(file.destination);
      logger.info('[uploadFile] client realPath: ' + realPath);
      logger.info('[uploadFile] Uploading file: ' + file.filename);

      await client.put(realPath, file.filename);

      logger.info('[uploadFile] Successfully saved file: ' + file.filename);
      return true;
    } catch (error) {
      logger.error('ssh2 - error - uploadFile', error);
      return false;
    } finally {
      try {
        await client.end();
        logger.info('[uploadFile] SSH client closed');
      } catch (e) {
        logger.error('ssh2 - error closing SSH client:', e);
      }
    }
  };

the different is real path value : "/documents/files/data"

[theophilusx]

There isn't sufficient information for me to tell what is goig on here. Please provide

• version of ssh2-sftp-client your using
• Version of nodejs this is running under
• The exact error message
• A debug log created by adding a debug logging function to the debug property in the connect config object (see the tests and example scripts for examples on using the debug property).

Some things to consider -

• The first argument to put should include the target filename, not just the path i.e. realPath + / + filename
• The second argument needs to be a complete path to the location of the file you want to upload, not just the filename
• Your 'uploadFile' function should be called with awsait and wrapped inside a try/catch as well (perhaps it is, I cannot tell based on info provided).

Have you tested your uploadFile function in a simple script without any of the web stuff and outside any class or object? To get to the root cause here, it will first be necessary to strip everything down to absolute minimum, verify basic function and then build up bit by bit.