Forum » Support and Bug Reports »

ProFTPD with mod_sftp fails with files > 2GB

Advertisement

MarisB
Joined:
Posts:
19

ProFTPD with mod_sftp fails with files > 2GB

Hi!

The latest Ubuntu LTS release (22.04.4) with its default ProFTPD Version 1.3.7c. WinSCP fails to upload files that are greater than 2GB. I am using SFTP (mod_sftp).

Clean Windows 11 installation with FileZilla and WinSCP 6.3.1 (no settings changed):
  1. . Uploading via user interface using both FileZilla and WinSCP works, but both show disconnect/reconnect at 2GB.
  2. Uploading from a script with -resumesupport=on also works.
  3. Uploading from a script with -resumesupport=off hangs indefinitely as it disconnects/reconnects at 2GB and starts uploading from the start again and again.
I believe that this issue is caused by the ProFTPD rekey feature:
http://www.proftpd.org/docs/contrib/mod_sftp.html#SFTPRekey

I have not tried to disable the rekeying feature, but it would be nice if WinSCP worked with the default ProFTPD configuration.

Attaching relevant log files. Currently I don't have a public facing test server, but I can make one if that helps.
  • logs.zip (9.31 KB, Private file)

Reply with quote

Advertisement

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,441
Location:
Prague, Czechia

Re: ProFTPD with mod_sftp fails with files > 2GB

Well, so it looks like ProFTPD issue, doesn't it? Did you report it to them?

Reply with quote

MarisB

I am not sure if it is a server side or client side bug. As they mention the rekeying in their documentation, so I thought this is a bug in WinSCP. Are the logs say that the bug is on the server side?

Reply with quote

martin◆
Site Admin
martin avatar

Well, you have the problem even with FileZilla. Does the rekey with your server work with any other SFTP client? How does mention of rekey in ProFTPD documentation indicate a bug in WinSCP?

Reply with quote

MarisB
Joined:
Posts:
19

I am sorry that my description was not clear enough. FileZilla shows a disconnection in the log, but it automatically handles this and the file is successfully uploaded. WinSCP also uploads successfully with -resumesupport=on. What is failing, is the option -resumesupport=off.

If you still think that this is a server's problem, I will try to report it to their team.

Reply with quote

Advertisement

martin◆
Site Admin
martin avatar

Sorry. Ok, so indeed, without resume support, WinSCP won't resume the transfer.
Is there a reason why you want to have the resume support turned off?

Reply with quote

MarisB
Joined:
Posts:
19

I was just testing around without a reason and found a bug. At first I thought that WinSCP is ignoring rekeying (I didn't see its support in the WinSCP docs), but now I have found that there is the rekeying support (https://winscp.net/eng/docs/ui_login_kex).

So there is still a bug, but on which side – WinSCP or ProFTPD? I know that WinSCP is a very reliable app, but not sure about ProFTPD, as I just needed one for testing a different server from OpenSSH, and I took one that is already on Ubuntu by default.

Not sure if this helps, but anyway – I just tested today using an OpenSSH client (scp) to ProFTPD, and it worked without rekeying (as far as -v param shows), however, when I try OpenSSH to OpenSSH the rekeying occurs.

Reply with quote

Advertisement

MarisB

I have just tested and OpenSSH can easily upload a 10GB file to ProFTPD. The verbose mode doesn't show any rekeying, however, it slightly pauses (uploading speed) at about 4096MB and 8192MB, so it seems like rekeying is taking place.

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,441
Location:
Prague, Czechia

@MarisB: Sorry for my late answer. It took me while to setup a test environment. And I've actually wasn't able to upload a big file to (local=fast) ProFTPD at all:
https://github.com/winscp/winscp/commit/7ae34b89576244e729e849a103c4624b5b2098a9
But once I've fixed that, I did not have any problem with rekey: 
. 2024-03-19 15:56:47.208 Opening remote file.
> 2024-03-19 15:56:47.208 Type: SSH_FXP_OPEN, Size: 43, Number: 3331
< 2024-03-19 15:56:47.219 Type: SSH_FXP_HANDLE, Size: 25, Number: 3331
> 2024-03-19 15:56:47.219 Type: SSH_FXP_WRITE, Size: 32757, Number: 3846
. 2024-03-19 15:56:47.231 Skipped 17 SSH_FXP_WRITE packets
< 2024-03-19 15:56:47.231 Type: SSH_FXP_STATUS, Size: 24, Number: 3846
< 2024-03-19 15:56:47.231 Status code: 0
. 2024-03-19 15:57:41.181 Initiating key re-exchange (too much data sent)
. 2024-03-19 15:57:41.220 Doing ECDH key exchange with curve Curve448, using hash SHA-512 (unaccelerated)
. 2024-03-19 15:57:41.512 Initialised AES-256 SDCTR (AES-NI accelerated) [aes256-ctr] outbound encryption
. 2024-03-19 15:57:41.512 Initialised HMAC-SHA-256 outbound MAC algorithm
. 2024-03-19 15:57:41.512 Initialised AES-256 SDCTR (AES-NI accelerated) [aes256-ctr] inbound encryption
. 2024-03-19 15:57:41.513 Initialised HMAC-SHA-256 inbound MAC algorithm
. 2024-03-19 15:58:34.913 Initiating key re-exchange (too much data sent)
. 2024-03-19 15:58:35.006 Doing ECDH key exchange with curve Curve448, using hash SHA-512 (unaccelerated)
. 2024-03-19 15:58:35.214 Initialised AES-256 SDCTR (AES-NI accelerated) [aes256-ctr] outbound encryption
. 2024-03-19 15:58:35.214 Initialised HMAC-SHA-256 outbound MAC algorithm
. 2024-03-19 15:58:35.214 Initialised AES-256 SDCTR (AES-NI accelerated) [aes256-ctr] inbound encryption
. 2024-03-19 15:58:35.214 Initialised HMAC-SHA-256 inbound MAC algorithm
. 2024-03-19 15:59:14.229 Skipped 85487 SSH_FXP_WRITE and 85473 SSH_FXP_STATUS packets
> 2024-03-19 15:59:14.229 Type: SSH_FXP_CLOSE, Size: 25, Number: 21893124
. 2024-03-19 15:59:14.269 Skipped 31 SSH_FXP_STATUS packets
< 2024-03-19 15:59:14.269 Type: SSH_FXP_STATUS, Size: 24, Number: 21893124
< 2024-03-19 15:59:14.269 Status code: 0
> 2024-03-19 15:59:14.270 Type: SSH_FXP_RENAME, Size: 50, Number: 21893394
< 2024-03-19 15:59:14.291 Type: SSH_FXP_STATUS, Size: 24, Number: 21893394
< 2024-03-19 15:59:14.291 Status code: 0
. 2024-03-19 15:59:14.291 Preserving timestamp [2024-03-13T09:20:47.000Z]
> 2024-03-19 15:59:14.292 Type: SSH_FXP_SETSTAT, Size: 34, Number: 3593
< 2024-03-19 15:59:14.306 Type: SSH_FXP_STATUS, Size: 24, Number: 3593
< 2024-03-19 15:59:14.307 Status code: 0
. 2024-03-19 15:59:14.307 Transfer done: 'C:\test\big2gb' => '/work/big2gb' [2797710400]
. 2024-03-19 15:59:14.311 Copying finished: Transferred: 2 797 710 400, Elapsed: 0:02:27, CPS: 19 009 929/s
Tested against the very latest (master) ProFTPD.

Reply with quote

Advertisement

MarisB

Thank you for your support! The latest ProFTPD with mod_sftp 1.3.9rc3, which I compiled from the most recent source, is working perfectly. It turns out Ubuntu LTS hasn't been keeping up with the newest features and fixes. I mistakenly thought they were more up-to-date.

Reply with quote

Advertisement

You can post new topics in this forum

Documentation

Support

Associations

Follow Us