S3 multipart uploads

Advertisement

Ysu
Joined:
Posts:
9
Location:
Australia

S3 multipart uploads

Could you guys please improve the S3 interface to use multipart uploads?
It's a great tool you have here, I love it, but the upload speeds are magnitudes lower than my upload bandwidth limit. Uploading a large backup file (tens of GBs) is taking 8–10 hours.

Thanks!

p.s. I wanted to donate again (as I did a long while back but it must have been on another account as it does not show), but PayPal has shat itself for me, and they've no support – it'd be great if there was another method for payment.

Reply with quote

Advertisement

bri
Guest

S3 multipart uploads

Hi,
as @Ysu asked, I'm also missing the multipart upload.
I would be very happy, if the multipart upload could be implemented.
Thank you very much!
Bri

Reply with quote

jones74
Donor
Joined:
Posts:
4
Location:
South Australia

multi-part uploads to s3 bucket

Hi Martin,

I would also like to request this enhancement. We use .NET wrapper to transfer files and appreciate it if multi-part uploads were available.

Thank you for a great product.

Regards, Chris Jones

Reply with quote

Advertisement

jones74
Donor
Joined:
Posts:
4
Location:
South Australia

Re: multi-part uploads to s3 bucket

Hi Martin,

I have upgraded to 5.9 and was expecting to see improved speed to Amazon S3.

I set my my region to ap-southeast-2, Asia Pacific (Sydney)

Prior to upload I transferred a 2gb folder and was getting a connection speed of between 7 and 10mbs

After upgrading to 5.19 my speed was reduced to around half of the previous speed averaging 5mbs

I then tried the same package with Cloudberry and was getting a connection of 36mbs. I want to continue to use WinSCP as we transfer files using the .NET package and it works well but the speed is an issue for us are we are transferring many GBs of data.

Do you know why my transfer speed to S3 is worse after upgrade, are there any additional configuration settings other than region that I need to change?

I can replicate the above issue and can provide screen shots/logs if required.

Thanks
Chris

Reply with quote

jones74
Donor
Joined:
Posts:
4
Location:
South Australia

Re: multi-part uploads to s3 bucket

Hi Martin,

Apologies upgraded to 5.19.5 from 5.17.9.

I am unable to edit the "Optimize connection buffer size" as the advanced options are greyed out for an S3 connection but not for FTP. I would like to try this before I downgrade incase it resolves it. How can I do this?

Thanks
Chris

Reply with quote

Advertisement

jones74
Donor
Joined:
Posts:
4
Location:
South Australia

Multi-part uploads to S3 bucket

Hi Martin,

I switched off the checkbox as you suggested.

I was still only able to get between 7-10 mbs versus cloudberry's 32mbs.

regards
Chris

Reply with quote

martin
Site Admin
martin avatar

Re: multi-part uploads to s3 bucket

Ok, thanks. So you got back the pre-upgrade speed, right? I'll look into this.

Reply with quote

Advertisement

You can post new topics in this forum