Forum » Support and Bug Reports »

Error reading response headers: Line too long

Advertisement

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Error reading response headers: Line too long

Just started getting the above error a few days ago. This is an automated process that runs daily and has been running fine for a few years.
I manually logged in to the WebDAV site and was able to authenticate with no issues. Here is the log file: 
open https://xxxxxxxxxxxxxxxx:***@files.xxxxxxxxxxx.net/webdav -timeout=120
Connecting to host...
Authenticated.
Starting the session...
Session started.
Active session: [1] xxxxxxxxxxxxxxx@files.xxxxxxxxxxx.net
synchronize local X:\FormularyDirectory\etc\DavClient\downloads\xxxxxxxxxxxxxxx\webdav\Formulary\FBS_3_0 /webdav/Formulary/FBS_3_0 -delete -criteria=either
Using configured synchronization options different from factory defaults.
Comparing...
Local 'X:\FormularyDirectory\etc\DavClient\downloads\xxxxxxxxxxxxxxx\webdav\Formulary\FBS_3_0' <= Remote '/webdav/Formulary/FBS_3_0'
Error listing directory '/webdav/Formulary/FBS_3_0'.
Error reading response headers: Line too long
(A)bort, (R)etry, (S)kip: Skip
Synchronizing...
Local 'X:\FormularyDirectory\etc\DavClient\downloads\xxxxxxxxxxxxxxx\webdav\Formulary\FBS_3_0' <= Remote '/webdav/Formulary/FBS_3_0'
It then proceeded to delete EVERY folder in the tree! Fortunately, I have a backup of those 200,000+ files and folders.

I'm running version 6.1.2

What's going on here?

Reply with quote

Advertisement

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,441
Location:
Prague, Czechia

Re: Error reading response headers: Line too long

Please attach a full session log file showing the problem (using the latest version of WinSCP).

To generate the session log file, use /log=C:\path\to\winscp.log command-line argument. Submit the log with your post as an attachment. Note that passwords and passphrases not stored in the log. You may want to remove other data you consider sensitive though, such as host names, IP addresses, account names or file names (unless they are relevant to the problem). If you do not want to post the log publicly, you can mark the attachment as private.

Reply with quote

Advertisement

Rv
Guest

Hello Team,

Can you advise how to solve this issue? Do you think that there is some issue on the Vendor site from where we are downloading the formulary file(s)?

Thanks

Reply with quote

johnmsch

I will check with the vendor, but I don't think its on their side.
As mentioned above, I can log in to the WebDAV site manually using the same credentials and access the entire folder tree. That's what I've been doing for the past week.

Reply with quote

martin◆
Site Admin
martin avatar

@johnmsch: I'm sorry, I somehow forgot to answer you.
I'm sending you an email with a development version of WinSCP to the address you have used to register on this forum.

Reply with quote

GregA
Joined:
Posts:
2

header line too long testing

I have also been seeing this issue. Would it be possible to get a copy of this dev version to test with?

Reply with quote

Advertisement

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Private attachment from testing with Development version.
Thank you
  • winscp.20240509141720.log (16.35 KB, Private file)

Reply with quote

martin◆
Site Admin
martin avatar

@johnmsch: That log is from the official WinSCP 6.3.3, not from the development version. It seems that you have extracted only the console-wrapper winscp.com, but not the main winscp.exe.

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Sorry @martin, I did miss copying the .exe to C:\Program Files (x86)\WinSCP.
Attached is the current log file with both the .com and .exe in place
  • winscp.20240510140251.log (16.42 KB, Private file)

Reply with quote

Advertisement

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Still not working. Different error this time. Private log file attahced.
  • winscp.20240513143657.log (21.96 KB, Private file)

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Thanks for the reply @martin.
The site we access at Surescripts is simply a data store that we synchronize some data files from. There are 10's of thousands of files and many of them update/add/delete on a daily basis and WinSCP has always been the perfect solution to synchronize all that data.
I just opened a support case with them and hope to have an answer within a few days.
I apologize for not contacting them first!

Reply with quote

Advertisement

GregA
Joined:
Posts:
2

Error reading response headers: Line too long

When trying these same links using other clients, the session cookie is coming back around 12KB so values that are greater than 256kb seem like they must be in error.

Also, I read the rfc6265 differently. I read "At least 4096 bytes per cookie" as a minimum rather than a max. That said, hopefully I can find out why that value is the size that it is.

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Thanks for that update @GregA.
I opened a case with Surescripts and they haven't made any changes to the formulary download process. They also say that out of the thousands of people downloading their formulary, you and me are the only ones having an issue. They have no way of knowing if any of the other clients are using WinSCP.
@martin – could you please take another look at this?

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

@martin Attached is a new private log file. I rebooted the server, which hasn't been done for months. There is a new error on line 207: Script: Invalid access to memory.
Would you please look at this again?
We are having to manually download the data and have 3 people working on this daily.
Thank you
  • winscp.20240514230001.log (21.97 KB, Private file)

Reply with quote

martin◆
Site Admin
martin avatar
Joined:
Posts:
41,441
Location:
Prague, Czechia

OK, I've sent you both new build again.

@GregA: The RFC says:
General-use user agents SHOULD provide each of the following minimum capabilities: ...
  • At least 4096 bytes per cookie
...
Servers SHOULD use as few and as small cookies as possible to avoid reaching these implementation limits
So imo even 12K is way over the "implementation limits".
WinSCP should support about 8K by default.
Note that common webbrowsers like Chrome or Firewall do not allow more than those 4K.

Reply with quote

Advertisement

jmb
Joined:
Posts:
1

Re: Error reading response headers: Line too long

Hi @martin. Is there a way to block or turn off accepting all cookies coming into WinSCP? Just wondering if this might solve this issue with the 12K cookie going over the current max size of 8K.

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

@martin - New private log file attached.
The latest error is: Error reading response headers: Line too long (262145 > 262144)
Also got an update from Surescripts engineering team and they said they were working with you on a fix. Evidently there was a deployment several weeks ago that affected other clients.
Thanks for your efforts to resolve this for us!

P.S. @GregA - Would you please email me? Something I'd like to discuss with you, but your email is blocked. Mine should be available on this board.
  • winscp.20240517141100.log (16.89 KB, Private file)

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

@martin - The line above that says:
Evidently there was a deployment several weeks ago that affected other clients.
should have said:
Evidently there was a deployment several weeks ago that also affected other clients.

Reply with quote

martin◆
Site Admin
martin avatar

Re: Error reading response headers: Line too long

I've sent another build.

I'm not in contact with Surescripts team.

Reply with quote

Advertisement

johnmsch

@martin - Happy to attach, but the log file is 3.6gb, zipped up is 1.9gb. Looks like around 165,000 files were processed in the latest run. It appears that the file attachment limit here is 10mb. If you want the whole file, let me know how to get it to you, otherwise, how much of that do you want?

Reply with quote

Advertisement

johnmsch
Joined:
Posts:
37
Location:
Atlanta

@martin - Here you go. Hopefully this is enough.
Your help has been greatly appreciated! If I can be of any further help, just let me know!
  • FileExtract.log (4.75 MB, Private file)

Reply with quote

johnmsch
Joined:
Posts:
37
Location:
Atlanta

Thanks again @martin
I still can't get a straight answer from Surescripts, other than
Evidently there was a deployment several weeks ago that also affected other clients.
I also got another reply from level 3 support:
WinSCP has a new build which should resolve this issue. WinSCP customers are recommended to use this new build: https://winscp.net/tracker/2289
Did they ever reach out to you directly?

Reply with quote

Advertisement

Riddhish
Guest

Hello Martin,

Thanks for clarifying the same.
Can you please provide timeline of next release which contains work-around changes to support server-side problem?

Thanks.

Reply with quote

Advertisement

Guest

Error reading response headers: Line too long

Can you tell me when a new version will be released?
Can you send me the version that fixes this issue?

Reply with quote

shail.shah
Joined:
Posts:
3
Location:
India

Error reading response headers: Line too long

Hi,

We're facing a similar error while downloading the file from Surescripts using WinSCP,
Message: Error listing directory '/webdav/Formulary/FBS_3_0'. Error reading response headers: Line too long
It looks like the same error, which is mentioned in this thread. We see there is a workaround already provided. We would need similar support to us ASAP as many of our clients are blocked.

Looking forward for your quick response.

Thanks,
Shail Shah

Reply with quote

martin◆
Site Admin
martin avatar

Re: Error reading response headers: Line too long

We will consider merging the fix into 6.3.4 and release it within few weeks.

If you need pre-release build, I need you to register here, so I have a means of contacting you.

Reply with quote

shail.shah
Joined:
Posts:
3
Location:
India

Re: Error reading response headers: Line too long

Hi @Martin,
I have already registered. Can you please provide me with the pre-released build?
Thanks,
Shail Shah

Reply with quote

Advertisement

Advertisement

shail.shah
Joined:
Posts:
3
Location:
India

Re: Error reading response headers: Line too long

Hi Martin,

I have tested the executable you provided, and it is working fine. However, we're using the NuGet package with the service, which requires both the WinSCP DLL and executable.
When I tried replacing just the executable and ran the service, it threw the following error message: 
Mdrx.Formulary.SynchronizationManager.exe	Information	0	
Session start at 06/11/2024 09:04:45
Connecting to SS WebDav............
Mdrx.Formulary.SynchronizationManager.exe	Error	0	
Error:
Message: The version of C:\Program Files (x86)\Mdrx.Hub.Formulary.V3.MultumDownload_WinSCP\winscp.exe (6.4.0.0) does not match version of this assembly C:\Program Files (x86)\Mdrx.Hub.Formulary.V3.MultumDownload_WinSCP\WinSCPnet.DLL (5.17.6.0).
StackTrace:    at WinSCP.ExeSessionProcess.CheckVersion(String exePath, FileVersionInfo assemblyVersion)
   at WinSCP.ExeSessionProcess..ctor(Session session, Boolean useXmlLog, String additionalArguments)
   at WinSCP.Session.Open(SessionOptions sessionOptions)
   at Mdrx.Formulary.SynchronizationManager.Synchronizer.Start()
Could you please provide me with the DLL for the pre-release build (version 6.4.0.0) along with the executable?

I appreciate your help!
Thanks,
Shail Shah

Reply with quote

Advertisement

You can post new topics in this forum

Documentation

Support

Associations

Follow Us