Tech question about FTP

[2MuchMark]

Help!

I have a really strange problem with FTP. I can't seem to FTP to certain servers.

The servers have nothing special in their configuration.
The server provider insists everything is fine.
My home Internet provider insists everything is fine, but rebooted my modem anyway.
Tested on 2 different computers, same result.

FTP times-out when using a client after about 10 seconds. I can FTP via a browser but it takes almost 100 seconds to connect. Clicking into sub dir's is just as dead-slow.

What can be going on? Others can FTP to the same severs with no speed problems. Only I, me, from this location, can't connect.

Even Traceroute and MTR reveal nothing special.

Help!

[lezinterracial]

That is weird. Did you try from more than one device?

edit= never mind saw where you said you tried on 2 computers.

[Sly]

Do you have the connection logs? Likely a FTP client setting.

[2MuchMark]

Do you have the connection logs? Likely a FTP client setting.

You mean connection logs on the server? No, but what would I look for exactly?

[BigFurry]

Try both Active Mode and Passive Mode in your FTP connection settings.

They use different schemes for establishing the control and data connections. It's possible that a firewall or router is blocking/throttling something.

https://stackoverflow.com/questions/...nd-passive-ftp

[GFED]

I prefer SCP. https://winscp.net

[2MuchMark]

Try both Active Mode and Passive Mode in your FTP connection settings.

They use different schemes for establishing the control and data connections. It's possible that a firewall or router is blocking/throttling something.

https://stackoverflow.com/questions/...nd-passive-ftp

Tried that! No go.

I prefer SCP. https://winscp.net

My machine is a Mac.

Here's an update: I installed Filezilla on the same machine. Problem is still there.

I took the laptop to a public wifi location: Problem still there!

I've ruled out any WAN issues at this point, and since others can connect I know its not my hosting provider. It seems to be localized to my computer only. I have no idea what it could be.

[GFED]

my machine is a mac.

It seems to be localized to my computer only. I have no idea what it could be.

[CurrentlySober]

My machine is a Mac...

Do you not have an old windows laptop in a cupboard that you could try with? Or give your login to a trusted windows user to try? At least you'll get your answer...

[BigFurry]

If you're using a host name for the FTP, also try the IP.

I had issues with a host one time, and it turned out it was IPv6 routing issues on their end. Using an IPv4 worked.

If you have a firewall, try checking its logs / disabling it, but I guess you already tried that.

You could also try 'HTTPing' to test for connectivity issues. I read that regular ICMP ping isn't really meaningful in a lot of cases. It's using a different protocol and its results can be misleading. (I think httping needs a http web server on the IP, but I'm guessing yours has one.)

https://www.vanheusden.com/httping/

[2MuchMark]

Do you not have an old windows laptop in a cupboard that you could try with? Or give your login to a trusted windows user to try? At least you'll get your answer...

Yep!

I hadn;t used my PC in a while but I fired it up, and same problem. It could not connect via FTP.

If you're using a host name for the FTP, also try the IP.

I had issues with a host one time, and it turned out it was IPv6 routing issues on their end. Using an IPv4 worked.

Checked that! I'm using IPv4 (I've never used 6...)

If you have a firewall, try checking its logs / disabling it, but I guess you already tried that.

Yup! Did that too. I even disabled the firewall of the server for a few minutes and removed all filters. Same problem.

You could also try 'HTTPing' to test for connectivity issues. I read that regular ICMP ping isn't really meaningful in a lot of cases. It's using a different protocol and its results can be misleading. (I think httping needs a http web server on the IP, but I'm guessing yours has one.)

Done and done! HTTP is fast and seems normal.

FTP'ing through a browser however is dead-slow.

-----------------

My service provider is Bell Canada. When I call tech support I of course only get the low-level support on the phone who barely know the basics. Essentially they won't support anything above "surfing the web". FUCK. I finally convinced the last one to have a manager call me back, which I'm waiting for now.

[BigFurry]

Well since another computer didn't help, nor another ISP (public wifi), it sounds like it's an issue on the server's end.

If you have access, check the server's logs too. Server-side FTP server logs, fail2ban logs if it's installed, server-side firewall logs, etc. And check the server's load / processes too.

Could be a network issue too of course.

As the others suggested above, I'd try SCP and SSL FTP too (ftp clients have this in the connection options, but the server needs to support it too).

[NickBaer]

Tried that! No go.

My machine is a Mac.

Here's an update: I installed Filezilla on the same machine. Problem is still there.

I took the laptop to a public wifi location: Problem still there!

I've ruled out any WAN issues at this point, and since others can connect I know its not my hosting provider. It seems to be localized to my computer only. I have no idea what it could be.

I have used Fetch on my Macs - for 20 years!

[NickBaer]

Is it always to the same FTP server?

In the "cPanel" does the FTP User section offer client configuration help?

Having SSL for https affect the secure version of FTP? (I forget what it is called now!)

[chaze]

Try your same log in through your phones Internet or another ISP. If it's faster than you know it's your ISP if it's the same you know it's the host.

[Helix]

"FTP connection timed out" - This most often happens when your Internet Service Provider is blocking the FTP port - port 21. More information on how to check if a port is blocked can be found here - https://www.siteground.com/kb/check-port-blocked-isp/

Another cause for this issue is if you are not using Passive mode with your FTP client. You can refer to the documentation of your FTP client for instructions how to change that.

Additionally, if you are use your domain as an FTP host to connect to, it might not be resolving properly. You can try to use your account's IP or the server hostname instead.

[Miguel T]

Mark, try changing your router DNSs to Google or Cloudflare (1.1.1.1 https://1.1.1.1) ;)

Let me know if that solved it.

[GFED]

I know you said you disabled the firewall and still had the same problem, but are you positive you don't need to have your IP address on the FTP whitelist? When was the last time it worked for you?

[Beaver1]

I installed Filezilla on the same machine.

Stay away from Filezilla!
Google Filezilla Hack.

[rowan]

You using IPv6?

[CurrentlySober]

What about not using a client at all, and going direct via the macs command line in console?

[2MuchMark]

Hi Everyone,

Thanks for all the suggestions - so far none of them help, so you all suck!

However we have narrowed it down. I can Traceroute to the server using only port21, and it replies, so connection is good, no firewall or other issues, etc. No ports blocked, nothing.

Something I completely missed though. If I try logging in via the command prompt in windows. I get a reply right away asking me for the username. POW, it's there, and its fast.

But, after I enter the Password is when it times-out in the ftp client, and responds very very slowly if I use the browser.

So to clariify:Only after I am logged in via FTP does it become dead slow.

Server is redhat linux. Any suggestions?

[rowan]

Does your IP resolve? The one you're connecting from.

Do "host x.x.x.x" (whatever your IP is) from the *nix commandline and see if there's a timeout or unusual delay.

[BigFurry]

SSH into the server and try logging in locally via the 'ftp 127.0.0.1', to see if it also hangs at/after the pass too.

Could be some issue with the FTP server.

[2MuchMark]

Does your IP resolve? The one you're connecting from.

Do "host x.x.x.x" (whatever your IP is) from the *nix commandline and see if there's a timeout or unusual delay.

Yes it resolves.

SSH into the server and try logging in locally via the 'ftp 127.0.0.1', to see if it also hangs at/after the pass too.

Could be some issue with the FTP server.

Yes this works.


I found something interesting: Get this:

According to the TCPDump, the server is sending a message after receiving the password, but the client is not responding. After 20 seconds, it times out. This is visible in the TCPDump itself.

So now wrap your head around this:

The CLIENT is not responding, to only that single message, and only from that IP Address.

In my particular case, the client is both a PC and a Mac, with different FTP Client softwares installed and tested.

The Client is also a few other computers too around the globe.

Get it? The server seems to be behaving normally. Certain clients, but not all, are not responding to the server after the password is sent.

I am so confused! This is the weirdest problem I have seen a really long time.

[BigFurry]

Are you monitoring every port with tcpdump, or only the initial connection?

Sounds like your side and the server side cannot create the second connection, the data channel. This is a second connection between the two IPs, usually created when you list a directory's contents or transfer a file. ACTIVE mode / PASSIVE mode decides how this second connection is created. Usually one of them works, if not, router/firewall somewhere in between or server-side is still the most likely cause. (As you already ruled out your PC by trying two different ones.)

As I recall/read:
- If you're using ACTIVE mode, the server would be trying to connect to your PC to establish the 2nd connection. If a firewall or router blocks it inbetween, you wouldn't see anything in tcpdump, as it doesn't reach your computer.
- In PASSIVE mode, you'd see your computer trying to establish an outgoing connection on a different port.

Does it really hang right after the password is sent? Isn't it the 'ls' FTP command that makes it hang?
It would be useful to see the 'ftp' command log with debug on and verbose on.

[HomerSimpson]

I had similar problem and it was hard to troubleshoot, but apparently the problem was the provider's modem that had some kind of firewall that would brake the connection. The only way to fix it was to ask them to give me different type of modem that didn't have this issue/feature.

[2MuchMark]

Are you monitoring every port with tcpdump, or only the initial connection?

Sounds like your side and the server side cannot create the second connection, the data channel. This is a second connection between the two IPs, usually created when you list a directory's contents or transfer a file. ACTIVE mode / PASSIVE mode decides how this second connection is created. Usually one of them works, if not, router/firewall somewhere in between or server-side is still the most likely cause. (As you already ruled out your PC by trying two different ones.)

As I recall/read:
- If you're using ACTIVE mode, the server would be trying to connect to your PC to establish the 2nd connection. If a firewall or router blocks it inbetween, you wouldn't see anything in tcpdump, as it doesn't reach your computer.
- In PASSIVE mode, you'd see your computer trying to establish an outgoing connection on a different port.

Does it really hang right after the password is sent? Isn't it the 'ls' FTP command that makes it hang?
It would be useful to see the 'ftp' command log with debug on and verbose on.

Hi Big Furry,

Thanks for the info! I thought I had solved the problem thanks to your post but it seems like it is still happening. I will get back to you.

[2MuchMark]

ok... some progress, but still strange.

I changed the time-out on my client to 100 seconds so I can now at least log in to the server.

But now: Transferring files to the server is fast. Refreshing the directory list is dead slow (about 80 seconds).

Sigh

Server is Redhat Enterprise 5 by the way.

[DirtyHarold]

If you have root access to the server then try something like bitvise, it's a terminal that comes with an SFTP window.

[rowan]

Random thought: it could be something weird like an MTU path problem. That's where your end is sending (say) 1500 byte packets, but some intermediate link can only support (say) a maximum of 1488. There are protocols to automatically determine the maximum common MTU, but they're often rendered useless by aggressive firewalling. MTU path discovery issues can be tricky to diagnose because often the initial setup of a protocol (such as the TCP handshake) will succeed, but then everything just stalls once it starts streaming full sized packets.

Then again, if the transfer is succeeding eventually, it's probably not that...

Are you absolutely sure that your IP - the one you're connecting from - resolves on the destination FTP server?

Examples: (do this on the server)

$ host 8.8.8.8
8.8.8.8.in-addr.arpa domain name pointer google-public-dns-a.google.com.

$ host 1.2.3.4
Host 4.3.2.1.in-addr.arpa. not found: 3(NXDOMAIN)

[BigFurry]

Is your client FileZilla?

You didn't say if you tried SSL FTP.
In the connection settings under Encryption there is
- Require explicit FTP over TLS
- Require implicit FTP over TLS
Worth trying both, it could change the communication a bit, and it might be a workaround.

Or better yet, use SFTP, which is really the best option if you have SSH access. Set SFTP in the FileZilla Protocol, and use your SSH credentials as user/pass.

[2MuchMark]

Random thought: it could be something weird like an MTU path problem. That's where your end is sending (say) 1500 byte packets, but some intermediate link can only support (say) a maximum of 1488. There are protocols to automatically determine the maximum common MTU, but they're often rendered useless by aggressive firewalling. MTU path discovery issues can be tricky to diagnose because often the initial setup of a protocol (such as the TCP handshake) will succeed, but then everything just stalls once it starts streaming full sized packets.

Then again, if the transfer is succeeding eventually, it's probably not that...

Are you absolutely sure that your IP - the one you're connecting from - resolves on the destination FTP server?

Examples: (do this on the server)

$ host 8.8.8.8
8.8.8.8.in-addr.arpa domain name pointer google-public-dns-a.google.com.

$ host 1.2.3.4
Host 4.3.2.1.in-addr.arpa. not found: 3(NXDOMAIN)

Hi Rowan,

Thanks I sent this to my network admin and hopefully I'll get an answer soon.

Is your client FileZilla?

You didn't say if you tried SSL FTP.
In the connection settings under Encryption there is
- Require explicit FTP over TLS
- Require implicit FTP over TLS
Worth trying both, it could change the communication a bit, and it might be a workaround.

Or better yet, use SFTP, which is really the best option if you have SSH access. Set SFTP in the FileZilla Protocol, and use your SSH credentials as user/pass.

Thanks - SFTP seems to work slightly better but it is still just as slow.

It's been almost a week now and no one seems to know what the issue can be. This is so frustrating..!

[Kafka]

Did you try in the command prompt ftp with a verbose option? like ftp -v your_host
Maybe you can see whats happening.

BTW, dont use ftp. All your data and password goes unencrypted over the net. Better use sftp.

[thommy]

Hi Rowan,

Thanks I sent this to my network admin and hopefully I'll get an answer soon.




Thanks - SFTP seems to work slightly better but it is still just as slow.

It's been almost a week now and no one seems to know what the issue can be. This is so frustrating..!

for me that sounds like a problem in your router. possibly the handshake port is blocked by a firewall or your ISP is using QOS on some ports.