OS X Server FTP Service and the OS X Client Firewall
I discovered a disturbing issue a couple of days ago after getting the Mac OS X 10.4 Server FTP service working. Mac users couldn't connect to the FTP server. I thought it was working once I could connect from my PC (which is my alternate machine.) But whenever I tried from my G5, or my iBook it would just hang. I tried reconfiguring FTP settings on the server, turning off the server firewall, etc… but nothing appeared to have any effect. But then one day in the shower it hit me!
The firewall on the client! Of course, this must be the issue. Sure enough I found that the firewall on the mac closes all ports other than the ones allowed explicitly in the exception list. The way FTP works is that the initial handshaking happens on ports 20-21. The handshaking is a way in which the two computers decide which ports they are going to use for the data transfer, sort of a trial and error. The communication goes something like this.
“Hello Mrs. Server, I'd like to manage some files today.”
The server replies “Good day Mr. Client, do you have the proper details?”
“Why yes Mrs. Server, my name is John, and my password is smarmy, which port would you like for me to use to enter?”
“Well Mr. Client, your credentials check out, why don't you try port 10271, it is a random port, but try this one.”
“Allright, one second while I try…. I'm sorry, but that port doesn't appear to be open, do you have another?”
“Try 9241, that one should be open.”
“Excellent, that one worked, I'll be sending my codes now, good day to you Mrs. Server.”
“Good day to you Mr. Client, happy file tranfers!”
That is the process, otherwise known as passive file transfer protocol, which is what most servers use by default. The benefit of this is that no one can know in advance which port the file transfer will be happening on, therefore enabling slightly more secure file transfers. It is possible to enable active file transfer protocol, where the actual data movement happens between ports 20 and 21, but this isn't as secure and sometimes doesn't work if there are services operating within these ports. With passive file transfer protocol the transfer happens somewhere between ports 1024 and 65535. The reason for this is that the ports below 1024 are used for internal processes and other pre-defined services. These must be blocked by default for security reasons. Many of the ports above 1024 do not connect to anything, or are used for low-level services that are contained and therefore the potential damage for a security breach is minimal.
The problem with the Mac firewall, which isn't actually a problem, is that these upper ports are shut off by default when the firewall is enabled. This complicates the process of FTP in passive mode because the server completes the handshake, but the looking for ports goes on forever, especially if stealth mode is enabled, until the preset timeout is reached on the client. This is what was happening. On the Mac client, there is a checkbox that can be found in “System Preferences” > “Network” > “Proxies” that says it will enable passive FTP mode for the firewall. This has never worked for me with the firewall enabled.
Today I opened all the ports above 1024 by adding a rule to the firewall. This would be somewhat dangerous, but all my client machines are behind another firewall anyway so it isn't as dangerous as it seems. In order for Mac users to make this consistently work, they will have to either enable and disable the firewall when they want to connect to the FTP server, or they will have to open the ports above 1024 with a rule as I have. While this isn't an ideal situation, it is the only one that will work for now.