Local/remote file mismatch

[gtwibell]

I'm using Yummy FTP to upload from Mac OSX 10.6.6 to a Linux host. Most files are binary around 450Mb in size. I have binary (raw data) selected as transfer encoding method.

In about 10% of cases, when Yummy says the transfer is completed there's a mismatch in the files. The local and remote directory listings in Yummy show identical file sizes. When I generate a .torrent file from the local directory, using Transmission, then begin seeding from the Linux host (Transmission again), Tranmission itself reports a file size mismatch - with the remote file typically 10Mb or 15Mb short. The Yummy directory listing continues to show exact same file sizes locally/remotely.

The only remedy has proved to be deleting the remote file(s) and re-uploading the same local original(s) again.

In the remaining 90% of cases everything proceeds perfectly. Even when the binary files are sometimes as much as 1.3Gb in size.

Many thanks,

GeoffT.

[JD]

Firstly, make sure you're using v1.8.4 of Yummy FTP.

When this happens again, you need to double check the file sizes - looking at the size in the file listing is insufficient because it's rounded to the nearest value (Gb, Mb, etc):

Select the Mac file in the Finder and choose Get Info from the File menu - in the resulting window it will show the size in *bytes*

In Yummy, select the Remote file and choose Get Info from the File menu - in the resulting window it will show the size in *bytes*

Are the two bytes values equal?

[gtwibell]

Hi and many thanks for the quick response.

Yes, I am on version 1.8.4. Just checked the file sizes with get info and they are same, to the bite. I guess the only way to check the actual contents is by checksumming. I wouldn't know how to do that on my remote host. But I could download the remote file back (into a seperate directory) overnight, then do a local check I guess.

It just occurred that this seems to happen most often when the FTP Watcher is in use. The files I'm uploading are video encodes that take about 45 minutes, and the files start out small and grow slowly as the encoding proceeds.

I'm not 100% certain whether I've had the problem with manual uploads. I'll keep a careful note from now on.

GeoffT.

[JD]

You're welcome

I wonder if the slow encoding speed of the app you're using might be triggering the FTP Watcher's upload prematurely, thereby initiating an upload of an incomplete file. The trigger for the Watcher to perform an upload is a new or a changed file - if an existing file's mod date or size changes then it will be uploaded provided the file's new stats remain unchanged for ten seconds. So, if your encoding app doesn't write to the output file for longer than 10 seconds it's possible for this to happen.

Unfortunately, there isn't a preferences setting for this so you can't adjust that value, but I'm going to add a way and send you a build which will, I think, fix the issue by increasing the 'wait' time to something a little more forgiving of the encoding app's speed.

Of course the trade off is that the Watcher's perceived responsiveness to upload files that need it will be decreased but in this case that is probably less important than to someone who makes frequent changes to, or adds, smaller files.

[gtwibell]

I seem to vaguely recall that it only happens when multiple transfers are enabled. Is it possible that FTP Watcher could be sending multiple progressively larger iterations of the same file into a single destination file? I guess the transfer terminates when the local/remote file sizes match, whether or not the contents match?

[JD]

If the Watcher is being triggered to upload prematurely (as I suspect)  then you'll definitely get odd behaviour.

Once I get you the new build I think this problem will be eliminated. I'm sorry for the delay in getting it to you, but it should be later today, then we can take it from there.