|
Pages: [1]
|
 |
|
|
Author
|
Topic: Yummy Eats my Processors (Read 177 times)
|
Scoob
Newbie
Posts: 2
|
I got Yummy as it seemed like the ideal candidate to schedule backups of remote dynamic websites, in that the site I want to back up have files added to them on an hourly basis.
Yummy seemed to fit the bill as it was capable of running a scheduled one-way sync - i.e. from server to my local back-up machine.
However, more often than not, Yummy starts to chew my processor, often gobbling up more than 70% of the CPU cycles over a prolonged period.
Does anyone know what might be causing this?
I am only connecting to about 7 sites, and even if I close connections to those not currently backing up I still get a massive chunk of processor used up.
The schedules I run should all be out of sync with one another, so no 2 are running at the same time and each runs only about once every 7-10 days.
Currently I have 2 connections open, "Analyzing Directories for Update Mac..." and still loosing over 70% of my processors to Yummy.
Admittedly, due to the nature of these sites there are a lot of files/folders... is it possible Yummy gets confused when a remote site has over a certain number of files/directories? One site has 600 or so at root level and the other is now up to 1100!! (I know that's quite a lot - but a shames Yummy can't handle it?)
I'm running an old Dual G4 500MHz, with over 1GB ram and an 8Mb connection to the Internet. I would have thought that was OK??
An help appreciated!
|
|
|
|
|
Logged
|
|
|
|
Scoob
Newbie
Posts: 2
|
I note that leaving Yummy to run eventually completes the process and processor usage drops.... I guess comparing a lot of files is pretty processor intensive?
|
|
|
|
|
Logged
|
|
|
|
|
JD
|
Are you assuming that Yummy doesn't examine the sub folders? How many files and folders are there contained within the entire folder hierarchy of your server(s)? I bet it's a lot more than 600, or even 1100, right?  Certainly, 600 or 1100 file compares with Yummy will be completed within a few seconds, so it must be that your server(s) have a lot more files to process than that. If you only want Yummy to compare the current folder (ie not compare sub folder contents) the un-check the 'Process sub-directories' checkbox in the Synchronize setup window. |
|
|
|
|
Logged
|
|
|
|
|
JD
|
I just wanted to clarify further... 70% processor usage is not unusual for the task at hand. It's a very intensive operation to compare thousands of files and folders. Certainly, I could make Yummy use less processor power, but if I did that then the sync would take a lot longer to complete. I don't think you want that  |
|
|
|
|
Logged
|
|
|
|
|
JD
|
Well, well, well ...  It turns out that I did find something in the sync logic which would spend an inordinate amount of time comparing files, if you have a complex directory structure with lots of files... I've fixed this in the latest beta build, and in my test case (which has 21,000 items) I've reduced the time to completion from 10minutes to 1minute. Yes, you read that right It still uses a lot of processor to do the comparisons, which is expected since it's an intensive task so no problem there, but since it completes the task so quickly now I don't think it's an issue any longer  |
|
|
|
|
Logged
|
|
|
|
|
paulc
|
After all, it IS called Yummy!
|
|
|
|
|
Logged
|
|
|
|
|
JD
|
good one!  |
|
|
|
|
Logged
|
|
|
|
|
Yuku
|
Please, release this beta, as it takes ages here to compare files (a CMS with about... hum... 4GB in more than 105.000 files).
Also, i'm wondering how the auto time offset is working? If i enable it, it always synchronize the whole site, whereas if i set a manual offset, it's okay (except for the CPU load and time it takes).
Anyway, keep up the good work!
Bye.
|
|
|
|
|
Logged
|
|
|
|
|
JD
|
The 1.5.2 update was released today Let me know if you still have issues with the auto time offset feature - it works here, and I have no other reports of problems. |
|
|
|
|
Logged
|
|
|
|
|
Yuku
|
Thanks JD, i've just installed it, and started a synchronize, i will tell you later how it goes.
But just for my information, how does Yummy negociate the time offset (in auto-mode)? Does the ftp server send a GMT info or something?
I ask this because when i set auto-offset, it reports a 2 hours offset, but the ftp server is in the same imezone as i am (GMT+1)... Strange...
Bye.
|
|
|
|
|
Logged
|
|
|
|
|
JD
|
Yummy sends a small file to the server, notes the local time when the upload completed, then fetches a directory listing to find the timestamp of the uploaded file on the server. The time offset is the difference between the local time recorded and the reported timestamp of the file by the server. Does this explain anything?
|
|
|
|
|
Logged
|
|
|
|
|
Yuku
|
Good job JD, it took about an hour to analyze the 105.000 files on my good old G4 933MHz. It was more than 4/5 hours with 1.5.0, when it didn't time out at least... Huge improvment for me, thanks a lot!
I'll have a try with the auto-offset later, and will report here when i have the time (i don't synchronize everyday as it's a quite long process).
Bye.
|
|
|
|
|
Logged
|
|
|
|
|
JD
|
Excellent results I look forward to your auto-offset findings. |
|
|
|
|
Logged
|
|
|
|
|
Yuku
|
Good news, the auto-offset feature now works beautifully . No whole download of the entire site anymore if auto-offset engaged! Bye. |
|
|
|
|
Logged
|
|
|
|
|
JD
|
Great! Thanks for reporting back |
|
|
|
|
Logged
|
|
|
|
|
|
Pages: [1]
|
|
|
|
