Synchronizing between local and FTP

G'day folks,

I am using DO 10.2.0.1 (4683) x86 and upload using the FTP from my location in Sydney Australia - New South Wales Tue 9:36:02 AM EDT * to my domain server in Chicago U.S.A. - Illinois Mon 4:35:13 PM CST.

As you can see there is a time difference from Sydney to Chicago of approximately -18 hours, i.e. Chicago is 18 hours behind my time in Sydney.

When I connect the 2nd panel on Opus to the FTP server and Synchronize, it will always not identify the anyname.txt files and will want to re-upload the 199 text files which did NOT change, in addition to the other newer files and folders which did change.

Is the wrong identification in the synchronization of the text (.txt) files a bug in Opus ?

Regards
pcbugfixer - The :imp: Devil made me ask this. :smiley:

What parameters are you using for the sync?

Have you configured the bookmark for the server appropriately for its timezone, so the file dates it shows are consistent with your local files?

Have you tried with other filetypes?

G'day Leo,

Used,
Mode: One way copy
compare: date (different) or size

  • Delete files from destination .....
  • Synchronize sub folder content
  • Ignore seconds when comparing by timestamps
  • Hide unaffected files
  • Automatic shrink panel

Not used,
Filter
Sync left to right
Only compare existing files
DST compensation

Current time settings,

Time on Web-hosting Server set to (cannot be changed)
Chicago
U.S.A. - Illinois
Mon 8:05:33 PM CST

My time,
Sydney
Australia - New South Wales
Tue 1:06:17 PM EDT *

NB: I did not (cannot recall) have this issue on previous version.

Regards
pcbugfixer :slight_smile:

NB: I do not have this problem between
Oxford United Kingdom - England Tue 2:10:43 AM GMT
and
Chicago U.S.A. - Illinois Mon 8:12:25 PM CST

Do you have the site timezone set correctly in its addressbook entry?

G'day Leo,

Did that - change the date / time to Sydney Australia.

So it is now copying the whole structure to the folder on the server however does not suggest that it is deleting any files or folders.

Regards
pcbugfixer

If the time on the server is Chicago time, why did you set it to Sydney? :slight_smile:
You should set the timezone in the address book to match the actual time used by the server (in this case it would be UTC-6).

G'day Jon,

The issue was one of file sync, i.e. synchronization, to be the same !
It seemed obvious that the time should be in sync, i.e. in parity with local time (the same)!
No one said otherwise, or did not specify which.
"Do you have the site timezone set correctly in its addressbook entry?" was interpreted to be the same as local time.

*So if you knew what to do, why jump in late ?

Regards
pcbugfixer *The :imp: made me say that.

You need to tell Opus which timezone the server is in, so the times the server reports can be adjusted from the server's timezone into your own timezone.

(Opus can get your own timezone information from Windows, but doesn't know where the server is.)

If you tell Opus the server is in the same timezone that you are in then there is no adjustment to be made, since that's saying the times the server reports are already in your timezone.

We're talking about the "Adjust directory and file dates for site time zone" checkbox and drop-down list here:


Even once that is set correctly, some servers will show the times that the files were uploaded instead of the timestamps that were on the local files before they were uploaded, and some servers ignore the command to set file timestamps to specific values.

Daylight saving time can also complicate things.

G'day Leo,

Thanks for the explanation, I did understand the logic of what servers do - Thanks
Obviously what Opus does is an other matter, however I have set the remote server in Opus to GMT -6 Central Time US and Canada and doing a complete new upload based on that.

BTW: My upload speed from Australia is currently averaging 97.4KB/s

Thanks for help so far, I will keep you posted on the new upload and then re-synchronization.

Regards
pcbugfixer :sunglasses:

G'day again Leo and Jon,

After some exhaustive uploads and many re-boots I again ran the FTP from both the Win XP Pro SP3 Workstation and the Win 7 Ultimate workstation.
The settings i used time Zone wise were as you recommended "GMT -6 Central Time US and Canada"

Other settings were set to,
Used,
Mode: One way copy
compare: date (different) or size

  • Delete files from destination .....
  • Synchronize sub folder content
  • Ignore seconds when comparing by timestamps
  • Hide unaffected files
  • Automatic shrink panel

Not used,
Filter
Sync left to right
Only compare existing files
DST compensation

    • I then performed a Synchronization from my local XP Pro SP3 Workstation and the D:\PCBF UPDate 03\0 0 UpdateCheckerFiles folder to the FTP site ftp://www.pcbugfixer.com//public_html/0 0 UpdateCheckerFiles folder.
  • After the comparison it wanted to copy ALL files and folder to the FTP site.
  • I allowed this to and continued - with many hours to wait until it was completed for the 4.9GB of files and folders. Total 734 files and 233 folders = 967 items.
  • so at the end of the night, 16 hours later I completed the upload, closed Opus and re-booted the system.
  • I then started the Opus, used the same Sync from the local to the FTP also same settings and compared, resulting in that the same uploads Total 734 files and 233 folders were listed to be performed, i.e. Copy from source to destination.
    • Believing that there might be an anomaly in the XP OS, I rebooted and loaded the Win 7 Ultimate Workstation.
  • Using the same time zone settings and other options i performed another sync from the loacal folder to the FTP folder.
  • The result was the same, i.e. it wanted to uploads Total 734 files and 233 folders were listed to be performed, i.e. Copy from source to destination.
    • I then re-tested the KVM / VPS in the UK to synchronize with the FTP site, using the same settings and time zone for the FTP site Synchronizing from the FTP site back to the KVM.
  • As I had uploaded the newer files and folders to the FTP site (ref. item 1) resulting on what I considered to be the correct number of downloads (Copy from source to destination.) and Deletions from the destination KVM site, I proceeded and as the speed from the KVM in the UK to the FTP site in the USA was faster (1.24bps to 1.64Mbps, the task went well.
  • I then closed the Opus and reloaded it to again Synchronize from the FTP to the KVM and there were no discrepancies, ie. the files and folder window showed no files or folder to be uploaded or deleted.
  • Thinking that this might have been a fluke, I closed Opus re-loaded it and performed a sync from the KVM to the FTP site resulting in no discrepancies, ie. the files and folder window showed no files or folder to be uploaded or deleted.
    NB the time zone is similar on both the KVM and FTP site.
    • Time Zone Details:

Chicago
U.S.A. - Illinois
Fri 10:44:27 PM CST

Sydney
Australia - New South Wales
Sat 3:45:06 PM EDT *

Oxford
United Kingdom - England
Sat 4:45:28 AM GMT

    • To day I started the XP workstation and again loaded the Sync parameters as described in Item 1. The result was the same, i.e. it wanted to uploads Total 734 files and 233 folders were listed to be performed, i.e. Copy from source to destination.
  • I then closed the Sync and started a new one with the reverse i.e. from the FTP site back to my workstation folder, result was the same, it wanted to uploads Total 734 files and 233 folders were listed to be performed, i.e. Copy from source to destination.
  • I then changes the options from compare: date (different) or size to compare: size which resulted in the scenario with which I started, it wanted to upload ALL the .txt files in addition to the other changes files and folder.
  • Reversing the sync yielded the same result.

************* OK, so what do I try next ??

I need this FTP sync to work correctly as I do not, due to the Australian restrictions on upload speed of max 100KB/s, have the hours it takes to upload ALL of the files ALL of the time. This was one of the main reasons why I purchased Opus, to only upload and remove the changed files and folders !

Regards
pcbugfixer

Are the file times on the FTP site actually shown as the same time as your local files?
(i.e. pick a file, any file at all, and see what "modified" time is shown for that file on both the FTP server and your local copy).

G'day Jon,

The answer is NO.

In the general display when the local and FTP sites are loaded the panels show,

  • Local folders and files dates are shown correctly.
  • FTP files and folders are either show as "yesterday" or "Thursday"

In the sync windows when it comes up,
E.G. a local "IntelliPoint 8.0 WARNING.txt" file date is shown as 13/06/2012 with 344 bytes, and on the FTP site the same file shows as 14/06/2012 with 333 bytes
.exe files as an example, "install_flash_player_ax.exe" shows on local as 15MB 07/11/2012 and on FTP as 15MB yesterday

If I change the option to compare size only, then
E.G. a local "IntelliPoint 8.0 WARNING.txt" file date is shown as 13/06/2012 with 344 bytes, and on the FTP site the same file shows as 14/06/2012 with 333 bytes
Other text files show either a day behind the current local file date or show the FTP file date as "yesterday" or is shown on FTP site as 1 day ahead of local file date, E.G. "ReadMe Notepad.txt" on local is 1.50KB 21/09/2012 and on FTP as 1.47KB and 22/09/2012

I trust this is clear for you.

Regards
pcbugfixer :sunglasses:

You need to get it to a state where the two sides show the same times (and EXACTLY the same times, not just the same dates). This is achieved by setting the timezone property in the address book entry as already described.

Although I don't know, I would say that either you haven't done this yet, or that the remote FTP server is not actually using the timezone you say it is. We don't know what timezone the server uses so we can't definitively tell you what to set it to - that's something you'll have to work out.

Jon,

Silly I am from time to time, stupid I am NOT !

  • and I am not a Novice, only where new Utilities are concerned, however with my 32 years experience in IT I pick things up very quickly !!

The time zone on the FTP site in Opus is set at "GMT -6 Central Time US and Canada" I believe I have said that several times in my posts, the timezone property in the address book entry as already described !!
Local time is set at "GMT +10 Canberra, Melbourne, Sydney"

Regards
pcbugfixer :grin:

Jon,

Do you have or can use TeamViewer ?
I will in this instant ONLY allow you to look at my XP Workstation and the Opus setup.

Regards
pcbugfixer :smiley:

Are you sure that the server is actually using that timezone?

Are you actually connecting to the FTP site using the addressbook? If you run the "Site Properties" command from the FTP menu when you are connected to the site, does the timezone shown in there match the one in the addressbook?

Assuming that, if you change the timezone and reconnect, do the timestamps change? If so, keep changing it until they're the same as your local files.

Jon,

Site Properties show the FTP site as "GMT -6 Central Time US and Canada"
I am in control of what the settings are on ALL my servers, however I had the Maintenance Guys that look after my USA server, re-check the time settings and the pcbugfixer.com FTP and the server it is on is also set to "GMT -6 Central Time US and Canada" i.e. uses the same as we specified for the connection.

If I do not use a time zone setting in the addressbook for the FTP connection, then only the 193 text files in the 176 folders want to update.

  • files display with the previously mentioned date differences.

Regards
pcbugfixer :confused:

Jon,
Current version for Opus is 10.2.0.1 (4683) x86
The previous version 10.2.0.0 (4645) x86 has this problem also. ( Installed on another XP Pro SP3 Workstation to test this out.

regards
pcbugfixer

The server may not preserve timestamps on files in the first place. If you upload a file and the server's copy gets the timestamp of the upload, instead if the timestamp the local copy had, then the times will never match up and it will not be possible to sync with the server based on timestamps.

Adjusting the timezone offset in Opus should shift all the times reported by the serve accordingly. If that is working then there is no bug. You then need to look at whether or not the server is actually recording and reporting timestamps in a way that makes sense for syncing by date.