Advanced Scheduled Synchronization with TurboFTP

FTP sync is a process where, through File Transfer Protocol, files are synchronized to or from an FTP or SFTP host so that it can be backed up, shared with certain parties or fed to a back-end system in a timely manner.

FTP Sync Benefits with TurboFTP Client:

  • Choose one of the flexible FTP sync schemes to do a selective sync or simply effective mirroring.
  • Protect your data in transit with a secure FTP/SFTP channel protected with industrial strength cryptography.
  • Built-in OpenPGP feature offers the option to add an extra layer of protection to your data in transit and even at rest. OpenPGP encryption is a much more reliable way to ensure the safety of data stored in the public or private cloud.
  • Relentless efforts to automatically retry interrupted transfer with checkpoint restart in any hostile network environment, which is extremely important to ensure delivery of your valuable data on schedule and complete.
  • Email notification upon successful delivery or error.

How to Synchronize Data?

With TurboFTP client, synchronizing data is easy to set up and test drive with the GUI without any scripting. An FTP or SFTP/SSH server such as TurboFTP Server must be running to allow TurboFTP to sync files against. The following guide will help you to properly sync data on FTP/SFTP.

Advanced Scheduled Synchronization with TurboFTP

1. Concepts

Source and Destination

The source and destination folders are defined based on the synchronization direction. For example, if the sync direction is Download, the remote folder is the source folder, and the local folder is the destination folder.


2. Synchronization Schemes

Synchronization Schemes define how TurboFTP chooses files to transfer after the file comparison is done. We will explain each of the available Synchronization Schemes in TurboFTP by examples.

2.1 Add only new files to destination

Only transfer new source files.

EXAMPLE 1: Only transfer new source files. No file time stamp comparison is involved.

2.2 Update only existing files at destination

Transfer files that overwrite old copies in the destination folder.

EXAMPLE 2: Under this scheme, TurboFTP will compare local and remote file timestamps. In this example, the local file root.txt exists at local and at remote and the local copy is newer. After the synchronization, the local copy of root.txt is uploaded while test.php is not.


2.3 Update and add new files to destination

Transfer new source files and updated source files that overwrite old copies in the destination folder.

EXAMPLE 3: the local copy of root.txt is newer and after synchronization, it is uploaded. In the local folder has test.php which the remote destination folder doesn't have, therefore test.php is uploaded as a new file.

Before action:

After action:

2.4 Update, add new and remove unique files at destination

Transfer new source files and updated source files that overwrite old copies in the destination folder and remove orphan files in the destination folder. Unique/orphan files are files that exist at the destination but not at the source.

EXAMPLE 4: Note that 'have.rar' at remote doesn't exist at local as is seen as a unique file. Therefore it is removed after the action.

Before action:


After action:

2.5 Transfer files added or updated since last sync

Compare time stamps of source files with last run time of the task, if it is newer then transfer them to the destination. With this scheme, file size comparison is not applicable. The first time the task runs, all source files will be transferred since there is no valid 'last runtime'.


After the 1st execution of task:


Then root.txt is modified and the task is executed again. This time the file 'test.php' is uploaded because its timestamp is newer than the last run time.

2.6 Remove duplicate files at destination

Find out files that exist at both local and remote and remove the copies at the destination.

Before action:


After action:

2.7 Transfer all source files

Transfer all files in source folder overwriting any existing files at the destination.

EXAMPLE 6: as illustrated below, all files are transferred regardless whether the destination folder has them or not.

Before action:


After action:

2.8 Transfer all source files and remove unique files at destination

Transfer all files in source folder overwriting any existing files at the destination. Remove orphan files in the destination folder. This is an option for 100% mirroring without caring about the result of file comparisons, in terms of either file time or file size, between both sides.


Before action: the orphan file 'have_no.rar' at the destination is removed after the action.


After action:

3. Other Options for Synchronization

Explanation of the above options:

3.1 Include subfolders: whether synchronize subfolders as well.

3.2 Ignore file name case: when comparing local and remote files, ignore file name case difference.

3.3 Compare file size rather than file time: by default, TurboFTP compares file time stamps during synchronization. Due to the limitation of regular FTP, where time in remote file listing is within the accuracy of a minute, you might want to decide which files to transfer based on the file size difference in bytes. Note that is this option is selected and the source file size is different from the destination file size, TurboFTP will consider the source file is 'newer'.

3.4 The last two options only relate to the visual comparison of Folder Synchronization in TurboFTP and will be ignored when doing automated synchronizations.

4. Execute external programs during synchronization

TurboFTP allows you to interface your own workflow processes with its synchronization action. External programs can be executed before the task, after task and against each downloaded file(locally) or uploaded file.

4.1 Execute Programs before Task

EXAMPLE 8: Pack local files into an archive, and upload the archive to a remote server.

We create a batch file to do archiving with a command line archiver like RAR.

"C:\Program Files\WinRAR\Rar.exe" a "C:\123\test\test.rar" "C:\123\test\test.php"

As illustrated below, we specify archive.bat to be executed before task:

As we only want to upload archives, we set up an inclusion file filter:

Before the task runs, both the local and the remote folders have the same set of files:

After it runs, the local files are packed into test.rar and uploaded:

4.2 Run program after task

EXAMPLE 9: Download a remote archive and extract it in a local folder.

"C:\Program Files\WinRAR\UnRAR.exe" e "C:\123\test\test.rar" "C:\123\test\"

We specify expand.bat to be run after task.

Before action, the local destination folder is empty. We use a sync task to download archives to the local folder, then extract all files from archive test.rar.

After action:


4.3 Execute external program against each downloaded file

EXAMPLE 10: Add each downloaded file to an archive

We have an archive back.rar and will add each downloaded file to it.

"C:\Program Files\WinRAR\Rar.exe" a "C:\backup\backup.rar" %1

Before task runs:

When the task was done, files are downloaded and added to the archive:

You can also delete or move each downloaded or uploaded file.

Please note under After Download the path we move files to is a remote folder path.

4.4 Execute program against each uploaded file

EXAMPLE 11: Extract a backup archive and uploaded extracted files to a remote folder.

We use the following script to extract a backup archive in a temporary folder.

"C:\Program Files\WinRAR\UnRAR.exe" e "C:\backup\backup.rar" "C:\123\test\"

While the task runs we delete each uploaded local file.

del %1

Before action: the local and remote folders are empty.

After action:

You can also use TurboFTP's build function to delete/move uploaded files. This is preferable because the task log shows whether the external program completed successfully.