Sync

Sync is very useful when you are importing or modifying files directly on the file system, e.g. with FTP. It basically compares the File Info (file size, modification time and MD5 checksum) stored in the database with the actual files.

New files will be added automatically, including ID3 tag detection and thumbnail generation. For files in sub-directories Sync creates the Category tree, i.e. for “folder1/subfolder/file.pdf” it creates the category named “subfolder” and it’s parent category “folder1”.

You find the Tool on WP-Filebase Dashboard:

Where to find the Sync Button

The Mechanism

Sync can be divided in mutliple steps:

  1. At first the plugin checks for changed or deleted files. In normal sync operation it just compares sizes and modification dates to detect a file changed. In hash sync it compares the MD5 hashes, which is more reliable, but can take much more time. When a file change is detected, it updates the values in the database including extended file info (ID3 tags).
  2. New files are added to the database. When adding new files with sync WP-Filebase looks for an image in the same directory with the same basename to use it as thumbnails. Example: document.pdf and document.png. Images named folder.png are used for category icons.
  3. Remote Syncs are executed (WP-Filebase Pro only). See Remote Syncs.
  4. In a final step file tags are refreshed and categories are synchronized. (updating file count and checking for missing folders)

Thumbnails

You can simple add thumbnails to your files by uploading an image with the same name as the file. For example mypdf.jpg will be used as thumbnail for mypdf.pdf. Also images with names like document-80×120.jpg are use as thumbnails. Any images named ‘folder.jpg’, ‘folder.png’, ‘folder.gif’, ‘cover.jpg’, ‘_caticon.jpg’, ‘_caticon.png’ or ‘_caticon.gif’ will be treated as category icon.

Troubleshooting

A complete successfull sync should end with the message:

 Filebase successfully synced.

If not something went wrong. In most cases its PHP running out of memory, especially when adding a great amount of files. To reduce memory usage, disable ID3 detection (WP-Filebase -> Settings -> Misc).

If you still have problems, append &debug=1 to the URL of the sync page, for example: //wpfilebase.com/wp-admin/admin.php?page=wpfilebase_manage&action=sync&debug=1 . The plugin will output backtrace debug info in HTML comments. Right click on the page, view source  and scroll to the end of the document to see the last backtrace info. Copy this text and include it in a support request to get help.

Batch Sync

WP-Filebase Pro has a batch sync mechanism to partition the files to add. The sync is executed with multiple HTTP requests so the overall memory usage is reduced. That allows to sync an unlimited number of files at once.

The normal sync would run out of memory after about 1000 files (with ID3 tag detection enabled and depending on server’s operating system and configuration). Batch Sync detects if the memory usage is critically and starts a new request if needed.

21 thoughts on “Sync

  1. VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)

    Hi there! I have a question… on one site where I’m testing WP-Filebase, to see if it meets my requirements, I had already uploaded all the files — around 126 GBytes or so. But there are not really many files, just 44 in total, each of them being huge (mostly they’re videos made from workshops). The problem here is that I don’t get any progress from the “sync” feature.

    Even with debug on, all I get is:

    Synchronisation
    Starting sync. Memory usage: 59.8 MiB – Limit: 90.0 MiB

    Checking for file changes… done!

    Searching for new files…
    44 Files found, 36 new.

    and a bar saying 0% complete. It stays that way for hours.

    The debug file just says “INIT SYNC” every time I initiate it. No errors on the logs. Although this is on a shared server with limited memory, I don’t see anything happening to give me a clue if the server is timing out somehow, or hitting a block somewhere but not throwing any errors…

    Nevertheless, it has worked for a while: after all, it did manage to find 8 files, tagged them correctly and so forth, and everything works well on the page where the tree view is placed with a shortcode, etc. So I know that the setup (permissions, etc.) is fine. Some of the files that were found on the first run are a couple of GBytes long, so WP-Filebase seems not to have any trouble with huge files (even if there are much bigger ones).

    My problem is that I get no feedback for the remaining 36 files, and no clue if WP-Filebase is actually doing anything or not.

    Short of directly hacking the database to “reveal” those files — something which I’m not familiar with and would definitely avoid at all if possible — is there anything else I can do to actually “see” if WP-Filebase is doing something or not?

    Note that this is for an educational site where students who have missed a workshop or two are able to access a password-protected area where they have access to notes and I wish to give them access to whole videos or audio transcripts.

    1. VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)

      After several refreshes, testing at different times of the day, I managed, so far, to reach 8% 🙂 — I turned ID3 detection off, to see if that improved things, and separated all files according to different folders/categories (to see if that helped). I think that now it’s just a question of being lucky until I get the whole process to complete at least once.

      It would be nice to get a bit more feedback on what’s failing, though.

  2. VA:F [1.9.22_1171]
    Rating: +1 (from 3 votes)

    Some further feedback. To populate the database, apparently WP-Filebase calls md5sum externally. Now when this is invoked for huge files (several GBytes), it seems to fail with a “broken pipe” error, which is understandable. My hosting provider limits the amount of memory a script can consume, and it’s penF" sty59_c_e- hon