Set up mergerfs (using ssh)
April 13th, 2021, 19:20

Introduction

In this tutorial we are going to setup mergerfs and rclone to be able to keep new files in a local folder while having your rclone mount accessible under the same folder.

In this specific setup we will use /mnt/shared/fusemounts/local for the local storage, /mnt/shared/fusemounts/mount for the rclone mount and /mnt/shared/fusemounts/merged as the merged path where all apps will read and write to.


Step 1 - Install Prerequisites

Make sure that you have all the needed dependencies such as:

  • Enable sudo from your cloudbox's dashboard

  • Install and setup an rclone mount. You may follow steps 1 - 3 from this guide.

  • An updated system - Start by making sure that your system is up-to-date by doing the following command:
    sudo apt-get update && sudo apt-get upgrade

  • Install mergerfs
    wget https://github.com/trapexit/mergerfs/releases/download/2.32.4/mergerfs_2.32.4.ubuntu-bionic_amd64.deb && sudo dpkg -i mergerfs_2.32.4.ubuntu-bionic_amd64.deb && rm mergerfs_2.32.4.ubuntu-bionic_amd64.deb

  • Install nano text editor:
    sudo apt-get install -y nano

* -y skips the confirmation you get while installing the packages


Step 2 - Running rclone mount and mergerfs

  • You will need to create the folders for the mounts. (This can be changed, but remember to change the future configs)
    mkdir /mnt/shared/fusemounts/local
    mkdir /mnt/shared/fusemounts/mount
    mkdir /mnt/shared/fusemounts/merged

  • Run a rclone mount to /mnt/shared/fusemounts/mount. (These options can be changed to your liking).
    If you have your rclone mount already mounted, skip this step.
    rclone mount <nameofrclonedrive>: /mnt/shared/fusemounts/mount --allow-other --buffer-size 256M --dir-cache-time 1000h --drive-chunk-size 128M --poll-interval=15s --timeout 1h --vfs-read-chunk-size-limit off --gid=911 --uid=911 &
    Change <nameofrclonedrive> to your remote

  • Following that command you will have to merge the local folder and the mount to /mnt/shared/fusemounts/merged.
    mergerfs /mnt/shared/fusemounts/local:/mnt/shared/fusemounts/mount /mnt/shared/fusemounts/merged -o async_read=false,use_ino,allow_other,auto_cache,func.getattr=newest,category.action=all,category.create=ff


Step 3 - Case A - Setup Backup on Cloud app (optional)

New files will be written under /mnt/shared/fusemounts/local. You can have them moved off to a cloud storage of your choice, by installing and setting up "Backup on Cloud (Rclone)". You can follow this guide.


Step 3 - Case B - Setup the moving script (optional)

To create the moving script you will need to run these commands.

Moving script will move files that are older than 3 days.

  1. Install cron. You may follow this guide.
    sudo apt-get install -y cron

  2. Install rclone on your system, if not already installed:
    sudo curl https://rclone.org/install.sh | sudo bash

  3. Create rclone config folder:
    mkdir /mnt/shared/config/rclone

  4. Create file with the moving script:
    cat >/mnt/shared/config/rclone/upload.sh <<-RCS
    #!/bin/bash

    lock_file="/mnt/shared/config/rclone/rclone-upload.lock"

    trap 'rm -f "\$lock_file"; exit 0' SIGINT SIGTERM
    if [ -e "\$lock_file" ]
    then
    echo "Rclone upload script is already running."
    exit
    else
    rm -f /mnt/shared/config/rclone/rclone-upload.log
    touch "\$lock_file"
    /usr/bin/rclone move /mnt/shared/fusemounts/local/ <nameofrclonedrive>: \
    --drive-chunk-size 64M \
    --tpslimit 5 \
    -vvv \
    --drive-stop-on-upload-limit \
    --delete-empty-src-dirs \
    --fast-list \
    --bwlimit=8M \
    --min-age=3d \
    --use-mmap \
    --transfers=2 \
    --checkers=4 \
    --log-file /mnt/shared/config/rclone/rclone-upload.log
    rm -f "\$lock_file"
    trap - SIGINT SIGTERM
    exit
    fi
    RCS

  5. Edit moving script if necessary:
    nano /mnt/shared/config/rclone/upload.sh

    Change <nameofrclonedrive> to your remote
  6. Set moving script as executable:
    chmod +x /mnt/shared/config/rclone/upload.sh

  7. Set script to run every day:
    crontab -e

    Running that command will open up the editor you will have to add this line (This will make the script run every day at 7am).
    0 7 * * * /mnt/shared/config/rclone/upload.sh > /dev/null 2>&1


Important Notes

  • If you want to unmount fuse itself:
    sudo fusermount -uz "/mnt/shared/fusemounts/<merged>"

  • Do not use "Repair Permissions" when you have a mounted folder running.

  • Do not restart apps (like Plex) when you have a mounted folder, as it can cause your app to hang. First you will have to unmount the folder, restart the app (let it fully start) and then mount the folder again.


Conclusion

Now that you have your rclone and mergerfs running you should be able to point your apps to /mnt/shared/fusemounts/merged (in this case) and mergerfs will show the contents of the local and mount folders under one path and it will write new files to the local folder.

You can ask more on our #mergerfs-plexdrive channel channel on Discord.