Synchronizing files across phone, laptop and NAS with Nextcloud and Syncthing

How-to guide

2022-09-03 9 min read Informatique, Organization

Introduction

I wrote other articles on how I organize my files across my phone, laptop and NAS (Network Attached Storage) using Nextcloud:

This system has kept my files safe and accessible remotely, but I have been frustrated by the lack of 2-way file synchronization in the Nextcloud client for Android.

I recently started using Syncthing to synchronize files between my Android phone and my laptop, and I found that the combination of Nextcloud and Syncthing covers my needs much better!

Many Nextcloud and Android users on the internet are also struggling with this and are looking for solutions. My post on configuring the Nextcloud client for Android is one of the most popular on my blog, with ~60 page views per month. So I’ve been motivated to write an update with a detailed review of my recent findings :nerd:.

Table of Contents

Why use two applications?

I need fast and reliable 2-way synchronization for my phone’s camera pictures, downloaded files and other documents so that I can process and organize them efficiently using either my phone’s gallery apps or my laptop.

I also need the remote access and selective downloading features of the Nextcloud client to access archived files from my phone.

Here’s the problem: unlike the desktop client, the Nextcloud client for Android doesn’t have good 2-way files synchronization. It has what they call instant uploads. The app watches some configured directories for new files, and moves them to Nextcloud folders; it’s a 1-way sync. This doesn’t work very well because it doesn’t allow me to process, organize or edit files synchronously on my laptop or my phone.

Syncthing is an awesome open-source tool for secure, fast, peer-to-peer, 2-way and multi-way file synchronization that works really well on Android and on desktop.

Because I have different file processing workflows that operate on different directories on the phone’s filesystem, the two file syncing apps don’t interfere and I can use the best tool for each process.

File processing workflows	Best files access method	Best tool
Organizing “inboxes” of pictures and files, either on my phone or my laptop.	2-way synchronization.	Syncthing
Accessing archived pictures and files.	Remote access and selective downloading.	Nextcloud

Architecture diagram

Diagram summarizing the synchronization and file access functionalities of Nextcloud and Syncthing across my phone, laptop and NAS.

Shortcomings of the Nextcloud client for Android

The Nextcloud client for Android is great for accessing files remotely. However, it doesn’t do 2-way sync:

GitHub issue: Allow 2-way-sync.
Reddit discussion: [Android] No two-way sync?.
Nextcloud forum discussion: To be clear…. no way to have 2way sync with android client.

…and it doesn’t play very well with Android’s MediaStore (the phone’s pictures gallery):

GitHub nextcloud/android issue #6387: Unable to see downloaded files in apps other than Nextcloud
GitHub nextcloud/android issue #10592: Media files downloaded with Nextcloud Client are not indexed/scanned by Android’s Media Storage

Syncthing vs other file synchronization apps

Lots of people recommend FolderSync to do 2-way synchronization between an Android phone and Nextcloud. It connects to Nextcloud using the WebDAV protocol. I tried it, I wasn’t pleased. The UI was a bit buggy, it is not documented well, there isn’t a public code repository to submit and track bug reports, and most importantly it’s not a free open-source program.

Comparison of Nextcloud client and Syncthing for Android

Nextcloud client for Android

Pros 👍	Cons 👎
Can browse a very large archive remotely.	Can’t do 2-way sync.
Can download directories or files selectively.
Automatically configure DAVx⁵ to synchronize contacts and calendar with Nextcloud

Syncthing for Android

Pros 👍	Cons 👎
Does 2-way and multi-way sync very well.	Can’t exclude subdirectories (other than editing a `.stignore` file) so it doesn’t work for accessing a huge archive.
Peer-to-peer protocol, no server involved (except for the public discovery and relay infrastructure).	Doesn’t sync directly to Nextcloud server, syncs only when the laptop is turned on.

File processing and organizing workflows

In order to identify how and when I would use Nextcloud or Syncthing, I need to describe my different workflows.

1. Creating content and collecting it to “Inbox” directories

Content is generated from the phone’s camera, and downloading content from applications.

The DCIM, Documents, Downloads, Pictures and Movies folders must be synchronized to ~/Inbox directories on my laptop.

Use Syncthing.

2. Organizing and archiving content

I can view, edit, delete, share or publish pictures and videos from my camera’s Gallery app.

I can also view, edit, delete or organize pictures and videos on my laptop using file browsers and video or picture editing programs. Changes are quickly synchronized back to the phone.

I organize pictures and files on my laptop using DigiKam and the file browser. Files are moved from ~/Inbox directories to the ~/Documents and ~/Pictures archive directories. As I do this, the files are synchronized out of my phone (deleted) and copied to their new location on the Nextcloud server.

My laptop synchronizes files with my phone using Syncthing, and to my Nextcloud server with the Nextcloud client.

3. Viewing archived content

If I need to access archived content on my phone, I can just browse my files using the Nextcloud client app.

Viewing archives for old pictures and projects:: Use the Nextcloud client app to browse files.
Downloading files for working locally:: Download files or directories in the Nextcloud client app. Nextcloud calls this feature “synchronize”, but it’s a 1-way and 1-time download. The Android MediaStore will pick up the files to make them accessible from other apps.

The Android filesystem

I spent some time exploring the phone’s filesystem and how the MediaStore interface works in order to understand what directories I need to sync.

Under the /DCIM and /Pictures directories, files are saved under a subdirectory by the name of the App. For example, /DCIM/Camera, and /Pictures/Reddit.

It looks like /Download files are not organized by source application.

Files downloaded in the Nextcloud app are stored in /Android/media/com.nextcloud.client.

(Syncthing) : Directories Synchronized with Syncthing.
(Nextcloud) : The root directory for files downloaded from Nextcloud.

Stockage interne
├── Android
│   ├── data
│   │   ├── com.nextcloud.client
│   │   └── ...
│   ├── media
│   │   ├── com.nextcloud.client
│   │   │   └── nextcloud
│   │   │       └── alex@nextcloud.deverteuil.net 
│   │   └── ...
│   └── obb
│       └── ...
├── DCIM 
│   ├── Camera
│   │   ├── 20220731_123922.jpg
│   │   ├── 20220731_124002.jpg
│   │   └── ...
│   ├── OpenCamera
│   │   ├── IMG_20220828_172707.jpg
│   │   ├── IMG_20220828_172746.jpg
│   │   └── ...
│   └── Screenshots
│       ├── Screenshot_20220713-100905.png
│       ├── Screenshot_20220713-100905.png
│       └── ...
├── Documents 
│   └── BeReal
│       ├── bereal-1661633012050.jpg
│       ├── bereal-1661713517923.jpg
│       └── ...
├── Download 
│   ├── 20210725_110806.txt
│   ├── 20220509_172413.jpg
│   ├── 2663ca9e73a6df29aa2b0a19a8a6eb31a1f1_211105_180324.pdf
│   ├── 4VRE68+Service-3877.pdf
│   ├── 8JYGH-3N2AK-WQZ8H@73199D34D2DBA813EE606BAED9EF03.pdf
│   ├── 9352e4c.jpg
│   └── ...
├── Movies 
│   ├── Instagram
│   │   └── VID_31191212_125936_000.mp4
│   └── Messenger
├── Music
├── Notifications
├── Pictures 
│   ├── Imgur
│   │   ├── xY580nx.jpg
│   │   ├── ywQtaZL.jpg
│   │   └── ...
│   ├── Instagram
│   │   ├── IMG_20220605_095002_084.jpg
│   │   ├── IMG_20220719_095108_107.jpg
│   │   └── ...
│   ├── Messenger
│   │   ├── received_885191768779949.jpeg
│   │   ├── received_956846471608708.mp4
│   │   └── ...
│   ├── Reddit
│   │   ├── RDT_20210707_0059423011284352944159678.jpg
│   │   ├── RDT_20210919_0934132522933891369918761.jpg
│   │   └── ...
│   └── Twitter
│       ├── 20210830_112325.jpg
│       ├── 20220104_133040.jpg
│       └── ...
├── Podcasts
├── Recordings
├── Ringtones
└── RW_LIB

Setting up synchronized directories

I set up five synchronized folders with Syncthing.

Phone directory	Laptop directory	Purpose
/DCIM	~/Inbox/Cell phone camera	Images created with camera apps and phone screenshots.
/Documents	~/Inbox/Cell phone documents	Documents created by various apps. On my phone, that’s where BeReals are automatically saved.
/Downloads	~/Inbox/Cell phone downloads	Files downloaded
/Movies	~/Inbox/Cell phone movies	Movie files created with camera apps.
/Pictures	~/Inbox/Cell phone pictures	Images saved from social media apps.

The /DCIM, /Documents, /Download, /Movies and /Pictures directories on my phone work like inboxes, so they are synchronized in subdirectories under ~/Inbox on my computer. I process inboxes on my computer by organizing files under the archive directories ~/Documents and ~/Pictures. The files get deleted from the phone, but they become accessible for viewing or downloading with Nextcloud.

Diagram of directories synchronized between my phone, my laptop and my Nextcloud server.
In the screenshot, I'm using the file browser on my Ubuntu laptop. I access my Android phone's filesystem over USB, and my Nextcloud files over WebDAV. — Diagram of directories synchronized between my phone, my laptop and my Nextcloud server.
In the screenshot, I’m using the file browser on my Ubuntu laptop. I access my Android phone’s filesystem over USB, and my Nextcloud files over WebDAV.

Keeping the MediaStore synchronized

The Android MediaStore is an index of pictures, videos and other documents found on the phone’s filesystem. Many applications—gallery apps in particular—use the MediaStore to browse media organized by date or by collection. Usually, when files on the phone are updated by Nextcloud or Syncthing, the MediaStore is updated to index new files, or de-index deleted files. However, sometimes this doesn’t happen, or there is a long delay before the index is updated.

Here are two methods to force a MediaStore re-scan.

1. Restart the phone

A MediaStore re-scan happens shortly after restarting the phone.

2. Delete the Media Storage app data

Go to Settings, Applications.
Open the Filter and sort options.
Select “Display system applications” and touch OK.
Search for the “Media Storage” app.
Go to the app’s Storage info.
Click “Delete app data”.

The MediaStore will start a re-scan of media files within seconds.

Go to Settings, Applications.
Open the **Filter and sort** options.

Select "**Display system applications**" and touch **OK**.

Conclusion

For managing and synchronizing files on my Android phone, Nextcloud and Syncthing complement each other very well.

I’ve been researching and trying different Nextcloud settings, and different file synchronization apps in the past few weeks. When I started using Syncthing, it was very easy to set up. It is extremely well designed and documented too. Syncthing was filling in exactly where Nextcloud was lacking. I’ve been using this workflow for a few days and I feel excited over the improvements I saw; from trying to work with the limited functionalities in Nextcloud, to seeing 2-way sync achieved between my phone and my laptop, using open-source software, the way I want it!

nextcloud android organizing