Linux

5187 readers

79 users here now

A community for everything relating to the linux operating system

Also check out !linux_memes@programming.dev

Original icon base courtesy of lewing@isc.tamu.edu and The GIMP

founded 1 year ago

MODERATORS

Ategon@programming.dev

anzo@programming.dev

dwraf_of_ignorance@programming.dev

Best practice for creating a unified view of data across multiple Linux systems? (programming.dev)

submitted 1 week ago by CoderSupreme@programming.dev to c/linux@programming.dev

3 comments fedilink hide all child comments

I'm looking to create a unified view of data across multiple Debian-based devices without fully replicating the data. My current setup includes:

Main computer with one HDD
Server with four drives
A couple of Raspberry Pis

I want a folder on each device that provides access to the contents from all drives, but without actually copying or syncing the data to each device. I'm aiming for a solution that allows browsing and accessing all files from any device while keeping the actual data in its original location.

I've been looking into using a combination of MergerFS and SSHFS. The idea is to use SSHFS to mount remote directories and then use MergerFS to combine these mounts with local directories into a single view. However, I'm not sure if I should merge the drives in the server and share the merged folder with all other systems or I should share each drive with each system and merge them in there. Is this the best approach or are there better alternatives?

I want to avoid solutions like Syncthing, Dropbox, or Google Drive that would clone the entire data set to each device. I'm trying to avoid data duplication and save storage space on devices with smaller capacities.

I'd love to hear your thoughts on the MergerFS + SSHFS approach, or if you have any other suggestions that might better fit my needs. Any insights, recommendations, or personal experiences would be greatly appreciated!

Thanks in advance for your help!

top 3 comments

sorted by: hot top controversial new old

[–] nous@programming.dev 12 points 1 week ago

Sounds like you just need to keep the data on your server and use samba or NFS and a network mount on the other devices.

[–] nullPointer@programming.dev 6 points 1 week ago

NFS

[–] sxan@midwest.social 1 points 1 week ago

You want Upspin. I want Upspin. But Upspin never went anywhere (it's at least 7 years old... ever heard of it?), and I personally believe that it was because it's a royal PITA to set up, and because the tutorial had instructions that expected you to be using GCS. If you wanted to do everything on your LAN, it was even harder.

It's got all the of the features you mention, and it's really the only system that does what it does; I really did try in the early days to get it running, and failed. It still has the caveat:

Upspin has rough edges, and is not yet suitable for non-technical users.

and, at 7 years old, if it hasn't gotten anywhere yet, I think it never will. Commits trickle in, but there's really no significant progress in usability.

Read the mission statement. It's glorious. And then wallow in despair that nothing else does this, and it's a zombie project.