A Story of Silent Data Corruption with Seafile
Lem453 @ Lem453 @lemmy.ca Posts 10Comments 436Joined 2 yr. ago
Next cloud has never stored the files themselves in a db. I've been using it since before it existed (own cloud) and then switched, it always has had a flat file storage that you can just backup and browse without the metadata from the database if you want.
Unfortunately that's also part of it's Achilles heel and why it's so slow, it's not optimized.
Seafile has an sshfs style client for windows, mac and Linux. Rather than a traditional folder sync like Dropbox (which seafile also has), seadrive mounts a remote connection to your library that you can browse in your file explorer. I've only used the windows version, it has little cloud icons that show the files are not local and then you can right click a folder of file and "make available locally" to have offline access. This sounds exactly like that you are looking for. Full gui access to all files with no local storage needed unless you want.
I haven't tried seadrive on Linux but they have the option on their site. I use the standard seafile-client on Linux and choose only certain libraries to use with no issues. On windows the seadrive is quite impressive in regard to how well it works.
https://help.seafile.com/drive_client/drive_client_for_linux/
The biggest issue with immutable OSs is the lack of containerized apps. Most devs simply don't distribute their apps in flatpaks etc. Install fedora atomic. Fist think I want to do is install xpipe to manage my servers. Can't be don't in an unprivileged flatpaks. Great layer it on.
Let's try seafile next to sync my files and projects...the flatpak is maintained by a random volunteer and most up to date version is from a year ago. Great, layer that in as well.
Let's install a command line tool, before it was 1 line, now it's a whole lot of googling only to discover that the best way is probably to just have a whole other package manager like brew
The concept is great and it has lots of potential, just it will only work if devs start packaging their stuff in a format that works with the new paradigm (containers)
It's not so much a seafile issue as it is a feature request for proper checksum verification of the files that are copied. The conditions where it it happened were an odd combination of having enough persistent storage, ram and cpu but lacking in ephemeral space...I think. The issue is that seafile failed silently.
Me messing about with other docker applications. Seafile is one of the first things I setup on my server. I've been adding and playing around with dozens of different apps since then, many of them have numerous containers each. Usually I make the container without a defined storage until I get the compose working, then I set the volumes to be zfs array. When that happens the old default docker volumes remains unused.
Need to remember to delete them periodically
This is one of the reasons I never use docker volumes. I bind mount a local folder from the host or mount and NFS share from somewhere else. Has been much more reliable because the exact location of the storage is defined clearly in the compose file.
Borg backup is set to backup the parent folder of all the docker storage folders so when I add a new one the backup solution just picks it up automatically at the next hourly run.
Awesome, take this close call as a kind reminder from the universe to backup!
Borg will allow incremental backups from any number of local folders to any number of remote locations. Borgmatic is a wrapper for it that also includes automated incremental borg backups.
I have a second server that runs this container: nold360/borgserver
Which works as a borg repository.
I also buy storage in borgbase and so every hour and incremental setup goes to both.
The other day I blew away a config folder by accident and restored it with no sweat in 2 mins.
Is it not in the immich_pgdata or immich-app_pgdata folder?
The volumes themselves should be stored at /var/lib/docker/volumes
For future reference, doing operations like this without backing up first is insane.
Get borgmatic installed to take automatic backups and send them to a backup like another server or borgbase.
For future reference, if you use docker compose you can setup up this as well for each application that has a postgres database
Give it the same user name and password for the database that the application uses. Give it a mounted volume then you backup the docker files and the folder where these backups are made at the same time
https://github.com/prodrigestivill/docker-postgres-backup-local
If you ever have a problem you will have several proper database dumps to work with
I have one of these for every docker app that runs postgres.
I believe there is a similar thing you can setup for mariadb and sqlite
This is the way. This is the video I followed.
https://www.youtube.com/watch?v=liV3c9m_OX8
I use traefik as reverse proxy. I have externally accessible domains for and then extra secure internal only domains that require wireguard connection first as an extra layer of security.
Authentik can be used as a forward auth proxy and doesn't care if it's an internal or external domain.
Apps that don't have good login or user management just get Authentik proxy for single sign on (sonarr, radar etc).
Apps that have oAuth integration get that for single sign on (seafile, immich, etc)
To make it work the video will talk about adding both the internal and external domains to the local DNS so that if you access it from outside it works and if you access from wireguard or inside the lan it also works.
Permanently Deleted
By zero maintenance they mean you don't even have to hit the update button. It all just happens automatically. Many Linux users won't like that but many windows and Mac users will.
Permanently Deleted
They advertise as being zero maintenance which is a huge deal with many windows and Mac users than don't want to think about the tool itself, they just want to use it. From the site:
What's the difference between Vanilla Kinoite and Aurora? Vanilla Kinoite is a very stock experience. Aurora includes many enhancements and tweaks, like included drivers for various printers, network adapters and more as well as included codecs. Aurora also features tweaks to enhance your battery life on a laptop.
Permanently Deleted
If you don't want to think about your computer and just want a tool to use there is Aurora. It's a variant of fedora but it uses an immutable file system which makes it super stable and reliable. If there are any issues you can easily roll back the entire os to a previous version.
This is true of all fedora atomic desktops. Aurora is a variant that takes it to the next level by making all updates and everything require as little human interaction as possible so you don't have to worry about how the computer runs and just use the computer for your actual tasks.
What makes butter better than btrfs for ostree systems?
I'm curious how using ansible to deploy docker containers is easier than just using docker compose?
Ansible makes sense to setup the OS the way it needs to be (file systems, folder structure etc), but why make every container through ansible instead of just making a docker compose and maybe having ansible deploy that?
Even easier is probably to just run something like portainer and run the compose file through there
If you submit the rpm to rpm-ostree then users can just find it from there with rpm-ostree install xpipe.
That requires an overlay but the alternative is a flatpak which won't work for an app like this I think anyways.
Users that install brew can just get it from here as a proper containerized install rather than an overlay.
The script is definitely not great as he primary way to install, everyone doing that should be doing so very reluctantly. Getting the rpm into package managers will go a long way.
That being said, xpipe is amazing. Only used it for a few hours and already love it and can't believe I didn't have it sooner.
Fedora atomic with kde (kionite) has been amazing on my laptop so far (recently moved from mint)
How do I install this on fedora? I'm not to keen on curling a bash script and running it. Thanks!
Edit: for fedora atomic, the answer is to download the rpm and overlay it with rpm-ostree install
Sounds like they actually changed it to Go language regex syntax instead of pearl syntax.
The documentation certainly makes it sounds like they just got rid of regex but this forum post seems to show otherwise.
https://community.traefik.io/t/pathprefix-regex/21819
I'm definitely in the wait for a month at least before attempting this upgrade camp...
Ya exactly this. I get optimizing for spee but there should at least be an option afterwards to check file integrity. Feels like a crucial feature for a critical system