cross-posted from: https://l.antiope.link/post/43914

Hi all. I’m trying to choose a configuration for my home storage. Speed is not a priority, I want a balance of stability and performance. I was thinking of making a raid 6 array with an ext4 file system for 4 disks of 2 TB each. Asking for advice, will this configuration be optimal?

Note - I am going to make a raid array based on external usb drives, which I will plug into the orange pi

  • Shdwdrgn@mander.xyzEnglish
    8·
    2 years ago

    Use ZFS instead of ext4… it has a lot built into it which protects data integrity, and it’s always been suitably fast for me. My current large arrays use raid-z2 (which is equivalent to RAID6, having two drives of redundancy), however I started out with the insane configuration of five drives in a raid0, lost power and dropped one of more drives several times, and was able to bring it back up without any data loss. I would NOT recommend this to anyone, but it took me time to learn that my drives were dropping due to a poor quality power supply, and the recovery impressed me so much that I have always used ZFS since then.

    There are a few optimizations you can do when you set up your array such as configuring the stripe width and sector sizes, although I believe the default settings on these are pretty ideal now and may no longer require tweaking.

    As for overall speed… My most recent array is built from eight 18TB drives, formatted out to around 90TB of usable space. While designing my external rack I knew that HDDs can’t really reach SATA3 speeds, but an array can out-perform that by accessing multiple drives at once. I built my assembly with cheap SATA2 backplanes and LSI SAS cards. Even right now with the array in use by multiple servers I am still getting speeds between 483-597MB/s while copying 10GB of random data and nearly 900MB/s copying from /dev/zero. You’re obviously not going to see that kind of speed from USB-connected drives, but the point is that ZFS itself will not slow you down in any way.

      • constantokra@lemmy.one
        1·
        2 years ago

        If you want to be able to grow, check out mergerfs and snapraid. If you’re wanting to use a pi and USB drives it’s probably more what you’re wanting than zfs and raid arrays. It’s what i’m using and I’ve been really happy with it.

          • constantokra@lemmy.one
            2·
            2 years ago

            I’ve been using linux for a long time, and I have a background in this kind of stuff, but it’s not my career and I don’t keep as current as if it was, so i’m going to give my point of view on this.

            A zfs array is probably the legit way to go. But there’s a huge caveat there. If you’re not working with this technology all the time, it’s really not more robust or reliable for you. If you have a failure in several years, you don’t want to rely on the fact that you set it up appropriately years ago, and you don’t want to have to relearn it all just to recover your data.

            Mergerfs is basically just files on a bunch of disks. Each disk has the same directory structure and your files just exist in one of those directories on a single disk, and your mergerfs volume shows you all files on all disks in that directory. There are finer points of administration, but the bottom line is you don’t need to know a lot, or interact with mergerfs at all, to move all those files somewhere else. Just copy from each disk to a new drive and you have it all.

            Snapraid is just a snapshot. You can use it to recover your data if a drive fails. The commands are pretty simple, and relearning that isn’t going to be too hard several years down the road.

            The best way isn’t always the best if you know you’re not going to keep current with the technology.

  • greengnu@slrpnk.netEnglish
    2·
    2 years ago

    Raid stopped being optimal now that btrfs and ZFS exist.

    If you plan on doing matching drives ZFS is recommended

    If you expect mismatched disks, btrfs will work.

    If you are most worried about stability get a computer with ECC memory.

    If you are most worried about performance, use SSD drives.

    If you want a bunch of storage for cheap, use spinning disks (unless you exceed the 100TB capacity range)