ESPresense: Easy Room Detection for Home Assistant

| Comments

I recently installed a Wink Relay in my office and “hacked” it to work with Home Assistant. This completed one of my home automation goals: all of my office’s lighting and its ceiling fan are accessible to Home Assistant for automating tasks.

Ever since first installing up Home Assistant, I’ve been wanting to automate turning on my ceiling fan. I very easily threw together this Node-RED flow. On each update of the temperature from office’s Zooz 4-in-1 sensor, it will use the temperature to decide whether to turn the fan on or turn it off.

There’s a shortcoming with this flow, though. Turning on a fan doesn’t lower the temperature in the room, it only makes it feel cooler thanks to the evaporative effect of air moving over your skin. The only time this automation would be beneficial was if someone (me) was already in the room.

Room-Specific Presence Detection in Home Assistant

Thanks to the Home Assistant iOS App and the GPS features in my iPhone 12 Pro Max, my Home Assistant installation has a pretty good idea of when I’m home. For the longest time, I’ve used this as a condition to either turn the lights on or off inside my office.

But my phone doesn’t necessarily know what room I’m in. I don’t think the GPS is accurate enough, and I definitely don’t want to try and figure out the GPS coordinates for the boundaries of each of the rooms in my house. This is problematic, but even more problematic is the fact that my phone isn’t always near me. I routinely leave my phone in other rooms as I nomadically wander around my house during the day.

I was in need of an easier—and better—method to implement room-specific presence detection inside Home Assistant.

Enter ESPresense

Thanks to a reply to one of my tweets from @HolgBarath, I learned of the existence of the project, ESPresense. After it caught my attention, I took a look at the ESPresense website, the ESPresense GitHub repository, and watched a few videos on YouTube. I immediately knew that I wanted to give it a closer look.

What is ESPresense? On their website, they say it’s “An ESP32 based presence detection node for use with the Home Assistant mqtt_room component for localized device presence detection.”

ESPresense accomplishes its goal by providing an interface to easily flash their firmware onto an ESP32 development board, which enables the ESP32 board to monitor nearby Bluetooth low-energy devices. Scatter a few of those ESP32 devices across your house and set up the Bluetooth device(s) in Home Assistant you want to track and you’re ready to unlock the room presence achievement!

I have two Bluetooth devices that are pretty much attached to me all the time: my Apple Watch SE and my Medtronic 770G Insulin Pump. Of those two devices, I figured the watch was the better device to use ESPresense to track.

Brian implements ESPresense at home

I am relieved to report that setting up ESPresense was easy enough and well-documented enough that I don’t think there’s much need for this blog to turn into a how-to guide. The ESPresense install page has all the information you need to get started, including Everything Smart Home’s excellent video on ESPresense embedded in the page.

The software prerequisites for ESPresense are pretty straightforward. I already had my own functional Home Assistant installation, which includes a MQTT server. For the hardware, I decided that I’d use the following hardware to build my ESPresense base stations:

  1. D1 Mini NodeMCU ESP32 ESP-WROOM-32 Development board (5 pieces) ($34.99)
  2. UorMe 1A 5V Single Port USB Power Adapters (6 pieces) ($10.96)
  3. Spater 6” Micro USB Sync Cable (5 pieces) ($7.98)
  4. MH-ET Live D1 mini ESP32 Case

Altogether, I wound up spending $60 and some time on my 3D printer to add Bluetooth tracking to 5 different rooms in my house. I definitely could’ve done it cheaper too. I didn’t really need all the USB power adapters or cables, as I probably have plenty of both stashed somewhere in the house.

Flashing ESPresense onto my ESP32 boards was a snap from their Install page. Their website allows you to flash the ESP32 with the latest version of ESPresense from right inside the browser and to open a serial terminal connection to the ESP32 after it is done flashing.

For the most part, everything went as smoothly as I expected from the documentation. I thought I’d share a few things that I encountered along the way that might have made it even smoother.

Bluetooth Chatter: I have a lot of Bluetooth devices in my office: my insulin pump, watch, phone, work laptop, personal laptop, smart speaker, etc.. Figuring out the Bluetooth details to create the sensor in Home Assistant wound up being a bit of a challenge. I used a couple different methods to try and sort that out.

  • MQTT Explorer, connected to my MQTT server on Home Assistant, and monitored the espresense\devices topic.
  • Took my laptop, watch, and an ESP32 board to a room with no BLE devices and used the ESPresense Terminal to determine the Bluetooth IDs
  • Bluetooth Scanner Apps were recommended a couple different places, and I expected them to be helpful. But I didn’t exactly find them to be especially useful—but everyone’s mileage may vary!

ESPresense’s very active development and automatic updates: By default, the auto-update feature is enabled on the ESPresense base station. It is also a very active project on GitHub. The combination of these two factors might occasionally work against you. On the day I was setting everything up for the first time, a release happened that caused my ESP32s to repeatedly crash and be quite unreliable. I wound up disabling the auto-update and using the ESPHome-Flasher to flash an earlier, more stable, version.

Each base station required calibration: This should be expected—especially in areas of the house where there were base stations near each other. I had to fine-tune each base station’s Maximum Distance to Report (in meters). It’s good to point out that this is an approximation based on the Bluetooth signals RSSI (Received Signal Strength Indicator). I ended up using Home Assistant’s developer tools to monitor the state and attributes of the sensor I created while I walked around each room.

When it was all said and done, I had ESPresense base stations in my office, the master bedroom, the living room, and our dining room.

What about that Ceiling Fan Automation?

Incorporating presence condition into the automation was a snap! I wound up adding a node to that flow to check which room ESPresense detected my watch was in. In order for the fan to get turned on in my office, two conditions would now need to be met: the temperature would need to be over 75 degrees and my watch would need to be nearest to the ESPresense base station in my office.

We had a rather warm day last week, and the automation worked great. I was working on writing this blog and noticed that the fan turned on. As the day progressed, I wandered in and out of my office to do other tasks. It was awesome to see that the ceiling fan was on when I was in the office—but off when I was somewhere else.

Final Thoughts

I enjoyed implementing ESPresense enough that I went ahead and ordered another 5-pack of the D1 Mini ESP32 boards. I don’t necessarily need them, but I like the idea that we could have ESPresense base stations in every room in our house. Adding presence detection in Home Assistant for about $12 per room is a tremendous value!

Reliable room-based presence detection is going to open the door for creating better automation that hasn’t been available to me before:

  1. Motion detection and room presence to turn the lights on in my office, keep them on, and turn them off shortly after I leave the office.
  2. Create new automation to automatically turn off the lights in my office when it’s empty
  3. Using my iPhone’s charging status and room presence in the bedroom to deduce whether I’m in bed.
  4. Personalize automations for other members of the household.

What other kinds of ideas am I overlooking? If you had presence detection enabled in your smart home, what kind of Bluetooth devices would you use for presence detection? What kind of tasks would you automate using presence detection? I’d love to hear what you think; share your ideas in the comments below!

A few weeks with TrueNAS SCALE

| Comments

About two months ago, I was putting the finishing touches on assembling and burning in the DIY NAS: 2022 Edition. Since then, I’ve been working on a couple tasks:

  1. Build some confidence in the hardware that I purchased.
  2. Establish some confidence in the two latest TrueNAS SCALE release candidates.

From the beginning, my plan was to create a fresh install of TrueNAS SCALE, export my 7-drive pool from my old NAS, import it into the DIY NAS: 2022 Edition, and rebuild everything from scratch. In addition to that, I also wanted to move my virtual machines from my Homelab server onto this new DIY NAS.


Since building it, the DIY NAS: 2022 Edition has been stable. However, I’ve had one hiccup. I experienced a single read error on two different hard drives one night (2 read errors total). One of those errors pushed the total error count of one of the two hard disk drives past a threshold, and ZFS kicked the drive out of the pool.


Out of an abundance of caution, I ran long SMART tests on both drives, examined the SMART data, and felt confident that I would be able to put the supposed degraded drive back into the array. After the resilvering was complete, I ran another set of long SMART tests on both drives, and haven’t had any issues since.

In the couple weeks since they occurred, I haven’t seen anything like it since, and I’ve been using my NAS pretty heavily since then. I’m not at all concerned about these errors. I often use my blog as a personal reference, so I’m noting them here for when—or if—they occur again.

TrueNAS SCALE does everything that I have been using TrueNAS CORE for

Above everything else, I needed TrueNAS SCALE to be able to replicate all of the features of TrueNAS CORE that I rely on. In the event that I couldn’t get SCALE to do this, I was not going to make any compromises and I planned to revert back to CORE immediately.

I’ve been using my DIY NAS as my primary storage for years now. I had a suite of snapshot tasks for the data stored on my NAS, SMB shares, an NFS share, cloud sync tasks to back up my NAS to Backblaze, a VM operating as a Tailscale Relay Node, and Nextcloud running in a VM and accessible via Tailscale.

It took me a handful of hours spread over a few days to recreate all of this from scratch on the DIY NAS: 2022 Edition. Setting everything from scratch was effortless, thanks to the SCALE UI. The folks at iXsystems have done a commendable job at making the management interface consistent between SCALE and CORE that rebuilding my NAS from scratch was a straightforward task.

TrueNAS SCALE quickly demonstrated to me that it was up to the task to meet all of the needs that FreeNAS and TrueNAS CORE have done so well since building my very first DIY NAS.

I was not able to consolidate all of my Virtual Machines onto TrueNAS SCALE

Between my Homelab and TrueNAS CORE machines, I had a few Virtual Machines: a Tailscale VM that I used as a Relay Node, Plex, and my Home Assistant virtual machine. I chose to deprecate the Tailscale Relay Node VM. I installed the “official” Plex app, I had some misadventures with the Nextcloud “apps” (more on that later), and ultimately wound up creating a new Nextcloud Virtual Machine to run on the DIY NAS: 2022 Edition.

After all of that, the only thing left running on my Homelab machine was my Home Assistant VM. I have a lot of Zwave home-automation devices which are controlled by a Zooz S2 USB stick (ZST10 700). On my Homelab machine, I’ve been passing through this single USB device to the Home Assistant virtual machine. But when it came time to do the same under SCALE, I learned it was going to be a bit trickier.

Currently, you cannot pass through an individual USB device using the SCALE interface. You can do it from the command-line, but anything you do at the command-line will get wiped out the next time the virtual machine restarts. The supported workaround is you need to pass through the entire USB controller to that virtual machine. Technically, I could potentially make this work but I’m currently unwilling to implement it. Redirecting the entire USB controller doesn’t seem like a very good solution. I’m fairly certain that other USB devices (keyboard, mouse, my UPS, etc.) that I wanted to plug into my NAS would get redirected to a virtual machine.

For the time being, I’m giving up my hope of decommissioning my Homelab server. I will continue to use it to host virtual machines that TrueNAS SCALE can’t support until SCALE resolves these requests. From the speculation I’ve read, it’ll be quite a few months before SCALE will support passthrough of an individual USB device to a virtual machine.

TrueNAS SCALE’s Apps Experience

I was pretty excited that TrueNAS SCALE was leveraging containerzation–even more so that they were going to try and make it simpler via its apps. I was pleased when Nextcloud and iXsystems announced that Nextcloud would be officially supported on TrueNAS. I think the concept of companies collaborating together like this will have a lot of benefit to their users.

I installed two official apps from the TrueNAS SCALE catalog; Plex and Nextcloud. As I mentioned previously, Plex was up and running without any issues in mere moments. The official Nextcloud app was a bit different.

Because my apps and VMs are being installed to an SSD mirror, I needed to use the Nextcloud option to use an external data source (a dataset on my 7x drive array). But anytime I picked the options to use an external storage location, the file-permissions inside the Nextcloud folder wound up being incorrect. It took a little bit of tinkering inside the container to change the file permissions for both the Nextcloud directories as well as my external data in order to get it working.

Because I worried that any future updates of the Nextcloud app would mean I’d need to be updating those permissions, out of curiosity, I thought that I’d try the Nextcloud app from a different catalog: TrueCharts.

TrueCharts

TrueCharts is a separate organization that has created an impressive catalog for use inside TrueNAS SCALE. Initially, I was excited because it appeared that there was a pretty substantial amount of documentation, including some guided walkthrough videos on YouTube. But ultimately, I found the documentation and videos to be pretty spartan, and I was frustrated by the number of bad links that all 404’ed on TrueCharts’s website.

Regardless of the state of the documentation, I was able to get their Nextcloud app functional on my NAS using my external storage. However, what I really wanted to do was share my Nextcloud app with friends and family using Tailscale, and that wasn’t working for me. I never thought this was TrueCharts’s fault, but I lacked the expertise to get to the bottom of it on my own, and I was going to need some help.

I’ve been a member of the TrueNAS Discord server for a while before installing SCALE. I was curious about listening in on what people had to share about SCALE. I was especially curious to hear about what people were doing with “apps” under SCALE. One of the recurring themes was users who were frustrated with how TrueCharts’s Discord server was being moderated. I mostly chalked it up to people being frustrated, but the repeating pattern was troubling.

When I ran into issues with TrueCharts’s Nextcloud app not listening on my NAS’s Tailscale IP address, I experienced this firsthand. Aware of the emphasis on following rules, I read through everything that I could, scoured each channel’s pinned posts, and searched all the past threads before creating my own support thread. Within a few moments, I was told what I was doing was hacky, unsupported, my support thread was archived, and I was told to go ask my question in a different “experts” channel. I quickly found that I lacked the ability to even post in the experts channel, so I messaged the moderator who gave me those directions. Their response was to scold me for using direct messages. Without any other options, I decided to delete the TrueCharts Nextcloud app and leave their Discord server.

A week or so later, I got a chance to privately discuss this experience with a different moderator of the TrueCharts Discord. It was a pleasant conversation, and I appreciate their admission that it was handled poorly and must have been either a bug or a mistake.

My experience with TrueCharts sapped any interest that I had in their app catalog. Even worse, it had a negative impact on my opinion of TrueNAS SCALE. I hope that the TrueCharts community matures, grows, and evolves to the point where experiences like mine are rare. However, that journey is just beginning for TrueCharts. Until they complete that journey, I’d recommend exploring all other possible options before relying on anything from the TrueCharts app catalog.

Things seem to break in TrueNAS SCALE when running Tailscale on the host

One of the biggest reasons that I’ve interested in TrueNAS SCALE is to install Tailscale on the host and use Tailscale to access my NAS from outside of my network and to share the VMs and apps that I’m running on my NAS. In migrating over to my new NAS, I learned that I wasn’t going to be able to share my Tailscale relay node with my friends and family. To accomplish what I wanted to do, I’d need to install Tailscale on the host itself.

It’s unsupported, but it is possible to install Tailscale on the TrueNAS SCALE host, which is an improvement over CORE. For a while, it even seemed to be working okay. I could access the official Plex app and all of SCALE’s hosted services (the web-management interface, SSH, SMB shares, etc.…) but in the days after installing Tailscale, I wound up running into several other problems:

  1. Tailscale got wiped out when I upgraded TrueNAS SCALE from 22.02-RC1 to 22.02-RC2
  2. The Nextcloud app from TrueCharts only listens on the IP address of the host itself; any traffic to the host’s Tailscale IP is ignored.
  3. I had one instance where my SMB shares stopped being accessible until I stopped Tailscale.
  4. Others reported issues with Tailscale preventing Kubernetes from starting up

This has been a tremendous disappointment to me. Tailscale has been so easy that I figured if I could get it installed on the host, everything on the host would be accessible on my Tailnet. But that didn’t turn out to be the case. I knew what I was trying wasn’t supported and I wouldn’t get much help for it, so I searched for a Tailscale feature request(s) on iXsystem’s JIRA page to upvote and found this:

If I want to host something on my NAS and share it via Tailscale, it will need to go in its very own virtual machine. This is how I was handling sharing things with TrueNAS CORE, and I’m content to keep doing it while using TrueNAS SCALE. I’m not thrilled with this, but I can accept it—for now.

Will Brian be using TrueNAS SCALE in 2023?

For as long as I have been using TrueNAS CORE (or FreeNAS), it has met all of my requirements. I’ve never even seriously considered looking at anything else. I like the ZFS file system, I like how easy TrueNAS CORE makes it to manage ZFS, and I remain excited about the possibility of completely consolidating my Homelab and NAS machines onto one platform.

But the minute I saw the “hyperconverged” buzzword in SCALE’s marketing material, my expectations for SCALE shifted a bit. After installing SCALE, I expected that I’d be able to retire my Homelab machine. I also expected that I’d be able to access and share all the applications hosted on my TrueNAS machine using Tailscale. Unfortunately for me, those expectations haven’t been met—at least, not yet. If the following happens this year, I’ll be really excited:

  • Incorporate support for Tailscale on the host for accessing/sharing the core SCALE services and any apps.
  • Enable features to leverage KVM’s pre-existing support for passthrough of unique USB devices to a Virtual Machine.
  • Expansion of the list of “official” apps in the TrueNAS SCALE catalog.
  • A maturation of the TrueCharts community to err on the side of inclusion, rather than exclusion.
  • Development of other catalogs of apps to compliment and compete with the current catalogs.

At the end of this year, I fully expect that I’ll still be using TrueNAS SCALE. By the time 2023 rolls around, I fully expect many of these items either have been resolved or a roadmap laid out for their resolution under SCALE.

Final Thoughts

Don’t let the words I’ve invested in the two areas that I’ve been disappointed with TrueNAS SCALE mislead you. Overall, I’m incredibly satisfied with TrueNAS SCALE. It fell short on a couple of my expectations, but I was also expecting that SCALE would far exceed what I have historically asked my own NAS to do. If I were asked to make a recommendation, I’d encourage others to take a look at TrueNAS SCALE first. I think its hardware support and its unrealized potential make it a great choice for DIY NAS builders today.

That being said, SCALE did not live up to my expectations, too. If my expectations aren’t being met by the end of this year, it will be time to seriously consider alternatives like UNRAID, Proxmox, or even building my own homebrew server from scratch.

After reading about my experience with TrueNAS SCALE, what do you all think? If you’re a TrueNAS CORE (or FreeNAS) user, are you excited about switching over to TrueNAS SCALE, or are you sticking with TrueNAS CORE? If you’re a prospective new builder, do you have a preference for SCALE or CORE? I’m interested to hear what you think down in the comments!

DIY NAS: 2022 Edition

| Comments

Early on in 2021, I started thinking about my own DIY NAS. I had 3D-printed a case, the MK735, a year or so earlier and I had been waiting for a good excuse to transplant my NAS into this awesome case.

As my DIY NAS kept reliably functioning, I decided that I would stop being patient and I would force the issue via upgrades. I calculated that the best upgrade for my NAS would be to max out the RAM. At the time, I was operating under the assumption that there would be all sorts of inexpensive secondhand DDR3 UDIMMs on eBay. But much to my chagrin, there was none to be found—especially inexpensively. Buying the four 16GB DDR3 UDIMMs to upgrade to 64GB of RAM (from 32GB) was going to cost me $800!

I wasn’t opposed to spending $800 (or more!) to upgrade my NAS, but I would need to get more value out of the upgrade then just doubling the amount of RAM, so I decided that the purpose of this DIY NAS build would be to replace and upgrade what has served me so well the past few years.

Update (01/13/2022): Pat and I devoted an episode to the DIY NAS: 2022 Edition on the Butter, What?! channel on Youtube. Check it out!

Motherboard and CPU

Every year, I spend a ton of time searching for the ideal motherboard for a DIY NAS build—and every year it is a challenge. Once I decided that I would be keeping DIY NAS for my personal use, it became almost impossible to find the perfect motherboard. I spent hours trawling through manufacturers’ websites, online vendors’ advanced search functions, and staring at my bank balance, looking for a motherboard that met this criteria:

  • Mini-ITX form factor
  • Integrated CPU, preferably passively cooled
  • Onboard support for at least 9 SATA drives
  • A substantial processor upgrade over my old Avoton C2550
  • No felonious assault of my bank account

I was searching for a unicorn. After trawling through every single MiniITX motherboard from 5—6 manufacturers, I concluded that my ideal motherboard doesn’t exist. My criteria narrowed it very quickly down to motherboards that were prohibitively expensive or would’ve required considerable concessions to my criteria. To make matters worse, the motherboards that were closest to meeting my criteria simply could not be found in stock anywhere.

Ultimately, I wound up choosing the Supermicro X11SDV-4C-TLN2F (specs), as it checked off most of the boxes for my criteria:

  • Mini-ITX form factor
  • Intel Xeon D-2123IT (4-cores, 8 threads, 2.2GHz, 8MB cache, and 60W TDP)
  • Up to 512GB ECC LRDIMM
  • Up to 8 SATA devices (4 onboard, 4 with OCuLink)
  • 1x PCI-E 3.0 X8
  • Vendors had listed it as low as $599

Unfortunately, the Supermicro X11SDV-4C-TLN2F would not support all 9 of my HDDs/SSDs. However it did carry a feature my current motherboard doesn’t have—two onboard 10Gb network interfaces—which freed up the PCI-e slot for an HBA, which kept a viable candidate. The integrated Xeon D-2123IT was quite a bit more powerful than my current Atom C2550, and the CPU is passively cooled. The fact that a few vendors had it listed at a somewhat reasonable price ($550—$600) encouraged me to place an order.

If finding the motherboard was difficult, actually buying it at a reasonable price was damn near impossible. Everywhere I found it listed either had an exorbitant price tag or the vendors with a reasonable price tag wanted you to wait for them to custom order it from Supermicro.

At first, I went the special order route—but chose to cancel my order after weeks went by with no updates and the supposed estimate kept incrementally going up the more time went by. I wound up finding the Supermicro X11SDV-4C-TLN2F listed on eBay and decided to buy it from there for about $650, but that listing has since climbed up up to nearly $800!

Case

Most years, shopping for the case for the year’s DIY NAS build blog is as much of a challenge that the motherboard is. Not because good options don’t exist, but because I seem to have used most common NAS cases already. When a manufacturer makes a good design, it will continue to be popular for years to come without much (or any) refinement.

The minute that I decided that I’d be keep this year’s DIY NAS to replace the DIY NAS I built back in 2016, I knew right away what case I’d be using: the 3DWebe.com MK735. The MK735 is so perfectly suited for my personal NAS that I printed my own MK735 and have been waiting for a chance to use it. Among the MK735’s many features, here are my favorite:

  • It is 3D-printable!
  • 7x 3.5” drive bays
  • 2x 2.5” drive trays
  • Drives under motherboard design
  • 3 independently cooled chambers, motherboard, hard drives, and power supply.
  • Case’s grill-laden design allows for excellent airflow.

Don’t have a 3D printer? Or don’t want to spend the time (and money) to manufacture and build your own DIY NAS case? I don’t blame you, 3D-printing a DIY NAS case is no small feat! If it is helpful, I wrote a blog about my favorite DIY NAS cases that contains a few ideas of other cases you might want to try.

Power Supply

Picking out a power supply for this year’s DIY NAS build was easy, too! Thanks to all of my DIY NAS building over the years, I’ve accidentally bought one or two ATX power supplies when I actually needed an SFX power supply instead. I have been saving those extra power supplies, assuming I’d eventually be able to use one of them.

In this case, I had purchased a SilverStone ET550-HG (specs) for a prior DIY NAS build. The 550 watt power supply should be more than ample to power my current DIY NAS which, is currently being powered by a 300 watt power supply.

I’m hopeful that the SilverStone ET550-HG’s 80 Plus Gold certification will be an upgrade in terms of power efficiency and that its fans run quieter than the 1U Power supply that I’m currently using today.

RAM

The whole reason I decided to keep the DIY NAS: 2022 Edition for myself can be found in RAM. Originally, I wanted to upgrade my DIY NAS from 32GB of RAM up to 64GB. As part of doing that upgrade, I decided I’d swap my NAS into the MK735.

As part of the new NAS build, I still wanted to reach 64GB of RAM. I achieved this by using 4 of the Micron 16GB DDR4-2666 ECC RDIMM (specs). While 2666MHz RDIMMs are supported by the Supermicro X11SDV-4C-TLN2F, they will only operate at the fastest speed supported by the motherboard, 2400MHz. It just turned out that when I was shopping, 2666MHz RAM was priced more competitively than its 2400MHz counterparts.

Host Bus Adapter and Cables

Because I had nine SATA drives that I wanted to bring over into the DIY NAS: 2022 Edition, a Host Bus Adapter (HBA) card was going to be required. If I were building my own NAS from scratch today, I probably would not have gone this route. Instead, I probably matched up a pair of SSDs with 6x 12TB (or larger) HDDs to achieve the same amount of storage that I currently have.

To achieve what I wanted, I was going to need to buy a HBA. In the DIY NAS: 2020 Edition, I purchased an IBM M1015 HBA and used it to make sure that every possible drive bay of its case could be filled. In order to use the IBM M1015 with TrueNAS, it is suggested that you reflash it in IT Mode to give the filesystem unfettered access to the drives themselves. Flashing that firmware wasn’t a tremendously difficult task—but it was challenging enough that I searched for something that had already been flashed for use with ZFS and I found this on Amazon: LSI 9211-8i P20 IT Mode for ZFS FreeNAS unRAID 6Gbps SAS HBA (specs). Spending $10—20 more than I spent on the IBM M1015 was an easy decision when it took me two to three hours just to flash the IBM M1015.

To go along with the LSI 9211-8i, I picked up a pair of Mini SAS to SATA cables (SFF-8087 to SATA Forward Breakout). All seven of my data drives would be plugged into the HBA, the motherboard’s SATA ports would be used for the OS drives, and between the HBA and motherboard there’d be 7 empty SATA ports for potential future use.

Storage

TrueNAS Scale Drives

Up until last year, I’ve been running the OS on USB flash drives (often mirrored across two thumb drives) on all of my DIY NAS builds. For a while now, this has been in contradiction to the suggested hardware recommendations. There’s apparently enough being written to the OS drive now that USB drives are likely to wear out, especially inexpensive flash drives.

Inexpensive SSDs are not difficult to find—but the really inexpensive ones are selling out and never being sold again, which makes it a challenge for me to recommend a particular model. Equally challenging is that SSDs are increasing their capacity and value. Assuming you can find an appropriately sized SSD (32-64GB) or SATA DOM, you’re not really getting the best value when buying it.

For me, that was just too much potential value in buying a pair of Crucial MX500 1TB to store the operating system on. At the time I bought the SSDs, I figured that I’d try and maximize their value by pre-partitioning the SSDs to use for the OS, ZFS cache (L2ARC or SLOG), and/or some fast storage on the NAS.

Do think this will work out? Or do you think these drives will wind up being 99.97% wasted? Keep on reading to find out!

NAS Hard Disk Drives

The most important part of your NAS build are the hard drives. Regardless of your budget, your storage will likely wind up being the most expensive component of your DIY NAS. It’s also impossible to make a one-size-fits-all recommendation to every prospective DIY NAS builder. Rather than make specific recommendations of what hard drives to purchase, I like to make suggestions:

  1. Measure and project your storage needs: How many hard drives you need for your DIY NAS ultimately depends on how much storage you currently need and how much you project that will increase over time. Oftentimes, spending more money now will be cheaper in the long run.
  2. Understand your data’s importance: Knowing the importance of your data is critical to choosing how many HDDs. Ask yourself, “What happens if someone steals my NAS?” Your answer to this question should help you understand what your hardware redundancy and backup plan should be. If your data is critical to you, your budget should include both hardware redundancy and some sort of off-site backup.
  3. Buy CMR Hard Drives for use with ZFS: Many Western Digital customers were shocked when their so-called “NAS-grade” drives were unreliable in their NAS systems as a result of Western Digital surreptitiously sneaking SMR technology into their Red products. Typically, the easy way to avoid this is by purchasing NAS-grade hardware, but the better advice is to buy drives using Conventional Magnetic Recording (CMR). Personally, I don’t really care if my HDDs are NAS-grade or not; if I find consumer-grade CMR HDDs, I’d happily store my data on them.
  4. Consider Shucking External HDDs: Usually the most cost-efficient method of buying hard drives is by buying external USB hard drives and removing the hard disk drive from inside the enclosure.

Here are a few recent deals that I’ve seen which have been compelling. I’ll be maintaining and updating this throughout as I learn about good deals on hard drives:

I purchased some hardware you probably won’t need, too!

Once I decided to keep this DIY NAS for my own use, I knew I would be buying a few components that I wouldn’t necessarily recommend that others buy.

RJ45 to SFP+ Transceivers

Five years ago, I built out an inexpensive 10Gb network using some secondhand 10Gb SFP+ network cards and setting up a point-to-point network between my desktop, my Homelab machine, and my NAS. About a year ago, I added a 10Gb SFP+ to my network, which simplified things quite a bit.

Unfortunately (for me) the Supermicro X11SDV-4C-TLN2F’s 10Gb interfaces are both RJ-45. As a result, I bought a 2-pack of iPolex 10G SFP+ RJ45 Copper Transceivers to allow the DIY NAS: 2022 Edition to work with my network.

LEDs

I’ve been impressed with putting LEDs inside computers. I think, when done well, they look pretty awesome. But I have never been tempted to do it myself. I don’t think a computer should be a focal part of your office’s decor. My preference has always been that computers are black or beige boxes, preferably out of view. But ever since starting The Butter, What?! Show with Pat—I’ve been looking for ways to make the background of my office a bit more interesting to look at and to try and feature my enthusiasm for DIY NAS topics. As a result, I had a brainstorm and asked myself “What if I put LEDs into my DIY NAS?”

We livestream the recording of The Butter, What?! Show on the first Tuesday of every month at 9 p.m. Central Time. What we wind up recording gets broken up into weekly episodes that are published to YouTube on Mondays. Come join us!

Naturally, there’s no hardware on Supermicro X11SDV-4C-TLN2F for controlling the LEDs, so I picked out the Nexlux WiFi Wireless LED Smart Controller to drive the LEDs. I purposefully picked something that I thought would work well with my Home Assistant server and began thinking of ways that I could incorporate my NAS into my Home Automation.

I bought the same LEDs that I’m currently using on the back of the sound-absorbing panels in my “recording studio.” I’ve been happy with how the SUPERNIGHT LED Strip Lights have worked out so far. I’m pretty certain that I’ll have quite a few feet of LEDs left over. What sort of LED lighting projects should I incorporate into my Home Automation? Let me know in the comments below! To round out my LED product purchases, I picked up a 4-pin molex to 2.1mm barrel jack to power the LED controller and LED strip from the computer’s power supply.

Final Parts List

Component Part Name Qty Cost
Motherboard Supermicro X11SDV-4C-TLN2F specs 1 $775.00
CPU Intel Xeon Processor D-2123IT specs N/A N/A
Memory Micron 16GB DDR4-2666 ECC RDIMM (MTA18ASF2G72PDZ-2G6E1) specs 4 $67.98
Case 3Dwebe.com MK735 specs 1 $19.99*
Host Bus Adapter LSI 9211-8i P20 specs 1 $89.00
Power Supply SilverStone SST-ET550-HG specs 1 $64.63
OS Drive Crucial MX500 1TB 2.5” SSD specs 2 $99.99
SAS Cable Internal Mini SAS SFF-8087 to Mini SAS High Density HD SFF-8643 N/A 2 $11.99
Misc. Network 10G SFP+ RJ45 Copper Transceiver 2 $50.98
LED Controller Nexlux WiFi Wireless LED Smart Controller 1 $10.99
LED Lights SUPERNIGHT LED Strip Lights, 5M SMD 5050 1 $14.99
Molex to 12V Power Cable CRJ 4-Pin Male Molex to 12V DC 5.5mm x 2.1mm 1 $5.99
TOTAL: $1,565.41


All the Parts MK735 MK735 with Door Open Supermicro X11SDV-4C-TLN2F Micron 4x16GB DDR4-2666 ECC RDIMM LSI 9211-8i HBA and Cables 2x Crucial MX500 1TB 2.5 LED Lights an WiFi LED Controller SilverStone ET550-HG Power Supply


Hardware Assembly, BIOS Configuration, and Burn-In

Assembly

Normally, when describing the assembly of a DIY NAS build, I strive to be really detailed for people who might choose to build their own DIY NAS using some of the parts from my blueprint, especially the case. 3D-printing and building the MK735 has been one of my favorite projects, but it took a lot of time, effort, and materials. I don’t expect that many people reading this blog will be using the MK735, so I am keeping the assembly notes a bit brief.

The two most difficult parts of assembling my DIY NAS in the MK735 was mounting the motherboard itself and the HBA. The trouble that I ran into in mounting the motherboard was twofold. First, it fits like a glove—there’s very little play for the motherboard once it is down in its tray. Second, you wind up threading the holes as you drive the screws into them for the first time. That makes putting the motherboard’s screws in pretty challenging—if I had to do it all over again, I’d remember to pre-thread each of these holes. However, it’s important to not over-tighten the screws too. Otherwise, you might strip the hole itself.

There’s enough room for the HBA, but I found that I couldn’t connect the cables when the HBA was installed. This is due to the MK735’s very compact nature—and I always expect these kinds of challenges when working with a small form factor case. Removing the HBA, installing the cables, and then bending the cables 180 degrees around the edge of the HBA allowed me to install the HBA.

I wound up deciding that I would save the installation of the LED controller and LED lights for a future blog. I think there’s an entire blog’s worth of content between installing the LEDs, incorporating them into my home automation via HomeAssistant, and automating some NAS-specific tasks. Plus there’s some exciting overlap with our Ooberlights project too!

Here is a time-lapse video of the hardware assembly, TrueNAS installation, and the configuration I did to set up a simple share on the DIY NAS: 2022 Edition. And if you’re really interested, I also made a nearly real-time version of the assembly video too after a few people had requested it previously.

BIOS Configuration

In the DIY NAS builds of the past, I seem to recall a sneaky BIOS configuration change that needed to be made in order to get the DIY NAS to function. I noted that and said to myself, I need to remember this when I write the blog! Every year since, I’ve kept this heading in the blog for similar sneaky configurations, but I’m still relieved that pretty much the only change I ever make in the BIOS is to change the order of which devices it’ll boot from—and this year’s NAS build was no different.

Burn-In

Typically, my DIY NAS builds aren’t in my possession for very long, so I like to torment the hardware a little bit to make sure there’s nothing wrong with it. However, since I’m planning to use the DIY NAS: 2022 Edition as a replacement for my current NAS, I can be a little more relaxed with my burn-in. I plan to run the DIY NAS: 2022 Edition (using some old drives) next to my current NAS for an extended period and evaluate its performance. Once I’m fully confident in the new machine, I will migrate my drives and settings over to the new NAS.

Regardless, I always burn-in any computer I build by running Memtest86+ for three single-threaded passes, and this year’s NAS build was no exception.

TrueNAS SCALE

I have been excited about TrueNAS SCALE since its announcement. What is TrueNAS SCALE and how does it differ from TrueNAS CORE (formerly known as FreeNAS)? I haven’t really used TrueNAS SCALE enough to pretend to have any expertise, but here’s a really high-level at stab some important features from my point of view (a DIY enthusiast):

  • Maturity: TrueNAS CORE (FreeNAS) has been around for a long time and TrueNAS SCALE just recently produced its first release candidate.
  • Operating System: TrueNAS CORE is built atop FreeBSD and TrueNAS SCALE is built atop Debian
  • Both use ZFS: OpenZFS is at the root of both CORE and SCALE. In my opinion, that makes them both pretty interchangeable for my usage.

Personally, I’m excited about TrueNAS SCALE for a few reasons:

  1. SCALE’s “hyperconverged” approach seems ideal for what many DIYers—myself included—are hoping to do with their NAS and/or Homelab builds.
  2. I’m less incompetent using Linux than I am using FreeBSD.
  3. SCALE should allow me to retire my separate Homelab server
    1. Migrating KVM VMs over to my TrueNAS SCALE box
    2. Replacing existing VMs (Plex, HomeAssistant, Nextcloud) with Linux Containers

Installing TrueNAS SCALE

In picking out components for the DIY NAS: 2022 Edition, I chose to buy a pair of Crucial MX500 1TB SSDs because the value of larger SSDs couldn’t be passed up. But in its current form, TrueNAS SCALE partitions and uses 100% of the OS drive’s space—which negates that potential value.

When I decided to use a pair of 1TB SSDs to act as the OS drive, I was determined to try and find a method that would allow me to create an appropriate-sized partition the SSDs for TrueNAS SCALE and then partition the remaining space to use for fast storage and/or ZFS caching (L2ARC and SLOG).

At any rate, I found this guide to partitioning a SSD as part of the TrueNAS Scale install in the /r/truenas sub-Reddit. Effectively, you modify the TrueNAS installation script to constrain the size of the boot partition created during the installation. After installing TrueNAS, you use the command-line to partition the remaining space, create a ZFS pool from those new partitions, export the pool, and finally import the pool using the TrueNAS SCALE web-interface. Following this guide worked perfectly for me, with one wrinkle: the specific line which needed editing. In the 9 months since this post was shared on reddit, the contents of /usr/sbin/truenas-install have changed. The author probably predicted this would change and shared the script’s location in the source code repository—so it was easy to go figure out where that line could be found in the current version of the script.

I named the pool made out of partitions on the SSDs “fast” and then created a new pool out of 7 high-mileage spare HDDs left over from my upgrades to larger hard drives. I created a RAID-Z2 pool named “slow” out of these old drives. After that, I created a new Local User for myself, created a Local Group called shareusers, and added my account to that new group. On each pool, I created datasets (fast-share and slow-share) and set the datasets’ permissions so that the shareusers group could read, write, and delete files contained in that dataset.

Finally, I opened up each share in Windows explorer, created a file, renamed that file, and then deleted that file to confirm that the share and file permissions had been set up correctly.

Just like that, my very first TrueNAS SCALE machine was up and running and ready to meet some network-attached-storage needs!

What’s Next, Brian?

In most years, I’d spend the end of the blog sharing some of the conclusions that I’ve reached based on the performance of the DIY NAS build, its cost, and its comparison to other products like it. I’m every bit as confident in making these declarative statements with the DIY NAS: 2022 Edition, but I’d rather take the opportunity to show it in 2022. So what’s next? More DIY NAS blogs, that’s for sure! I’m going to be running the DIY NAS: 2022 Edition in parallel alongside my current NAS for a while. Typically, I rush to do some throughput testing in an effort to demonstrate the performance of the new machine for prospective buyers. But because I’m keeping the DIY NAS: 2022 Edition for my personal use, I’ve got a few ideas for new blog topics:

  1. Comparing/contrasting the performance of my old NAS and the new one.
  2. My migration from TrueNAS CORE to TrueNAS SCALE
  3. Using TrueNAS SCALE’s “apps” to retire Virtual Machines
  4. Consolidating my NAS and Homelab onto the same hardware
  5. Incorporating my NAS into my Home Automation

What other kinds of DIY NAS/Homelab content would you be interested in? If you’ve been interested in building your home server, what’s been holding you back? I would love to hear about your own DIY NAS and Homelab journeys in the comments below. Even better, come join the Butter, What?! Discord server and share your thoughts in the #diynas-and-homelab channel!

Building my own Drone Race Timer: RotorHazard on Delta 5 Hardware

| Comments

For most of the past five years, one of my favorite hobbies has been flying first-person view (FPV) quadcopters. It’s been a fun little obsession which ticks off a lot of my favorite things: going fast, doing tricks, building things, and perpetually tweaking them.

Back in 2017, Pat and I taught a class at a local makerspace for building quadcopters which caught the eye of the local drone-racing group, Dallas Drone Racing, and we got a chance to chat with the group’s leadership. Since then, I’ve spectated several races, including watching Pat race his own quadcopter.

I’ve also had the chance to hang out a few times with Alex Vanover, of Drone Racing League fame, and each time he’s encouraged me (and everyone else) to get out to Dallas Drone Racing’s, field and start racing. I give Alex credit for convincing Pat to try racing a couple years ago—and one of these days, I’ll join them in racing. I’ve got parts for two racing quadcopters that have been waiting to get assembled for a couple months now.

However, I am NOT a Racer

Let’s get this admission out of the way early: I am slow—very slow. Real quadcopter racing looks like this DVR recording of Alex Vanover’s FPV feed. This is what Alex saw in his goggles while running some laps on a track set up for a recent MultipGP race:

I won’t even burden you with a comparison video of my own DVR footage what I call “racing.” Among the people that I fly with regularly, just about everybody is better and faster than I am. Ultimately, the person that I’m competing against is myself. As long as I’m consistently improving, I’m perfectly happy with how I’m performing. If I just so happen to beat Pat along the way, well, that’s just icing on the cake!

I feel the need, the need to understand my speed!

If I’m to improve, there’s no shortcut. I need to spend more time flying fast, predetermined courses. Being able to invest the time it takes to improve will always be my biggest challenge. I have far too many responsibilities, hobbies, and other distractions working against me. However, there are some efficiencies to be found.

A few of us race in Velocidrone every Thursday night for fun. I’m convinced that this extra “stick time” has been a tremendous help improving my flying both virtually in the simulator and also in real life. If you’re interested in joining us on simulator night, come join our Discord server’s #drone channel!

But what’s also been helpful is the timing built in to the simulator. Being able to discern between what feels faster and what’s actually faster is huge. Recently, Joshua Bardwell reviewed the ImmersionRC LapRF Puck on his YouTube channel. I was nearly ready to buy the LapRF Puck based on Joshua’s review, but then he mentioned that a do-it-yourself open-source lap timer existed and provided links to it. Knowing that DIY solutions existed piqued my curiosity, so I checked it out!

Enter the Delta 5 Race Timer

Disclaimer: While I ultimately had a great experience with the hardware for the Delta 5 Race Timer, the software wound up being a bit of a different story. It seems as if the Delta 5 Race Timer is a bit of a digital ghost town. There haven’t been any recent changes in Github, no responses to issues submitted, the Facebook group seems to be either gone or closed, and most of the tutorials are a bit dated. If you’re interested in building your own DIY race timer—please make sure to read all of my blog to avoid what I ran into!

The Delta 5 Race Timer seemed like it was right up my alley! It’s an open-source project to build your own DIY race timer using a custom PCB and some fairly common electronics components. Here’s a parts list and prices of the components (or equivalent components) that I wound up ordering. Thanks to my stash of bits and pieces from prior Raspberry Pi, Arduino, quadcopter, and custom computer projects, I was able to avoid having to buy several of the components.

Product Qty Brian’s Vendors Alternative Vendors
Delta 5 Race Timer PCB (set of 5-10) 1 $4.90 N/A
Raspberry Pi 4 (2GB version) 1 $56.99 $35.00
Geekstory Mini Nano V3.0 ATmega328P (set of 4) 1 to 2 $14.99 ($3.75/ea) $3.96/ea
3D-printed D5RT Housing – 4 and 8 Node by FreeFormFPV 1 N/A N/A
Boscam RX5808 Video Receivers 4 to 8 $5.64 $5.99
Pololu 5V, 2.5A Step-Down Voltage Regulator 1 $11.95 N/A
Pololu 3.3V, 2.5A Step-Down Voltage Regulator 1 $11.95 N/A
Elegoo Resistor Kit Assortment, 0 Ohm-1M 1 $11.99 $15.99
M3x30mm Hex Socket Flat Head Countersunk Bolts Screw (set of 25) 1 $9.49 $6.83
Dupont Connectors, Male-to-Male and Male-to-Female Kit 1 $4.49 $6.94
Jumper Wires: Female to Female 1 $6.49 $3.49
XT-60 Pigtail (Male) 1 $4.22 $3.22
2-pin Standard Jumpers (set of 120) 1 $5.99 $1.25

Just by looking at the Delta 5 Race Timer’s parts list, it was fairly obvious to me that I could potentially get some great value out of investing a few extra dollars and a little bit of work at my soldering station! Especially when I realized that I already owned most of the wires and connectors, a Raspberry Pi, and a spare Arduino Nano or two. Regardless of how many components I already owned, I still think the DIY race timer is a great value:

  • ImmersionRC LapRF Personal Race Timing System: $99.99 (price per node: $99.99)
  • 4-Node DIY Race Timer (Delta 5 hardware): $166.01 (price per node: $41.50)
  • 8-Node DIY Race Timer (Delta 5 hardware): $203.56 (price per node: $25.45)

Delta 5 Hardware Assembly

All things considered, the assembly of the hardware went smoothly, despite my clumsy solder-work. Part of what compelled me to take on this project was realizing that it would also provide some great practice to increase my soldering skill by a little bit.

TweetFPV on YouTube has a great three-part tutorial on assembling the Delta 5 Race Timer’s hardware and software.

It took me an entire Saturday evening after my son went to bed to assemble the Delta 5 Race Timer hardware using 4 nodes. But I’m reasonably confident that if I were to do it again, I would be considerably faster. Nothing about the build was complicated, but my soldering is pretty slow, inefficient, and required a lot of double-checking. While I may have been slow, the work paid off! After soldering everything together, but before putting in the video receivers and Arduino Nanos, I powered it up and measured the voltages with my multimeter. I was pleasantly surprised to see that the appropriate voltages were being read on the different points in the PCB. Later on that week, I put it all together and powered it on for the first time:


Delta 5 Software Installation and Configuration

My biggest disappointment in this project was that the Delta 5 Race Timer appears to be abandoned. The project itself has many open issues, hasn’t had any new changes in the Github project in over 2 years, and there are a number of issues that have been opened without really receiving much attention. At first I was a little concerned, but I figured that I had enough knowledge that I could follow the old directions and get it working.

I wound up running into one of the open issues on Github, and while I managed to apply some Google-fu and seemingly moved past a problem, I’m not really certain if I actually solved that problem or caused a different problem. I never could get the Delta 5 Race Timer web interface functioning. The logs were full of error messages that made me think that my Raspberry Pi’s operating system had packages that were much newert than what would work with the Delta 5 Race Timer project. After tinkering with it for a few hours across a couple days, I was beginning to worry that I’d need to seek out someone in the FPV community to let me copy the SDCard image from their working Delta 5 Race Timer.

Thankfully for me, one of my Google searches lead me down a rabbit’s hole and I stumbled across someone asking questions very similar to my own. Somebody had a very simple answer to that question; they suggested that the person go take a look at a project called RotorHazard instead of trying to get the Delta 5 Race Timer functioning.

RotorHazard Race Timer Saves the Day

The minute that I saw the RotorHazard project page, I breathed a sigh of relief knowing that I was not going to have to fumble through resuscitating a project that had gone stale and that I wasn’t well-equipped to accomplish. You might be asking, “What’s RotorHazard?” and I read this answer from someone on their Facebook page: “RotorHazard is basically Delta 5 Race Timer 2.0”. Given what I’ve seen of the RotorHazard so far, I think that answer might be a bit basic and too modest!

After skimming through RotorHazard’s project page, I began to wish that I had come across it much sooner. Thankfully, there wasn’t any harm done, as the Delta 5 Race Timer hardware setup is 100% compatible with RotorHazard. If I had the chance to do it all over again, I probably would be inclined to build my own using the RotorHazard S32_BPill PCB instead. The hardware design seems more sophisticated, and I especially liked the fact that the RotorHazard PCB can support up to 8 video receivers as opposed to daisy-chaining two Delta 5 Race Timer PCBs together to get 8 nodes. The RotorHazard S32_BPill PCB’s features include replacing the 4 to 8 Arduino Nanos with a single STM32 processor.

Beyond these key attributes, there’s an assortment of other features that I would have also been interested in:

  • Better 3D-printed case design with more cooling
  • Support for power/battery monitoring
  • Compatibility with LED strips and other LED options

All that being said, I don’t plan to make any changes to my hardware in the near term!

Let’s Race!

The weekend prior to publishing this blog, Pat came over and we set up an indoor track inside my house. We set up my RotorHazard race timer, and within a few minutes, we were racing laps around the track that we laid out in my house. We had to tweak the calibration of the video receivers and adjust the placement of the RotorHazard timer once or twice. But once we had everything dialed in, we were having a riot racing around the house.


It was awesome listening our times get read off by my laptop as we zipped around the somewhat-complicated course that we set up. Moreover, I can totally see bringing this out to an open field and setting up a crude course outside and doing the same exact thing.

What’s Next?

Thanks to having quite a few of the parts laying around the house already, I was able to buy enough parts to build an 8-port race time for about $125-150. Because of the size of our group and the extra effort it takes to get eight quadcopters in the air at the same time, I opted to only include 4 ports in mine. But if I ever wanted to grow mine to eight ports, I have enough spare components to make it happen.

I plan on doing a bit more research. My next goal is to use the Raspberry Pi 4’s wireless interface to act as a WiFi access point. That way when we’re in the field, I can fire up the race timer and access it from my laptop, without any Internet access.

I really enjoyed our racing session. We didn’t really do any “real” racing, we wound up using the “Open Practice” mode. Nevertheless, it was a riot racing Pat (and myself) around the house. As more of my drone-flying buddies are fully vaccinated, I’m looking forward to inviting a couple more people over and see what it’s like with four pilots racing at the same time!

What do you all think? If you’re interested in racing, what do you do to try and measure your own speed when casually practicing? Is this a project that you’d be interested in building on your own? Is a product like the ImmersionRC LapRF Personal Race Timing System more suited for what you’re wanting to do? Or maybe you’re timing yourself a different way? Let us all know in the comments below!

Self-Hosting my own Cloud Storage: FreeNAS, Nextcloud, and Tailscale

| Comments

Until recently, I’ve never really felt the urge to access the contents of my DIY NAS from outside of my own network. The way that I’ve used my NAS, it has been simpler to use services like Google Drive to have access to my most critical data on my various computers, tablets, and phones.

Between COVID-19 and my new job being 100% remote, I have spent nearly all of my time on my own network. If anything, I have less of a need to access my NAS’s contents remotely. However, I’m hopeful about our vaccination efforts and I’m cautiously optimistic there might be light at the end of that tunnel. My growing catalog of content (primarily video footage) has long since eclipsed what can be stored on Google Drive or Dropbox—and even if I could buy the space, I’d much rather invest the cash into improving my NAS!

In my blog about implementing Tailscale at home, I installed Tailscale on my OpenWRT router and used Tailscale’s relay node feature to allow my other Tailscale nodes to use all of the resources on my network. But SMB’s performance over the Internet isn’t that great, and I wanted to make sure that important content—like my blog—continued to be synchronized across all of the machines that I’d want to be able to access it from.

I wound up deciding that I’d try and host my own cloud storage using Nextcloud and access it via Tailscale. This decision was instigated by the shift in what I was asking of my own DIY NAS and encouraged by the numerous questions I’ve been asked about self-hosting cloud storage over the years.

My DIY NAS is running FreeNAS-11.2-U8, has an Avoton C2550 CPU, 32GB of RAM, and a 10Gb NIC. Everything in this article was written in the context of using my NAS. I would expect that the same—or similar—steps would work with different versions of FreeNAS (now known as TrueNAS Core), but your mileage may vary.

Please share your experiences with different versions in the blog’s comments!

Plugin vs. Jail vs. Virtual Machine

The most difficult decision I made was whether to use the Nextcloud plug-in for FreeNAS/TrueNAS, to create a FreeBSD Jail, or to host it within a Bhyve virtual machine. Each option had its own benefits and drawbacks, and I as I explored this topic, I experimented with each of the three options.

  • Nextcloud Plug-in: Setting up the plug-in for FreeNAS/TrueNAS was incredibly easy; I had Nextcloud up and running in mere moments. I even briefly exposed my Nextcloud VM to the Internet through my router. I had hoped it would be a simple task (for me) to add the Tailscale client to the jail created by the Nextcloud plug-in, but quickly learned that wouldn’t be the case.
  • FreeBSD Jail: Having run into challenges tinkering with the Nextcloud plug-in, I figured I could just install and host my own Nextcloud alongside Tailscale from inside a jail on my NAS. Setting up Nextcloud in this jail was easy—but the Tailscale client wound up being difficult. Tailscale would crash any time I launched it, and I wasn’t having much success debugging it on my own—or finding helpful information to help me stumble through resolving it. Unfortunately for me, I’m simply not a savvy enough of a FreeBSD user to get Tailscale functioning.
  • Bhyve Virtual Machine: I would have preferred running Nextcloud and Tailscale in a FreeBSD Jail, mostly because it’s less resource-intensive. But after not having much luck before, I ultimately decided that I would use the Bhyve hypervisor to host a virtual machine. For that virtual machine, I’d choose Ubuntu 20.04 for its operating system. I am a bit much more familiar with Linux vs. FreeBSD and infinitely more capable of finding answers to questions that I run into since Ubuntu is fully supported by both Tailscale and Nextcloud.

Why is Brian hosting the Nextcloud Virtual Machine on his DIY NAS instead of his Homelab server?

This is a good question! All things being equal, I would choose to host Nextcloud on my homelab server. But I’ve made exceptions to this rule before! For example, there are things (Home Assistant, Octoprint, etc.) in my house running on Raspberry Pis that probably should be running in a virtual machine on my Homelab server instead.

Ultimately, the reason my Nextcloud VM is running on my DIY NAS is: all of you! I’m quite grateful that so many people find their way to my blog when researching various DIY NAS topics. I know that hosting your own cloud storage is a huge point of interest in the NAS community, so it made sense to do it in a way that a fellow NAS enthusiast might want to follow!

Besides, I routinely advocate building over-the-top DIY NAS machines with more processing power than a NAS might need. Using that extra CPU power to host a cloud storage VM is a great way to leverage that extra capability.

Creating a Virtual Machine to run Nextcloud and Tailscale

On my DIY NAS, I logged into the FreeNAS interface and created a Virtual machine. I allocated the VM with a single Virtual CPU, 512MB of RAM, created a new 64GB disk image, pointed it to the Ubuntu 20.04 ISO I picked and, started the virtual machine up.

Using the FreeNAS UI’s built-in VNC viewer, I attempted to begin the installation and immediately ran into this error:

1
2
3
Initramfs unpacking failed: write error
Failed to execute /init (error -2)
Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.

The culprit to this error was the fact that I only allocated 512MB of RAM (the minimum suggested for Nextcloud) and Ubuntu Server’s minimum hardware requirements list a gigabyte of RAM. Increasing the Virtual Machine’s RAM to a gigabyte moved me past this error.

For the most part, I simply paged my way through the Ubuntu 20.04 Server setup picking primarily default options, naming my server nextcloud and setting up my root username and password. On the Featured Server Snaps screen, I was delighted to see that there was an option for Nextcloud, but as a result of my experimentation I decided to recommend skip over install the Nextcloud Snap at this point.

The installation rebooted the virtual machine, and after the completion of the reboot, the VM was a little annoyed at me and said it wasn’t able to unmount the CDROM and then told me to eject the installation media. My solution for this was probably a bit brutish: I shut down the virtual machine and deleted the CDROM device entirely. A bit crude, perhaps, but still very effective.

I started the Virtual Machine up one more time, confirmed that I was able to log in via FreeNAS’s built-in VNC client, and then switched over to using SSH for the remainder of my work.


Choosing VM Wizard Type Guest OS, Servername, Boot Method, etc... Allocation of Virtual CPUs and RAM Creating a Virtual Disk Image Virtual NIC Setup Mounting the Boot Media First Boot of the VM in VNC Viewer


Installing and Configuring Tailscale on the Nextcloud VM

On my freshly installed Ubuntu 20.04 virtual machine, the first thing that I set out to do was to get Tailscale up and working. As I had learned from my prior experience with setting up Tailscale on other devices, it was incredibly easy.

I simply followed the few steps from the Setting up Tailscale on Ubuntu 20.04 LTS (focal) documentation and my new virtual machine was up and running without any fuss.

Set up a ZFS Dataset and NFS Share on my NAS to house my Nextcloud Data

My intention all along was to create a dataset on my DIY NAS, to share it with the Nextcloud VM, and then to configure Nextcloud to use that shared path to store all of its data, primarily because what I valued was the data that was being stored and less so the virtual machine. In the long run, having the dataset will save me some storage space (example: space wasted in over-allocating space in a virtual hard drive and saving some administrative work in growing the virtual hard drive as more data is stored on Nextcloud), it enables me to set up ZFS snapshots for my Nextcloud data, and I’m poised to include this Nextcloud dataset among what I’m already backing up on Backblaze B2.

  1. Created a user/group to grant permissions to access the Nextcloud Data directory as.
  2. Created a new ZFS dataset on my NAS for the Nextcloud data.
  3. Set the permissions on the ZFS dataset, setting the User and Group to match what was created in the prior steps.
  4. Created an NFS share, pointed it to the Nextcloud dataset’s path (/mnt/volume1/nextcloud), restricted it to the Nextcloud VM’s hostnames (nextcloud, nextcloud.lan, and nextcloud.briancmoses.com.beta.tailscale.net), and set the Mapall User and Mapall Group to the User and Group created in the first step.

Setting up user/group Adding ZFS Dataset Setting Dataset Permissions Creating NFS Share


Mount the NFS share to the Nextcloud Virtual Machine

Having created the dataset and NFS share on the NAS, I swapped over to the Nextcloud VM to mount the share. I went through a few different iterations before landing on these steps. I wound up learning that the local directory to mount the NFS share in needed to belong under /mnt or /media thanks to how the Snaps work.

  1. Installed the nfs-common package on the Nextcloud VM (sudo apt update and sudo apt install nfs-common)
  2. Created a new Nextcloud data directory (mkdir -p /mnt/nextcloud) on the Nextcloud VM.
  3. Change ownership to root of the new Nextcloud data directory (sudo chown -R root:root /mnt/nextcloud) on the Nextcloud VM.
  4. Changed Permissions on the new Nextcloud data directory (sudo chmod 0770 /mnt/nextcloud)
  5. (Optional) Confirmed that the new NFS share existed, could be mounted, and that files could be viewed/edited/deleted on the Nextcloud VM.
    1. Validate presence of NFS share via showmount -e drteeth.lan.
    2. Manually mounted the NFS share via sudo mount drteeth.lan:/mnt/volume1/nextcloud /mnt/nextcloud.
    3. Created, viewed, and deleted a test file in the /mnt/nextcloud/ path.
    4. Manually unmounted the NFS share via sudo umount /mnt/nextcloud.
  6. Set up mounting the NFS share at boot by editing /etc/fstab.
  7. Executed sudo mount -a to mount the newly added line from /etc/fstab.

Installing the Nextcloud Snap and configuring it to use the custom Nextcloud Data Directory

In my stumbling and tinkering, I deleted everything and started over from scratch a few times. I had made the mistake of too excitedly installing Nextcloud and immediately started using it before figuring out how make it use the network share that I had mounted for its data storage.

But in following the Nextcloud Snap’s directions on changing the data directory to use another disk partition, I wound up overlooking two equally important details:

  1. I needed to connect the removable-media Snap to Nextcloud.
  2. The local path to my share needed to exist beneath either /mnt or /media on my Nextcloud VM.

Not understanding these two details had me scratching my head at a couple different points, running into permissions errors, and flailing trying to get Nextcloud working

  1. Executed the Nextcloud Snap Installation (sudo snap install nextcloud)
  2. Connected the removable-media to Nextcloud Snap (sudo snap connect nextcloud:removable-media)
  3. Edited the Nextcloud autoconfig (/var/snap/nextcloud/current/nextcloud/config/autoconfig.php) and updated the directory variable to ‘/mnt/nextcloud’.
  4. Restarted the Nextcloud PHP Service (sudo snap restart nextcloud.php-fpm)
  5. From my browser I opened the Nextcloud VM’s URL
  6. I set up the primary administrator’s account in the Nextcloud UI.

Now What?!

The possibilities are really endless! First, I need to get the Nextcloud client installed on my desktop computer, laptop, my phone, and tablet. But after that, I’m curious about using Tailscale’s sharing to maybe provide some cloud storage to family and friends. I’m very interested in untangling my rat’s nest of synchronization tasks and cloud storage providers and relocating my blog’s storage into Nextcloud. The same goes for my recent FPV quadcopter footage, I’d like it to get uploaded to Nextcloud so that I can more easily edit those videos.

Final Thoughts

This blog was supposed to be mostly about Nextcloud, but I can’t stop raving about Tailscale. Prior to using Tailscale, hosting my own cloud storage solution was going to be too much investment—of dollars and time! The sum total of effort in setting up a VM, configuring up Nextcloud, maintaining SSL certificates, dealing with the opening ports on my firewall, and any fallout from aggravating my Internet service was just too much. Combining Nextcloud with Tailscale eliminated or mitigated the hassles associated with many of those factors.

I’m excited to tinker with Nextcloud, and at the rate that Tailscale keeps announcing new features, I’m excited to see what feature I’ll get to try on this VM next! For the time being, I’m going to prioritize how I can leverage Nextcloud to make my most important data more ubiquitous, but I’m open to any possibilities!

Are you using Nextcloud with your NAS in order to host your own cloud storage? How have you tackled keeping your data synchronized between many devices? Are you interested in Nextcloud—but haven’t yet taken the plunge? What’s standing between you and adoption of Nextcloud? What sort of functionality would you like to see featured in future blogs? Let me know down in the comments below, I’d love to hear your thoughts!

Tailscale: A VPN that even Brian can use!

| Comments

I built and blogged about building my own DIY NAS back in 2012 and I’ve been repeating building a new DIY NAS once or twice a year ever since. One of the most frequently asked questions across those blogs has been, “How do you access your DIY NAS from the Internet, Brian?”

My answer to that question has always been “I don’t.” For the most time I’ve had a DIY NAS, I just didn’t have much need or interest in accessing the contents of my NAS from outside my home. At first, I only used my DIY NAS for backing up the computers that I had at home. But slowly over time, I’ve transitioned to using my DIY NAS as my primary storage for all of my data.

Consequently, accessing my data remotely has been more and more important over the years. Primarily, I’ve used services like Google Drive or Dropbox to access critical data and synchronize changes made to it across all my machines. For quite a while this has been both easy and cheap. But I started creating more content, especially videos for my YouTube channel, and this solution slowly began to break down and become more expensive.

Over the past two years, I’ve made a few halfhearted attempts to install and configure a VPN endpoint within, or at the edge of, my own network. Every time that I tried, I ran into issues—mostly all related to my lack of expertise—and set it aside to figure out another day.

For a while now, a few people have been encouraging me to check out Tailscale. In fact, Pat’s been telling me routinely about how he’s made his life easier with Tailscale and insisting that it was really simple to set up. While I have had no reason to doubt Pat’s assessment, I’ve also learned that there’s a cornucopia of topics that Pat thinks are painfully simple which completely short-circuit my brain.

What is Tailscale?

Over on Tailscale’s website they describe Tailscale as “A secure network that just works. Zero config VPN. Installs on any device in minutes, manages firewall rules for you, and works from anywhere.”

You create an account with Tailscale, you install a client on each machine, associate those clients with your account, and Tailscale encrypts traffic between any of your endpoints.

Tailscale was as easy as Pat made it sound every time he told me about it. In fact, I think it was easier!

How am I using Tailscale?

Out of curiosity, I set up Tailscale on a few different devices and without any effort I had Tailscale up and running on my phone, my tablet(s), two different Raspberry Pis, my laptop, my desktop computer, and on my OpenWRT router.

Remote Desktop access to my Tradewars 2002 game server

A couple years ago, I wrote a nostalgic blog about Tradewars 2002 which convinced me to spin up my own Tradewars game server on a small virtual machine hosted in Azure. For a long time, I’ve lived with exposing more than just the Tradewars game server’s port in order to remotely access the machine.

Implementing Tailscale on the game server let me close down the port(s) that I had exposed—and probably never should have—on the TW2002 virtual machine.

This is something that I could’ve accomplished in the Azure portal on my own by setting up some firewall rules in Azure and also on the virtual machine. But it was so much easier to just completely close it down and use Tailscale instead.

As an added benefit, my super-secret outer space trading strategy is now happening over an encrypted tunnel!

Pi-KVM

If you’ve read my two-part series of blogs about Pi-KVM then you’re already aware that Pi-KVM is an awesome little project that allows you to build an inexpensive KVM-over-IP using a Raspberry Pi 4 Model B (2gb version), a video-capture device, and an assortment of USB cables.

Combining the Pi-KVM and Tailscale is a really compelling pairing which demonstrates the value of both products. If I had a friend who was having a computer problem in another location (near or far), I could give them the Pi-KVM, they could hook it to their computer and network, and thanks to Tailscale I could access the Pi-KVM interface remotely. I could access the machine’s BIOS, boot from an ISO, or remotely access the native operating system without much effort at all.

A shotgun approach to access the contents of my DIY NAS remotely

Tailscale’s recommended approach is to put the Tailscale client on all of the devices and assemble a mesh network of connected endpoints. This assumes that you’re able to install the client directly on each machine. Unfortunately, FreeNAS (or TrueNAS) does not include the Tailscale client in their base operating system and they don’t really want you tinkering with the operating system at all, so the Tailscale-recommended approach is not as straightforward.

In evaluating my options, I knew I’d need to use Tailscale’s relay node if I wanted to be able to access my NAS using Tailscale. It seemed like I had a few options using Tailscale as a relay node:

  1. A jail hosted on my FreeNAS server.
  2. A virtual machine on my homelab server.
  3. On my OpenWRT router.

For the time being, I have opted for the third option. I knew that running it on my own OpenWRT router was possible thanks to Pat’s blog about putting Tailscale on his Mango OpenWRT router.

I configured the relay node to relay for the subnet that my NAS (and the rest of my home network) is on. I wound up deciding that if I was going to compromise by moving away from Tailscale’s mesh of encrypted network endpoints, then I would do so in a way that provided the maximum of possible utility. In this way, I was using Tailscale pretty similarly to a traditional VPN—everything on my network at home is accessible from outside of my network—provided the computer I’m using it from (my laptop) is running a Tailscale client too.

So what do I think?

Ultimately, I would’ve preferred adhering to the full-Tailscale method and installing client(s) on every machine that I want to be reachable from my other machines running Tailscale. My lack of understanding of FreeBSD and how FreeNAS is architected were significant enough of obstacles that I made a compromise.

I expect that it is possible that a FreeNAS enthusiast could create a jail, install the FreeBSD Tailscale package in the jail, and tighten the scope of the Tailscale relay node down to only the IP of the NAS. Perhaps this will be a topic of a future blog? Or even better—maybe somebody will answer this question in the comments below?!


Regardless of my obstacles with FreeNAS, Pat absolutely was correct—Tailscale makes all of this so much easier! I’ve installed Tailscale on devices with very different hardware, running a number of different operating systems, and numerous different use cases. All of it was really straightforward and easy to set up. I didn’t have to create any firewall rules—it all just worked.

I’m excited about the possibilities that Tailscale presents. They recently added endpoint sharing as a Public Beta, which is a really useful concept that I’m going to be exploring as part of future blogs. So please stay tuned!

Pi-KVM: Controlling a 4-port KVM and setting up Tailscale

| Comments

In a previous blog, I raved about how awesome Pi-KVM is. If you’re not aware, Pi-KVM is an open-source project that allows you to turn a Raspberry Pi into an IP-KVM.

You plug the Pi-KVM into your network and into a computer and then from anywhere else on that network, you can remotely control that computer as if you’re sitting in front of it, including doing things like accessing the remote machine’s BIOS.

When I did my initial research, I learned that building a Pi-KVM can be done incredibly inexpensively—it would cost less than $80 to build one from the recommended parts list. By building mine from a CanaKit Raspberry Pi 4 Model B (4GB) Pro Kit, I wound up spending quite a bit more than that $80. Regardless of how much I spent, I felt like I got a great value out of what I built.

Enhancing my Pi-KVM Setup

After building a Pi-KVM for Pat for Christmas, I was hooked and quickly built one for myself. Following that, I built another one that I could easily use with other computers outside of my office. Most notably, I wanted one to use in my “recording studio” where I do most of the work in assembling my DIY NAS builds.

I also began contemplating improving the Pi-KVM I was planning to use with my DIY NAS and my homelab servers in my office. When I started out, I planned on just sharing the same Pi-KVM with both machines, but the more I thought about it, the more I realized that I wanted to avoid having to swap the cables between the two servers.

This got me thinking. Either I’d need to build yet another Pi-KVM, or find a KVM switch that I could trigger from Pi-KVM to swap between the two hosts.

Adding a 4-port KVM Switch, the ezcoo EZ-SW41HA-KVM

Thanks to Novaspirit Tech on YouTube and his Q&A video about the Pi-KVM, I was already aware of what I wanted to try next. Among the topics discussed in the video was an inexpensive 4-port KVM switch made by ezcoo, the ezcoo EZ-SW41HA-KVM. This KVM switch has its own USB management interface which allows for firmware updates and switching which port is active on the KVM switch.

What set this particular switch apart from others was that Pi-KVM has functionality built-in that can issue the commands to the ezcoo EZ-SW41HA-KVM. You can configure the web interface’s elements to suit your needs and then pick which of the four ports is active with your Pi-KVM.

By using my Pi-KVM, the ezcoo EZ-SW41HA-KVM, some HDMI cables, some USB A to B cables, a couple VGA-to-HDMI adapters, and a generous helping of obsessive-compulsive cable management, I now have a 4-port IP-over-KVM setup functioning.

Parts List

A Pi-KVM can be built for under $80, and for an additional $150, your Pi-KVM can be extended to work with up to 4 different computers. This was ideal for me because I often wind up tinkering with a third—and sometimes fourth—computer at my desk, particularly when I’m working on one of my DIY NAS builds.

Setting up Tailscale on my Pi-KVM

Tailscale is a simple and easy-to-use VPN service built atop Wireguard. By installing and configuring a device, you can access that device over a secure VPN connection on any machine that you’re running the Tailscale client on. Pi-KVM has incorporated Tailscale as a configurable option. Once it is set up and on your Tailscale account, you can access your Pi-KVM from another machine as long as it is connected to the Internet.

Being able to remotely access your Pi-KVM is a handy way to remotely access a machine without having to directly expose it to the Internet. This isn’t a critical feature to me, but I think it’s really quite interesting. I don’t really access anything on my network from outside of my house, but having that as an easy option is really intriguing.

Pat’s been telling me to check out Tailscale for ages. Every time that he tells me about it, it has sounded really interesting—but I didn’t really have a good use-case for it. But now that I’ve set it up for using with my Pi-KVM, I know that it’s a matter of time before I’m using it with other machines too!

What’s Next?

After writing two blogs about it, I’m still quite excited about the Pi-KVM project. Pi-KVM is working on their own hardware to build in a bunch of extra features and are preparing to have the hardware manufactured. I’m excited enough about what’s been shared that I’ve signed up to preorder that hardware. As I have been working on this second blog, Novaspirit Tech published a video reviewing the Pi-KVM v3 hardware. Here are a few things I plan to do with my Pi-KVM:

  • Build another Pi-KVM for use on my workbench in my studio
  • Set up network passthrough to share the Raspberry Pi’s wireless interface for use where a network drop isn’t convenient.
  • Upgrade one (or more?) of my Pi-KVMs to the version 3 of the hardware
  • Other ideas? What would you do with a Pi-KVM that I haven’t considered yet? Please share your ideas in the comments, I’d love to see them!

Final Thoughts

Between building and using three different Pi-KVMs, reading about the Pi-KVM version 3 hardware, and now seeing it in action, I am doubling down on my previous position. I have no reservations whatsoever about capitalizing on the opportunity to pre-order the version 3 hardware once I can. I’m also happy that I can help support Pi-KVM on Patreon.

Pi-KVM is an awesome project. Pi-KVM has joined OctoPi and HomeAssistant at the top of my list of favorite Raspberry Pi projects. I’m excited to see what’s in store for the future of Pi-KVM.

Giveaway

I think that Pi-KVM is an interesting enough project that as part of this pair of blogs, I’m going to give away three kits which will hopefully get someone well on their way to building their own Pi-KVM. Each kit will contain:

Here are the giveaway details (Note: There’s a new way to enter that’s unique to this blog!_):

briancmoses.com: Raspberry Pi 4 Model B (2GB version) with Customized 3D-Printed Case Giveaway

Pi-KVM: an inexpensive KVM over IP

| Comments

I recall griping at Pat one day he was over for dinner that I wished everything had an IPMI interface or that nothing did. Only two of the computers at my house have an IPMI interface, all the rest did not. When the DIY NAS: 2020 Edition was burning in, I had to get up and go into the other room to see what it was up to, but I didn’t think it was worth the price premium to move up to a motherboard which included IPMI among its features.

Pat nodded in agreement and remarked, “There’s this neat project, Pi-KVM, that lets you build a cheap IP-KVM out of a Raspberry Pi! You should check it out.” We talked about different possible uses for it and then got busy repairing a quadcopter or playing some video games.

A few days later, I recalled this conversation later when wracking my brain for a Christmas gift idea for Pat and immediately decided then that I’d build a Pi-KVM for Pat. As a bonus, I’d get to play with his Christmas present a bit before deciding if I wanted to build one of my own!

KVM over IP (IP-KVM)

A KVM (or KVM switch) is basically a device that allows you to share a keyboard, video, and mouse between two computers. For a very long time, I had my own DIY NAS, primary workstation, and work laptop all plugged into a KVM switch. Whenever I needed to use one of those three computers, I’d hit a button and the KVM would switch me between the three computers.

An IP-KVM is very similar: you plug the IP-KVM device into the keyboard, video, and monitor of a computer but then you access it over a network. The keyboard, mouse, and display that you’re using aren’t actually plugged into the remote computer.

Both a KVM and IP-KVM are superior to remote access (TeamViewer, Remote Desktop, VNC, etc.) because you’re accessing the actual hardware remotely. You’re able to see the machine POST, access the BIOS, and watch it load the operating system. Most remote access options require the operating system to be up and running first. In enough scenarios, that’s simply not enough.

IPMI

The most controversial parts of my DIY NAS build blogs is how frequently I recommend consumer-grade hardware. When people advocate for using enterprise hardware, the two reasons that resonate the most to me are support from the manufacturer (potentially including longer warranties) and that most server-grade motherboards have an IPMI interface.

Simplifying things a bit—maybe tremendously—IPMI is quite similar to having a built-in IP-KVM incorporated on the motherboard. The motherboard has a dedicated network interface that gets plugged into a switch and assigned an IP address by your router. Using a web browser or an IPMI client, you’re able to access this interface and interact with the hardware as if you were sitting in front of the computer with a keyboard, mouse, and monitor plugged into it.

The DIY NAS: 2016 Edition was the first motherboard I used with an IPMI interface. That feature (among others) helped convince me to use the same motherboard when I upgraded my personal NAS later the same year. That upgrade also meant that I was able to retire my KVM switch and all of its cables. When I built my homelab server, I made sure to pick a motherboard that included an IPMI interface. Since then, you know how much I’ve successfully used those IPMI interfaces? Almost zero!

Thankfully, I’ve rarely actually needed to use the IPMI interfaces. Both machines have been tremendously stable and do their jobs without much interaction from me. However, each time that I have attempted to access their IPMI interfaces, I have run into minor issues. When I encountered these difficulties, I simply reverted to old behavior and grabbed my spare monitor and keyboard from my closet for the following reasons: it’s less effort, and the IPMI’s web interfaces have been pretty terrible.

When I bought an extension for my desk, a new switch for my 10Gb network, and moved around both my DIY NAS and homelab servers, I decided to just leave the IPMI interfaces disconnected and removed the network cables on each machine. It’s almost like I knew that a couple months later, I would be tinkering with something I liked way better than any IPMI interface that I’d used.

Pi-KVM

So what’s Pi-KVM all about anyways? It’s an open-source project for building your own IP-KVM. So far, Pi-KVM has been through a couple different hardware variations. All of the hardware variations have been built around different Raspberry Pi models and a varying amount of do-it-yourself electronics. The current hardware version (v2) can be built around either a Raspberry Pi 4 Model B (2GB or higher) or a Raspberry Pi Zero W. Depending which Raspberry Pi option you pick, you’ll also need a video-capture device and some USB cables/adapters.

When it’s all said and done, the Raspberry Pi is connected to your computer’s display and USB ports. You pull up the Pi-KVM web interface in your browser, and you’re then in control of remote computer as if you’re physically standing right there. It’s really quite fantastic!

I looked for a comparable off-the-shelf piece of equipment, but there’s really nothing quite like it. I suspect that there’s just not much consumer demand for IP-KVM hardware right now. For most consumer users, there are acceptable enpough methods for accessing computers remotely, like VNC, Remote Desktop, and many others.

Nevertheless, after assembling the Pi-KVM that I gave to Pat for Christmas, I was immediately convinced that I wanted at least one for myself. Seeing these features in action were what sold me:

  • Incredibly easy to build the hardware (version 2)
  • The web interface was really responsive and easy to use.
  • The latency was low.
  • CD-ROM or Flash Drive emulation to pass through to the connected host.


These features aren’t all-encompassing either! They’re just the features that I was immediately zeroed-in on. There’s a whole cargo container full of other features that I haven’t leveraged yet too. The ATX controls sound really intriguing—having the ability to remotely press the power and reset buttons seems like it could come in really handy. Securely accessing my Pi-KVM from the Internet sounds interesting, but I’d rather not open ports on my router’s firewall in order to do so. However, there is a Tailscale client available. The idea of being able to access Pi-KVM from any device that I have a Tailscale client running on seems fascinating. Plus, Pat keeps telling me about how Tailscale makes these kinds of things easy, so this sounds like an excellent opportunity to prove Pat correct!

Brian’s Pi-KVM Parts List

When I ordered parts for Pat’s (and then again for my own) Pi-KVM, I made a mistake (or did I?) and bought a Raspberry Pi 4 Kit with 4GB of RAM. A Raspberry Pi 4 2GB meets Pi-KVM’s hardware requirements and would’ve worked just fine.

I decided that having 2GB of extra RAM might be useful in case there was other functionality that I wanted to add to my Pi-KVM down the road. Maybe one of these five awesome headless Raspberry Pi uses are good candidates to run alongside my Pi-KVM?

Brian Spent Too Much Money!

It’s important to keep in mind that {“I wasn’t a very thrifty shopper and wound up spending way more than I needed to. A Pi-KVM can easily be built for about $80!”} This can be done by more closely following the suggested hardware list:

I mentioned before that I couldn’t really find a comparable product when I searched for one. About the closest thing I could find were USB Crash Cart Adapters, like this one from Startech.com. This crash cart adapter is over $200 more than what I paid, doesn’t allow remote access over the network, is only VGA, has a much smaller set of features, and requires a custom application installed on the machine you’re accessing the remote machine from.

A price tag of under $80 is inexpensive enough that I’d gladly lend my Pi-KVM to friends who need my help with something on their PCs. It’s cheap enough that I’m going to definitely build another one just to have a loose spare for whenever it might be handy. For example, when I’m working through one of my DIY NAS builds!

What’s Brian think? I’m all in on Pi-KVM!

This is all Pat’s fault; he suggested I look into Pi-KVM awhile back. Once I did, I knew I wanted to build my own. In building one for both Pat and I, I’ve also learned that Pi-KVM is working on their own hardware and I now know that I want that too. Their hardware will include an extra ethernet interface to act as a pass-through, its own low-latency video capture ability, wider hardware support for finicky BIOSes, and many other features.

I’ve signed up to pre-order it and I’ve also become a Patron of pikvm on Patreon. The next iteration of hardware is going to make a fine upgrade—and another blog—down the road when I get my hands on it. It might be a fun project to design my own 3D-printed case for, or even maybe collaborate with Pat and mill something on his CNC machine.

But wait, there’s more!

In Novaspirit Tech’s YouTube review of the Pi-KVM and his subsequent video Q&A about Pi-KVM, he mentioned that Pi-KVM can also interact with a traditional KVM to allow you to switch between numerous different machines. I have purchased the ezcoo EZ-SW41HA-KVM 4-port KVM switch, a couple VGA-to-HDMI adapters, HDMI cables, and USB cables to hook into it.

Adding the ezcoo KVM switch on to my own Pi-KVM is something I’m looking forward to building and blogging about in the very near future!

Giveaway

briancmoses.com: Raspberry Pi 4 Model B (2GB version) with Customized 3D-Printed Case Giveaway

DIY NAS: EconoNAS 2020

| Comments

For a long time, I’ve been building two different NAS builds every year: an impressive premium DIY NAS with a premium price tag and a more economical DIY NAS that’s more friendly to the bank account. The whole point of all of these different DIY NAS builds has always been to encourage people to build their own do-it-yourself network attached storage (NAS) server. Got a big budget and want to build something to show off? I’ve got a DIY NAS for that! Or do you want to squeeze every little drop of storage out of your budget? Well, I’ve got a DIY NAS for that too!

In true 2020 fashion, the DIY NAS: 2020 Edition found itself neglected until very late in the year. This created the likelihood that there would be no EconoNAS at all in 2020. Rather than skipping the EconoNAS in 2020, I decided that I would frantically write a blog containing the parts list of what I would have put in the EconoNAS.


Disclaimer!

A long time ago, in a blog post far, far, back in history—I wrote a blog about a fictitious DIY NAS that I had not actually built. Because of this, I wound up picking a CPU that physically fit in the motherboard, was listed by the motherboard’s specifications as compatible, but would not boot up unless the BIOS was up-to-date enough. At least one person shared an experience where they received a motherboard with a BIOS version old enough that it would require a BIOS update before that CPU could be used.

I was mortified to learn this, and I continue to be embarrassed about it to this day. Since that mishap, I’ve built each DIY NAS that I’ve blogged about and I have done everything in my power—assuming my bank account allows it—to make sure that I was able to assemble what I’m recommending to others. Here at the end of 2020, that streak is temporarily coming to an end.

I did not build this year’s EconoNAS. While I’m quite confident that these parts will all work together to build a great machine, I think it’s worth warning you all that I did not actually build it. I put in quite a bit of effort to try and confirm hardware compatibility and also to seek out others’ shared experiences with this same hardware. Hopefully, someone will build this DIY NAS and share their experience in the comments below!

Motherboard & CPU

Originally, I was really tempted to build the EconoNAS using the same exact motherboard as the DIY NAS: 2020 Edition, but its supported CPUs were just too expensive for my tastes. I couldn’t justify the amount of expense that would’ve been incurred by the motherboard and CPU alone. Instead, I opted to realize the opportunity that existed in saving money on both components.

I wound up selecting the GIGABYTE B450 Aorus M (specs). The price, the wide range of supported CPUS, the Micro ATX form factor, the support for up to 128GB of RAM, and the six onboard SATA3 ports were all features that made it easy to choose the B450 Aorus M motherboard.

Of all the components in the DIY NAS: 2020 Edition, the CPU was by far the most over-the-top. My friend, Pat, wrote in his NAS building tips and rules of thumb blog that “Serving files doesn’t require a gargantuan processor.” In putting together this parts list for an economical NAS build, I wanted to focus in on that point—because I’m not quite sure if Pat believes that I actually comprehend it!

The AMD Athlon 3000G CPU (specs) is a 3.5GHz dual-core CPU. Compared to the AMD Ryzen 9 3900X CPU that I used in the DIY NAS: 2020 Edition it is dirt-cheap to purchase (about 11% of the price of the AMD Ryzen 9 3900X) and power-sipping to run (35W TDP vs 105W TDP). Inexpensive to purchase and inexpensive to operate are hallmarks of a good decision in building an economical NAS.

RAM

For RAM, I opted to match the 32GB of memory found in the DIY NAS: 2020 Edition, but went with more affordable non-ECC RAM by choosing a Crucial 32GB (16GBx2) DDR4 2666 MT/s (PC4-21300) kit (specs). The TrueNAS Core hardware suggestions suggest 8GB as a bare minimum, and the bare minimum is usually what I opt for in building an economical NAS. However, I wanted to build a NAS that measured up to its big brother (as long as the budget allowed it), and giving the file system more RAM to cache data to is a great way to squeeze out extra performance in the budget.

Case and Power Supply

The Fractal Design Node 804 (specs) is routinely suggested at me for NAS builds—and for good reason—it’s a fantastic case. But it’s lacked a feature that I appreciate in my “big” NAS builds—hot swap bays—and it’s always been a little bit more expensive than I’d like to spend on a case for an economical build. However, this year I splurged on the case.

If I were rebuilding my NAS today, and I couldn’t 3D-print my own custom Mini-ITX NAS case, then the Fractal Design Node 804 would be at or near the top of my list. I’ve been tempted to incorporate it into a NAS build for a while. Selfishly, I’m a little disappointed I’m not actually building this NAS, because I really would have liked to build a NAS out of it.

The Fractal Design Node 804’s key features are:

  • 8x 3.5” Internal HDD bays
  • 2x 2.5” Internal HDD bays
  • Fits Micro ATX or Mini ITX motherboards
  • Fits a standard ATX Power supply up to 260mm deep.

I’m not a huge fan of power supplies—I never have been. They’re not fun to shop for because it’s hard to quantify what the differences are between a $30 power supply and a $150 power supply. The features that do differentiate power supplies are infuriating to me; modular cables and LEDs.

I picked the Corsair VS500 power supply (specs) because it was from a manufacturer that I recognized/respected, its wattage (500W) would be roughly twice the amount of power the CPU and eight hard disk drives would consume (about 235 watts), and it seemed to be fairly well reviewed from what I was able to gather.

Host Bus Adapter and Cables

When I built the DIY NAS: 2020 Edition I thought to myself: “Getting rid of the IBM M1015 SAS Controller and specialty SAS cables will be an easy cost savings for the EconoNAS.

But after pricing out all the components, I realized that it’d only cost about $100 to add an enterprise-grade HBA to support 8 additional hard disk drives. Being able to fill up every drive bay in the Fractal Design Node 804 by adding an IBM M1015 SAS Controller and a pair of SFF-8087 to SATA Forward Breakout cables seemed like a very good return of investment of another $100 dollars.

If five drive bays (reserving at least one for the OS) is enough hard drives bays for you, this is your best place to start saving money on your own EconoNAS!

Storage

In my DIY NAS builds, there are two types of storage used: storage for the operating system and the actual storage that is used and exposed to the network. Over the years, I’ve come to the conclusion that it’s easy to make specific suggestions for the operating system’s storage device and it’s foolhardy to try and make specific suggestions with regards to the actual drives that data is being stored on.

TrueNAS CORE OS Drive

In the past year (or longer), the using a USB flash drive for the OS drive in TrueNAS CORE has gone from not suggested, to outright discouraged. In 2020, I’ve adjusted my NAS builds and said goodbye to my beloved SanDisk Ultra Fit USB drive. In the DIY NAS: 2020 Edition I used a pair of m.2 SSDs to mirror the OS on. The m.2 SSDs were not expensive, but their use prevent the simultaneous use of some of the onboard SATA ports. To maximize the number of SATA devices the EconoNAS could accommodate, I opted to pick a 2.5” SATA SSD to hold the operating system.

A better, more economical, choice is the Kingston 120GB A400 SSD (specs). A better—but less economical (and still a good value)—choice would be to mirror TrueNAS CORE across a pair of SSDs. I think a single SSD is a fine choice due to the quality and reliability of modern SSDs. Failure of the OS drive is further mitigated by the fact that the TrueNAS Core configuration is automatically backed up daily and it is possible to manually back up the system configuration as well.

NAS Hard Disk Drives

How much storage you need and how expensive it is makes it difficult to present an economical choice. The hard drives with the best price-per-terabyte usually carry a higher price tag, and the fewer drives that you buy, the higher percentage of your storage winds up being dedicated to redundancy. Buying cheap hard drives sometimes means settling for obsolete or refurbished drives. Here are some tips that I have for buying hard drives for an economical NAS.

  1. Shuck External Drives: External hard drives tend to be cheaper than their internal counterparts and tend to go on sale more often. While only the hard drive manufacturers can tell exactly why they’re less expensive, many people speculate that it’s in part because external HDDs typically have a much shorter warranty period. Whatever the reason, shucking an external hard drive is oftentimes the best value in acquiring storage for your NAS.
  2. Two Drives of Redundancy: Regardless of your NAS build, I recommend that you pick a configuration that has two drives’ worth of redundancy. There are a few different ways to achieve this, but my favorite is RAID 6. When using ZFS on TrueNAS CORE, the equivalent is raidz2.
  3. Lots of Smaller HDDs > Fewer Larger HDDs: While the largest hard drives are the best value, you’ll get far more net storage by buying more smaller drives. As an example, consider two different configurations of a 48TB array each with two disks of redundancy (RAID 6/RAID-Z2) using either a 12TB Western Digital Red Plus or a 6TB Western Digital Red Plus.
    1. 8x 6TB HDDs (RAID-Z2)
      1. Cost: about $1,432 ($179.00/ea)
      2. Usable Storage: 36TB
      3. Price per Usable TB: $29.83
    2. 4x 12TB HDDs (RAID-Z2)
      1. Cost: about $1,219 ($304.99/ea)
      2. Usable Storage: 24TB
      3. Price per Usable TB: $50.79
  4. Try and Avoid Buying Drives from the same Batches: This is a challenge and there’s no guarantee, but when you buy the same model hard drive all from the same vendor, the chances are that all of the drives you bought came from the same batch. If there was a flaw in that batch, you should expect to see that flaw across all of those drives. You can increase your chances of avoiding this by: buying the same model of hard drive from different vendors or by buying equivalent hard drives from different manufacturers.
  5. Read Backblazes HDD Stats: I can’t stress this enough. Backblaze is buying enough drives and sharing the data about those drives with everyone. Let these statistics guide you in assessing the hard disk drives you’re considering. These statistics are an awesome resource.

Final Parts List

Component Part Name Count Cost
Motherboard GIGABYTE B450 Aorus M specs 1 $92.99
CPU AMD Athlon 3000G specs 1 $99.99
Memory Crucial 32GB Kit (16GBx2) DDR4 specs 1 $148.29
Case Fractal Design Node 804 specs 1 $149.98
OS Drive Kingston 120GB A400 SATA 3 2.5” SSD specs 1 $24.99
Host Bus Adapter IBM Serveraid M1015 SAS/SATA Controller specs 1 $70.96
SAS Cable Internal Mini SAS to SATA Cable (SFF-8087 to SATA Forward Breakout) N/A 2 $11.99
Power Supply Corsair VS500 specs 1 $49.99
TOTAL: $661.17

NAS Software

In my mind, there are two attributes of a NAS that differentiate it from other computers: a large amount of hard drives, and some sort of user-friendly interface to help you manage it. Both attributes are a bit subjective; you could definitely build a NAS and with only one hard drive in it and you could install nearly any operating system on the server and add some sort of method for sharing files across your network.

But, in my opinion, what really turns your DIY NAS into a competitor to the QNAP, Synology, Drobo, etc. NAS offerings is being able to easily manage the DIY NAS without sinking much time into performing any command-line wizardry and mastering the different products that make a computer into a functioning NAS.

TrueNAS CORE

I’ve been using TrueNAS CORE (formerly known as FreeNAS) in my own NAS and nearly every single DIY NAS build I’ve built since. TrueNAS Core’s interface runs atop the FreeBSD operating system and aside from the interface, the next most important feature is the ZFS filesystem. TrueNAS Core is bundled with everything you need to get started to create a redundant array of hard drives, share the content to all of the computers on your network, and includes a comprehensive plug-in library.

TrueNAS is not without its own drawbacks. The ZFS file system does not like SMR (shingled magnetic recording) drives, and many/most inexpensive hard drives out there are SMR drives. PMR (Perpendicular magnetic recording) drives are strongly recommended when working with ZFS and they tend to be pricier than SMR drives, which isn’t the most budget friendly. Additionally, I’ve sometimes found the community of users to be closed off to the value of DIY NAS builders building their NAS out of consumer-grade components.

None of these drawbacks have discouraged me in the past. I was (and have been) willing to run my own NAS on hardware that you’re more likely to find in a home user’s computer, rather than a server in a datacenter.

The Rest

I’m perpetually curious about NAS distributions, and while I clearly favor TrueNAS, there’s no shortage of equally viable alternatives. I’ve tinkered with a few, but not extensively enough to perpetually keep up with all the new features of all the different NAS distributions out there. If you find TrueNAS isn’t your cup of tea, here’s a list of a few others you might want to look at:

There are no bad choices here; all of the options look pretty good. When I built my first DIY NAS back in 2012, I planned to experiment with a few options to see what was out there. Unfortunately for all the others, I tried FreeNAS first and didn’t feel it was critical for me to try any of the others once I had everything set up and working.

What’s Brian Think?

I’m every bit as excited about the DIY NAS: EconoNAS 2020 as I am with the DIY NAS: 2020 Edition, but for different reasons:

It demolishes off-the-shelf solutions

The key features of the DIY NAS: EconoNAS 2020 (listed below) put it at a sticker price of right around $500. At that price point I can’t find a single off-the-shelf NAS that matches up well. The closest equivalent NAS that I could find was the QNAP TS-873-8G and the QNAP TS-932PX-4G, but both are expensive (between $550 and 900), have less compute horsepower, less memory, fewer (or no) upgrade options, and have far fewer options for customization. In comparison, the DIY NAS: EconoNAS 2020 has:

  • 10 Internal Drive Bays (8x 3.5” bays and 2x 2.5” bays)
  • AMD Athlon 3000G CPU
  • 32 GB of DDR4 RAM
  • 120GB SSD for the operating system
  • Upgrade Options: more HDDs, more CPU, more RAM, 10Gbit network, etc.
  • Is less expensive at $513

Update (03/14/20): As I expected, prices have climbed dramatically for the motherboard, the case, and the CPU! This has driven the price up to just over $650. If you’re good shopper, I think you can beat the prices I’m seeing on Amazon right now. But I’d also encourage you to tweak the blueprint and look for better values on these components—especially the CPU and Case!

Extremely flexible

If you wanted something as equally potent as this year’s over-the-top DIY NAS, it’d be as easy as swapping out the CPU (see caution below about BIOSes and CPU support), but it’d still be a bit more economical by skipping out on the expensive case, premium low-profile CPU cooler, and the two m.2 SSDs. If this list of parts does not meet your compute needs, it’d be pretty inexpensive (around $180) to add an AMD Ryzen 1600AF CPU and the GPU from the DIY NAS: 2020 Edition and turn this EconoNAS into a little mini homelab server.

Upgrades, upgrades, and more upgrades

Nearly everything about the DIY NAS: EconoNAS 2020 can be upgraded, either before you build it or down the road as new releases push the prices down on all hardware:

  • CPU: The motherboard supports a tremendous number of different CPUs, too many to list succinctly.
    • Note: When picking or upgrading a CPU, make sure to research which BIOS is needed for which CPUs and chart out your upgrade path. What happens when a BIOS upgrade is needed to support a newer CPU? What happens if making that update renders your old CPU incompatible?
  • RAM: Upgradeable to 128GB of RAM total.
  • More HDDs: If a bigger case were selected, it’d support up to 14 SATA devices (6 onboard, 8 on the IBM M1015.)
  • PCI Express: one PCI Express x16 and one PCI Express x1 slots available to do things like building your own inexpensive 10Gb network.

I would build the EconoNAS for myself and I’d tell my friends to build it too

If for some reason I needed to replace my NAS today or a friend asked me about building their own NAS, I would ultimately wind up building this EconoNAS and not the bananas DIY NAS that I built earlier in the year. I’d be really tempted by all the bells and whistles of the DIY NAS: 2020 Edition, but ultimately I’d wind up building this EconoNAS.

What do you all think? Which of the two would you be inclined to build? Would you build the DIY NAS: 2020 Edition? Or would you build this more economical variation on the same theme? Please let us know in the comments below!

Giveaway

Every NAS that I’ve built in the past few years, I’ve wound up giving away. But because I didn’t build the EconoNAS this year, I can’t give it away. However, the giveaway for the DIY NAS: 2020 Edition is still ongoing at the time this EconoNAS blog was published. Check out the giveaway details below. Entering a giveaway and winning the DIY NAS: 2020 Edition is quite budget-friendly!

Update (01/01/2021): I rang in the New Year by picking a winner in the giveaway of the DIY NAS: 2020 Edition! I hope that everyone joins me in congratulating Matt H. of Orlando, Florida for winning the TrueNASGiveaway! Matt’s visitation of [my Youtube page][bcm_yt] (I hope he clicked like and subscribe!) in early December is the entry that wound up turning him into the winner of the DIY NAS. Altogether, 1,636 people combined for 7,893 entries into this year’s giveaway. Thanks to everyone for making the giveaway a big success!

DIY NAS: 2020 Edition

DIY NAS: 2020 Edition

| Comments

For many years now, I’ve been building, blogging, and giving away DIY network-attached storage (NAS) builds. I was started down this path when I couldn’t find a relevant and recent build parts list to follow when I built my first DIY NAS back in 2012. In blogging about my own experience building my NAS, I surprisingly find myself atop Google’s search rankings for search terms like “DIY NAS”. Ever since, I’ve been regularly building and blogging about my different DIY NAS builds. My perpetual hope is to encourage potential DIY NAS enthusiasts to build and design their own custom DIY NAS solutions.


When I got done with last year’s DIY NAS build, I thought that I had built the most bananas over-the-top DIY NAS build that I could possibly imagine. Upon finishing it, I committed to myself that the 2020 DIY NAS build would be far more restrained.

But then I built my first AMD-based DIY NAS, the 2019 EconoNAS, and in building that NAS I realized that the extreme flexibility of AMD’s CPU architecture was suited quite well for DIY NAS enthusiasts to take advantage of. When I published last year’s EconoNAS blog, I remarked to myself, Well I guess I have to build an even more bananas AMD DIY NAS now, don’t I?!

And then 2020 happened…

…and thankfully my wife, son, and I remained healthy—something I wish for all of you too. As a Type 1 Diabetic, I’m among the highest risk for bad outcomes. As I began working from home, I thought I’d be able to spend a few of the weekly hours that I spent commuting working on blogs instead.

But what I found was the opposite—surviving a pandemic, being a productive remote worker, plus trying to help keep an eye on my feral 4-year-old son was a tremendous energy drain. I did my best to emulate my amazing wife and focused my efforts on our household, and unfortunately the DIY NAS: 2020 Edition suffered for it.

All of the hardware that I’d purchased so far languished in my “other office” while I re-acclimated to this new normal!

If you’ve ordered anything online recently, you’ve certainly noticed that COVID-19 has disrupted a lot of the availability of items. This is especially true of the components in the DIY NAS: 2020 Edition. I was disappointed to see how much of it has been difficult to find in stock. If you decide to emulate this build, please use the comments below and help each other out in finding vendors with the parts in stock—or suitable replacements for hard-to-find components!

Case and Power Supply

For every NAS build, I always like to lead off with the key component, which in most years is the motherboard. Especially with my preference for smaller motherboards and integrated CPUs. But this year is different! In late 2019, SilverStone contacted me and asked if I’d review the SilverStone CS381 (specs) if they sent one to me.

The SilverStone CS381 is really impressive on paper. There’s room in the case for a total of 12 different hard disk drives. With 8 of those hard drives being accessible in hot-swap drive bays. The case accommodates Micro-ATX, Mini-ITX, and Mini-DTX motherboards. While I prefer smaller cases for my DIY NAS builds, the CS381 is not a large case by any stretch of the imagination. Moreover, its bigger footprint allows it to accommodate cards up to 267mm or even a 240mm radiator for a water-cooling setup.

Of the many components in the DIY NAS: 2020 Edition, I’ve been most excited about getting my hands on the SilverStone CS381. Will it warrant an update to Brian’s Top 3 DIY NAS Cases on butterwhat.com? Stay tuned!

Power supply was a bit of a headache, but only because I didn’t pay attention to the details of the SilverStone CS381 and initially bought a full-size ATX power supply instead of the SFX or SFX-L power supply that the case supports. I wound up choosing the be quiet! BN639 SFX-L Power Supply primarily because of its wattage and price. I am not a fan of modular power supplies. I’d much rather use a couple zip ties to manage an extraneous power cord or two, rather than have to dig around my office months—or even years—down the road to find where I stored the extra cables. Unfortunately for me there just wasn’t any non-modular option within what I was shopping for. I would’ve considered spending an extra $5-10 for a non-modular option.

Motherboard, CPU, and CPU Cooler

One of the things that I was most excited about after selecting the SilverStone CS381 as the case was the additional motherboards that I’d get to shop for. Being able to include Micro-ATX motherboards more than doubled the number of motherboards that met the criteria of what I feel is important for a DIY NAS build. Since I’d already decided I wanted to build a DIY NAS with an AMD CPU and that I’d picked out a case that could support up to 12 hard disk drives, my ideal criteria for the motherboard was:

  • Mini-ITX or Micro-ATX
  • AMD AMD4 CPU Socket
  • Support for 12 SATA Devices
  • Documented support for ECC RAM
  • Support for M.2 SSD(s)

Of the criteria, I knew that the 12 SATA devices and the documented support of ECC RAM would present the biggest challenge. While the AMD Ryzen CPUs support ECC RAM, it’s not necessarily implemented on all of the motherboards nor is it really something that the motherboard’s marketing departments have put a lot of effort including in their marketing materials. In doing my research, the best point of advice I read was to read reviews of the motherboards and focus on whether they tested the ECC functionality.

In regards to the SATA devices, I knew that I wasn’t going to find a motherboard that supported 12 SATA devices—especially at a “reasonable” price point. Moreover, I also wanted to use M.2 SSD(s) for the operating system, and that typically knocks out the use of some of the available SATA controllers on the motherboard.

With all of that in mind, I very quickly narrowed in on the ASRock X570M Pro4 motherboard (specs). In reading about the motherboard, I was confident it supported ECC RAM, and I liked that it had enough onboard SATA to support the SilverStone CS381’s 8 hot-swap bays. The motherboard very nearly met all of my ideal criteria by itself and at a fairly reasonable price. The only criteria it wasn’t able to meet—support to fill up all of the case’s internal and external drive bays—would get handled in the rest of my hardware purchases.

If you wind up using this build as a template for your own DIY NAS, be aware that ASRock has several X570-based motherboard with similar model numbers. Specifically, there’s an ASRock X570 Pro4 which is an ATX form factor and will not fit in the case.

Before you purchase, double-check to make sure that you are buying the ASRock X570M Pro4—the “M” is important! I nearly made this same exact mistake myself and decided to post this update after a commentor shared their unfortunate experience and wanted to warn others.

In building a bananas AMD NAS, I instantly scrolled to the top of AMD’s processor offerings and observed to myself, “Now that’s just a bit TOO bananas, Brian.” But in doing a bit of browsing of benchmarks of high-end CPUs, I was drawn to the price-to-performance of the AMD Ryzen 9 3900X (specs).

Moreover, whatever way you slice it, the AMD Ryzen 9 3900X is complete and absolute overkill for the processing power needs of NAS. The selection of this processor really makes the machine capable of much more than just being a NAS. I’d encourage people who follow this blueprint to fully leverage the extra processing power to experiment with virtualization and host things that complement the storage capabilities.

An eager patron on Patreon started building his own DIY NAS from my parts list and helped me realize (Thanks, Alex!) that because I’d selected the SilverStone CS381, I’d need a low-profile CPU cooler. Because I wound up selecting the AMD Ryzen 9 3900X, I opted for what’s widely regarded as one of the best low-profile AM4 CPU cooling solutions; the CRYORIG C7 Cu (specs). The 105W TDP of the CPU convinced me that I’d need to make sure to pack the most amount of cooling in the space I was allowed.

RAM

Very little expense was spared in the building of the DIY NAS: 2019 Edition, but in building it I intentionally saved a few dollars by going with the bare minimum in recommended RAM. In fact, had I experienced any difficulty in the benchmarking of the NAS, I was ready to buy more RAM and talk about both of those decisions.

In making sure this year’s DIY NAS was more bananas than the prior year’s significantly upgrading the RAM was a no-brainer for me. I picked two 16GB DIMMs of DDR4 2666MHz PC4-21300 Unbuffered ECC RAM (specs) for the DIY NAS: 2020 Edition. A total of 32 GB of RAM would be sufficient for the needs of this year’s DIY NAS build, although I would advise the virtual machine enthusiast to consider more, depending on the number and workload of virtual machines they plan to run.

In this year’s DIY NAS, I’d also think that RAM would be one of the places where you’d see some opportunity to find savings. With my prior builds, especially the EconoNAS builds, I’ve been routinely pleased with how everything performs when using RAM that’s at the minimum side of the hardware recommendations.

Host Bus Adapter and Cables

My ideal motherboard would’ve had enough onboard controllers to support 12 SATA devices and two M.2 SSDs. My ideal motherboard likely doesn’t exist or comes with such a price tag, that I’d never even consider it. Rather than try and find that motherboard, I opted to add a host bus adapter (HBA) to add support for the additional devices that I wanted the DIY NAS: 2020 Edition to support. I chose an IBM M1015 (specs) to add those additional drives. The IBM M1015 is widely recommended for use with FreeNAS/TrueNAS, once you reflash its firmware, and adds support for an additional 8 SATA devices.

As always is the case, neither the motherboard nor the case includes enough SATA cables to support all of the drive bays. I complemented the standard SATA cables that are shipped with the motherboard with two 3-packs of 18” SATA3 cables with locking latches. But because I’d purchased the IBM M1015, I’d need more cables than just the extra SATA cables.

Because of the drive backplane inside the SilverStone CS381 and the IBM M1015, an additional type of cable was needed. Effectively, the cable needs to connect from the two SFF-8087 Mini-SAS ports on the IBM M1015 to the two SFF-8643 Mini-SAS ports on the drive backplane inside the SilverStone CS381 case.

When everything was all said and done, SilverStone CS381’s 8 external bays would be handled by the IBM M1015and the 4 internal bays would be handled by the SATA controller included on the ASRock X570M Pro4.

Graphics

One of the drawbacks of choosing from the tippy top of AMD’s CPU offerings is that you lose the integrated graphics options. While I’m a big fan of integrated graphics in building DIY NAS builds, none of AMD’s compelling CPUs support the integrated graphics. I went online and found the least expensive low-profile PCI-e GPU that I could find, the MSI Gaming GeForce GT 710 1GD3H LPV1 (specs).

I picked the MSI Gaming GeForce GT 710 1GD3H LPV1 so that someone could follow along at home and build their own DIY NAS. If I were building this for myself, I’d strongly consider rummaging around my spare parts bin. Or potentially, just borrowing a GPU from another machine, getting it up and running, and then running the NAS headless indefinitely into the future.

Storage

TrueNAS CORE Drives

The constant from my very first DIY NAS build to the DIY NAS: 2019 Edition has been my use of the SanDisk Fit and Cruzer Fit flash drives to hold the FreeNAS/TrueNAS OS. In my own NAS, I’ve mirrored the USB boot device and have recommended others do the same for years.

In building a ridiculous NAS, it seemed like I should wade into unchartered territory and consider something a bit more stable and durable than a USB flash drive. For this year’s DIY NAS, I decided I’d pick a pair of Corsair Force Series MP500 120GB M.2 SSDs (specs) with the intention of mirroring the OS across both of the SSDs like I’ve done for years on my trusty USB flash drives.

Update (03/14/21): The 120GB Corsair Force drive has been pretty hard for some to find. The price of the 240GB variant the Corsair Force Series MP500 240GB is cheaper than what I paid for the 120GB version and seems like it would be a fine substituion for those of you wanting to use the DIY NAS: 2020 Edition as a blueprint. I’ve updated the parts list below to reflect the 240GB version.

Final Parts List

Component Part Name Count Cost
Motherboard ASRock X570M Pro4 specs 1 $169.99
CPU AMD Ryzen 9 3900X specs 1 $429.99
CPU Cooler CRYORIG C7 Cu specs 1 $101.00
Memory Crucial 16GB DDR4 DIMM 2666 MHz / PC4-21300 ECC (CT16G4WFD8266) specs 2 $87.98
Case SilverStone Technology CS381B specs 1 $349.99
Host Bus Adapter IBM Serveraid M1015 SAS/SATA Controller 46M0831 specs 1 $70.96
Power Supply be quiet! BN639 600W SFX L Power Supply specs 1 $119.90
OS Drive Corsair Force Series MP500 120GB specs 2 $109.00
OS Drive Corsair Force Series MP500 240GB specs 2 $99.99
SATA Cable BENEI SATA3 18” Straight-through Cable with Locking Latch (3 pack) N/A 2 $6.99
SAS Cable Internal Mini SAS SFF-8087 to Mini SAS High Density HD SFF-8643 N/A 2 $13.30
GPU MSI GT 710 1GD3H LPV1 specs 1 $69.94
TOTAL: $1,728.29



NAS Hard Disk Drives

For the majority of DIY NAS builders, the most expensive component that you’ll wind up buying is the hard disk drives. More importantly, how much storage a DIY NAS builder needs and how much redundancy they need are both very personal decisions. Because of these factors, I’ve decided to stop buying hard drives for each year’s DIY NAS build and instead make some recommendations, instead.

Here are few tips and considerations that I have to aid picking out hard drives for your DIY NAS

  1. Quanity how much data you need to store and how quickly you accrue additional data.
  2. Assume that you’ll be replacing drives at or near the end of their warranty.
  3. With FreeNAS/TrueNAS, growing your array is easiest by replacing smaller drives with bigger ones.
  4. Decide how much redundancy you want within your array (Note: Brian strongly recommends at least 2 drives’ worth of redundancy!)
  5. Buy drives from different manufacturers and/or vendors to try and maximize the chance that your different drives came from different batches.

Going through each of the above should give you an idea of how much storage capacity your array should need at the end of the hard drives’ warranty period.

Please also keep in mind that I have zero qualms about putting consumer-grade hard drives into a DIY NAS. Most people who find themselves looking at my DIY NAS builds are probably already storing all of their data on consumer-grade hard drives. It’s important to remember that the “I” in RAID originally stood for inexpensive, and it’s indisputable that consumer-grade hard drives are usually the best value when it comes to price per terabyte.

However, it’s also worth understanding and doing a bit of research comparing and contrasting the both shielded magnetic recording (SMR) and perpendicular magnetic recording (PMR)—sometimes also known as conventional magnetic recording (CMR)—technologies used in hard drives. SMR drives achieve higher data density, but because data is laid down much like shingles on your roof, there’s a substantial performance decrease when certain tracks of data are written. The tracks being changed plus its neighboring tracks need to be read and rewritten.

I won’t begin to proclaim to full understand it, but the real world application here is that ZFS doesn’t necessarily play nicely with all SMR drives. This recently came to a head when Western Digital sneakily started using SMR in its Red drives. In making suggestions for hard drives in NAS builds using ZFS, I tend to suggest CMR drives.

Hardware Assembly, BIOS Configuration, and Burn-In

Assembly

Putting the DIY NAS: 2020 Edition together was a rather straightforward event. I spent a couple hours over five different nights and had everything put together without too much frustration. The most difficult part of the assembly was easing the motherboard into the case, thanks to the sheer weight of the CRYORIG C7 Cu. Between the heft of the CRYORIG C7 Cu and the horizontal support pieces connecting the SilverStone CS381, I had to incrementally inch the motherboard onto the case’s standoffs to get it mounted and aligned.

Reflashing the IBM M1015

Reflashing the IBM M1015 was by far the most challenging part of assembling and configuring the DIY NAS: 2020 Edition. The IBM M1015 has been around for a really long time and has been a go-to choice for DIY NAS builders for a very long time. The one problem with the card is that it’s recommended that the card is reflashed with a different firmware to put it into IT mode—especially if you’re using ZFS.

There’s no shortage of “how-to” guides on getting this done. When I did my research and bought the card, I thought it’d be no big deal and something I’d be done with in a matter of minutes, but it actually wound up being a bigger hassle than that.

At first, I read through a few guides:

  1. /r/DataHoarder: Flashing an IBM M1015 to IT mode
  2. How-to: Flash LSI 9211-8i using EFI shell
  3. ServeTheHome: IBM ServeRAID M1015 Part 4: Cross flashing to a LSI9211-8i in IT or IR mode

The problem with all of these guides is that the content has been around for so long that a lot of it has become stale. Links are dead, technologies have changed, and weird manufacturer-specific incompatibilities made working through any of these guides impossible!

Basically, I wound up using a mish-mash of steps from all of these guides to bungle my way through what was needed for the hardware that I picked out. I’m not going to try and reinvent the wheel and add my own guide to make the waters more murky. But here are the important parts that I learned:

  1. Made a FreeDOS-bootable USB disk using the utility Rufus, and formatted it FAT32
  2. Extracted the various utilities (DOS and EFI) and firmwares to the USB disk.
  3. Used the v1 (important) version of the Tianocore EDK2 Shell_Full.efi, renamed it to Bootx64.efi, and placed it in the /efi/boot path on the USB drive.
  4. Used Legacy-mode to boot into FreeDOS on the USB drive to perform the steps using the megacli.exe and megarec.exe executables to preserve the SAS ID and clear the HBA’s memory.
  5. Rebooted and booted from the USB drive using UEFI-mode.
  6. Ran the steps using the sas2flash.efi utility for flashing the firmware and to restore the SAS ID.

BIOS Configuration

Over the years, there’s been a handful of critical BIOS settings that have wound up being a game-changer for the different DIY NAS builds, and I’ve always made sure to capture those kinds of changes as part of these blogs. However, the DIY NAS: 2020 Edition proved to be rather simple.

The only thing I changed in the BIOS was to juggle the boot order of the devices. All I did was set the NVMe drive to be the primary boot device, but then used the function key for anytime I needed to boot from the USB drive when installing TrueNAS CORE.

Burn-In

In my rush to get the DIY NAS: 2020 Edition into my rear view mirror, I opted to run Memtest86+ overnight using the default configuration. My two concerns in building my DIY NAS machines is flaky hardware and poor installation. Running Memtest86+ overnight completed 5 passes without any problems, and I rebooted into the BIOS to check the system temperatures.


The next morning, there’d been zero errors captured by Memtest86+ and after rebooting into the BIOS, I didn’t have any concerns with any of the temperatures that were reported within.

TrueNAS Installation

Way back in 2012, I was disappointed in the information that was out there regarding FreeNAS. In the time since, lots of people (me included, I hope!) have shared their experiences setting up FreeNAS/TrueNAS CORE. More importantly, the content that iXsystems has created and shared is quite helpful.

It seemed inefficient to try and recreate the same content, especially when iXsystems has done such a good job with theirs. Please take a look at How to Set up and Install TrueNAS CORE and check out the FreeNAS and TrueNAS Youtube channel.

Benchmarks

When I started building DIY NAS systems, I was particular interested in the throughput and power consumption of my DIY NAS machines. As time has gone by over the years, I’ve learned a couple things:

  1. Network is your first bottleneck: Year after year, nearly every single DIY NAS I’ve built has easily saturated the Gigabit network interface that the overwhelming majority of our computers are connected with.
  2. Power consumption depends on usage: The biggest single power-consuming component the DIY NAS has is the CPU (about 105W), but it’s important to consider that a typical 7200 RPM hard drive uses up to 25 watts. I put a lot of effort into trying to test and gather the same data from every DIY NAS build that I build and blog about, but those tests don’t really reflect how I use my own DIY NAS, and, more importantly, probably don’t reflect how you’ll wind up using your own DIY NAS.

Nevertheless, it’s still fun to grab all of the video I’ve recorded for the DIY NAS: 2020 Edition, copy it over to the NAS, and see it saturate the gigabit network interface on the NAS! Mission accomplished!

TrueNAS CORE

I love playing with the new hardware—it’s the most fun part of any DIY NAS build that I wind up doing. But getting a chance to evaluate the latest version of TrueNAS Core (formerly known as FreeNAS) is a huge perk! I’m always very reluctant to make changes to my own NAS, since it has become my primary place to store data. Having a sandbox machine to evaluate the latest and greatest offering is a huge value to me.

For the first time ever, I wished that I had a stopwatch running. From the time that I plugged in the USB drive with the TrueNAS CORE installer ISO on it and turned on the DIY NAS: 2020 Edition to the point where I was copying files over to the NAS itself was definitely under 30 minutes. I was surprised at how smoothly it went.

The release notes for TrueNAS CORE 12 contains a few items which have me intrigued. I’m particularly interested where they state “Virtually every area of the platform has been updated and includes some major performance improvements, including SMB, iSCSI, ZFS and more.” But on top of that, the polish and refinement of the TrueNAS interface is a nice upgrade in itself.

It is probably worth pointing out that along the way, I was asked this in Twitter by @JonathonMoore:


Similar to what Jonathon reported, I also found that TrueNAS CORE’s reporting wasn’t working particularly well. I didn’t see any of the graphs get populated with data and when I tried clicking around to the other categories of reports, it didn’t seem to have any effect in the TrueNAS interface. If reporting and monitoring are important to you, you may want to wait for what’s coming next.

Conclusion

If you thought I went overboard when I built the DIY NAS: 2019 Edition, then you’re definitely going to think that I completely overdid the DIY NAS: 2020 Edition. If this is what you think, I agree with you! This year’s DIY NAS build is going to go down as a missed opportunity for a course correction by returning to a more pragmatic approach to the DIY NAS.

Instead, I outdid last year’s DIY NAS build in nearly every regard—especially the price:

  • AMD Ryzen 3900X CPU (12 cores up to 4.6GHz)
  • 32GB DDR4 ECC RAM
  • 8 hot-swap drive bays.
  • TrueNAS CORE 12.0 (mirrored on two 120GB NVMe SSDs)
  • A price tag of nearly $1,700 (and climbing!)

This machine is ultimately much more of homelab server than it is a NAS. It has plenty of potential to handle quite a bit of computing responsibilities beyond network-attached-storage.

If you’re thinking that this is just too much money to spend on a network-attached-storage device, I agree with you. If you’re thinking that this is a top-notch homelab server to run your NAS (among other things) with a price tag to match it, I agree with you too! The AMD Ryzen 9 3900X did not break a sweat in anything that I asked it to do. There’s a tremendous amount of room to grow in to in the DIY NAS: 2020 Edition.

The DIY NAS: EconoNAS 2020 blogbut no build—shortly followed the publishing of this over-the-top NAS. Among the things I liked most about the DIY NAS: 2020 Edition was its use of AM4 CPUs and the wide variety of CPUs that it could support. I tried to leverage that flexibility in picking out components for the EconoNAS.

What do you all think of the DIY NAS: 2020 Edition? Is it way too overboard for you, or do you think there’s a ton of potential and you’re excited to build your own little data center inside it? I’d love to hear in the comments below!

Giveaway

Update (01/01/2021): I rang in the New Year by picking a winner in the giveaway of the DIY NAS: 2020 Edition! I hope that everyone joins me in congratulating Matt H. of Orlando, Florida for winning the TrueNASGiveaway! Matt’s visitation of my Youtube page (I hope he clicked like and subscribe!) in early December is the entry that wound up turning him into the winner of the DIY NAS. Altogether, 1,636 people combined for 7,893 entries into this year’s giveaway. Thanks to everyone for making the giveaway a big success!

DIY NAS: 2020 Edition