Story of my NAS

Updated on 20/11/2020
Updated on 10/12/2020

I think every computer/networking enthusiast have to get a server, sooner or later. If it’s used for hosting Minecraft parties, web servers, file servers or whatever else, you’ll very quickly discover the charm of having such a thing. I mean you can do basically anything with it. And if you do mind paying a monthly or hourly fee to a big corporate, like I do, then you’ll wonder.. How can I get a private, owned by myself server? Well I wondered the same, and got the answer(s). So buckle up, because this is the story, of my NAS.

Why would I need such a thing?

As I’ve mentioned, you can host many, MANY things on it. From different game servers (Minecraft, Terraria, Arma3, CS:GO,…), web servers, file servers (OMV, FreeNAS), home media server (Plex, Kodi), DHCP (piHole (which is also a DNS level ad-block)), SMTP mail server, torrent server.. and the list goes on and on. So yes, there are a lot of uses for such a thing. Personally I think most people would use it as a file server, to backup their main computers and/or extended storage, media server and some nerdy people for game servers. If you’re extra nerdy, then I for examply, use it also for dedicated work/test/dev environments.

But there are some drawbacks, for sure.. right?

Yes. Yes there are, and unfortunately, they aren’t minor. First of all, you have to consider cost of electricity. This machine will probably run 24/7, so getting a 200W TDP CPU, beefy af GPU and whatever is not recommended. Usually the 105W TDP CPU is the golden maximum, where it won’t hurt that much. Then there’s noise. If you don’t want to spend money on expensive and quality fans, there will be noise. So either have a place where it won’t bother anyone, or be prepared to be annoyed by whiny noise all the time. And at last, you have to maintain it. That applies to software and hardware. That means updating OSes, packages, software, dusting it, replacing faulty hardware, etc. But if all of that seems acceptable to you, then continue reading.

The story of Rack

First Server

A few years back, my very good friend approached me with a trade offer. He somewhere got Dell Poweredge 1950, and was willing to basically give it to me, for a very, very small price (I think it was around $20). So naturally I said yes, knowing next to nothing about that particular server. As it turned out, it had a quad-core CPU with 4GB of DDR2 RAM on socket 771. So pretty old, but still usable. But the major problem was the fans. The server is 1U form factor (the smallest ones), thus it used small fans with high airflow. That means a lot of RPMs, and a lot of noise. I mean A. LOT. OF. NOISE. We could barely hear each other, standing next to ourselves. So no, this was quickly discarded.

Second server

This triggered a need for something quiet. So from him, I’ve got (for free) a nice 2U case with 450W 80+Gold power supply. Great starting point. So I scavenged marketplaces, and eventually found nice SuperMicro motherboard on socket 775. It came with 4GB of RAM and a Pentium D CPU. I was originally planning on swapping the CPU from the Dell, but after few failed attempts I figured out that the maximum FSB frequency of the SuperMicro motherboard was too small for the CPU. So I was stuck at the Pentium D, because I was unable to find any good Xeon on 775, somehow. This was painful, but the server managed to run for a good 2 months, running just a single Minecraft server. I mean, the CPU was basically pinched at 100%, but it was pretty quiet (I’ve bought some random low-profile cooler that was reviewed as very quiet, which was. Shoutout Raijintek). But having a big-ass server laying on your desk was pretty annoying. So I had to do something about that. And here comes

The Rack

Yep. I bought a rack. Not just any. Wait for it. I wanted a rack, because a few weeks prior, my switch died and I’ve stumbled upon an extremely good deal for a rackmounted one. So at this point I was like “well why the heck not”. I scoured marketplaces for I think 2 months, before I’ve found that some company on the other side of town was upgrading, and selling their servers and racks. One in particular was priced at just $15. I immediately jumped on it, called the person, made a deal. And that was.. a mistake. I called a friend with a car, he picked me up, and we went there. The rack I bought weight literally 135kg and it was 42U. 42U goddamn rack. These racks are used in data centres. I should’ve looked better at the pics. Well it was too late, the person I was buying from had already left. And so here we were, with a small hatchback and a rack almost as big as a car. We had to pay the doorman to keep it for a bit, while we went back to town, bought a power drill and some drill bits, came back, dismantled the whole thing, stripped almost all of the cars interior, somehow put it all in, drove all the way back, assembled it and the car. What originally was planned as a 2-3hr trip turned out to be whole day adventure. But it was worth it, and now I had a giant rack that I couldn’t put anywhere, because it was massive. Eventually we found a place, in my room, in between tables, as you can see on the picture.
rack in place
So this was functional, non-problematic, and pretty quiet. As the time passed by, I’ve bought some random stuff into the rack, such as a 17” monitor, keyboard, some cable management holders and whatnot. It was decent, and it was running perfectly for about 3 months.
Then, we had to move.
Into a flat, where we can barely fit. So a huge ass rack was not possible. At this point, I was certain I’ll sell it. I mean, these racks go for around $150, so I would make a ton of money from it. But my friend (the one that helped me take it home) said they could use a rack in their house. So I lend it to him for unknown amount of time. They are using it for servers and 3D printer and stuff. So it’s not sitting somewhere, collecting dust and getting rusty. Anyways, moving on.

Buying 12 servers is unnecessary.. trust me

While scavenging marketplaces, I’ve found a guy selling the exact servers I’ve got from my friend. For $80, a Dell Poweredge 1950. Well, 12 of them. He didn’t say if it was price for one or for all, so for the lulz, I emailed him with that exact question. He replied that it was the price for all of them. One Poweredge, even that old, goes for around $30-$50 a piece, depending on config. And he was selling 4 top-spec ones, 2 almost top spec and 4 mixed. For a $80 total. Somehow I gathered $80, and emailed him that he got a deal. Only problem was that he lived about a 2hr drive from where I lived. And the night before, we got totally plastered and went to sleep at 4AM. Great time, when we had to be at his place at 12. So after very not good 4 hours of sleep, we got up and drive there. I had a license at this time, and I was driving the whole time. We got there, completely tired. Somehow loaded the servers (mind you, each weight around 30kg), payed and went back. On the way back, it started snowing, and I was really starting to feel lack of sleep. But somehow, we made it without killing anyone, unloaded the servers, overloaded the elevator and there was I, with a 13 identical outdated servers. They were stacked by my bed for 2 months, and then the very same friend that now have my rack offered to store them in his storage unit. So now they are collecting dust. I mean, they are not bad, but are way too loud and consume a lot of power. So yea.. But I sold 2 of them and basically got my money back, so it was worth it. If you for some reason want some Poweredges 1950 and you are from Czech Republic, hit me up :D

this was a big, big mistake

Third (final) server

Now I didn’t had a server, and I was dedicated to have one, whatever it takes. I just wanted that Minecraft server, but not only that. I discovered Plex, which everyone in the family agreed to use (which I’m surprised by, holds). And also a NAS/backup unit. So I contacted a few tech friends, and I’ve got hooked up with a case, power supply, some fans and drives. I recently upgraded my main rig, so I had my old components at disposal. I used them as a first NAS build (i5 4670, 8GB DDR3, 80GB boot HDD, 1TB storage, ASUS Rampage VI Gene). But my brothers PC was getting outdated, so I swapped them around. Now my NAS/server was basically an electrical heater. running FX-8350, MSI 970 Gaming MOBO and the same 8GB of RAM. This was fine, having 8 physical cores, but it was heating up way too much and drawing at around 240W at almost idle. So nope. This wasn’t an option. Again, while scavenging interwebs, I’ve found a guy selling pair of 3TB drives for 3$ a piece. So I instabought them, they are pretty worn and have few reallocated sectors, but I’ve put them in mirrored RAID. They are still working today. I’ve started looking at something else, preferably not 775 or AM3 as I wanted to use it more extensively and 4 cores was just not enough. I was looking at socket 1366, but motherboards are too expensive. Eventually I said frick it, I’ll go on mainstream platform. Z270 mobos are pretty cheap, CPUs are getting cheaper by day and there are plenty of them. At the same day, I found a guy selling ASUS X99-S motherboard (yes, X99 motherboard, probably the most expensive mainstream platform) with 4C xeon for just $120. I couldn’t resist. This motherboard rocks a 2011-v3 socket support, which created a huge possibility to upgrade later, up to 18 cores. It supports 3200MHz DDR4 RAM, up to quad channel. Awesome. So I’ve bought that. With that, I also upgraded the case (previous could hold only 3 drives, and I’ve got some more meanwhile) and cooling. Friend of mine gave me NH-D15 for free (he uses me as taxi, so instead of money, he gave me the cooler), so that was a massive upgrade from the random noisy cooler I had before.

What’s next?

Here is the full specs of my current NAS/Server:

  • Intel Xeon E5-1620 v3
  • Noctua NH-D15
  • ASUS X99-S
  • 128 Kingston DDR4 @ 2133MHz (SC)
  • BlueStorm 450W PSU
  • unknown Thermaltake case
  • 120GB Kingston A400 SSD (boot drive)
  • 2x 3TB Seagate ST3000VX (RAID 0 (stripe)*, file server)
  • 1TB WD Blue (VMs, personal stuff)
  • this was previously in RAID1 (mirror). I changed it to RAID0 (striped) because I capped the 3TB maximum pretty quickly, and also, I’m expecting of getting a pair of new 3TB drives, so I’ll probably hook them into RAID10.

I’m planning on getting a new CPU. As I said, I’m running a lot of VMs and stuff and 4 cores is just.. not enough. I have a deal on Xeon E5-2650 v3, which is 10C/20T CPU, at even lower TDP. Also, I’m planning on expanding storage to at least 6TB (probably by getting another pair of 3TB drives and creating RAID10) with highspeed SSD cache (maybe 1TB) and of course, 10Gbit NIC. This would be only for my PC, as I’m using the server the most, and transfer a lot of data. Gigabit is fine but slow, and when you can have 10Gbit.. why not. And I want to get Quadro GPU, because 1. I was quoted for making a Blender render machine for a friend (which will cover electricity bills) and 2. NVENC/NVDEC for Plex. I was looking at Quadro K4000/4200, those are pretty affordable and with good performance. I’ll let you know how the upgrades goes.

Software

On first machine, I was running OpenMediaVault, because it was used solely for storage. But because it was discarded quickly, and I’ve found that OMV doesn’t support virtualization (at least at the time), I’ve moved onto Ubuntu Server for my second server. This was OK, as I could just screen it. For Minecraft server and vsftpd, that was plenty. In between second and third iteration of my server, I’ve found out about FreeNAS. I was a bit scared because it was based on BSD and I’ve never worked with it before, but I gave it a shot in third iteration. And oh my. That was probably one of best choices in my life. It runs like a butter, supports VMs, Jails (something like Docker containers but BSD based, i think), real-time monitoring, reports, CRON jobs, plugins, VNC, multiple services (SMB, AFP, DNS, FTP, NFS, SSH,…) and a lot, lot more. Also it have nice UI, and recently it merged with its enterprise part and formed TrueNAS Core, which is free like freeNAS but migrates some enterprise functions which were paid to the free version. I’ve experimented with it for a bit, but I’m still at the better supported FreeNAS.

VMs

As for virtuals, I have several running. Here a quick rundown:

  • piHole
    That is DNS level adblocker and DHCP server. This autostarts, and have to run, otherwise my internet wouldn’t work, as this replaces the DHCP server in my router. It’s a great project and I definitely recommend it.
  • Arma 3 server
    For my brother and his friend. They wanted to play modded Arma, but no one of them have strong enough PC. So I hosted it for them. Sicne I dont’t have public IP address, I’ve used ZeroTier VPN for tunneling.
  • Bunch of Minecraft servers
    I lend one to a friend, and on second we play daily.
  • WiFi FTP autoinserter
    Related to my other project about WiFi vulnerabilities, I’ve created this VM which runs some helper scripts for the project.
  • DevVM1, 2 and 3
    Just 3 development VMs, each running different OS (Ubuntu, Arch and Debian) for developing, testing and messing around with things like Docker, Node, SMTP, custom APIs and such.
  • Plex
    Home media streaming service. Basically your local Netflix.
  • Jellyfin
    Replaced Plex, because I wanted HWENC/HWDEC for free. Really good home media streaming service. Better than Netflix (you don’ have to pay)
  • BitTorrent
    Where do you think I’m getting movies for the Plex from? Just kidding, of course all of them are from legal sources ;)
  • Guacamole
    A remote SSH/VNC/RDP webUI manager.

The ending…?

That is basically everything I have to say. You got the stories behind the hardware, current specs, plans for future and what software I’m running. I’ll obviously keep updating this every time I make an upgrade or change worth of updating the page. So be sure to visit this once a month to see the progress.

Update 20/11/2020

So it’s been a couple of weeks now, and I’ve done some.. things. First of all, I lost all of the data. You may be wondering how. Well that’s simple. FreeNAS just wasn’t cutting it, and I’ve got my hand on Quadro K2000, which supports NVENC/NVDEC. I found out I can use that with Plex, because with CPU encoding, I was able to pull out only 1-2 1080p streams at once. And FreeNAS can’t do hardware passthrough. So I decided to migrate to proxmox, which is awesome hypervisor system. Well, I exported my data zpool, idea being that I’ll install OpenMediaVault (much simpler, smaller and Debian based) and just import the pool there.. But as you pobably now know, drives are a bit old and not really reliable. I still don’t know what exactly happened, but I think one of the drives experienced massive failure while exporting the data, and stuff got corrupted.. of course, zpool was already offline so it couldn’t self-restore. Thus, after a solid day of trying, I wasn’t able to restore any of the data. Which is kinda sad, but I hadn’t had anything really important on it, so.. no big loss.

Okay, I instaleld proxmox, virtualized basically all of VMs I’ve had running before and it’s okay. Everything is working as expected, only exception being Plex, which I hadn’t had time to do yet. But the VM is running and installed, ang the GPU is showing up fine, I just need to set up Plex and somehow connect it to the network drive.

These are basically the only updates I have so far. No hardware changes yet.

Update 12/10/2020

BIG UPDATES. Or not so big but .. updates nontheless. My friend sold me 128GiB DDR4 2300MHz RAM (for 5000 czk, around 180$), so I’ve plenty of memory to do crazy things. I was thinking of getting 10Gbit NICs for a while (if you happen to be Czech and have pair of them, and you are willing to sell them for reasonable price, hmu), but I have no SSDs capable of running at 10Gbps R/W, nor enough money to build NVMe RAID. As funny as it may sound, it’ll be probably better for me to just make like 48GB ramdisk and use that as a high speed cache. Anyways. I have the RAM, nothing else changed in the hardware. I’ll be getting a new PSU and case tho very soon. I also dumped the idea of Plex as my home media center, because NVENC/NVDEC (or basically any hardware acceleration) can be only enabled if you pay for it. Which is ridiculous. So I was searching for some alternatives, and found Jellyfin. IT IS SO GOOD. It’s community-driven, open-source, extremely well supported and it just looks good and much less bloated than Plex. I’ve managed to set it up in under 30 minutes, and that’s with Nvidia drivers installation, ffmpeg configuration and configuring the media libraries inside it. Switching to NVENC was as simple as chossing that option from menu. And it actually works! It’s amazing. Fuck plex. Also I’ve set up Apache Guacamole VM. No need for it now, as all of my VMs are managed via SSH and I have putty for that on my main machine. But since I’ll be finally getting a public IP (or VPN tunnel to someone with public IP rather), it’ll be good to have, so I can manage my servers from one nice webUI.