Recently I got a hold of 6 x Intel DC S3610 SSD drives that I wanted to play around with, and see what performance I could get out of them on my workstation PC at home.

To give you a little background on my use case I decided to write little bit about my setup and use case. My workstation is mainly used for graphics work and map making for a project I have www.iskort.is . Creating those maps takes up a lot of resources in all categories, I always need more RAM, more CPU power, and specially Disk performance and space. For example, recently I was working on a new 3D map data files for Iceland and the dataset when uncompressed was 6TB of 3D files, and the final 3D dataset that I saved out of this raw data was approx. 400GB of Lidar “like” data. My workstation is pretty beefy, but some projects I have to run on a VM on my server that has dual 6 core CPU’s, and 288GB of RAM where I have also around 20TB of storage.

My workstation is based on a Gigabyte GA-X99-UD4 motherboard. There I have 64GB of RAM and an Intel i7 5930K CPU (3.5 GHz, overclocked to 4.4 GHz). I have a GeForce 1080 graphics card and an older GeForce 980 card in the system as well as some of the workflow I use utilizes the CUDA cores on both GPU’s. For the OS and temp files I have been using a 512GB Samsung 950 Pro NVMe M.2 based drive. I had a Mushkin Scorpion deluxe 480GB PCIe based drive for my working dataset, but recently that good old card died on me.

So back to what I have now, 6 x Intel DC S3610 SSD! I wanted to find out the best configuration of those drives, find out what RAID levels would get me the best performance, and also to see if I should try out Storage Spaces that is in Windows 10. I don’t have a dedicated hardware RAID card in my setup, and for a long time I have used the Intel Chipset Rapid Storage Technology RAID (chipset/software raid) to do stripes of 2 drives or mirrors. So far that has worked great for me. But now with 6 drives I needed to see where my bottleneck would be. – Would it be the drives, the chipset or even my old, (but pretty powerful when overclocked) CPU.

Initially I decided to test RAID 0 and see how linear the performance would be when adding more than 2 drives. I used FIO with multiple files and threads to make sure I wouldn’t cap my results on single file or single thread on the CPU.

(click on images for larger version)

The graph shows Random Read IOPS. Here I saw a clear benefit of having 2 x drives in Raid 0, and a little more benefit of having 3 drives in Raid 0. – 4 or more drives resulted in worse performance except for 4k and 8k block size.

Same trend when doing random writes. – 3 Drives would give some performance boost over 2 drives, but 4-6 drives resulted in worse performance. I then went into the BIOS, change the SATA ports to AHCI mode instead of RAID mode, and tested out using Windows Disk management software RAID. I tested out “Simple” storage spaces profile as well, as that is also based on striping as RAID 0

Here I got more performance and more linear growth. Performance up to 6 drives was showing on up to 16k block size but little was added on block size of 32K after 4 drives and on 128k block size 3-6 drives gave the same performance. Storage spaces with “Simple” profile and 6 drives were a little behind the software raid level in 4-16k block size, but minimal difference were on larger block sizes.

Same goes for random writes. More linear performance in 4-8K, but after 3 drives in 32k almost no gain.

Looking at MB/Sec graphs I noticed there were an obvious bottleneck in the system at approx. 1600MB/sec read, and 1250MB/Sec write. No matter the block site or number of drives, I could not get more throughput out of the system. My finding on this is that the X99 Chipset is at its maximum there and basically with 3 SSD drives like the Intel S3610 drives, more than 3 drives would saturate the maximum throughput of the chipset.

When looking at the average numbers below it’s clear that software raid in Windows outperforms the Intel chipset raid. Especially in the lower block sizes, where my system could deliver approx. 230.000 IOPS at 4K. At that rate my CPU was 100% busy doing those IO’s and having more powerful CPU would probably get me some more IOPS.

Since I didn’t want to run Raid 0 in production, I tested out different RAID options and also the options in Storage spaces in Windows 10 to see what would give me the best performance but also reasonable level of protection in case of a drive failure.

When using the X99 chipset Raid levels, I was able to do Raid 5 with 6 drives, but with Raid 10 I was only able to use 4 drives. I also tested Mirror with 2 drives, and 3 sets of mirrors, and then creating a Raid 0 in Windows Disk manager, to emulate Raid 10 with 6 drives. With Windows Storage spaces I created a 6 disk “Parity” drive, and a 3 Disk Parity drive. A 6 x 2Way Mirror and a 6 x 3 Way Mirror, a 2 x 2Way mirror and a 5 x 3 Way mirror.

The useable space varies greatly on those options and of course when using fewer drives I could have to options to create more volumes

On those different RAID levels I got pretty good read performance over all but as expected 2 drive mirror did not hold up agains the Raid 5 or Raid 10 options.

When looking at writes Raid 5 took a huge write penalty and that option did not look very promising.

Turning over to Windows Storage spaces, I got more options to test.

Read performance was better than the Intel Raid options on the same number of drives as before.

On write performance, both Parity options were terrible, and approx. 10 time worse than the Intel Raid 5 option. 2 Way mirror with 6 drives looked good but less than half of the write performance in 4k than the Simple profile with no redundancy but still better than 6 drive Raid 0 option on the Intel chipset RAID.

Looking at MB/Sec values, most options were capping at 1550MB/sec as before as the Different Raid 0 options.

Looking at writes MB/Sec it’s obvious that the RAID 5 options had the worst performance and 6 x 2Way Mirror were just above the “Mixed” Raid 1 / 0 Mode where both Intel Raid 1 and Windows Raid 0 were bundled togeather.

Looking again at the average numbers

Windows storage spaces were showing better performance when using all 6 drives

Again 6 x 2 Way storage spaces gave me the best average write performance.

Overall, the performance of Windows Storage spaces was a nice surprise. Previously I didn’t gave it much though as I had been using Intel chipset Raid for 2 disk configuration with almost 2 x performance boost over 1 drive. Also what storage spaces allowed me to do, is to bundle all the 6 disks into one pool, but then carve volumes that had different protection, or no protection at all, and that is exactly what ended up doing.

I had my OS drive from the Samsung 950 Pro NVMe M.2 drive, I then created a 1.3TB volume with 2 way protection for my workflow data where I would work on the mapping data files. I created a 500GB volume with “Simple” profile, where I placed all temp files, Photoshop scratch disk file and my mapping software’s temp folder. I also have an 4TB SSHD “hybrid” drive from Seagate (ST4000DX002) for archives and other stuff like my drone flight videos or images from my DSLR camera.

I hope this blog was useful for you, especially if you had plans to do RAID with the Intel chipset with more than 2 SSD drives. My suggestion is to use Windows Storage Spaces instead.

Share if you like this blogpost or write a comment below.