Building a Storage Server with OpenIndiana

This post describes how to build a storage server using the OpenIndiana operating system. OpenIndiana is free, open-source, and based on OpenSolaris. It includes ZFS which provides features like software RAID. OpenIndiana can also act as iSCSI target. For my setup, I’m using 1 SSD and 4 HDDs. I created 2 partitions on the SSD. One partition is for the operating system. The second partition is for the ZFS read cache.

Install OS

Download the latest version from the OpenIndiana web site. Create two partitions on the SSD for the operating system and L2ARC. Continue through the installation process. After it has completed, reboot and login as root.

Configure Network Settings

Use this command to view your network interfaces:

dladm show-phys

Configure the management network on e1000g0.

svcadm disable svc:/network/physical:nwam
svcadm enable svc:/network/physical:default
ipadm create-addr -T static -a 10.3.16.10/20 e1000g0/v4
route -p add default 10.3.16.1
echo nameserver 8.8.8.8 > /etc/resolv.conf
cp /etc/nsswitch.dns /etc/nsswitch.conf

iSCSI supports multi-path which allows for redundant physical paths and better utilization of multiple network links. Configure e1000g1 and e1000g2 and place them on different subnets.

ipadm create-addr -T static -a 10.3.0.10/24 e1000g1/v4
ipadm create-addr -T static -a 10.3.1.10/24 e1000g2/v4

Now that your network is setup, you can install updates.

pkg update

Configure iSCSI Target

Run the following commands to setup iSCSI on the server.

pkg install -v SUNWiscsit
svcadm enable -r svc:/network/iscsi/target:default
itadm create-tpg tpg1 10.3.0.10 10.3.1.10
itadm create-target -t tpg1

Add Storage Pool

Find your disks using this command:

ls /dev/dsk | grep -P "d0$"

Using the disks found in the last command, create the new storage pool with the following commands:

zpool create bigpool mirror c3t5Ad0 c3t5Bd0
zpool add bigpool mirror c3t5Cd0 c3t5Dd0

Use a SSD for read cache.

zpool add bigpool cache c5t0d0p2

Manage Volumes

Create a new volume with the following command:

zfs create -V10G bigpool/vol1
LU=`sbdadm create-lu /dev/zvol/rdsk/bigpool/vol1 | grep -P"^[0-9]" | \
awk '{print $1}'`
stmfadm add-view $LU

Monitor performance.

zpool iostat -v bigpool 2

You can grow your volumes online using the follow commands:

zfs set volsize=20G dpool/vol1
sbdadm modify-lu -s 20g /dev/zvol/rdsk/bigpool/vol1

For more information, check out these links: