in SunOS and Solaris

setting up Solaris zones

I promised to follow up on the last article about Solaris Logical Volume Manager with one about setting up Solaris zones, so here it is.

For those of you not in the know, Solaris zones (or containers; the terms are used interchangeably) is Sun’s virtualization technology, similar to Microsoft Virtual Server or VMWare‘s products. However, the “guests” (or “non-global zones” in Solaris-speak) must also be Solaris, and effectively run the identical base system as the “host” (or “global zone”). This is quite similar to the way FreeBSD’s jails work.

Sun is pushing the zone technology very hard these days, due to virtualization technology being the hot topic in IT at the moment. Solaris Zones do have some interesting advantages over even FreeBSD jails, namely:

  • patches applied in global zone are automatically applied to the non-global zones (for the most part), easing maintenance;
  • ability to share the pkgdb from the global zone to the non-global zones;
  • ability to easily loopback-mount global zone filesystems from within non-global zones;
  • ability to do some resource control (CPU shares only) upon the non-global zones

I predict that Sun engineers are working very hard on adding more knobs to the last item, so that you’ll eventually be able to control how much swap, RAM, etc. that the non-global zones are using.

The two main commands one uses to configure and install zones are zonecfg(1M) and zoneadm(1M). The way I am partitioning this new Sun Fire v210 server is as follows:

zone purpose
zn00 DB2 master instance
zn01 WebSphere 6.0 (WAS6) instance 1
zn02 WAS6 instance 2
zn03 WAS6 instance 3

(Yes, I know I will probably need some more RAM for this machine!)

All the zones are being stored under /opt/zones/zone-tag.

Alrighty, so let’s configure a zone. zonecfg(1M) seems to be a tree-like interactive interpreter similar to Cisco IOS, but you can pass it a command file to execute as well. Let’s set up a command file:

create -b
set zonepath=/opt/zones/zn00
set autoboot=true
add inherit-pkg-dir
set dir=/lib
end
add inherit-pkg-dir
set dir=/sbin
end
add inherit-pkg-dir
set dir=/usr
end
add inherit-pkg-dir
set dir=/opt/sfw
end
add inherit-pkg-dir
set dir=/platform
end
add net
set address=10.0.0.7
set physical=bge1
end
add rctl
set name=zone.cpu-shares
add value (priv=privileged,limit=1,action=none)
end
add attr
set name=comment
set type=string
set value=DB2
end

From the top down, this configuration means I’m setting up a zone under /opt/zones/zn00 and that I want it booted on system startup. Underneath this metadata, there are some inherit-pkg-dir directives, which means that I want to inherit any packages installed from the global zone in those directories in this non-global zone. It’s important to note that those directories become read-only on the non-global zone.

Once this is done, configure the zone using the zonecfg(1M) command:

# zonecfg -z zn00 -f zn00.cfg

This configures the zone, but does not actually install it. “Installing” means actually setting up the system configuration within the /opt/zones/zn00 hierarchy, fixing up the vfstab(4), and so on. Let’s install the newly-configured zone:

# zoneadm -z zn00 install
Preparing to install zone <zn00>.
Creating list of files to copy from the global zone.
Copying <2591> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <977> packages on the zone.
Initialized <977> packages on zone.
Zone is initialized.
Installation of these packages generated warnings:
The file contains a log of the zone installation.

Usually I create a sysidcfg file at this point to avoid having to interactively configure the zone when it boots up (just like you do when you install a new Solaris box). You can just copy the sysidcfg into the zone root:

# cp zn00.sysidcfg /opt/zones/zn00/root/etc/sysidcfg

Then, you can boot up the zone:

# zoneadm -z zn00 boot

Attach to the console with zlogin(1) and you can watch the zone booting, just like a regular Solaris box:

[Connected to zone 'zn00' console]
Hostname: zn00
Loading smf(5) service descriptions: 45/103
. .
etc.

That’s basically it. There are a few limitations that I’ve found with zones thus far:

  • You can’t move a zone on the filesystem after you’ve created it
  • You can’t edit many elements of the zone configuration after you’ve created it. This includes the pkg-dir directives, which can be a real pain (say, if you added all of /opt)
  • The only resources you can control in a non-global zone are CPU shares and the number of LWPs, and setting up resource limitations often requires some custom hackery
  • If you share a removable device from the global zone (e.g. /cdrom) by virtue of a add device directive, you cannot eject the removable device from the global zone without shutting down the non-global zone
  • DTrace does not work in a non-global zone
  • Zone interfaces must have hardcoded network settings (the zone cannot use DHCP)
  • Adding zones to a system really bloats the mnttab and the output of df(1) in the global zone
  • Sun really needs to find different terminology for global zone vs. non-global zone(s) because it’s just too hard to say 🙂 and also gives the impression that you can have more than one global zone, which isn’t true — the global zone would be called the “guest” under a VMWare environment, or Domain 0 under Xen

That said, the zone technology seems to be very powerful and Sun has overcome some major design problems. The one I am most impressed with is that Sun has managed to stop the global zone daemons that bind to *:[port-number] from binding to all the non-global zone aliases as well, without having to edit system startup files, etc. to tell daemons like sshd from binding only to the global zone’s IP. I wonder if they do this by ordering service startup ahead of zone startup within SMF?

More information on setting up Solaris Zones can be found in the relevant Sun documentation.