Fast Track to Solaris 10 Adoption: Solaris Grid Containers
Performance Issues
Please click on a question below or download a pdf version.
- If I have a DB zone, an application zone, and a Web server zone, instead of running everything in the global zone, what are the performance penalties incurred from all communication going across zones via the network?
- Just a note: if anyone asks about running other operating systems inside a zone I know Sun says you can't but I have. What I did was compile up the Bochs ("box") app inside a zone. This is an x86 system emulator, and I successfully ran FreeDOS inside a zone. Now running x86 code on a SPARC system isn't the most efficient use of resources, but you can do it. I would be interested to see how it runs on a x86 system.
- When you say that all IPC is virtualized, does that mean I can allocate more shared memory to one zone vs. another, and does that amount get pulled from the total allocated to the global zone, meaning it would have to be pre-allocated to the global zone first?
- I am convinced that zones has growth potential. Are there any limitations; other than thinking of it as a VM clone, that I should be aware of?
- Where do you see customers using Solaris Grid Containers as opposed to simply using the resource management features of the Solaris 9 OS? Why?
- Do Solaris Grid Containers have any implications/benefits for massive multiplayer game networks?
- I am seeing that Solaris Grid Container technology will or could potentially be set up to be fault tolerant between containers. Any feedback on this?
- Is a zone panic isolated to the zone or does it affect the entire server?
- Let's say I share /usr with all zones. Is it possible to exclude certain directories under the /usr filesystem? I don't want to share /usr/openwin with the zones, but I need it globally.
- How does this compare to User-Mode Linux?
- Can I move a zone from one physical machine to another? If so, does it require a reboot?
- Will zones help if an application "triggers" a CPU crash (panic)?
- Does each container have separate cron daemons running so that users of each zone can have crons kick off?
- I'm surprised that the Solaris Grid Console software isn't mentioned within the answers. It has a GUI and can move Solaris Grids easily. When will it be available for the Solaris 10 OS?
- Do IP Filters (or packet filter or Sun's equivalent) work in zones to isolate them from each other?
- Does the grid container get tied to a CPU? If so, how would failover work?
- The discussion so far seems based on a single system. Can multiple systems make up a grid definition and can they interact?
- Is one able to capture and save Container configuration information so that, for example, it could be easily migrated to another physical server?
- When you say a zone "boots," does that imply that each zone is running a separate instance of the OS, like domains on your larger servers?
- The biggest weakness I've spotted on zones is that there is still a global /etc/system (for some variables). Can you comment on the future of /etc/systems if you agree that it is a weakness?
- Are the memory regions isolated per zone? Is this isolation available on both the Solaris OS and x86/AMD?
- What happens when you create two non-global zones and they both share /usr, but then you need to apply some patch because zone A needs it (maybe because it's being used for compiles and the newest compiler needs it), but you don't want to disturb zone B?
- Where can I get the list of files being copied from the global zone when a zone is created?
- What criteria would I look at to determine what apps to place in containers as compared to separate domains?
- With a grid container, are all kernel resources virtualized? For example, are the kernel IPC data structures or other fixed-size kernel data structures like the proc table?
- Can each zone be on a different subnet with its own NIC? If so, how do I add a default route for each zone within the zone?
- What if a hacker somehow gains control over the global zone? He could wipe out all zones and get access to disk and CPU. How can we be sure zoneadmd can't be broken from a zone?
- Is there a per-server or per-process limit on disk I/O requests that an application can make?
 |
Q: If I have a DB zone, an application zone, and a Web server zone, instead of running everything in the global zone, what are the performance penalties incurred from all communication going across zones via the network?
A: Cross-zone network communication is handled in the IP layer of the kernel, rather than going out over the wire, so performance shouldn't be significantly different from running the applications together in the global zone (and should be much better than running on separate systems, assuming sufficient CPU, memory, etc. is available).
Back to top
Q: Just a note: if anyone asks about running other operating systems inside a zone I know Sun says you can't but I have. What I did was compile up the Bochs ("box") app inside a zone. This is an x86 system emulator, and I successfully ran FreeDOS inside a zone. Now running x86 code on a SPARC system isn't the most efficient use of resources, but you can do it. I would be interested to see how it runs on a x86 system.
A: Sure, this certainly works. Generally we tell people "other operating systems don't run inside zones," to emphasize that zones is not actually a virtual machine technology basically, you've started your own virtual machine.
Back to top
Q: When you say that all IPC is virtualized, does that mean I can allocate more shared memory to one zone vs. another, and does that amount get pulled from the total allocated to the global zone, meaning it would have to be pre-allocated to the global zone first?
A: IPC limits can now be specified on a per-project basis, so shared memory can be allocated for individual applications without affecting the global zone. We're also working on providing per-zone limits for locked memory (including ISM) and IPC limits in general.
Back to top
Q: I am convinced that zones has growth potential. Are there any limitations; other than thinking of it as a VM clone, that I should be aware of?
A: Since I don't know anything about your environment, it's hard to know what you might need that we haven't provided. Remember, it isn't a virtual machine, so the benefits and limitations are different (don't forget that virtual machines have their own problems). I think the best thing you could do to answer your question is to give zones a spin via the Solaris Express program. Please let us know via the Zones BigAdmin Forum what is or isn't working for you.
Back to top
Q: Where do you see customers using Solaris Grid Containers as opposed to simply using the resource management features of the Solaris 9 OS? Why?
A: Solaris Grid Containers is really the combination of both the resource management features available in the Solaris 9 OS (and extended and improved in the Solaris 10 OS), as well as the isolation and security features provided by the Zones facility in the Solaris 10 OS. You can use each independently, but we feel that the combination provides the fullest solution.
Back to top
Q: Do Solaris Grid Containers have any implications/benefits for massive multiplayer game networks?
A: Sun is developing some novel new designs for multiplayer game networks. We think that zones fits well with this, as it may allow game hosting providers to create more dynamic provisioning systems and also to host multiple customers in a highly isolated fashion on the same nodes in the network.
Back to top
Q: I am seeing that Solaris Grid Container technology will or could potentially be set up to be fault tolerant between containers. Any feedback on this?
A: Containers are already fault tolerant between themselves, since application failures in one zone will not affect another zone. Even better, Zones and the new Predictive Self-Healing technology (available in the current Software Express) will be used together, so certain hardware faults may cause only one particular zone to be affected rather the whole system.
Back to top
Q: Is a zone panic isolated to the zone or does it affect the entire server?
A: Because zones don't have independent kernels, there really is no such thing as a "zone panic." If there is a bug in the (system-wide) kernel, or a catastrophic hardware fault, the kernel will panic, and all zones will be affected.
Back to top
Q: Let's say I share /usr with all zones. Is it possible to exclude certain directories under the /usr filesystem? I don't want to share /usr/openwin with the zones, but I need it globally.
A: One way of doing this as part of the zone configuration is to create an empty directory in the global zone and configure a loopback mount for the zone on top of the directory in question: add fs set dir=/usr/openwin set special=/empty set type=lofs add options ro.
Back to top
Q: How does this compare to User-Mode Linux?
A: UML creates multiple OS instances, each of which can run an application. With containers, we have a single OS instance, with isolated application environments. This results in a more powerful administrative model (we think), as well as performance advantages.
Back to top
Q: Can I move a zone from one physical machine to another? If so, does it require a reboot?
A: You can't at this time (although you can roll your own solution by having two identical zones on separate machines). We realize this is inconvenient, and we plan to improve this in the future.
Back to top
Q: Will zones help if an application "triggers" a CPU crash (panic)?
A: If the kernel panics, either due to kernel software problems or unrecoverable hardware failure, then the entire system (including all zones) will go down. On the other hand, if a hardware failure or software fault can be restricted to a single zone, only that zone will be rebooted (which happens very quickly).
Back to top
Q: Does each container have separate cron daemons running so that users of each zone can have crons kick off?
A: Yes, each container has its own cron daemon and can be configured with its own crontabs.
Back to top
Q: I'm surprised that the Solaris Grid Console software isn't mentioned within the answers. It has a GUI and can move Solaris Grids easily. When will it be available for the Solaris 10 OS?
A: Most of our docs and discussions of the Solaris 10 OS are synced to the currently available features being shipped via the builds in the Solaris Express program. Solaris Grid Console is a future capability.
Back to top
Q: Do IP Filters (or packet filter or Sun's equivalent) work in zones to isolate them from each other?
A: Not at the present time. However, the same effect can be achieved either by setting up "reject" routes from the global zone (see the route(1M) main page for details) or by setting up IPsec in the global zone and configuring it to deny traffic the IP addresses configured in the zones you are trying to separate.
Back to top
Q: Does the grid container get tied to a CPU? If so, how would failover work?
A: Yes, a container can be bound to an individual CPU (or set of CPUs). If that CPU needs to be taken offline due to hardware issues, the container will be unbound and the (global zone) administrator will be notified.
Back to top
Q: The discussion so far seems based on a single system. Can multiple systems make up a grid definition and can they interact?
A: Multiple systems can be tied together using other software in the Solaris Grid product suite, particularly the clustering software and Solaris Grid Engineer. The Solaris Grid Containers feature only applies within a single system, allowing that system to be subdivided to run multiple applications in isolation.
Back to top
Q: Is one able to capture and save Container configuration information so that, for example, it could be easily migrated to another physical server?
A: Yes, the tools for managing the container (zone) configuration allow the configuration to be written out and transferred between machines. We're also working on ways to make this easier, particularly if the contents of the zone (files, etc.) can be placed on shared storage.
Back to top
Q: When you say a zone "boots," does that imply that each zone is running a separate instance of the OS, like domains on your larger servers?
A: Each zone runs on a single version of the Solaris OS for that server; the zone contains the application processes and a small amount of OS resources required for that zone. Booting a zone essentially reboots the application(s), and it only takes seconds, not minutes.
Back to top
Q: The biggest weakness I've spotted on zones is that there is still a global /etc/system (for some variables). Can you comment on the future of /etc/systems if you agree that it is a weakness?
A: We're working on eliminating the need for tunables in /etc/system; our view is that these should either be automatically determined (without any need for administrative control) or turned into dynamically controllable parameters such as those provided by the resource controls facility. In the Solaris 10 OS, we've removed the need for the most prominent usage of /etc/system: the System V IPC (semaphore, shared memory, message passing) tunables are now either removed (if they could just be made dynamic) or are per-project resource controls. In either case, there's no need to configure these on a system-wide basis. Other /etc/system parameters still exist, but we're working on this for the future.
Back to top
Q: Are the memory regions isolated per zone? Is this isolation available on both the Solaris OS and x86/AMD?
A: We do plan to do this in a (near) future release; in fact the memory isolation will be available even without using zones. Zones can already be bound to resource pools, and in a future release you will be able to associate a memory set with a resource pool. We're also working on some other memory-related limits and controls. And yes, it will all work on all platforms.
Back to top
Q: What happens when you create two non-global zones and they both share /usr, but then you need to apply some patch because zone A needs it (maybe because it's being used for compiles and the newest compiler needs it), but you don't want to disturb zone B?
A: For the present, all zones need the same patch level for Solaris OS packages, and the patch commands will automatically keep zones in sync. Patch levels for unbundled and layered products like the compilers and Java Enterprise System can be at "different" levels in different zones.
Back to top
Q: Where can I get the list of files being copied from the global zone when a zone is created?
A: The list of files copied from the global zone come from the packaging database which can be seen in /var/sadm/install/contents. Some files are not copied those in packages marked with a SUNW_PKG_HOLLOW pkginfo(4) attribute and others (editable and volatile) are copied from an archive so they're installed in a factory default condition.
Back to top
Q: What criteria would I look at to determine what apps to place in containers as compared to separate domains?
A: The criteria I would look at are application size, the fault boundary, and level of isolation required. For application size, what are the resource requirements? If your application requires only a small amount of resources (let's say 1 CPU), zones will definitely save you money, since domains are typically at a rougher granularity (the system board). For the fault boundary and isolation, what are the consequences of a fault? A domain is going to give you hardware-level isolation. The downside is that you'll have to manage different OS instances on each domain.
Back to top
Q: With a grid container, are all kernel resources virtualized? For example, are the kernel IPC data structures or other fixed-size kernel data structures like the proc table?
A: System V IPC is virtualized. This means that two applications running in different containers can use the same IPC key without conflicts. In other cases, we've kept the same basic data structures, but filtered access; for example, searching /proc in a container will only see processes from that container, but the actual kernel data structure contains all processes. In general, we've made decisions based on the virtualization requirements and performance concerns, and reworked subsystems that seemed to require it.
Back to top
Q: Can each zone be on a different subnet with its own NIC? If so, how do I add a default route for each zone within the zone?
A: Each zone can be on its own subnet, but at the present time the global zone itself must have an IP address on each of those subnets as well. Note that the network interface in the global zone can be in a "down" state and so only the local zone with the "other" IP address on the physical interface will be using it. The global zone can add a default route tied to each such interface so non-global zones can have their own default route.
Back to top
Q: What if a hacker somehow gains control over the global zone? He could wipe out all zones and get access to disk and CPU. How can we be sure zoneadmd can't be broken from a zone?
A: This is really two questions. Obviously, if a hacker were able to gain control (superuser access) over the global zone, he could control the whole system this is part of the architecture. Security sensitive installations should be careful to protect the global zone by limiting services, using firewalls, etc. However, the overall zone design prevents intruders from being able to go from a non-global zone to the global zone. This isn't enforced by zoneadmd (which just manages the zone lifecycle); it's enforced by the same kernel subsystems that control cross-process, uid, etc., access.
Back to top
Q: Is there a per-server or per-process limit on disk I/O requests that an application can make?
A: Not at present. However, we plan on continuing to add additional resource controls, and this may include disk I/O requests. Please review the Resource Management and Zones AnswerBook resources for more info.
|