Introduction to Crash Dump Analysis and the SunOS Kernel (ST-375)

Who Can Benefit

Back to top

Prerequisites

• Administer the Solaris OS on a Sun server
• Understand basic operating system concepts
• Use a World Wide Web (WWW) browser, such as the Netscape browser
• Manipulate files and directories on a UNIX file system
• Be familiar with fault analysis and problem-solving techniques
• Access the messages file
• Boot a Solaris OE machine
• Modify OpenBoot PROM (OBP) variables
• Use the SunSolve(SM) program effectively
• Have a SunSolve account to access other than the free collections
• Modify swap partition sizes
• Have some familiarity with C programming concepts and syntax

Courses and reading to help students obtain these prerequisites include:

• SA-239: Intermediate System Administration for the Solaris 9 Operating System
• SA-299: Advanced System Administration for the Solaris 9 Operating System
• The book Panic! by Chris Drake and Kimberly Brown

Back to top

Skills Gained

• Distinguish software-caused system crashes from hardware-caused system crashes
• Retrieve initial analysis information from a crash dump using Solaris CAT, ACT, the adb debugger, and the mdb debugger
• Distinguish among unique panic string, bad trap, and hang crash dumps
• Identify appropriate SunSolve search strings for unique panic string and bad trap crash dumps
• Use the SunSolve program to find bugs and cases related to the crash dumps or error messages
• Given a C structure declaration, describe how the data structure is stored in memory and how it looks when displayed by Solaris CAT, the adb debugger, and the mdb debugger
• Describe the major kernel subsystems and use Solaris CAT, the adb debugger, and the mdb debugger to display the subsystems' related data structures

Back to top

Related Courses

Before:

• Intermediate System Administration for the Solaris 9 Operating System (SA-239)
• Advanced System Administration for the Solaris 9 Operating System (SA-299)

After:

• Solaris 9 Operating System Internals (SP-365)
• Advanced Crash Dump Analysis and SPARC Architecture (ST-475)

Back to top

Course Content

Module 1 - Gathering Information

• Obtain system environment information
• Collect system crash data
• Determine if a system is hung
• Obtain a crash dump from a hung system
• Force crash dumps for hard hangs
• Describe the mechanism and system resources necessary to create crash dumps
• Describe how to configure a system to save crash dumps
• Troubleshoot crash dump failures
• Describe and run SunExplorer software
• Describe system information used to analyze a system crash

Module 2 - Analyzing Information

• Differentiate between software and hardware problems
• Interpret the output from the prtdiag utility
• Interpret the output from the Sun Explorer software
• Analyze patch information
• Describe the types of device names and their components
• Display the device tree
• Interpret information in the messages file
• Interpret memory error messages

Module 3 - Understanding Data Structures

• Describe how a C language structure looks in memory
• Describe how a pointer is used in the C language
• Describe linked lists
• Describe stacks
• Describe hashing

Module 4 - Analyzing System Crash Dumps

• Define common terms used in describing information in crash dumps
• Describe the information used for initial crash dump analysis
• Describe information sources within the SunSolve program that are useful for resolving system crashes
• Describe the type of information available in bug reports
• Use the SunSolve program to locate technical information, bugs, and patches related to system crashes
• Appropriately use the kenv tool
• Use Automated Crash Tool (ACT) to obtain initial crash dump information
• Use the adb debugger to obtain initial crash dump information
• Use the Modular Debugger (MDB) to obtain initial crash dump information
• Use Solaris Crash Analysis Tool (CAT) to obtain initial crash dump information

Module 5 - Introducing Kernel Features and Organization

• Describe the advantages of a modular kernel
• Define the difference between the SunOS software package and the Solaris OS package
• Describe the location of kernel modules on disk
• Describe the components of the SunOS software
• Describe user processes
• Describe interrupts
• Describe multithreading
• Describe the location of structure definitions

Module 6 - Exploring Kernel Services

• Describe the major kernel services
• Describe what traps are and how they are processed
• Describe how system calls are processed
• Describe what synchronization and protection are and why they are needed
• Describe the Solaris OS locking mechanisms
• Describe signals and how they are processed
• Describe how interrupts of various levels are processed
• Describe kernel clock handling
• Describe what callouts are and how they are stored and processed
• Describe what cyclics are and what parts of the kernel are clients of this service

Module 7 - Examining the Process Subsystem

• Draw an image of a running multithreaded process
• Describe multithreading in the kernel and processes
• Identify the structures used by the kernel to support a process
• Describe the process life cycle, including thread creation
• Describe how threads are scheduled
• Describe priority inversion and how the kernel resolves the problem
• Describe how time stamps are used in the kernel
• Describe how the kernel controls processors

Module 8 - Examining the Memory Subsystem

• Describe memory management issues from the kernel perspective
• Describe paged memory
• Identify the process structures used in address translation
• Describe anonymous memory
• Describe kernel page structures and their use
• Describe the MMU and the HAT layer
• Describe the kernel address space
• Describe kernel memory allocation
• Describe paging and swapping

Module 9 - Examining the File and I/O Subsystem

• Describe the user's view of the UNIX file system
• Describe the basic file system structure
• Describe the virtual file system framework
• Describe how file system mounting and path name resolution are accomplished
• Describe the use of the Directory Name Lookup Cache (DNLC)
• Describe the UFS file system specific data structures
• Describe process file management
• Describe the use of the specfs file system
• Describe the kernel device tree
• Draw a simple stream and explain its elements

Back to top

Browse Other Course Topic Areas