Linux ошибка kernel

Kernel Panics

A kernel panic occurs when the Linux kernel enters an unrecoverable failure state. The state typically originates from buggy hardware drivers resulting in the machine being deadlocked, non-responsive, and requiring a reboot. Just prior to deadlock, a diagnostic message is generated, consisting of: the machine state when the failure ocurred, a call trace leading to the kernel function that recognized the failure, and a listing of currently loaded modules. Thankfully, kernel panics don’t happen very often using mainline versions of the kernel—such as those supplied by the official repositories—but when they do happen, you need to know how to deal with them.

Note: Kernel panics are sometimes referred to as oops or kernel oops. While both panics and oops occur as the result of a failure state, an oops is more general in that it does not necessarily result in a deadlocked machine—sometimes the kernel can recover from an oops by killing the offending task and carrying on.

Tip: Pass the kernel parameter oops=panic at boot or write 1 to /proc/sys/kernel/panic_on_oops to force a recoverable oops to issue a panic instead. This is advisable is you are concerned about the small chance of system instability resulting from an oops recovery which may make future errors difficult to diagnose.

Examine panic message

If a kernel panic occurs very early in the boot process, you may see a message on the console containing «Kernel panic — not syncing:», but once Systemd is running, kernel messages will typically be captured and written to the system log. However, when a panic occurs, the diagnostic message output by the kernel is almost never written to the log file on disk because the machine deadlocks before system-journald gets the chance. Therefore, the only way to examine the panic message is to view it on the console as it happens (without resorting to setting up a kdump crashkernel). You can do this by booting with the following kernel parameters and attempting to reproduce the panic on tty1:

systemd.journald.forward_to_console=1 console=tty1

Tip: In the event that the panic message scrolls away too quickly to examine, try passing the kernel parameter pause_on_oops=seconds at boot.

Example scenario: bad module

It is possible to make a best guess as to what subsystem or module is causing the panic using the information in the diagnostic message. In this scenario, we have a panic on some imaginary machine during boot. Pay attention to the lines highlighted in bold:

kernel: BUG: unable to handle kernel NULL pointer dereference at (null) [1]
kernel: IP: fw_core_init+0x18/0x1000 [firewire_core] [2]
kernel: PGD 718d00067 
kernel: P4D 718d00067 
kernel: PUD 7b3611067 
kernel: PMD 0 
kernel: 
kernel: Oops: 0002 [#1] PREEMPT SMP
kernel: Modules linked in: firewire_core(+) crc_itu_t cfg80211 rfkill ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG nf_conntrack_ipv4 ... [3] 
kernel: CPU: 6 PID: 1438 Comm: modprobe Tainted: P           O    4.13.3-1-ARCH #1
kernel: Hardware name: Gigabyte Technology Co., Ltd. H97-D3H/H97-D3H-CF, BIOS F5 06/26/2014
kernel: task: ffff9c667abd9e00 task.stack: ffffb53b8db34000
kernel: RIP: 0010:fw_core_init+0x18/0x1000 [firewire_core]
kernel: RSP: 0018:ffffb53b8db37c68 EFLAGS: 00010246
kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
kernel: RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffffc16d3af4
kernel: RBP: ffffb53b8db37c70 R08: 0000000000000000 R09: ffffffffae113e95
kernel: R10: ffffe93edfdb9680 R11: 0000000000000000 R12: ffffffffc16d9000
kernel: R13: ffff9c6729bf8f60 R14: ffffffffc16d5710 R15: ffff9c6736e55840
kernel: FS:  00007f301fc80b80(0000) GS:ffff9c675dd80000(0000) knlGS:0000000000000000
kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 0000000000000000 CR3: 00000007c6456000 CR4: 00000000001406e0
kernel: Call Trace:
kernel:  do_one_initcall+0x50/0x190 [4]
kernel:  ? do_init_module+0x27/0x1f2
kernel:  do_init_module+0x5f/0x1f2
kernel:  load_module+0x23f3/0x2be0
kernel:  SYSC_init_module+0x16b/0x1a0
kernel:  ? SYSC_init_module+0x16b/0x1a0
kernel:  SyS_init_module+0xe/0x10
kernel:  entry_SYSCALL_64_fastpath+0x1a/0xa5
kernel: RIP: 0033:0x7f301f3a2a0a
kernel: RSP: 002b:00007ffcabbd1998 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
kernel: RAX: ffffffffffffffda RBX: 0000000000c85a48 RCX: 00007f301f3a2a0a
kernel: RDX: 000000000041aada RSI: 000000000001a738 RDI: 00007f301e7eb010
kernel: RBP: 0000000000c8a520 R08: 0000000000000001 R09: 0000000000000085
kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000c79208
kernel: R13: 0000000000c8b4d8 R14: 00007f301e7fffff R15: 0000000000000030
kernel: Code: <c7> 04 25 00 00 00 00 01 00 00 00 bb f4 ff ff ff e8 73 43 9c ec 48 
kernel: RIP: fw_core_init+0x18/0x1000 [firewire_core] RSP: ffffb53b8db37c68
kernel: CR2: 0000000000000000
kernel: ---[ end trace 71f4306ea1238f17 ]---
kernel: Kernel panic - not syncing: Fatal exception [5]
kernel: Kernel Offset: 0x80000000 from 0xffffffff810000000 (relocation range: 0xffffffff800000000-0xfffffffffbffffffff
kernel: ---[ end Kernel panic - not syncing: Fatal exception

• [1] Indicates the type of error that caused the panic. In this case it was a programmer bug.
• [2] Indicates that the panic happened in a function called fw_core_init in module firewire_core.
• [3] Indicates that firewire_core was the latest module to be loaded.
• [4] Indicates that the function that called function fw_core_init was do_one_initcall.
• [5] Indicates that this oops message is, in fact, a kernel panic and the system is now deadlocked.

We can surmise then, that the panic occurred during the initialization routine of module firewire_core as it was loaded. (We might assume then, that the machine’s firewire hardware is incompatible with this version of the firewire driver module due to a programmer error, and will have to wait for a new release.) In the meantime, the easiest way to get the machine running again is to prevent the module from being loaded. We can do this in one of two ways:

• If the module is being loaded during the execution of the initramfs, reboot with the kernel parameter rd.blacklist=firewire_core.
• Otherwise reboot with the kernel parameter module_blacklist=firewire_core.

Reboot into root shell and fix problem

You’ll need a root shell to make changes to the system so the panic no longer occurs. If the panic occurs on boot, there are several strategies to obtain a root shell before the machine deadlocks:

• Reboot with the kernel parameter emergency, rd.emergency, or -b to receive a prompt to login just after the root filesystem is mounted and systemd is started.

Note: At this point, the root filesystem will be mounted read-only. Execute # mount -o remount,rw / to make changes.

• Reboot with the kernel parameter rescue, rd.rescue, single, s, S, or 1 to receive a prompt to login just after local filesystems are mounted.
• Reboot with the kernel parameter systemd.debug-shell=1 to obtain a very early root shell on tty9. Switch to it with by pressing Ctrl-Alt-F9.
• Experiment by rebooting with different sets of kernel parameters to possibly disable the kernel feature that is causing the panic. Try the «old standbys» acpi=off and nolapic.

Tip: See Documentation/admin-guide/kernel-parameters.txt in the Linux kernel source tree for all options.

• As a last resort, boot with the Arch Linux Installation CD and mount the root filesystem on /mnt then execute # arch-chroot /mnt.

Disable the service or program that is causing the panic, roll-back a faulty update, or fix a configuration problem.

Introduction

Kernel panic is a system crash or a kernel crash that prevents the system from continuing its normal operation. The crash occurs when the kernel encounters an unrecoverable error or a condition it cannot handle.

Kernel panic is associated with Unix-based operating systems, such as Linux, BSD, and macOS. The error is similar to the blue screen of death (BSOD) error in Windows OS.

This article explains what kernel panic is, lists the possible causes, and helps users troubleshoot this type of error.

What is Kernel Panic?

Kernel panic is a type of boot error in Unix-based systems that prevents the system’s operating system from recovering quickly. The state occurs when a fatal error prevents the kernel from loading properly, causing the system boot to fail.

The following image is an example of kernel panic in Linux:

Kernel panic also serves as a safety measure to prevent any hardware or software damage after a major system error. It prevents further issues to the software, hardware, and memory and writes the kernel dump to disk, which technicians can debug to determine the issue.

During a kernel panic state, the OS prints different messages on the display. The messages may include information about error type, the code module running when the error occurred, and which drives were loaded.

The following image is an example of kernel panic in macOS:

Usually, no user action is required in case of kernel panic, and the issue is resolved by rebooting, either manually or the OS reboots automatically.

Kernel Panic vs. System Crash

Kernel panic and a system crash both indicate abnormal states of a system, and both may cause severe issues for the user. However, the two states refer to different aspects of system failure.

Kernel panic is a system error in Unix-based systems that happens when the kernel, the core component responsible for managing system resources, encounters an error and terminates abnormally. During kernel panic, the kernel stops all system processes and displays a diagnostic message.

The kernel panic state is often accompanied by a «panic» sound or a distinctive visual pattern to capture the user’s attention. Kernel panic is usually triggered by critical errors such as hardware failures, severe software bugs, or faulty device drivers.

On the other hand, a system crash, also known as system failure, happens when the kernel terminates software (an app or the OS) that encountered an error that prevents it from functioning properly. The error usually causes the entire system to become unresponsive or shut down abruptly.

A system crash occurs due to software bugs, hardware failure, incompatible drivers, or conflicts between different software components.

The image below depicts the BSOD error in Windows, which is similar to kernel panic in Unix-based systems:

There are multiple scenarios and factors that cause kernel panic. The most prominent causes of kernel panic are listed below:

• The kernel is unsupported on not installed properly.
• Recent patches have issues.
• The OS fails to access or write to memory.
• Various software bugs.
• Malware infected the system.
• A corrupted initramfs file (an archive file of the initial file system that is loaded to memory during startup), or if it hasn’t been created properly for the specified kernel version.
• System updates or patches that contain bugs.
• RAM or CPU failure or improper installation.
• Hard disk damage or data corruption.
• A defective or corrupted system file, such as missing partitions.
• Unsupported hardware.
• Incompatible device drivers.

How to Troubleshoot Kernel Panic

After a kernel panic event, the system restarts either automatically or requires manual restart by the user. After the restart, it is essential to troubleshoot the crash and investigate the cause of kernel panic.

The investigation usually involves reviewing the system log, examining hardware components, and running diagnostics. Kernel panic generates a log file with information that helps decipher what triggered the crash. Most operating systems prompt the user to send the log file to the OS developer, which can assist them in preventing kernel panic errors in future releases.

A single, isolated kernel panic error doesn’t mean something is wrong with the hardware or software. However, if the system goes into a state of kernel panic often, tracking down the root of the issue is highly recommended.

Follow the steps below to troubleshoot different kernel panic causes.

How to Fix Hardware Issues Causing Kernel Panic

The following steps help determine if a hardware error is causing kernel panic:

1. Check the RAM. Ensure that RAM has been installed correctly, especially if you have recently upgraded the RAM in your machine. Check if the sticks of RAM are placed firmly into the slots. If anything seems out of place, reinsert the RAM sticks.
2. Unplug Peripheral Devices. Any peripheral device attached to your system can cause an issue that leads to kernel panic. Try unplugging peripheral devices one by one to determine if any of them are causing the issue.
3. Check for Disk Errors. Every OS has a built-in disk check and repair software that helps determine if the hard drive is faulty.

How to Fix Software Issues Causing Kernel Panic

The steps below help troubleshoot kernel panic in case of software issues:

1. Update Outdated Software. Ensure all software is updated and disable any programs installed just before the kernel panics started.
2. Check On-Boot Launches. Check which programs are set to launch on boot. Disable any newly added software and check if the problem persists.
3. Check for System Updates. Download the latest updates for your operating system and device drivers. If you are running an unstable beta version of the OS, it could be the cause of kernel panic.
4. Use a Rescue Image. If you have recently made many changes to the system, consider using System Restore or Time Machine to roll back the changes to a point before the kernel panics started. Some operating systems offer rescue images by default. Boot up a rescue image to troubleshoot the issues.
5. Check Crash Log. Linux log files provide useful insight into system errors. Examine the logs and check for anything that may cause an error with the kernel.
6. Remove Third-Party Extensions. If you have installed any third-party kernel extensions, try removing them to identify if they are causing the issue.
7. Free Up Disk Space. Free disk space is necessary to ensure the smooth operation of apps and processes. If there is no free disk space on the system, try manually removing old files and unused software to minimize the possibility of future kernel panics.
8. Boot Up in Safe Mode. All Unix-based systems have a safe mode. If the system keeps going into kernel panic, try booting up in safe mode to have the opportunity to troubleshoot the issue.
9. Reinstall the OS. If none of the steps above helped, reinstall the operating system to ensure no software bugs or malware will interfere and cause kernel panics. However, make sure to back up your data before reinstalling.

Conclusion

This tutorial explained what kernel panic is, how to troubleshoot it, and the possible causes of the error. While kernel panics can be frustrating and disruptive, with proper understanding and troubleshooting, you can address them effectively and maintain a stable and reliable computing environment.

From Wikipedia, the free encyclopedia

A kernel panic (sometimes abbreviated as KP^[1]) is a safety measure taken by an operating system’s kernel upon detecting an internal fatal error in which either it is unable to safely recover or continuing to run the system would have a higher risk of major data loss. The term is largely specific to Unix and Unix-like systems. The equivalent on Microsoft Windows operating systems is a stop error, often called a «blue screen of death».

The kernel routines that handle panics, known as panic() in AT&T-derived and BSD Unix source code, are generally designed to output an error message to the console, dump an image of kernel memory to disk for post-mortem debugging, and then either wait for the system to be manually rebooted, or initiate an automatic reboot.^[2] The information provided is of a highly technical nature and aims to assist a system administrator or software developer in diagnosing the problem. Kernel panics can also be caused by errors originating outside kernel space. For example, many Unix operating systems panic if the init process, which runs in user space, terminates.^[3][4]

History[edit]

The Unix kernel maintains internal consistency and runtime correctness with assertions as the fault detection mechanism. The basic assumption is that the hardware and the software should perform correctly and a failure of an assertion results in a panic, i.e. a voluntary halt to all system activity.^[5] The kernel panic was introduced in an early version of Unix and demonstrated a major difference between the design philosophies of Unix and its predecessor Multics. Multics developer Tom van Vleck recalls a discussion of this change with Unix developer Dennis Ritchie:

I remarked to Dennis that easily half the code I was writing in Multics was error recovery code. He said, «We left all that stuff out. If there’s an error, we have this routine called panic, and when it is called, the machine crashes, and you holler down the hall, ‘Hey, reboot it.'»^[6]

The original panic() function was essentially unchanged from Fifth Edition UNIX to the VAX-based UNIX 32V and output only an error message with no other information, then dropped the system into an endless idle loop.

Source code of panic() function in V6 UNIX:^[7]

/*
 * In case console is off,
 * panicstr contains argument to last
 * call to panic.
 */
char    *panicstr;

/*
 * Panic is called on unresolvable
 * fatal errors.
 * It syncs, prints "panic: mesg" and
 * then loops.
 */
panic(s)
char *s;
{
        panicstr = s;
        update();
        printf("panic: %s\n", s);
        for(;;)
                idle();
}

As the Unix codebase was enhanced, the panic() function was also enhanced to dump various forms of debugging information to the console.

Causes[edit]

A panic may occur as a result of a hardware failure or a software bug in the operating system. In many cases, the operating system is capable of continued operation after an error has occurred. However, the system is in an unstable state and rather than risking security breaches and data corruption, the operating system stops to prevent further damage and facilitate diagnosis of the error and, in usual cases, restart.^[8]

After recompiling a kernel binary image from source code, a kernel panic while booting the resulting kernel is a common problem if the kernel was not correctly configured, compiled or installed.^[9] Add-on hardware or malfunctioning RAM could also be sources of fatal kernel errors during start up, due to incompatibility with the OS or a missing device driver.^[10] A kernel may also go into panic() if it is unable to locate a root file system.^[11] During the final stages of kernel userspace initialization, a panic is typically triggered if the spawning of init fails. A panic might also be triggered if the init process terminates, as the system would then be unusable.^[12]

The following is an implementation of the Linux kernel final initialization in kernel_init():^[13]

static int __ref kernel_init(void *unused)
{

         ...

        /*
         * We try each of these until one succeeds.
         *
         * The Bourne shell can be used instead of init if we are
         * trying to recover a really broken machine.
         */
        if (execute_command) {
                if (!run_init_process(execute_command))
                        return 0;
                pr_err("Failed to execute %s.  Attempting defaults...\n",
                        execute_command);
        }
        if (!run_init_process("/sbin/init") ||
            !run_init_process("/etc/init") ||
            !run_init_process("/bin/init") ||
            !run_init_process("/bin/sh"))
                return 0;

        panic("No init found.  Try passing init= option to kernel. "
              "See Linux Documentation/init.txt for guidance.");
}

Operating system specifics[edit]

Linux[edit]

Kernel panics appear in Linux like in other Unix-like systems, but they can also generate another kind of error condition, known as a kernel oops.^[14] In this case, the kernel normally continues to run after killing the offending process. As an oops could cause some subsystems or resources to become unavailable, they can later lead to a full kernel panic.

On Linux, a kernel panic causes keyboard LEDs to blink as a visual indication of a critical condition.^[15]

macOS[edit]

When a kernel panic occurs in Mac OS X 10.2 through 10.7, the computer displays a multilingual message informing the user that they need to reboot the system.^[16] Prior to 10.2, a more traditional Unix-style panic message was displayed; in 10.8 and later, the computer automatically reboots and displays a message after the restart. The format of the message varies from version to version:^[17]

• 10.0–10.1: The system displays text on the screen, giving details about the error, and becomes unresponsive.
• 10.2: Rolls down a black transparent curtain then displays a message on a white background informing the user that they should restart the computer. The message is shown in English, French, German and Japanese.
• 10.3–10.5: The kernel panic is almost the same as version 10.2 but the background of the error screen is black.
• 10.6–10.7: The text has been revised and now includes a Spanish translation.
• 10.8 and later: The computer becomes unresponsive before it immediately reboots. When the computer starts back up, it shows a warning message for a few seconds about the computer restarting because of a kernel panic, and then the computer restarts back up. The message now includes a Chinese translation.

Sometimes when there are five or more kernel panics within three minutes of the first one, the Mac will display a prohibitory sign for 30 seconds, and then shut down; this is known as a «recurring kernel panic».

In all versions above 10.2, the text is superimposed on a standby symbol and is not full screen. Debugging information is saved in NVRAM and written to a log file on reboot. In 10.7 there is a feature to automatically restart after a kernel panic. In some cases, on 10.2 and later, white text detailing the error may appear in addition to the standby symbol.

• 

  Mac OS X 10.0–10.1 kernel panic

• 

  Mac OS X 10.2 kernel panic

• 

  Mac OS X 10.3–10.5 kernel panic

• 

  Mac OS X 10.6 and 10.7 kernel panic

• 

  Message shown after the computer restarts because of a kernel panic in OS X 10.8 and later versions

See also[edit]

• Core dump
• Blue screen of death
• Screen of death
• Machine-check exception (MCE)
• Reliability, availability and serviceability (RAS)

References[edit]

Linux is used everywhere in the IT world. You’ve probably used Linux today, even if you didn’t realize it. If you have learned anything about Linux, then you know it is indeed a kernel. The kernel is the primary unit of the Linux operating system (OS) and is responsible for communications between a computer’s hardware and its processes.

In this article, you will learn about one situation related to the Linux kernel: The kernel panic. The term itself can make you panic, but if you have the proper knowledge, then you can remain calm. Every system admin faces this issue at least once in their career, but reinstalling the system is not the first solution you should turn to.

What is a kernel panic?

A kernel panic is one of several Linux boot issues. In basic terms, it is a situation when the kernel can’t load properly and therefore the system fails to boot. During the boot process, the kernel doesn’t load directly. Instead, initramfs loads in RAM, then it points to the kernel (vmlinuz), and then the operating system boots. If initramfs gets corrupted or deleted at this stage because of recent OS patching, updates, or other causes, then we face a kernel panic.

If we dig into the boot process more, then we encounter the Linux «chicken/egg problem.»

[ Readers also liked: Terminals, shells, consoles, and command lines ]

When a Linux system boot process starts after the Master Boot Record (MBR) step, GRUB is loaded. The kernel needs to be loaded into RAM to start the OS, but the kernel is situated on the hard disk (/boot/vmlinuz), and the hard disk is not yet mounted on /. Without mounting, no files can be accessed, even the kernel. To overcome this, first initramfs/initrd loads in RAM directly and mounts the /boot partition in read-only mode. Next, it mounts the hard disk on the / partition, and the process continues.

# ls -lrth /boot/

This process emphasizes the importance of initramfs/initrd in the Linux boot process.

Why do kernel panics occur?

Kernel panics occur:

1. If the initramfs file gets corrupted.
2. If initramfs is not created properly for the specified kernel. Every kernel version has its own corresponding initramfs.
3. If the installed kernel is not supported or not installed correctly.
4. If recent patches have some flaws.
5. If a module has been installed from online or another source, but the initrd image is not created with the latest installed module.

How to troubleshoot?

The first thing to do after seeing a kernel panic error is not to panic ,because now you are aware of the image file related to the error.

Step 1: Boot the system normally with your given kernel version.

Then you may see this error:

Press Enter or any key, and then you will see the following:

This is your kernel panic situation.

Step 2: Reboot your machine again and select the rescue prompt.

In RHEL 6 or earlier versions, we do not have this option, but in RHEL 7 and onwards, we have a built-in rescue image.

This image boots your OS normally.

Step 2.1: Go to /boot and list all files. Here you will see there is no initramfs file for your kernel, but there is an initramfs file for rescue by which you have booted your system, and another is for kdump.

The initramfs for the kernel is missing.

Step 3: You will need to create a new initramfs file that corresponds to your kernel version.

Step 3.1: First check your kernel version:

#uname -r

Step 3.2: Next, run the dracut command:

#dracut -f <initrd-image> <kernal-version>

3.3) List the /boot directory contents again. The initramfs file for the kernel is now created.

Step 4: Now, when you boot normally, your machine starts without a kernel panic error.

Step 5: There might be a situation that occurs when you boot your system with a rescue image with creating a new initramfs file where you couldn’t make a new file because it was already present.

At this point, we need to create an initramfs image with the mkinitrd command or dracut command.

Step 5.1: Check your kernel version first using the uname -r command.

Step 5.2: Run the mkinitrd command with the --force option and your kernel specification:

#mkinitrd --force <initrd-Image> <Kernel-Version>

Your initramfs file is regenerated by these short steps, and you can now start your OS without any errors.

[ Free ebook: Manage your Linux environment for success ]

Wrapping up

Now, anytime you see a kernel panic error, you will definitely not panic because you know why this error occurred and how to resolve it. This article covers one of the common Linux boot problems: kernel panic. There are so many other potential boot problems that can occur in Linux, but resolving those issues will become much less of a panic when you gain some advanced knowledge of your system.