Broken pipe ошибка что значит

В настоящее время Python считается зрелым языком программирования, который широко используется специалистами по обработке данных и инженерами по искусственному интеллекту (ИИ) из-за его простоты и легкочитаемого синтаксиса.

В данном руководстве мы обсудим [Errno 32] Broken pipe в Python, известное сообщение об ошибке, которое мы часто видим при взаимодействии с файловой системой. Мы разберем причину ее возникновения, а также способы ее избежать и исправить в коде.

«Сломанный канал» обычно считается ошибкой IOError (сокращение от «Ошибка ввода-вывода»), которая произошла на уровне системы Linux. Обычно она возникает при чтении и записи файлов или, другими словами, при выполнении ввода / вывода файлов или сетевого ввода / вывода (через сокеты).

Эквивалентная системная ошибка Linux – EPIPE, взятая из кодов ошибок GNU libc.

Макрос: int EPIPE

“Broken pipe.” означает, что на другом конце конвейера нет считывания процесса. Каждая функция библиотеки, вызывающая код ошибки, также выдает сигнал SIGPIPE; этот сигнал завершает программу, если не обрабатывается или не блокируется. Следовательно, программа никогда не отобразит EPIPE до тех пор, пока она не обработает или не заблокирует SIGPIPE.

Из приведенного выше утверждения мы можем сделать вывод, что система, отправляющая сигнал SIGPIPE, вызывает ошибку [Errno 32] Broken pipe в механизме межпроцессного взаимодействия Linux.

Например, система Linux внутренне использует другой сигнал, называемый SIGINT. В Linux команда Ctrl + C отправит сигнал SIGINT, чтобы завершить процесс, или мы можем использовать команду kill для достижения того же эффекта.

Python по умолчанию не игнорирует SIGPIPE. Однако он преобразует сигнал в исключение и вызывает ошибку – IOError: [Errno 32] Сломанный канал каждый раз, когда он получает SIGPIPE.

Ошибка “сломанный канал” при подключении к терминалу Linux

Всякий раз, когда мы сталкиваемся с ошибкой [Errno 32] Broken pipe при попытке передать вывод скрипта Python другой программе, например:

 
$ python file_name.py | head 

Объяснение:

Вышеупомянутый синтаксис конвейера создаст процесс, отправляющий данные в восходящем направлении, и процесс, читающий данные в нисходящем направлении. Когда нисходящему потоку не нужно читать данные восходящего потока, он отправит сигнал SIGPIPE процессу восходящего потока.

Когда нисходящий поток не должен читать данные восходящего потока? Давайте разберемся в этом на примере. Команда head в этом примере должна прочитать достаточно строк, чтобы сообщить восходящему потоку, что нам больше не нужно его читать, и она отправит сигнал SIGPIPE процессу восходящего потока.

Всякий раз, когда восходящий процесс является программой Python, возникает ошибка типа IOError: [Errno 32] Broken pipe.

Как избежать ошибки “сломанный канал”?

Если мы не заботимся о правильном перехвате SIGPIPE и нам нужно быстро запустить процесс, вставьте следующий фрагмент кода в начало программы Python.

Синтаксис:

 
from signal import signal, SIGPIPE, SIG_DFL  
#Ignore SIG_PIPE and don't throw exceptions on it...(http://docs.python.org/library/signal.html) 
signal(SIGPIPE,SIG_DFL)  

Объяснение:

В приведенном выше фрагменте кода мы перенаправили сигналы SIGPIPE на стандартный SIG_DFL, который система обычно игнорирует.

Однако рекомендуется остерегаться руководства Python по библиотеке сигналов, чтобы предостеречь от такой обработки SIGPIPE.

Перехват IOError во избежание ошибки Broken pipe

Поскольку ошибка Broken pipe является ошибкой IOError, мы можем разместить блок try / catch, чтобы ее перехватить, как показано в следующем фрагменте кода:

Синтаксис:

 
import sys, errno 
try: 
    ### IO operation ### 
except IOError as e: 
    if e.errno == errno.EPIPE: 
        ### Handle the error ### 

Объяснение:

В приведенном выше фрагменте кода мы импортировали модуль sys и errno и разместили блок try / catch, чтобы перехватить возникшее исключение и обработать его.

Возможное решение проблемы в многопроцессорной программе

В программах, которые используют рабочие процессы для ускорения обработки и многоядерные процессоры, мы можем попытаться уменьшить количество рабочих процессов, чтобы проверить, сохраняется ли ошибка или нет.

Большое количество рабочих процессов может конфликтовать друг с другом при попытке взять под контроль ресурсы системы или разрешение на запись на диск.

In this article, we will discuss Pipe Error in python starting from how an error is occurred in python along with the type of solution needed to be followed to rectify the error in python. So, let’s go into this article to understand the concept well. 

With the advancement of emerging technologies in the IT sector, the use of programming language is playing a vital role. Thus the proper language is considered for the fast executions of the functions. In such a case, Python emerges as the most important language to satisfy the needs of the current problem execution because of its simplicity and availability of various libraries. But along with the execution, the errors during the execution also comes into existence and it becomes difficult for the programmers to rectify the errors for the processing of the problem.

The Emergence of Broken Pipe Error

A broken Pipe Error is generally an Input/Output Error, which is occurred at the Linux System level. The error has occurred during the reading and writing of the files and it mainly occurs during the operations of the files. The same error that occurred in the Linux system is EPIPE, but every library function which returns its error code also generates a signal called SIGPIPE, this signal is used to terminate the program if it is not handled or blocked. Thus a program will never be able to see the EPIPE error unless it has handled or blocked SIGPIPE.

Python interpreter is not capable enough to ignore SIGPIPE by default, instead, it converts this signal into an exception and raises an error which is known as IOError(INPUT/OUTPUT error) also know as ‘Error 32’ or Broken Pipe Error.

Broken Pipe Error in Python terminal

python <filename>.py | head

This pipeline code written above will create a process that will send the data upstream and a process that reads the data downstream. But when the downstream process will not be able to read the data upstream, it will raise an exception by sending SIGPIPE signal to the upstream process. Thus upstream process in a python problem will raise an error such as IOError: Broken pipe error will occur.

Example:

When we run this file from unix commands:

python3 main.py | head -n3000

Procedure to avoid Broken Pipe Error

Approach 1: To avoid the error we need to make the terminal run the code efficiently without catching the SIGPIPE signal, so for these, we can add the below code at the top of the python program.

from signal import signal, SIGPIPE, SIG_DFL  
signal(SIGPIPE,SIG_DFL) 

Output:

0
1
20
1
2
3
4
5
6
7
8
9
3
4
5
6
7
8
9

Explanation:

The above code which is placed on the top of the python code is used to redirect the SIGPIPE signals to the default SIG_DFL signal, which the system generally ignores so that the rest part of the code can be executed seamlessly. But Approach 11 is not effective because in the Python manual on the signal library, which is mentioned that this type of signal handling should be avoided and should not be practiced in any part of the code. So for this reason we will go for the second approach.

Approach 2: We can handle this type of error by using the functionality of try/catch block which is already approved by the python manual and is advised to follow such procedure to handle the errors.

import sys, errno  
try:  
   # INPUT/OUTPUT operation #
except IOError as e:  
   if e.errno == errno.EPIPE:  
       # Handling of the error  

Example:

Output:

0
1
2
3
4
5
6
7
8
9

Explanation:

In the above piece of code, we have used the built-in library of python which is the Sys and Errno module, and use the try/catch block in order to catch the raised SIGPIPE exception and handle it before it stops the program execution.

Last Updated :
23 Sep, 2021

Like Article

Save Article

I’ve been dabbling in the Linux world for a good decade now, and it never ceases to surprise me with its quirks and nuances. I mean, who wouldn’t love the charm of the terminal, the power of the command line, and the satisfaction of troubleshooting a complex problem? Today, we’re going to dive headfirst into one of the most common issues that Linux users encounter: the dreaded ‘Broken Pipe’ error.

Trust me, I know how frustrating it can be when you’re working on a crucial task, and bam! The terminal throws this error at you. But rest assured, my friends, we’re not helpless here! As overwhelming as it may seem, with a little patience and understanding, it’s absolutely fixable. So, let’s roll up our sleeves and get down to business!

The ‘Broken Pipe’ error: What is it?

Just to give a brief overview for beginners (and a refresher for the veterans), the ‘Broken Pipe’ error typically occurs when one process is trying to write data to another process that is no longer available to receive it. In other words, the communication channel (or “pipe”) between the two processes has somehow been “broken.”

One thing I’ve learned throughout my Linux journey is that Linux is all about communication. That’s what makes it so powerful yet sometimes so tricky. And the ‘Broken Pipe’ error is a prime example of communication gone awry.

Example that demonstrates the ‘Broken Pipe’ error

Let’s use a simple case involving two popular Unix commands: yes and head.

The yes command continuously outputs a string until it is killed, and head command outputs the first part of files. When we pipe the output of yes into head, head will stop after it has printed out the first ten lines (which is its default behavior), and it will close its input pipe. But yes will still try to write to the pipe, and that’s when we get a ‘Broken Pipe’ error.

Here’s the command you can try:

yes | head

Now, if you run this command in a terminal, you may not see an error. That’s because the shell automatically ignores the ‘Broken Pipe’ signal (SIGPIPE). However, if you run it in a script, the script will exit due to the error.

Let’s put it in a script to see the error:

#!/bin/bash
yes | head
echo "Script finished"

If you run this script, you’ll see that “Script finished” does not get printed because the script exits when the ‘Broken Pipe’ error occurs.

Now, let’s handle the error using trap as we discussed earlier:

#!/bin/bash
trap 'echo "Broken pipe signal detected" >&2' PIPE
yes | head
echo "Script finished"

This time, the script doesn’t exit when the ‘Broken Pipe’ error occurs. Instead, it prints “Broken pipe signal detected” and continues to the end, printing “Script finished”. This is a simple but clear illustration of the ‘Broken Pipe’ error and how to handle it.

Identifying the cause: The first step towards a solution

To fix any error, we first need to understand its cause. One common reason for this error, which I personally detest because it always seems to happen at the worst possible time, is network instability. You might see this error if you’re SSH-ing into a remote server, and your internet connection is unstable or drops out for a moment. The server tries to send data, but since your computer isn’t connected anymore, the pipe is “broken.”

Another cause can be when a command tries to write output to a pipe or file, but the pipe has been closed or the file has been removed. This often happens when you’re piping the output of one command into another, and the second command ends before the first one does. As a quick example, let’s say we’re using the yes command piped into head. If head finishes execution before yes, it closes the pipe, leading to the ‘Broken Pipe’ error. Oh, the number of times this has caught me out!

Fixing the error: Time to get our hands dirty

Now, onto the most exciting part, at least for me – fixing the error! Depending on the cause, there are a few ways to handle this:

Case 1: Network instability

If you’re dealing with an unstable network causing your SSH connections to drop, you can use tools like autossh, mosh, or screen.

• autossh: This handy tool automatically restarts SSH sessions and port forwarding if they crash, helping to maintain the connection.
• mosh: An excellent alternative to SSH, mosh provides a robust and responsive connection, even with intermittent network connectivity.
• screen: This utility allows you to start a screen session, run your command, and then detach from the session. You can later reattach to the session, and it’s as if you never left!

I must confess, I’m a huge fan of mosh for its simplicity and robustness. But feel free to choose the one that suits your needs and preferences!

Case 2: Commands writing to a closed pipe

For the scenario where a command is trying to write to a closed pipe, we can trap the ‘Broken Pipe’ signal in our scripts and handle it gracefully. To do this, we use the trap command in bash scripting.

Here’s a simple example:

trap 'echo "Pipe has broken, but we're not going to crash and burn!" >&2' PIPE
yes | head

In this script, if a ‘Broken Pipe’ signal is detected, the message “Pipe has broken, but we’re not going to crash and burn!” is printed to standard error.

Keeping a watchful eye: Prevention is better than cure

Lastly, I’d like to share a piece of wisdom I’ve gathered over the years: An ounce of prevention is worth a pound of cure. It’s far better to prevent errors than to fix them. Keep your scripts clean, ensure you handle exceptions, and regularly check your network connectivity if you’re working on remote servers.

Wrapping up

In conclusion, while the ‘Broken Pipe’ error can be a nuisance, it’s not the end of the world, nor is it the end of your Linux journey. In fact, it’s just the beginning of a deeper understanding of how Linux operates. It’s these little challenges that, in my opinion, make Linux not just an operating system, but an adventure!

Remember, every problem has a solution, and every error is a stepping stone to becoming a better Linux user. I hope this blog post helps you navigate the ‘Broken Pipe’ error with confidence and ease. Until next time, happy troubleshooting!

SSH, an acronym for Secure Shell, is a remote network protocol that is used to securely connect to remote devices such as servers and network devices over a TCP/IP network.

It is a cryptographic network protocol that provides strong encryption technologies and hashing to secure communication between two devices on a network.

SSH uses two main authentication methods: password authentication and public key authentication. With password authentication, a user provides the remote host’s IP address or FQDN (Fully Qualified Domain Name) and password to authenticate.

Public key authentication uses an SSH key pair for authentication, which comprises two SSH keys: private and public keys.

The private key resides on the user’s machine and should always be kept confidential and secure. The public key is saved on the remote host that a user connects to. During authentication, the identity of the two keys is compared and access is granted.

When connecting to a remote system via SSH, you might encounter the error Client_loop: send disconnect: Broken pipe.

In this tutorial, we will see why this happens and address the error.

The error is simply a disconnection message that notifies you that your SSH connection timeout has been exceeded.

This is a period of inactivity during which no Linux command is executed or issued from the client side. When this happens, the SSH session is terminated, effectively disconnecting you from the remote server.

Most users will usually press ‘ENTER’ or a key on the keyboard to avoid having an idle SSH session which will cause the disconnection to the host. However, this can tedious and time-wasting.

Thankfully, SSH default configuration settings provide a few parameters that you can configure to keep your SSH connections active for longer periods of time.

Fix Client_loop: send disconnect: Broken pipe Error

To resolve this issue, you need to increase the SSH connection timeout on the client. To do so, modify the default SSH configuration file which is usually at /etc/ssh/sshd_config.

$ sudo vi /etc/ssh/sshd_config

Be sure to locate these two parameters: ClientAliveInterval and ClientAliveCountMax. Let’s check out what they do.

• ClientAliveInterval – This is the period of inactivity after which the SSH server sends an alive message to the remote client that is connected to it.
• ClientAliveCountMax – This is the number of attempts that the server will make to send the alive message from the server to the client.

We will set the two values as follows:

ClientAliveInterval	300
ClientAliveCountMax	3

This means that after the first 300 seconds (5 minutes) of inactivity from the client, the server will send an alive message to the client to keep the SSH session active.

If no data or response is received from the client for the next 300 seconds (at the 600-second mark), the server will again send another alive message. Finally, after 900 seconds of inactivity from the client, the SSH connection will be terminated or dropped.

Be sure to save the changes made to the file and then exit. Then restart the SSH daemon.

$ sudo systemctl restart sshd

Alternatively, you can connect to your remote client Linux system by specifying the ServerAliveInterval parameter in seconds (300 seconds), which means your SSH session is active for up to 5 minutes.

$ ssh -o ServerAliveInterval=300 username@server_ip_address

In this tutorial, we demonstrated how to resolve the Client_loop: send disconnect: Broken pipe error. As you have seen all you need is to perform a few tweaks in the SSH configuration file.