Логи ошибок cron

As others have pointed out, cron will email you the output of any program it runs (if there is any output). So, if you don’t get any output, there are basically three possibilities:

1. crond could not even start a shell for running the program or sending email
2. crond had troubles mailing the output, or the mail was lost.
3. the program did not produce any output (including error messages)

Case 1. is very unlikely, but something should have been written in the cron logs. Cron has an own reserved syslog facility, so you should have a look into /etc/syslog.conf (or the equivalent file in your distro) to see where messages of facility cron are sent. Popular destinations include /var/log/cron, /var/log/messages and /var/log/syslog.

In case 2., you should inspect the mailer daemon logs: messages from the Cron daemon usually appear as from root@yourhost. You can use a MAILTO=... line in the crontab file to have cron send email to a specific address, which should make it easier to grep the mailer daemon logs. For instance:

MAILTO=my.offsite.email@example.org
00 15 * * *  echo "Just testing if crond sends email"

In case 3., you can test if the program was actually run by appending another command whose effect you can easily check: for instance,

00 15 * * * /a/command; touch /tmp/a_command_has_run

so you can check if crond has actually run something by looking at
the mtime of /tmp/a_command_has_run.

cron already sends the standard output and standard error of every job it runs by mail to the owner of the cron job.

You can use MAILTO=recipient in the crontab file to have the emails sent to a different account.

For this to work, you need to have mail working properly. Delivering to a local mailbox is usually not a problem (in fact, chances are ls -l "$MAIL" will reveal that you have already been receiving some) but getting it off the box and out onto the internet requires the MTA (Postfix, Sendmail, what have you) to be properly configured to connect to the world.

If there is no output, no email will be generated.

A common arrangement is to redirect output to a file, in which case of course the cron daemon won’t see the job return any output. A variant is to redirect standard output to a file (or write the script so it never prints anything — perhaps it stores results in a database instead, or performs maintenance tasks which simply don’t output anything?) and only receive an email if there is an error message.

To redirect both output streams, the syntax is

42 17 * * * script >>stdout.log 2>>stderr.log

Notice how we append (double >>) instead of overwrite, so that any previous job’s output is not replaced by the next one’s.

As suggested in many answers here, you can have both output streams be sent to a single file; replace the second redirection with 2>&1 to say «standard error should go wherever standard output is going». (But I don’t particularly endorse this practice. It mainly makes sense if you don’t really expect anything on standard output, but may have overlooked something, perhaps coming from an external tool which is called from your script.)

cron jobs run in your home directory, so any relative file names should be relative to that. If you want to write outside of your home directory, you obviously need to separately make sure you have write access to that destination file.

A common antipattern is to redirect everything to /dev/null (and then ask Stack Overflow to help you figure out what went wrong when something is not working; but we can’t see the lost output, either!)

From within your script, make sure to keep regular output (actual results, ideally in machine-readable form) and diagnostics (usually formatted for a human reader) separate. In a shell script,

echo "$results"  # regular results go to stdout
echo "$0: something went wrong" >&2

Some platforms (and e.g. GNU Awk) allow you to use the file name /dev/stderr for error messages, but this is not properly portable; in Perl, warn and die print to standard error; in Python, write to sys.stderr, or use logging; in Ruby, try $stderr.puts. Notice also how error messages should include the name of the script which produced the diagnostic message.

As a Linux user, you are probably already familiar with crontab. You can automate tasks by running commands and scripts at a predefined schedule. Want to automatically take backups? Crontab is your friend.

I am not going into the usage of crontab here. My focus is on showing you the different ways to check crontab logs.

It helps investigate whether your cronjobs ran as scheduled or not.

Method 1: Check the syslog for crontab logs

As per the Linux directory hierarchy, the /var/log directory in Linux stores logs from the system, services, and running applications.

While the cron logs are also in this directory, there is no standard file for these logs. Different distributions keep them in different files.

For Debian-based distributions, the file /var/log/syslog contains the logs from the cron job execution and should be consulted in case your cron jobs are not working:

cat /var/log/syslog | grep -w 'cron’

You will see all cron jobs listed on your terminal when the above command is run. The grep command will filter the cron related messages apart from the rest.

For RedHat-based distros, the cron logs have dedicated file /var/log/cron.

In both cases, you will probably need to specify the sudo keyword or use the root account to access the logs.

Method 2: Use a custom log file (recommended)

Using a separate custom file for logging cron jobs is a recommended practice.

For this, you can configure ‘rsyslog’ to forward cron logs. Rsyslog is a Linux service that has features similar to Syslog logging.

Simply create a file cron.log under the directory /etc/rsyslog.d:

touch /var/log/cron.log

Now open the file /etc/rsyslog.d/50-default.conf for editing:

nano /etc/rsyslog.d/50-default.conf 

and locate the line starting with #cron.* and remove the # at the start of the line.

To make the changes work, save and close this file and finally restart the rsyslog service and check its status:

sudo systemctl restart rsyslog

sudo systemctl status rsyslog

The status of the service should be highlighted as active (running).

Now, whenever you need to access the crontab logs, just read the content of this log file:

less /var/log/cron.log

Method 3: Use dedicated services like Cronitor monitor cron jobs

Cronitor is a service that can be deployed to monitor any type of cron job.

Many of the cron versions will start logging when the scheduled job executes or if there are any problems with the crontab. However, the output from the cron job or its exit status is not logged.

Here Cronitor comes in handy and works perfectly. It is a complete solution for all your crontab needs. It captures logs, metrics, and status from all the jobs and creates instant alerts for crashed or failed to start cron jobs.

You can see all this through a web-based interface.

For Cronitor or CronitorCLI installed on Kubernetes, logs can be captured for as high as 100MB for a single execution. Here, you can find the detailed installation steps on Linux for CronitorCLI.

Other monitoring tools and services like Better Uptime also provide the feature to monitor the cron jobs automatically.

Conclusion

System log files are very crucial for troubleshooting and diagnosing system-related issues, cron logs are thus no exception for this.

The Syslog keeps the logs related to crontab however, having a dedicated log file for cron is recommended. A web-based service like Cronitor, Better Uptime, and Uptime Robot also helps.

Cron can generate logs, which are very useful in troubleshooting your cron jobs.
In this quick tutorial, we will take a look at cron logs – how to find them and
how to read them.

Where are cron logs stored?

By default, all cron logs are stored in the main system log which is located in
/var/log/syslog on Ubuntu and Debian systems and in /var/log/cron on CentOS.

You can filter cron logs in the system log by running the grep command.

cat /var/log/syslog | grep cron

This applies to Ubuntu and Debian. On CentOS, simply replace the system log
directory to /var/log/cron and you are good to go.

How to set up cron.log file?

Another way you can monitor cron logs is to set up separate log file especially
for cron logs.

To do this, open the /etc/rsyslog.d/50-default.conf configuration file in the
nano editor using the following command:

sudo nano /etc/rsyslog.d/50-default.conf

This will open the configuration file for editing. In this file, uncomment the
highlighted line to enable logging cron logs into the /var/log/cron.log file.

Output:
#  Default rules for rsyslog.
#
#  For more information see rsyslog.conf(5) and /etc/rsyslog.conf

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
#daemon.*                       -/var/log/daemon.log
kern.*                          -/var/log/kern.log
#lpr.*                          -/var/log/lpr.log
mail.*                          -/var/log/mail.log
#user.*                         -/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
#mail.info                      -/var/log/mail.info
#mail.warn                      -/var/log/mail.warn
mail.err                        /var/log/mail.err

#
# Some "catch-all" log files.

...

The highlighted line should look like this:

We are not done yet. Now we need to create the /var/log/cron.log file. To do
this, simply run the following command:

sudo nano /var/log/cron.log

Save the empty file and exit. Then restart the rsyslog service:

sudo systemctl restart rsyslog

At this point, all cron logs are stored in the /var/log/cron.log file.

How to watch cron logs in real-time?

To view cron logs in real-time, create watchcron file using the following
command:

Add the following line in the file. Then save and exit the file.

watch -n 10 tail -n 15 /var/log/cron.log

This will refresh the logs event page after 10 seconds and displays the last 15
events on the page.

Add the executable permission to the newly created watchcron file:

And finally, copy this file in /usr/sbin location using the following command:

sudo cp watchcron /usr/sbin

Now to watch cron log in real-time, simply type watchcron in the terminal and
hit enter.

Explore more

Imagine living in a world where the tedious task of manually backing up your data or running routine tasks on your Linux system is a distant memory. Sounds like a dream, right? With the power of crontab in Linux, this dream can become your reality. Crontab, a robust utility in Linux, enables you to automate tasks, liberating you from the shackles of monotonous, repetitive work.

However, as with any powerful tool, it’s not just about setting it up and forgetting it. You need to keep a keen eye on these automated tasks to ensure they’re running seamlessly. This is where crontab logs come into play. They assist you in monitoring the tasks and troubleshooting any issues that might crop up.

In this guide, we’ll delve into the fascinating world of crontab logs in Linux, exploring various methods to monitor them effectively. So, whether you’re a seasoned Linux user or a newbie, prepare for an enlightening journey into the realm of task automation and monitoring in Linux.

TL;DR What are crontab logs and how to monitor them?

Crontab logs are records created when tasks are run by the Linux crontab. Reading these logs helps check on the health of these automated tasks and troubleshoot any issues. Methods of monitoring these include syslog, creating a custom crontab log file, using a service like Cronitor, or real-time monitoring with watchcron.

Understanding Crontab

Before we delve into the specifics of monitoring crontab logs, it’s crucial to understand what crontab is and why it’s such a vital tool in Linux. Crontab, short for ‘cron table’, is a configuration file that specifies shell commands to run periodically on a given schedule. It’s like your personal assistant in the Linux world, always ready to take care of routine tasks without needing constant reminders.

Imagine having to manually back up your data every day at a specific time, or having to run a script every hour. Sounds tedious, right? This is where crontab comes in. It allows you to automate these tasks, leaving you free to focus on more important things. You just need to tell your ‘assistant’ what to do and when, and it will take care of the rest.

However, like any good manager, you need to keep an eye on your assistant. This is where monitoring crontab logs becomes essential. These logs provide you with a record of all the tasks that crontab has executed, along with any errors or issues that may have occurred. By keeping a close eye on these logs, you can ensure that your automated tasks are running smoothly and troubleshoot any issues that arise.

In essence, crontab is a game-changer when it comes to managing your Linux system. When used effectively, it can streamline your workflow, save you time, and reduce the risk of errors. But to truly harness its power, you need to understand and regularly monitor crontab logs. Let’s dive deeper and explore how to do just that.

Accessing Crontab Logs through Syslog

Having grasped the importance of crontab and its logs, let’s explore how to access these logs. A prevalent method is through syslog, a standard for message logging in Unix-like operating systems. Syslog offers a centralized repository where various system messages, including crontab logs, are stored.

So, how do we access these logs? Let’s break it down for different Linux distributions.

For Debian or Ubuntu users, you can use the following command:

grep CRON /var/log/syslog

For CentOS, Fedora, or RHEL users, the command would be:

grep CRON /var/log/messages

Executing these commands will display the cron logs from the syslog.

Please note that you’ll need root access or the sudo keyword to access these logs. Without these privileges, you won’t be able to view the logs. Therefore, always remember to prefix your commands with sudo if you’re not logged in as the root user.

You might wonder why we’re using syslog when there’s another logging system called journal logs. While both syslog and journal logs serve the same purpose of storing system logs, they differ in a few ways. Syslog is a more traditional and universally supported logging system, while journal logs are a newer system introduced with systemd. Journal logs offer advanced features like indexing and binary logging, but for accessing crontab logs, syslog serves the purpose just fine.

Understanding and accessing syslog is a crucial step in effective crontab log monitoring. It enables you to keep track of your automated tasks and troubleshoot any issues that may arise, ensuring your Linux system runs smoothly and efficiently.

Creating a Custom Log File for Crontab

While accessing crontab logs through syslog is effective, it can sometimes be overwhelming due to the large volume of logs from various system activities. What if we could have a separate, dedicated log file just for crontab? Well, the good news is, we can! Creating a custom log file for crontab can significantly streamline log management, making it easier to monitor and debug cron jobs.

The process involves configuring the ‘rsyslog’ service to forward cron logs to a separate file. Rsyslog is an open-source software utility used on UNIX and Unix-like computer systems for forwarding log messages in an IP network. It offers high-performance, robust security features, and a modular design. While it might seem daunting at first, don’t worry! We’ll guide you through the process step by step.

First, you need to edit the rsyslog configuration file. You can do this by using the following command:

sudo nano /etc/rsyslog.d/50-default.conf

In this file, you need to add a line that instructs rsyslog to save all cron logs to a separate file. Add the following line:

cron.* /var/log/cron.log

This line directs all cron-related logs to a file named cron.log in the /var/log directory. After adding this line, save and close the file.

Next, you need to restart the rsyslog service to apply the changes. Use the following command:

sudo service rsyslog restart

And there you have it! You’ve successfully created a separate log file for crontab. You can now view your crontab logs by using the following command:

cat /var/log/cron.log

Creating a custom log file for crontab not only improves log management but also allows you to direct output from specific crontab jobs to a selected log file. This simplifies the process of verifying whether a crontab is running and makes it easier to isolate and troubleshoot any issues. Essentially, a custom log file offers an organized and efficient way to monitor crontab logs, making your life as a Linux user much easier.

Utilizing Dedicated Services for Cron Job Monitoring

Creating a custom log file for crontab undeniably simplifies log management. However, there are dedicated services available that can enhance crontab monitoring even further. One such service is Cronitor. Cronitor is a monitoring service specifically designed for cron jobs, boasting a range of features that simplify the task of keeping an eye on your automated tasks.

So, how does Cronitor work? Once you’ve set up Cronitor, it begins monitoring your cron jobs, providing real-time alerts if a job fails to run on schedule or encounters an error. It also offers detailed logs and metrics to help you comprehend your cron job’s performance and troubleshoot any issues. This can be incredibly valuable in a production environment where prompt and accurate monitoring is crucial.

There are also other monitoring tools available, such as Better Uptime, that offer similar functionalities. Like Cronitor, Better Uptime provides real-time alerts and detailed reports, assisting you in keeping your cron jobs running smoothly.

But why do we need dedicated services like Cronitor when we have cron logs? While cron logs are indeed useful for debugging and auditing, they don’t record job output or exit statuses. Cronitor fills this gap. It captures this vital information, providing you with a more comprehensive view of your cron jobs.

For those using Kubernetes, CronitorCLI can be installed to capture logs and monitor cron jobs. This can be particularly useful in a containerized environment where traditional cron logs may not be readily available.

Dedicated services like Cronitor offer a comprehensive solution to crontab monitoring. They not only simplify the process but also provide additional features like real-time alerts and detailed metrics, making them an indispensable tool in any Linux user’s toolkit.

Real-Time Monitoring with watchcron

While syslog, custom log files, and dedicated services like Cronitor provide excellent ways to monitor crontab logs, there’s another method that might prove particularly useful: real-time monitoring with watchcron.

Watchcron is a command-line tool that offers real-time monitoring of cron jobs. It enables you to view your cron logs as they’re being written, providing immediate insights into cron job execution. This can be incredibly beneficial for troubleshooting issues as they occur, rather than after the fact.

So, how do you use watchcron? The first step is to install it. This can be done using the following command:

sudo apt install watchcron

Once watchcron is installed, you can commence monitoring your crontab logs in real-time by simply typing watchcron into your terminal. This will display a live feed of your cron logs, allowing you to keep an eye on your automated tasks as they’re being executed.

Real-time monitoring with watchcron offers a unique advantage: instant feedback. This enables you to promptly identify and resolve any issues, ensuring your cron jobs run smoothly and efficiently. It’s akin to having a live CCTV feed of your crontab, providing you with instant updates on your automated tasks. So, if you’re seeking a way to keep a close eye on your cron jobs, give watchcron a try. It might just be the tool you’ve been searching for.

Conclusion

We’ve journeyed deep into the world of crontab logs in Linux in this guide. We’ve uncovered the essence of crontab, its indispensable role in Linux, and the significance of monitoring its logs. We’ve dissected various methods for monitoring crontab logs, from checking logs through syslog, creating a custom log file for crontab, to employing dedicated services like Cronitor and real-time monitoring with watchcron.

Each of these methods brings its own unique advantages to the table, and the optimal one for you will hinge on your specific needs and circumstances. The key takeaway, whether you’re a seasoned Linux user or just dipping your toes in, is this: monitoring crontab logs is pivotal for efficient task automation in Linux. It ensures your automated tasks run without a hitch, alerts you to any potential issues, and simplifies troubleshooting.

So, remember not to simply set your cron jobs and dismiss them. Keep a watchful eye on them, monitor their performance, and troubleshoot any issues that arise. With the strategies and tools we’ve unpacked in this guide, you’re well-prepared to do just that. As we revisit our initial analogy, think of yourself as a manager overseeing a personal assistant. With the right monitoring tools and practices, you can ensure that your ‘assistant’ (the crontab) is performing its tasks efficiently. Happy monitoring!