Syslog system and logs in Linux. Linux system logs (logging management) Ubuntu logging

Before starting with this guide, it is recommended that you log into your Linux and follow this guide by looking directly at the files, as this is the best way to learn and remember the topic.

Learn about log files when your system is running smoothly, how this understanding of log files will help you successfully diagnose and fix any problems that may arise in the future.

And finally, ‘ Linux log files‘ is a fairly broad topic and is unlikely to be fully covered in one article. This article can probably only serve as a general guide. Every application installed on the system has its own logging mechanism; whenever you need specific information about an application, then the application's documentation is the best place to look for it.

General Log Files

As a common standard on almost every Linux system, log files are located in the /var/log directory. Any other applications you may later install on your system will likely dump their log files here. After logging into your system, type the command

Ls -l /var/log

To view the contents of this directory.

/var/log/messages– Most general system messages are logged here, including messages during system startup.
/var/log/cron– Messages from the cron daemon are logged here. Tasks are created and stopped, as well as error messages.
/var/log/maillog or /var/log/mail.log– Registration of mail server information running on the server.
/var/log/wtmp– Contains the history of all inputs and outputs.
/var/log/btmp– Records of failed login attempts.
/var/run/utmp– Logging of the actual login state of each user.
/var/log/dmesg– Contains very important messages about the ring buffer core. In human terms, this means that when the core spins up, it records all the information here. Team dmesg can be used to view messages from this file.
/var/log/secure– Security related messages will be stored here. This includes messages from the SSH daemon, failed passwords, non-existent users, etc.
/var/log/mariadb– If MariaDB is installed on the system, then this is where it will throw logs by default
/var/log/mysql– If a database is installed, this is the default data recording directory.

View and manage log files

The initial registration of the Linux daemon process rsyslogd and its configuration is in /etc/rsyslog.conf.

For all plaintext log files, the logs can be viewed using the cat command. However, if the log file is very large, then you may want to use the tail command, which can only show the last part of the log.

To view the last 500 entries of a file, enter the following command:

Tail -n 500 /var/log/messages

For monitoring logs in real time, tail -f is also a very useful command that will track messages as they are logged. This is especially useful in troubleshooting mail flows and mail delivery errors.

Tail -f /var/log/maillog

Some Linux logs are like binary files that must be parsed by another application specifically tailored to view these logs. These logs are saved in /var/log/wtmp /var/log/btmp and /var/run/utmp .

To view the contents of /var/log/wtmp use: last
To view the contents of /var/log/btmp use: lastb
To view the contents of /var/run/utmp use: who

Cpanel specific log files

Apache log files:

/usr/local/apache/logs/ – General logs.
/usr/local/apache/domlogs/ – Domain-specific logs.

Exim log files:

/var/log/exim_mainlog
/var/log/exim_rejectlog

CPanel Log Files:

/usr/local/cpanel/logs/ – All Cpanel-related messages in this location.

DirectAdmin specific log files

DirectAdmin Log Files

/var/log/directadmin/ – DirectAdmin related logs.

Log files

/var/log/httpd/ – The Apache web server is logged into the standard directory.
/var/log/httpd/domains/ – For all other domains, logs are located in this subdirectory.

Log files

/var/log/proftpd/ – If ProFTPd is used.
/var/log/pureftpd.log – If PureFTPd is used.

Exim log files

/var/log/exim/ – Exim mail transfer agent logs in this directory.

Log files

/var/lib/mysql/server.hostname.com.err – This is the directory that is logged for .

Log files

/var/log/yum.log – Logging of the Yum package manager.
/var/log/httpd – On systems based on /RedHat CentOS this is where the Apache web server will store logs by default.

Log files

/var/log/apache2/ – On Ubuntu systems, Apache web server logs are stored in this directory.
/var/log/apt/ – Logs from package management in Ubuntu.

Placing a pipe symbol (|) in front of the file name will allow you to use fifo (first in - first out, first in, first out) or named pipe as a receiver for messages. Before starting (or restarting) syslogd, you must create a fifo using the mkfifo command. Sometimes fifo are used for debugging.

Terminal and console

A terminal such as /dev/console.

Remote machine

To forward messages to another host, precede the host name with an (@) symbol. Please note that messages are not forwarded from the receiving host. (for this assignment to work on the client and server in the file /etc/services line must be written syslog 514/udp, and UTP port 514 is open)

a list of users

Comma separated list of users receiving messages (if the user is logged in). This often includes the root user.

All registered users

To notify all registered users using the wall command, use the asterisk (*) character.

An example of a simple syslog.conf:

# Print all kernel messages to the console. #kern.* /dev/console # All logs at info level or higher, except email messages, and # do not log authentication messages or cron daemon messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # Log messages containing sensitive # authentication information to a separate file, regardless of their level. authpriv.* /var/log/secure # All messages from the mail system are also written to a separate file. mail.* -/var/log/maillog # Log scheduler messages to /var/log/cron cron.* /var/log/cron # Emergency messages should be immediately received # by all system users *.emerg * # Save messages news of crit level and higher into a separate file. uucp,news.crit /var/log/spooler # Save boot messages to boot.log local7.* /var/log/boot.log

As with many configuration files, the syntax is as follows:

  • Lines starting with # and empty lines are ignored.
  • The * symbol can be used to indicate all categories or all priorities.
  • The special keyword none specifies that logging for this category should not be performed for this action.
  • A hyphen before the file name (like -/var/log/maillog in this example) indicates that the log should not be synchronized after each entry. If your system crashes, you may lose information, but disabling synchronization will improve performance.

In the syntax of the configuration file, you can put before the priority sign! to indicate that the action should not be applied, from this level and above. Similarly, priority can be given sign = to indicate that the rule applies only to this level, or != to show that the rule applies to all levels except this one. Below are a few examples (there are many other examples in man syslog.conf):

# Send all kernel messages to /var/log/kernel. # Send all critical and higher level messages to the remote sysloger machine and to the console # Send all info, notice and warning level messages to /var/log/kernel-info # kern.* /var/log/kernel kern.crit @sysloger kern .crit /dev/console kern.info;kern.!err /var/log/kernel-info # Send all mail system messages except the info level to /var/log/mail. mail.*;mail.!=info /var/log/mail

I tried to show the operation of syslogd as clearly as possible in the diagram:

Starting the syslogd daemon

Starting the logging daemon is started during system initialization using a script /etc/rc.d/init.d/syslog, however, in order to set launch parameters, there is no need to adjust this script - starting from version 7.2, launch options are read from a separate configuration file /etc/sysconfig/syslog (/etc/default/syslogin debian).

Here are some possible syslogd daemon launch parameters:

  • -a /folder/socket- specifying an additional listening socket (don't forget to create the socket first)
  • -d- debugging mode. In this case, the daemon does not go into the background and issues all messages to the current terminal;
  • -f config-file-name. Specifies the name of an alternative configuration file that will be used instead of the default /etc/syslog.conf;
  • -l host-list- specifying a list of hosts whose names should not be written with the full domain name (FQDN - Full Qualified Domain Name);
  • -m minutes- sysklogd running without this option writes messages of the mark category to the log every 20 minutes. Using the -m option, you can either change the interval between marks or completely cancel the issuance of such messages;
  • -p socket- setting an alternative UNIX socket (instead of the default listening /dev/log);
  • -r- permission to receive messages from remote hosts;
  • -x- prohibition of determining a host name by its address to prevent freezing when working on the same host with a DNS server.
  • -v- show version and finish work

After the syslogd daemon starts, a status file is created /var/lock/subsys/syslog zero length, and a file with a process identification number /var/run/syslogd.pid.

Using the command
kill -SIGNAL `cat /var/run/syslogd.pid`

can you send syslogd daemon one of the following signals: SIGHUP- restart the daemon; SIGTERM- completion of work; SIGUSR1- enable/disable debugging mode.

Actually, there are two logging daemons running on the system - syslogd And klogd. Both daemons are included in the package sysklogd.

Demon klogd is responsible for logging events occurring in system core. The need for a separate klogd daemon is because the kernel cannot use the standard syslog function. The fact is that the standard C libraries (including the library in which the syslog function is located) are intended to be used only regular applications. Since the kernel also needs logging functions, it includes its own libraries that are not available to applications. Therefore, the kernel uses its own message generation mechanism.

Demon klogd is designed to organize the processing of these messages. In principle, it could do this processing completely independently and independently of syslogd, for example by logging these messages to a file, but in most cases the default setting of klogd is used, in which all messages from the kernel are forwarded to the same syslogd daemon.

Automatic rotation (updating completed files) and archiving logs

Over time, the log file tends to grow, especially when a service is running intensively. Accordingly, it is necessary to be able to control the size of the logs. This is done using logrotate commands, which is usually performed cron daemon. I will talk about the work of cron in the following articles. the main objective logrotate commands is to periodically back up the logs and create new clean logs. Multiple generations of logs are saved and when the last generation log reaches the end of its lifespan, it can be archived (compressed). The result can be sent by mail, for example, to the person responsible for maintaining archives.

To determine the order of rotation and archiving of logs, use configuration file /etc/logrotate.conf . You can set different frequency for different logs, such as daily, weekly or monthly, you can also control the number of generations accumulated, and you can also specify whether and when copies of the archives will be sent to the archive manager. Shown below example file /etc/logrotate.conf:

# first the default parameters are set (global options) # update log files weekly weekly # store an archive of logs for the last 4 weeks rotate 4 # create a new (empty) file after rotation (update) create # uncomment if you want the saved ones files were compressed #compress # enable rotation settings from the specified directory include /etc/logrotate.d # do not store wtmp, or btmp -- log data rotation settings are as follows: /var/log/wtmp ( missingok monthly create 0664 root utmp rotate 1 ) / var/log/btmp ( missingok monthly create 0664 root utmp rotate 1 ) # specific syslogs can be configured below

Global options are placed at the beginning logrotate.conf file. They are used by default unless something more specific is specified elsewhere. In the example, the logs are rotated weekly and backups are retained for four weeks As soon as the journal is rotated, a new one is automatically created in place of the old journal. logrotate.conf file may contain specifications from other files. So, it includes all the files from the directory /etc/logrotate.d.

This example also contains special rules for /var/log/wtmp And /var/log/btmp(storing information about successful and unsuccessful attempts to log into the system), which are rotated monthly. If files are missing, no error message is generated. A new file is created and only one backup is saved.

In this example, when the backup reaches the last generation, it is deleted because it is not determined what to do with it.

Log Backups can also be created when logs reach a certain size, and scripts can be created from sets of commands to run before or after a backup operation. Example:

/var/log/messages ( rotate 5 mail logadmin@sysloger size 100k postrotate /usr/bin/killall -HUP syslogd endscript )

In this example the rotation /var/log/messages produced when it reaches 100 KB in size. Five backups are accumulated, and when the oldest backup expires, it is emailed to logadmin@sysloger. The postrotate command word enables a script that restarts the syslogd daemon after rotation has completed by sending a HUP signal. The endscript command word is required to terminate the script, and also if there is a prerotate script. See the man pages for logrotate for more information.

Options specified in the configuration file logrotate.conf:

  • compress| nocompress(older versions may or may not be compressed using gzip)
  • compresscmd(specifies the compression program, default is gzip)
  • uncompresscmd(specifies the decompression program, default is ungzip)
  • compressext(specifies the suffix for compressed files)
  • compressoptions(sets the parameters of the compression program; default is "-9", i.e. maximum compression for gzip)
  • copytruncate| nocopytruncate(usually the old version is renamed and a new version of the log is created; when this parameter is set, logrotate copies the log to a new file and then truncates the old one; used if the program that created the log does not know how to close it; entries made in the interval between copying and truncating are lost ; but would it help if the program creating the log, instead of the append mode, simply writes to a file using an internal pointer?)
  • create[permissions owner group] | nocreate(immediately after renaming the old version of the log and before calling postrotate, a new log is created with the specified attributes - access rights are set in octal form, as in chmod.2; if the attributes are not specified, then they are taken from the old log)
  • daily(versions in the series change daily)
  • delaycompress| nodelaycompress(some programs do not immediately close the journal, in which case compression must be postponed until the next cycle)
  • errorsemail(who to send bug reports to)
  • extensionsuffix(sets the suffix added to file names during rotation before the compression suffix)
  • ifempty| notifempty(change versions even if the file is empty; default)
  • includefile name| directory-name (textually substitute a file or all files from the specified directory; subdirectories, special files and files with suffixes from the exclusion list are not included; cannot be used inside a section)
  • mailaddress| nomail(when changing versions leads to the need to delete the old journal, then send it to the specified address)
  • mailfirst(send not the deleted version of the log, but the first one)
  • maillast(send deleted log version; default)
  • missingok| nomissingok(do not send error messages if log is missing)
  • monthly(versions change monthly)
  • olddirdirectory| noolddir(during a version change, the log is moved to the specified directory; it must be on the same physical device)
  • postrotate(all further lines up to the endscript line are executed as shell commands after the version change process)
  • prerotate(all further lines up to the endscript line are executed before the version change process)
  • rotatenumber(how many old versions to store; if 0, then none)
  • sizebyte(a version change occurs if the log size exceeds the specified number; you can use the suffixes "k" - kilobyte - and "M" - megabyte)
  • sharedscripts| nosharedscripts(execute the prerotate and postrotate commands only once for all files described in the section)
  • tabooext[+] suffix-list(specifying a list of exclusion suffixes for include; if a plus sign is specified, then addition, otherwise replacement; default: .rpmorig, .rpmsave, .rpmnew, ",v", .swp and "~")
  • weekly(versions change weekly)

Examining and monitoring logs

Log entries typically contain a timestamp, the name of the host on which the process being described is running, and the name of the process. View logs you can use a pager program, for example, less, you can search for specific entries (for example, kernel messages from a specific daemon) using the command grep:

# less /var/log/messages # grep "ppp" /var/log/messages | tail Dec 17 16:34:25 proxy pppd: Connection terminated. Dec 17 16:34:25 proxy pppd: Exit. Dec 17 16:35:57 proxy pppd: LCP terminated by peer (^P]kV^@

The computer may not work constantly and turn off, say, at night. Therefore, entries in /var/log/messages are stored cyclically from computer startup to shutdown, this can be seen in the messages:

Dec 17 08:32:56 syslog-server syslogd 1.4-0: restart. Dec 17 08:32:56 syslog-server syslog: starting syslogd succeeded Dec 17 08:32:56 syslog-server kernel: klogd 1.4-0, log source = /proc/kmsg started. Dec 17 08:32:56 syslog-server syslog: starting klogd succeeded

Dec 17 08:32:56 syslog-server kernel: Kernel command line: auto BOOT_IMAGE=linux ro root=303 BOOT_FILE=/boot/vmlinuz-2.4.2-2 Dec 17 08:32:56 syslog-server kernel: Memory: 125652k/130560k available (1365k kernel code, 4200k reserved, 92k data, 236k init, 0k highmem) Dec 17 08:32:56 syslog-server kernel: CPU: Intel(R) Pentium(R) 4 CPU 1.60GHz stepping 02

Also, in this file you can find information about disk memory (including information about disk geometry, partition structure and interrupts used), information about peripheral devices, about the launch of individual services, information about connecting file systems and messages about user logins , as well as error messages.

Sometimes it may be necessary monitoring system logs to search for current events. For example, you can try to catch a rarely occurring event at the moment when it happens. In this case, you can use the command tail with option -f to monitor the contents of the system log. Example:

# tail -f /var/log/messages | grep syslog-server Dec 17 16:46:09 syslog-server pppd: pptpd-logwtmp.so ip-up ppp0 maikop 94.77.0.150 Dec 17 16:46:09 syslog-server pppd: Script /etc/ppp/ip-up finished (pid 12552), status = 0x0 Dec 17 16:46:49 syslog-server pptpd: CTRL: Client 85.175.197.65 control connection started Dec 17 16:46:49 syslog-server pptpd: CTRL: Starting call (launching pppd, opening GRE) Dec 17 16:46:49 syslog-server pppd: Plugin /usr/lib/pptpd/pptpd-logwtmp.so loaded.

In addition to the log files specified in /etc/syslog.conf, there are also other files, for example a file that stores information about the system boot process before syslogd was started, as well as files that have a binary format and store information about the last login user into the system, about all successful user logins and all unsuccessful user logins, respectively. Also in the /var/log/ directory there may be log files of such daemons as a web server or a proxy server. The format of these files is similar to syslogd logs.

Finally, I would like to emphasize that this protocol is not very secure, because syslog does not contain any protection against message forgery. Even worse, using the UDP protocol allows attackers to send messages on behalf of any host. Your local network should be shielded from receiving packets with spoofed local addresses (although this will not prevent you from sending spoofed messages from inside the local network) and from receiving packets from outside on port 514/udp. There are known cases of disk overflow with false messages.

The syslog and UDP protocols do not provide guaranteed delivery (messages may be lost due to network congestion or intercepted, corrupted messages are deleted without warning), correct delivery sequence (a message about the completion of a process may arrive before the message about its start), or priority delivery.

Confidentiality of messages is not ensured, since they are transmitted in clear text.

If, when setting up a message generator, you specify an incorrect collector or relay address, then there will be no error messages - the messages will be deleted (or written to someone else's log).

Several projects have been proposed to improve the syslog protocol. For example, RFC 3195 proposes a TCP-based logging system (syslog-conn) that ensures messages are delivered in the correct order. The syslog-sign project proposes to provide authentication, ordering, message integrity, and detection of missing messages by generating special messages containing a digital signature of a block of previous messages, preserving the standard syslog protocol and format, and using UDP.

Let's summarize:

Linux has a single daemon responsible for logging events on the local system and remote systems. All events are collected from the /dev/log socket, UDP port - 514, as well as from the “helper” - the klogd daemon, which sends messages from the kernel. All collected messages are filtered by the syslogd daemon through rules in the /etc/syslog.conf file and distributed according to the rules to the appropriate destinations. Log files are periodically truncated. The frequency is determined by the logrotate.conf file and the logrotate command, which is launched by the system scheduler - cron.

That's all for today. I hope I described everything as clearly as possible. I will add to the article over time!

Best regards, McSim!


When viewing log files (log files) in Linux, sometimes you need to monitor new log entries in real time. That is, you specify which log file (or files) you want to view and monitor in real time for new entries in that file.

Monitoring log files with the tail command

If you run the command tailFileName without any additional arguments, the last 10 lines of the file will be printed and the command will complete its work.

In order for the tail command to continuously display the latest entries in the file, that is, if new entries appear in the file, the information on the screen is updated, the -f option is used:
tail -f LogFileName

Run the tail -f command to output the log file /var/log/syslog

tail -f /var/log/syslog

Since the option is used -f , the tail command does not complete its work, but waits for new entries to appear in the log file. As soon as new entries are added to the log file, they will immediately be displayed in the terminal.

To interrupt the execution of a command, press the keyboard shortcut Ctrl+C

tailf command

The tail -f command is equivalent to the tailf command.

Usage:

Tailf /var/log/mylogfile.log

An important difference between the tailf command and tail -f is that tailf does not access the file when it is not modified. As a consequence, the file access time is not updated and the system does not continuously flush the file to disk when the file is not updated.

The description of the tailf command indicates that it is useful for monitoring log files on laptops. Since the disk is not accessed unnecessarily and battery life is preserved.

tail -F. If the file has been renamed or deleted

Typically, in Linux, log files are not written indefinitely, otherwise such a file would be very inconvenient to use in the future. Instead, so-called file rotation is used. When the log file becomes large, it is either deleted or renamed (a backup copy of the file is created), and further messages begin to be written to a new, empty file.

The tail command has two options: -f and -F

  • If used option -f and the tracked file is renamed, the tail command continues to track the already renamed file. Team tail in this case it is tied to the identifier (inode) of the file.
  • If used option -F and the tracked file is renamed, the tail command will detect this, and as soon as a new log file is created (with the name that we specified to the command tail), team tail will start tracking this new file.

Let's look at an example.

We will monitor the log file /var/log/apache2/error.log. Execute the tail command with the -F option

Tail -F /var/log/apache2/error.log

If the system moves (renames) the error.log file to error.log.1 and creates a new error.log file, then our tail command will continue to monitor the new error.log file

If we were to use the option in this example -f , then the tail command would continue to monitor the error.log.1 file, which is no longer relevant for us when viewing logs in real time.

Monitor multiple log files simultaneously

The tail command supports tracking multiple files at once. To do this, you must specify file names separated by spaces.

Tail -f /var/log/apache2/error.log /var/log/apache2/access.log

As soon as any of the files changes, the name of this file and the new entries in it are displayed on the screen.

For simultaneous monitoring of several log files, there is a very convenient multitail utility

It does not just display data, but creates its own window (area) for each file and displays the data in this window. With its help, it is very convenient to track many log files at once and see them in one terminal window.

The multitail utility can be installed from the standard repositories of your distribution. To install, run the command (select the appropriate command for your distribution):

Sudo apt install multitail sudo yum install multitail sudo dnf install multitail

Usage:

Multitail /var/log/apache2/access.log /var/log/apache2/error.log

To exit the utility, press the q key

It is impossible to imagine a user and administrator of a server, or even a Linux-based workstation, who has never read log files. The operating system and running applications constantly generate various types of messages, which are logged in various log files. Knowing how to identify the log file you need and what to look for in it can save you a lot of time and help you resolve the error faster.

Logging is the main source of information about system operation and errors. In this short guide, we will look at the basic aspects of operating system logging, directory structure, and programs for reading and reviewing logs.

Basic log files

All log files can be classified into one of the following categories:

  • applications;
  • events;
  • services;
  • systemic.

Most log files are contained in the /var/log directory.

  • /var/log/syslog or /var/log/messages contains a global system log in which messages are written from the moment the system starts, from the Linux kernel, various services, detected devices, network interfaces and much more.
  • /var/log/auth.log or /var/log/secure- information about user authorization, including successful and unsuccessful login attempts, as well as the authentication mechanisms involved.
  • /var/log/dmesg- device drivers. Using the command of the same name, you can view the output of the file contents. The log size is limited, when the file reaches its limit, older messages will be overwritten by newer ones. By setting the --level= switch, you can filter the output by significance criterion.
Supported logging levels (priorities): emerg - system unused alert - action must be performed immediately crit - criticality conditions err - error conditions warn - warning conditions notice - common but significant conditions info - informational debug - debug messages (5:520)$ dmesg -l err usb 1-1.1: 2:1: cannot get freq at ep 0x1 usb 1-1.1: 1:1: cannot get freq at ep 0x81 usb 1-1.1: 1:1: cannot get freq at ep 0x81
  • /var/log/alternatives.log- The output of update-alternatives, which contains symbolic links to the default commands or libraries.
  • /var/log/anaconda.log- Entries recorded during system installation.
  • /var/log/audit- Records generated by the auditd audit service.
  • /var/log/boot.log- Information that is written when the operating system boots.
  • /var/log/cron- Report from the crond service about executed commands and messages from the commands themselves.
  • /var/log/cups- Everything related to printing and printers.
  • /var/log/faillog- Failed login attempts. Very useful when checking security threats, hacker attacks, and brute force hacking attempts. You can read the contents using the faillog command.
  • var/log/kern.log- The log contains messages from the kernel and warnings that can be useful in troubleshooting errors in custom modules built into the kernel.
  • /var/log/maillog/ or /var/log/mail.log- Log of the mail server used on the OS.
  • /var/log/pm-powersave.log- Battery saving service messages.
  • /var/log/samba/- Samba file server logs, which is used to access Windows shared folders and provide Windows users with access to Linux shared folders.
  • /var/log/spooler- For the old school, contains USENET messages. Most often it is empty and abandoned.
  • /var/log/Xorg.0.log- X server logs. Most often they are useless, but if they contain lines starting with EE, then you should pay attention to them.

There will be a separate package manager log for each distribution.

  • /var/log/yum.log- For programs installed using Yum on RedHat Linux.
  • /var/log/emerge.log- For ebuilds installed from Portage using emerge on Gentoo Linux.
  • /var/log/dpkg.log- For programs installed using dpkg on Debian Linux and the entire family of related distributions.

And some binary logs of user sessions.

  • /var/log/lastlog- Last user session. You can read it with the last command.
  • /var/log/tallylog- Audit of failed login attempts. Output to the screen using the pam_tally2 utility.
  • /var/log/btmp- One more log of unsuccessful login attempts. Just like that, just in case you haven’t figured out where to look for traces of hacker activity.
  • /var/log/utmp- List of currently logged-in users.
  • /var/log/wtmp- Another log to record user logins. Displayed on screen using the utmpdump command.
(5:535)$ sudo utmpdump /var/log/wtmp [Tue Aug 11 16:50:07 2015] [~~ ] [Tue Aug 11 16:50:08 2015] [~~ ] [Tue Aug 11 16: 50:57 2015] [Tue Aug 11 16:50:57 2015] [~~ ] [Tue Aug 11 16:50:57 2015]

And other magazines

Since an operating system, even one as wonderful as Linux, in itself does not provide any tangible benefits, most likely a database, a web server, and various applications will be running on a server or workstation. Each application or service can have its own event and error log file or directory. Naturally, it is impossible to list all of them, only a few.

  • /var/log/mysql/- MySQL database log.
  • /var/log/httpd/ or /var/log/apache2/- Apache web server log, the access log is in access_log, and errors are in error_log.
  • /var/log/lighthttpd/- Lighttpd web server log.

The user's home directory may contain logs for graphical applications, DE.

  • ~/.xsession-errors- Stderr output from X11 graphics applications.
Initializing "kcm_input" : "kcminit_mouse" Initializing "kcm_access" : "kcminit_access" Initializing "kcm_kgamma" : "kcminit_kgamma" QXcbConnection: XCB error: 3 (BadWindow), sequence: 181, resource id: 10486050, major code: 20 (GetProperty) , minor code: 0 kf5.kcoreaddons.kaboutdata: Could not initialize the equivalent properties of Q*Application: no instance (yet) existing. QXcbConnection: XCB error: 3 (BadWindow), sequence: 181, resource id: 10486050, major code: 20 (GetProperty), minor code: 0 Qt: Session management error: networkIdsList argument is NULL
  • ~/.xfce4-session.verbose-log- XFCE4 desktop messages.

What to view - lnav

Almost everyone knows about the less utility and the tail -f command. The vim editor and the Midnight Commander file manager are also suitable for these purposes. They all have their drawbacks: less doesn't handle logs with long strings well, mistaking them for binaries. Midnight Commander is only good for quick browsing when there is no need to search through a complex pattern and go back and forth a lot between matches. The vim editor understands and highlights the syntax of many formats, but if the log is updated frequently, distracting messages about changes to the file appear. However, this can easily be circumvented using<:view /path/to/file> .


Recently I discovered another useful and promising, but still slightly damp, utility - lnav, as transcribed by Log File Navigator.




Install the package as usual with one command.


$ aptitude install lnav #Debian/Ubuntu/LinuxMint $ yum install lnav #RedHat/CentOS $ dnf install lnav #Fedora $ emerge -av lnav #Gentoo, must be added to package.accept_keywords file $ yaourt -S lnav #Arch

Log Navigator lnav understands a range of file formats.

  • Access_log web server.
  • CUPS page_log
  • Syslog
  • dpkg.log
  • strace
  • Custom entries with timestamps
  • gzip, bzip
  • VMWare ESXi/vCenter Log

What does understanding file formats mean in this case? The trick is that lnav more than a utility for viewing text files. The program can do something else. You can open multiple files at once and switch between them.


(5:471)$ sudo lnav /var/log/pm-powersave.log /var/log/pm-suspend.log

The program can directly open an archive file.


(5:471)$ lnav -r /var/log/Xorg.0.log.old.gz

Shows a histogram of informative messages, warnings and errors when you press a key . This is from my syslog.


Mon May 02 20:25:00 123 normal 3 errors 0 warnings 0 marks Mon May 02 22:40:00 2 normal 0 errors 0 warnings 0 marks Mon May 02 23:25:00 10 normal 0 errors 0 warnings 0 marks Tue May 03 07:25:00 96 normal 3 errors 0 warnings 0 marks Tue May 03 23:50:00 10 normal 0 errors 0 warnings 0 marks Wed May 04 07:40:00 96 normal 3 errors 0 warnings 0 marks Wed May 04 08 :30:00 2 normal 0 errors 0 warnings 0 marks Wed May 04 10:40:00 10 normal 0 errors 0 warnings 0 marks Wed May 04 11:50:00 126 normal 2 errors 1 warnings 0 marks

In addition, syntax highlighting, tab addition and various useful features in the status bar are supported. Disadvantages include unstable behavior and freezing. Hope lnav will be actively developed, a very useful program in my opinion.

System administrators, and regular Linux users, often need to look at log files to troubleshoot problems. In fact, this is the first thing any system administrator should do when any error occurs in the system.

The Linux operating system itself and the running applications generate various types of messages that are logged in various log files. Linux uses special software, files and directories to store log files. Knowing which files contain the logs of which programs will help you save time and solve the problem faster.

In this article we will look at the main parts of the Linux logging system, log files, as well as utilities with which you can view Linux logs.

Most Linux log files are located in the /var/log/ folder. You can list the log files for your system using the ls command:

Rw-r--r-- 1 root root 52198 May 10 11:03 alternatives.log
drwxr-x--- 2 root root 4096 Nov 14 15:07 apache2
drwxr-xr-x 2 root root 4096 Apr 25 12:31 apparmor
drwx------ 2 root root 4096 May 5 10:15 audit
-rw-r--r-- 1 root root 33100 May 10 10:33 boot.log

Below we will look at 20 different Linux log files located in the /var/log/ directory. Some of these logs are only found on certain distributions, for example dpkg.log is only found on Debian based systems.

/var/log/messages- contains global Linux system logs, including those that are recorded at system startup. Several types of messages are recorded in this log: mail, cron, various services, kernel, authentication and others.

/var/log/dmesg- contains messages received from the kernel. Logs many messages during the boot phase, they display information about hardware devices that are initialized during the boot process. You can say this is another log of the Linux system. The number of messages in the log is limited, and when the file is full, with each new message the old ones will be overwritten. You can also view messages from this log using the dmseg command.

/var/log/auth.log- contains information about user authorization in the system, including user logins and authentication mechanisms that were used.

/var/log/boot.log- Contains information that is logged when the system boots.

/var/log/daemon.log- Includes messages from various background daemons

/var/log/kern.log- Also contains messages from the kernel, useful in troubleshooting errors in custom modules built into the kernel.

/var/log/lastlog- Displays information about the last session of all users. This is a non-text file and you must use the lastlog command to view it.

/var/log/maillog /var/log/mail.log- logs of the email server running on the system.

/var/log/user.log- Information from all logs at the user level.

/var/log/Xorg.x.log- X server message log.

/var/log/alternatives.log- Information about the operation of the update-alternatives program. These are symbolic links to default commands or libraries.

/var/log/btmp- Linux log file contains information about failed login attempts. To view the file, it is convenient to use the command last -f /var/log/btmp

/var/log/cups- All messages related to printing and printers.

/var/log/anaconda.log- all messages recorded during installation are saved in this file

/var/log/yum.log- Logs all information about package installations using Yum.

/var/log/cron- Whenever the Cron daemon starts executing a program, it writes a report and messages from the program itself in this file.

/var/log/secure- contains information related to authentication and authorization. For example, SSHd logs everything here, including failed login attempts.

/var/log/wtmp or /var/log/utmp - Linux system logs , contain a log of user logins. Using the wtmp command you can find out who is logged in and when.

/var/log/faillog- Linux system log, contains unsuccessful login attempts. Use the faillog command to display the contents of this file.

/var/log/mysqld.log- Linux log files from the MySQL database server.

/var/log/httpd/ or /var/log/apache2- log files of linux11 Apache web server. Access logs are in the access_log file, and error logs are in the error_log

/var/log/lighttpd/- Linux logs of the lighttpd web server

/var/log/conman/- ConMan client log files,

/var/log/mail/- this directory contains additional mail server logs

/var/log/prelink/- Prelink links libraries and executables to speed up the loading process. /var/log/prelink/prelink.log contains information about .so files that were modified by the program.

/var/log/audit/- Contains information generated by the auditd daemon.

/var/log/setroubleshoot/ - SE Linux uses the setroubleshootd daemon (SE Trouble Shoot Daemon) to report security problems. This log contains messages from this program.

/var/log/samba/- contains information and logs from the Samba file server, which is used to connect to Windows shares.

/var/log/sa/- Contains .cap files collected by the Sysstat package.

/var/log/sssd/- Used by the system security daemon, which manages remote directory access and authentication mechanisms.

Viewing logs in Linux

To view logs on Linux, it is convenient to use several Linux command line utilities. This can be any text editor, or a special utility. Most likely, you will need superuser rights to view logs in Linux. Here are the commands that are most often used for these purposes:

  • zgrep
  • zmore

I will not go into detail on each of these commands, since most of them have already been discussed in detail on our website. But I will give a few examples. Viewing Linux logs is very simple:

We look at the log /var/log/messages, with the ability to scroll:

less /var/log/messages

View Linux logs in real time:

tail -f /var/log/messages

Open the dmesg log file:

cat /var/log/dmesg

First lines of dmesg:

head /var/log/dmesg

We only output errors from /var/log/messages:

grep -i error /var/log/messages

In addition, you can view logs on Linux using graphical utilities. System Log Viewer can be used to easily view and monitor system logs on a Linux laptop or personal computer.

You can install the program on any system with an X server installed. Also, any graphical test editor can be used to view logs.

conclusions

In the /var/log directory you can find all the necessary information about the operation of Linux. From today's article you have learned enough to know where to look and what to look for. Now viewing logs in Linux will not cause you problems. If you have any questions, ask in the comments!



Have questions?

Report a typo

Text that will be sent to our editors: