Rsyslog

rsyslog is the linux logging tool for all system activity. There are a number of useful things you can do with it to evaluate and debug problems with your server with the logs

Directories

Main system logs are located: /var/log/messages This is a catchall log for any logging on the system
rsyslog configuration file: /etc/rsyslog.conf
rsyslog daemon: /sbin/rsyslogd
Root logins, user logins, su attmpts are located: /var/log/secure
Mail traffic is logged to: /var/log/maillog
Error Message from uuvp and news server (innd) daemons: /var/log/spooler (This one is not used by most systems anymore)

Some versions of Fedora don't have any of these log files aswell and instead use the journalctl command to access and filter through the logs. rsyslog though can be installed aswell to work side by side with journalctl and will not interfere

Configure Remote Logging

Remote logging is an important security functionality so that hackers can not alter the log files and remove their presence on the machine. By sending copies of your logs to a machine completely dedicated and locked down to only archive logs will ensure that logs will not be tampered with

You can configure a system to send its logs to a remote server by editing the /etc/syslog.conf file. Add an entry to the bottom of the conf file that looks something like this

<facility>.<priority> @<ip-of-log-server>:514

Facility and prioiryt are used to specify and filter what log information is sent to the log server. See the man pages for full documentation on the different facilites and priorities are available to filter the send logs. Note that priority is used as a threshhold and logs from that threshold and up are sent to the server.

To send all logs from each facility and each priority level you can just use *.*

Example:

*.* @144.272.38.38:514

Note also the importance of port 514. This is the typical default port number most loggings servers are configured to use. You can though of course configure this on your logging server.

Configure Logging Server

With rsyslog you can easily configure your logging server to recieve external log files with a few commands to rsyslog on startup

Open /etc/rsyslog.conf.
Uncomment or add the following lines:

$ModLoad imudp
$UDPServerRun 514

$ModLoad imtcp
$InputTCPServerRun 514

If you know what transportation protocol your server uses you only need to add/uncomment the corresponding lines.

If you wanted to change the port number to receive log files, change the value assigned to the $UDPServerRun variable

For the changes to take effect, restart rsyslog

sudo systemctl restart rsyslog #Fedora/CentOS
sudo service rsyslog restart #Ubunt

Configure RSysLog To Use A Different Conf Directory

RSysLog can be configured to load its configuration details form a different directory. Unfortunately this implementation so far has to be done manually and does not persist past a restart.

copy the rsyslog.conf file (by default in /etc/rsyslog.conf) to the new location you would like it to be referenced by
Shutdown the currently running rsyslog service with the following command
```
sudo service rsyslog stop
```
Startup the rsyslog daemon manualy with the following command
```
rsyslogd -n -f /new/conf/directory/rsyslog.conf &
```
Press Enter Twice. Once Loads the program, Second releases terminal from the process Note the '&' at the end is important to ensure the daemon executes in its own process

This setup may appear a bit obvious as someone may login and find rsyslog turned off, attempt to turn it on, and recieve errors. To resolve this issue execute step 3 with the following command instead

rsyslogd -n -i 1447 -f /new/conf/directory/rsyslog.conf &

By default rsyslogd uses PID 1446 to loadup. If you are to run multiple instances of rsyslog on your system, you will need to provide each new instance with their own PID. This can be done simply with the -i parameter when loading rsyslogd

Facilities

Facility	Message Category
auth or security	Security / Authorization
authpriv	Private Security / Authorization
cron	Cron Daemon Messages
daemon	System Daemon-Generated Messages
ftp	FTP Server Messages
kern	Kernel Messages
lpr	Printer SubSystem
news	Network News Subsystem
syslog	Syslog-Generated Messages
user	User Program-Generated Messages
UUCP	UUCP SubSystem
mail	Mail SybSystem
## Priorities

Priority	Message type
debug	Debug messages
info	Informational status messages
notice	Normal but important conditions
warning or warn	Warning Messages
err or error	Error Messages
crit	Critical Conditions
alert	Immediate Attention Required
emerg or panic	System is unusable
## Configure Remote Logging to A Database

RSysLog can log to a database. Documentation online is poor and there are a number of careful steps that must be met. But the procedure is possible. This procedure will install and configure remote logging to a MySQL database

Configure MySQL Database

Find the version of rsyslog on the logging server by restarting it and checking the /var/log/syslog file. Rsyslog prints its version when it initializes.
Go to the rsyslog website and search the archives for a download matching the server version. Download it to the server that hosts the MySQL database.
Extract the archive and locate createDB.sql, usually under /rsyslog-8.4.2/plugins/ommysql/createDB.sql. Run it against MySQL from the console or with the MySQL Toolbox. The script normally creates the database, but ensure that the database user has access.

Configure Logging Server

Install the MySQL output module on the logging server:
```
apt-get install rsyslog-mysql
```
Skip any prompted automated setup procedure.
Open the mysql.conf file created under /etc/rsyslog.d/:
```
nano /etc/rsyslog.d/mysql.conf
```
Add or update the following configuration, substituting your own values where necessary:

$ModLoad ommysql
*.* :ommysql:<host>,<database>,<username>,<password>

For my setup this looks something like this. Note that the host could either be localhost, in which case the logging server also is hosting the MySQL database, or it can be a domain or IP of another system hosting the MySQL database. RSysLog will resolve hostnames so a domain is acceptable aswell

$ModLoad ommysql
*.* :ommysql:192.168.20.100,Syslog,username,supersecret

Restart rsyslog on the logging server:
```
service rsyslog restart -r
```
Log in to the MySQL server and verify that events are arriving.

Raspberry Pi Example

My router can send logs remotely to a syslog server on its network, so I set one up on my Pi. It took about 15 minutes.

Edit /etc/rsyslog.conf to listen for connections on the default port 514 (see above).
Create the log file:
```
sudo touch /var/log/router.log
```

Configure rsyslog to use the new log file. Under /etc/rsyslog.d/, create a file with the .conf extension:

sudo nano /etc/rsyslog.d/router.conf

Add the following lines:

$template NetworkLog, "/var/log/router.log"
:fromhost-ip, isequal, "192.168.0.1" -?NetworkLog
& ~

Restart rsyslog (see above).
Enable remote logging in your router's web interface.

Configure log rotation. It is a good idea to compress and remove the log file when it becomes too large. Create a file under /etc/logrotate.d/:

sudo nano router

Add the following lines:

/var/log/router.log {
    rotate 7
    size 500k
    notifempty
    compress
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

Notes

If RSyslog is not recieivng logs try the following: - On the recieving host check and entry for udp/514 exists under /etc/services - RSysLog may need to be started with the -r flag to allow remote logging. This may need to be done on the sender and/or reciever - Example: systemctl restart rsyslog -r - Clear the firewall with:

```bash
iptables -F
iptables -X
```

Sources

https://linux.die.net/man/8/rsyslogd

https://linux.die.net/man/5/rsyslog.conf

https://linux.die.net/man/8/syslogd

http://serverfault.com/questions/542379/how-to-change-rsyslog-configuration-file-directory

http://opensourceforu.com/2015/10/remote-logging-using-rsyslog-and-mysql/

http://www.rsyslog.com/doc/v8-stable/tutorials/recording_pri.html