Keeping logs tidy with newsyslog

[Shogun 4475]

Hi there,

 

I'm sure you've found yourself into the situation where logs grow out of proportion and maybe fill your disk, especially if your metin2 server binary went into a loop of some kind. Or maybe your nginx or mysql logs grow so big that it takes 10 seconds just to open them. Of course you could just delete them and let them recreated, but a much cleaner solution is to let the syslog daemon keep your logs neatly ordered for you. For example, we can rotate the current log to a backup file (such as myserver.log.1) once it reaches a certain size, or we can do it every day, or every week; we can even use both criteria at the same time.

 

Once the criteria is met for a second time, myserver.log will become myserver.log.1 and myserver.log.1 will become myserver.log.2, and so on. We can keep as many backups as we want too, and the oldest file will get deleted once rotation time comes. This ensures that the logs will never take more space on disk than the size limit we set, multiplied by the number of log files to keep.

 

So let's say we want to rotate our metin2 server logs. First take a look at /etc/newsyslog.conf to see how it looks like. You can write directly to this file, but it would get overwritten when upgrading FreeBSD, so let's do this the proper way and create a new file called /etc/newsyslog.d/<application>.conf

 

Each line corresponds to a log and each value is separated by a TAB:

 

<location of the log file>      <user:group>      <mode>      <logs to keep>        <size limit>      <time>       <flags>        <pid>

 

Where user:group is the user owning the log file; mode the permissions set (such as 644); size limit is the size in kilobytes at which the log will be automatically rotated; time is a specially formatted string defining a time of the day, month or week for the logs to be rotated; flags define whether we want to compress the old logs and other options; and pid the path to the process id we have to signal.

 

If this means nothing to you don't worry, I will give you a couple of ready made examples right now.

 

Example A > Rotate game logs every time they grow to 4MB in size, keep 6 old logs, and compress them with gzip:

 

ee /etc/newsyslog.d/metin2.conf

 

/home/metin2/channel1/game1/syslog           metin2:metin2                 644              6               4096               *                 CZ                    /home/metin2/channel1/game1/pid.game

/home/metin2/db/syslog           metin2:metin2                 644              6               4096               *                CZ                    /home/metin2/db/pid.db

 

(you can repeat this for each core)

 

Example B >  Rotate MySQL logs every monday at 4am, keep 3 old logs, and compress them with gzip:

 

ee /etc/newsyslog.d/mysql.conf

 

/var/db/mysql/myhostname.err       mysql:mysql          644          3         @W1D4          *             CZ                  /var/db/mysql/myhostname.pid

 

Example C > Rotate NGINX logs every day at midnight, do not keep any old log:

 

ee /etc/newsyslog.d/nginx.conf

 

/var/log/nginx-access.log       www:www         644      0         @T00            *            C                /var/run/nginx.pid

/var/log/nginx-error.log       www:www         644      0         @T00            *            C                /var/run/nginx.pid

 

How do we know if we did it properly? Well, the system will send us a message if something is wrong, but only if we set a mail address for root mail to be forwarded to. By default FreeBSD sends error messages to the root account which we can read in the console using the "mail" command but since this is quite primitive let's have all mail sent to our e-mail:

 

ee /etc/aliases

 

Below this text:

 

# Pretty much everything else in this file points to "root", so
# you would do well in either reading root's mailbox or forwarding
# root's email from here.
 

Add this line, replacing the bracketed text with your email address:

 

root:<your email here>

 

Save and exit and then enter this command in the shell:

 

newaliases

 

Ready! Now you should receive system email in your own mailbox.