Skip to content

Configure TLS Encrypted Tunnel For Remote Logs Using Syslog-ng

Avatar photo

https://www.linkedin.com/in/gineesh/ https://twitter.com/GiniGangadharan

You might be a Sysadmin, developer, DBA or whatever, logs are like treasure boxes for anyone working in IT. And the best practice to keep logs in a central location together with local copy. Most of the logging programs have the ability to send logs to a remote logging server (as well as receive logs from remote machines); eg rsyslog, syslog-ng etc.

But, still there is a concern for sending server/application/database logs sending over tcp as plain text; yes indeed. But no need to worry as most of the logging programs will have simple mechanisms to implement TLS Tunnels for sending and receiving logs. In below demo, we will implement TLS tunnel to send logs from one machine (using syslog-ng) and receive the logs on another logging server (syslog-ng).

On Logging Server

Install Syslog-ng

You will most likely need to enable Extra Packages for Enterprise Linux (EPEL)

# cd /etc/yum.repos.d/
# wget https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng321/repo/epel-7/czanik-syslog-ng321-epel-7.repo
# yum install syslog-ng
# systemctl enable syslog-ng
# systemctl start syslog-ng

Note : If you have rsyslog or other logging systems running, you need to stop that first and configure custom items (if any) in syslog-ng manually.

Generate SSL Certificate

Generate Key and CSR

We will store the keys in /etc/syslog-ng/ssl

# cd /etc/syslog-ng
# mkdir ssl
# cd ssl
# openssl genrsa -des3 -out logserver.key 2048
# openssl req -new -key logserver.key -out logserver.csr

Remove the passphrase from the key

cp logserver.key logserver.key.org
openssl rsa -in logserver key.org -out logserver.key

Generate a self-signed certificate

openssl x509 -req -days 365 -in logserver.csr -signkey logserver.key -out logserver.crt 

Syslog-ng configuration on Server side

Configure a Source to receive logs over TLS

Edit /etc/syslog-ng/syslog-ng.conf and add below section.

(You can either directly edit /etc/syslog-ng/syslog-ng.conf or add separate configuration file under conf.d for easy configuration file management. eg: /etc/syslog-ng/conf.d/mylog.conf)

# Step 1 - TLS Source Listen
source source_514_tls {
 tcp(port(514)
 tls(
   # SSL Certificates which we have created in previous steps
   key_file("/etc/syslog-ng/ssl/logserver.key")
   cert_file("/etc/syslog-ng/ssl/logserver.crt")
   peer_verify(optional-untrusted)
   )
 flags(no-multi-line)
 );
 };

Configure a Destination

This can be another logging server (eg: Elastic, Splunk, ArcSight Connectors etc) or a local destination.

# Step 2 - Configure Destinations

# Local Destination on same server
destination dest_514_local { file("/var/log/messages_514"); };

# Destination on another server:1514
destination dest_1514 {
 tcp("10.1.10.100" port (1514)
 };

Configure Logging

# Step 3 - Configure Source to Destination Logging

# Forward Logs from 514 to another server:1514 
log { source(source_514_tls ); destination(dest_1514); };

# Log same logs received on 514 to locally.
log { source(source_514_tls ); destination(dest_514_local); };

On Client Machine

Here client means any machine who send data to connector machine

Configure Certificate

You need to get the certificate from logging server and configure to send data over TLS.

Download the certificate from server

Download logserver.crt and keep it under directory /etc/syslog-ng/ssl/ssl-for-client/ (or any other suitable location).

# ls -l /etc/syslog-ng/ssl/ssl-for-client/
total 4
-rw-r--r-- 1 root root 1257 Jun 27 09:52 logserver.crt

Find hash for your key

# openssl x509 -noout -hash -in /etc/syslog-ng/ssl/ssl-for-client/logserver.crt
acd0d3bb

Create Symlink to Certificate

Create a symbolic link to the certificate that uses the hash returned by the previous command, with an added .0 suffix.

# ln -s /etc/syslog-ng/ssl/ssl-for-client/logserver.crt /etc/syslog-ng/ssl/ssl-for-client/84d92a45.0
# ls -l /etc/syslog-ng/ssl/ssl-for-client/
total 4
lrwxrwxrwx 1 root root   47 Jun 27 09:54 acd0d3bb.0 -> /etc/syslog-ng/ssl/ssl-for-client/logserver.crt
-rw-r--r-- 1 root root 1257 Jun 27 09:52 logserver.crt

Syslog-ng configuration on Client Side

Please note the certificate path mentioned at client side in destination d_mesg_copy.

Define a Destination to Logging Server

# Step 1 - Destination 10.1.10.200:514
destination d_mesg_copy {
 tcp("10.1.10.200" port (514)
 tls(
   ca_dir("/etc/syslog-ng/ssl/ssl-for-client/")
   )
 );
 };

Enable Logging to Remote Target

# Step 2 - Send Copy of var-log-messages to d_mesg_copy
log { source(s_sys); filter(f_default); destination(d_mesg_copy); };

Make sure you have enabled and restarted syslog-ng daemons on both servers after config updates.

Disclaimer:

The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.

Avatar photo


https://www.linkedin.com/in/gineesh/ https://twitter.com/GiniGangadharan
Gineesh Madapparambath is the founder of techbeatly and he is the co-author of The Kubernetes Bible, Second Edition. and the author of 𝗔𝗻𝘀𝗶𝗯𝗹𝗲 𝗳𝗼𝗿 𝗥𝗲𝗮𝗹-𝗟𝗶𝗳𝗲 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻. He has worked as a Systems Engineer, Automation Specialist, and content author. His primary focus is on Ansible Automation, Containerisation (OpenShift & Kubernetes), and Infrastructure as Code (Terraform). (aka Gini Gangadharan - iamgini.com)

Comments

2 Responses

  1. Doesnt work. problem with key certificate.
    Error setting up TLS session context; tls_error=’digital envelope routines:EVP_DecryptFinal_ex:bad decrypt’

  2. This is something wrong with the certificate generation. You can try any methods to generate the certificate.

    Also, share the output of history command so that we can see if anything wrong with command executed.

    You may contact us via telegram: t.me/techbeatly

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.