Juniper SRX Logging Methods and Configuration: Stream Mode vs Event Mode

JunOS has strong flexibility on many features. One of them is logging. It support flexible logging options. This post summarizes some concepts I learned from my work and studying. 

1.Understand Juniper SRX logging Type:

1.1 System Logging

Junos OS supports configuring and monitoring of system log messages (also called syslog messages). You can configure files to log system messages and also assign attributes, such as severity levels, to messages. Reboot requests are recorded to the system log files, which you can view with the show log command. SRX Series devices can send system log messages from the control plane (Routing Engine) to one or more destinations. Destinations can include local files on the SRX Series device (because the SRX Series device is a syslog server), remote syslog servers, user terminals, and the system console.

1.2 Traffic Logging (Event Mode)

You can use traffic logs to track usage patterns or troubleshoot issues for a specific policy. You can configure a policy so that traffic information is logged when a session begins (session-init) and/or closes (session-close). To generate traffic logs for multiple policies, you must configure each policy to log traffic information. You also must configure syslog messages with a severity level of info or any. In the default configuration, these messages and all other logging messages are sent to a local log file named messages.

1.3 Notes from Juniper KB:

2. Understand Juniper SRX Logging Methods:

Control Plane and Data Plane

2.1 Control Plane Logging

The control plane logs have to do with events triggered by daemons on the control plane. This includes messages about the underlying hardware (chassisd), general-purpose messages (messages), and various protocol daemons like IDPD, appidd, and so on. Control plane logging is on by default to log locally, but you can override this with your own logfiles, syslog hosts, and criteria for different log messages. All logs are stored in the /var/log directory on the control plane. The configuration has been described at section 1.1

Services on the control plane:

• Management Daemon (MGD):  Provides the interface between the UI components and the backend configuration and is responsible for acting on the Junos configuration to the system itself.
• Routing Protocol Daemon (RPD) : All routing protocols including RIP, OSPF, IS-IS, BGP, PIM, IPv6 counterparts, and so on.
• User interfaces: Console, Telnet, SSH, J-Web, NetConf.
• Filesystem interfaces: FTP/SCP.
• Syslogd: Logging subsystem on the control plane, different than what is on the data plane. This generates the OS and application logs on the control plane.
• Networking services: DNS, DHCP, NTP, ICMP, ARP/ND, SNMP.
• Chassisd: Controls the hardware operations of the data plane and interfaces with the components to ensure they are active and operating properly.
• JSRPD: This is the high availability daemon that runs the HA functionality between two SRX chassis in an HA cluster.

2.2 Data Plane Logging

Data plane logs are primarily those generated by components that process traffic on the data plane. These include the firewall logs (RT_LOG, which stands for Real-Time Log because it is not stored on the data plane) from the flowd process, IPS logs, UTM logs, and logs from other security components like Screens. Data plane logging is off by default and must be configured. Typically, it is recommended that you send logs off the SRX to a syslog host due to the large volume of logs that can be generated from the data plane, particularly on high-end SRX platforms like the 5800. In fact, it can take an entire infrastructure of syslog servers to handle the large volume of syslog messages that the high-end SRX can generate per second. For this reason, there are two different mechanisms that we can use to log messages to the control plane, as discussed in the next section.

Services on the data plane:

• Intrusion Detection and Prevention Daemon (IDPD)
• IKED
• PKID

2.2.1 Event Mode 

Event mode  – control plane log processing – used on low end devices. Optionally even rate can be specified. Once event mode is enabled under “security” then the logging to local file can configure under “system syslog” as above at section 1.2.  You also can configure that security traffic logs are handled through the eventd process and sent with system logs though control panel Routing Engine.

2.2.2 Stream Mode

Stream mode – data plane logging – Normally used on high end SRX devcies but can be configured on any SRX devices. Under security the syslog parameters can be specified, e.g. syslog server, syslog format, facility.

Note: SRX can only log to the control plane (Event mode) or log out the data plane (Stream mode) at one time

Security logs such as traffic and IDP logs are able to be streamed through the traffic interface ports to a remote syslog server. SRX devices do not send streamed session logs to the Routing Engine (RE). Because system logging is performed on the RE, session or traffic logs cannot be written to the RE file system. Therefore, all traffic logging must be sent to a remote syslog server. Because fxp0 belongs to the RE, the remote syslog server must be reachable by an interface on an IOC. Traffic logging cannot be sent out through fxp0.

When the logging mode is set to stream, security traffic logs generated in the data plane are streamed out a revenue traffic port directly to a remote server. That also means your local log file will stop logging. Match condition configuration in System -> Syslog part does not work in Stream mode.  Its as per design, the Routing engine is the one which puts the match condition and filters the log,
since when we use stream mode the traffic is streamed out of the data plane itself and doesn’t reach the RE the match condition dose not work when using stream mode and only works in event mode.

Basically, only thing works at System – Syslog section are those generated from control plane.

Please keep one thing in mind. Maximum stream destination is three. If you are configuration more than three destination, you will get following error messages from CLI. If you are using JunOS Space, it wont allow you add more than three destination either.

Control plane pushing configuration to data plane

“show security log” will show you something about audit log but not policy logging after enabled cache in the security log section, else SRX will show you Security Log disabled.

After you enabled cache under security -> log configuration, as shown at the configuration of section 2.2.2, you will get output like below once you use command show security log:

Reference:

• Security logging is disabled
• System Services
• [SRX] Match condition for logging in system syslog does not work when mode in stream and works with event mode
• Stream logging problems in SRX