Category: syslog

Someone made a really interesting remark on my Disable Console Logging blog post: Cisco IOS has log message rate limiting – all you need is the logging rate-limit configuration command.

read more see 7 comments

Mike Nipp has wondered which syslog message to use to reliably detect router reload under all circumstances:

The problem I had with the SYS-5-RESTART message is I don't think you will get one if the power is suddenly pulled from the router. It does do a SNMP-5-COLDSTART and SYS-6-BOOTTIME on boot up.

I did an actual power-cycle test of a router and the SYS-5-RESTART message is reliably generated at every startup, be it from the power cycle or the reload command (I was not able to provoke an on-demand crash ;).

If you use EEM applets with action syslog commands, the messages generated by those commands will be displayed in the same format as the debugging messages, regardless of their severity. This might be a problem if you decide to use different timestamps for debugging and regular syslog messages (or disable debugging timestamps completely).

Cisco IOS release 12.4(15)T brought (among a plethora of voice features) the logging to non-volatile storage, a nice-sounding name for the ability to write syslog messages into files on your flash memory (or an embedded disk, if you have one). To configure it, use the logging persistent [url directory] [size filesystem-size] [filesize logging-file-size] global configuration command:

• The directory argument specifies where you want the files to be stored (for example, flash:/logging).
• The filesystem-size specifies the maximum disk space the logging files can consume (once you exceed the limit, the oldest file is deleted)
• The logging-file-size parameter specifies the maximum size of each file (once the file grows too large, a new file is created).

Note: You can store the log files on the router's flash memory if it appears as a disk file system (check with the show file systems command). Wouldn't it be great if this feature would also work on USB drives ...

Cisco has in IOS release 12.3(4)T (integrated into 12.4) finally introduced features (long available in Unix and Windows) that slow down dictionary attacks on a router. On top of logging of login failures, you can also slow down the login process by delaying the router response after a login failure with the login delay seconds command.

On top of that, the you can configure the router to enter quiet mode after several login failures have been detected in specified timeframe with the login block-for seconds attempts tries within seconds configuration command.

The log-adjacency-changes OSPF configuration command was improved with the detail command that logs every step of OSPF adjacency establishment (sample printout below), making it a great troubleshooting tool.

%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from DOWN to INIT, Received Hello
%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from INIT to 2WAY, 2-Way Received
%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from 2WAY to EXSTART, AdjOK?
%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from EXSTART to EXCHANGE, Negotiation Done
%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from EXCHANGE to LOADING, Exchange Done
%OSPF-5-ADJCHG: Process 1, Nbr 172.16.0.21 on Serial0/0/0.100 from LOADING to FULL, Loading Done

The ability to replace router uptime with date and time in the logging messages with the service timestamps log datetime command was present in IOS for a long time, but I was always annoyed at timestamps when collecting syslog messages for demonstration purposes. The command to turn them off has also been available "forever", but was too obvious for me to try out ... the no service timestamps log command.

In a previous post, I've shown how you can log the changes in interactive user's privilege level. With the Cisco IOS Login Enhancements (introduced in IOS release 12.3(4)T, integrated in 12.4), you can also log all login successes and failures, even when using local user database (a similar functionality was previously achievable only when using central TACACS+ or RADIUS server).

The configuration commands to enable terminal access logging are login on-success log and login on-failure log. You can also specify that you want send SNMP traps in these circumstances (with the trap option) or that you only want to log every Nth attempt with the every n option.

The logging userinfo configuration command introduced in IOS release 12.3T (integrated into 12.4) starts logging of all changes in user privilege levels (as requested by enable or disable commands).

As part of Configuraton Change Notification and Logging feature, Cisco IOS stores the most recent configuration commands in a circular buffer and (optionally) sends them to syslog streams.

This feature is configured under the archive configuration mode with the log config command, which brings you to yet another configuration mode where you can fine-tune the parameters (they are obvious, on-router help is sufficient), for example:

archive
 log config
  logging enable 100
  notify syslog
  hidekeys

After you've enabled configuration command logging, you can use the show archive log config all command to inspect the logging buffer. You can also display commands entered in a particular session or by a selected user.

If you've configured notify syslog, every configuration command also triggers a syslog message similar to this one:

3d03h: %PARSER-5-CFGLOG_LOGGEDCMD: User:console logged command:interface loopback 0

Note: This feature logs only the configuration commands, if you want to log all commands, use TACACS+ or Embedded Event Manager.

The Cisco IOS’s AAA architecture contains many handy features, including authorizing and logging every CLI command executed on the router. Unfortunately, the AAA command accounting only supports TACACS+ as the AAA transport protocol, making it unusable in RADIUS environments.

You can use Embedded Event Manager as a workaround. The following configuration commands will log every command executed on the router.

Syslog has always been considered an undependable means of reporting network problems by serious network administrators as it runs over unreliable UDP transport. Sometime in the twilight zone between IOS releases 12.3T and 12.4, Cisco IOS got the capability to transport syslog messages over TCP with the logging host ip-address transport tcp port configuration command (the command is documented in 12.4 manuals but missing in 12.3T manuals).

Note: IOS implements standard syslog stream over TCP, not the more complex RFC 3195.

To support syslog over TCP, you also need TCP-capable syslog server. In Unix environments, you can use syslog-ng, on Windows, Kiwi syslog daemon is a perfect choice.

Note: to enable syslog over TCP in Kiwi Syslog Daemon, go to File/Setup/Inputs/TCP, click Listen for TCP Syslog messages and enter the desired TCP port number.

If you're still stuck with frame-relay connections (or use them in test environment, because it's easy to set up any-to-any connectivity between a larger number of routers), you were probably used to subinterface logging events reporting when the line protocol on a point-to-point subinterface would go up or down based on LMI DLCI status.

Very quietly, these logging events disappeared, first on 7500-series routers in IOS release 12.1(14), now they're gone by default on all platforms. If you still want to see what's going on with your frame-relay subinterfaces, you have to enter logging event subif-link-status configuration command on every subinterface.

I can only guess that some people that used the syslog events for network management were very surprised by the first (undetected) frame-relay failure following an IOS upgrade :)