rsyslogd dynamically create log file based on msg content

This is tested on a Ubuntu 10.04 system (should work on newer Ubuntu/Debian system too)

1. Create file /etc/rsyslog.d/30test.conf, with the following content:

$template DynFile,"/tmp/test-%msg:7:18%.log"

:msg,startswith," ABCD-001122334455" ?DynFile

#:syslogtag,startswith,"test" ?DynFile

#:syslogtag,startswith,"test" /tmp/test.log

2. Open file /etc/rsyslog.conf, and make the following modification for the following line:

$FileCreateMode 0644

comment out the following lines:

#$PrivDropToUser syslog

#$PrivDropToGroup syslog

3. sudo service rsyslog restart

4. logger -t test "ABCD-001122334455sdfsfsdfsdfsdfsdfs-----------"

5. Now check file /tmp/test-*.log, and you should see file /tmp/test-001122334455.log

To Log message with a defined template
1. First define the template
      $template shortlog,"%msg:10:1000%"

2. Use the template by appending ";template-name" to the end of the output file

Notes:

a. Property msg starts with a space " ". 

b. Eventually we should try to figure out why dropPriviledge does causes issues here.

TIPS:

A good debugs tip:

1. add /etc/rsyslog.d/debug.conf

*.* /var/log/all.log;RSYSLOG_DebugFormat
#Note here ";RSYSLOG_DebugFormat" is the output template

2. restart rsyslog

3. watch the file /var/log/all.log to see all log messages with property names

Log with UNIX Timestamp
$template unixTS,"%timegenerated:::date-unixtimestamp%,%msg%\n"
:msg,contains,"[UFW " /var/log/ufw.log;unixTS

#Note here ";unixTS" is the output template

http://www.rsyslog.com/tag/use-a-template/
Also refer to the rsyslog PDF version of the manual (Chapter 1) for overview of how rsyslogd processes messages)

Syntax Check

rsyslogd -N1

===============UPDATE: New Syntax for Syslog version 7 and newer =============
rsyslog's new syntax is called "RainerScript". To generate dynamic files using output templates, generate a file in /etc/rsyslogd.conf/10-mytest.conf with the following content, and then restart service rsyslogd. The following example only logs message from the application "abc" and the message itself starts with "ABC" to a dynamic file name.

template(name="myfilename" type="string" string="/tmp/my-%programname%.log")
template(name="shortlog" type="string" string="%msg:10:$%\n")
if  $programname == 'abc' and $msg startswith " ABC"  then {
    action(type="omfile" dynaFile="myfilename" template="shortlog" )

}

=============UPDATE: More examples==================
1. variables are set using "set" and they start with "$!" in Rainer Scripts
2. variables only work inside the if block
3. Inside template the variables are surrounded by %%. e.g. "%$!devid%"
4. "stop" has replaced the "~" to discard a particular message.

module(load="omprog")
template(name="devid" type="string" string="%$!devid%")
if ($programname == 'charon') then {
    if ($msg contains "deleting IKE_SA")  then {
        set $!devid= re_extract($msg, "([A-Z0-9])+-([A-Z0-9])+.vpnclient.meetcircle.co", 0, 0, "unknown");
        #action(type="omfile" file="/var/log/delete.log" template="devid" )
        action(type="omfwd" Target="192.168.1.2" Port="15140" Protocol="tcp" template="devid" )
        #action(type="omprog" binary="/usr/bin/rsyslogpost.sh" template="devid")
    }
}
:programname,isequal,"charon" stop