【logstash】解析 sendmail 的 log grok pattern

動機

---

上一篇 【logstash】解析 postfix 的 log
既然 postfix 好了，打鐵趁熱
sendmail 也來一發

---

想法

---

最基掰的地方是， sendmail 會把收件人擠成一條
你得用些技巧去拆開他

---

動手做

---

先看一下log長怎樣

Nov 27 01:19:14 server server sendmail[16234]: tAQHJCEb016230: to=, ctladdr= (0/0), delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=114312, relay=a.b.c. [255.255.255.255], dsn=2.0.0, stat=Sent (OK 1448558354 z6si4967775igz.21 - gsmtp)

直接曬修改過後的 pattern
sendmailPattern/sendmail.grok

COMPID \[%{POSINT:pid}\]:
QUEUEID [A-za-z0-9]{14}
LOGIN [.a-zA-Z0-9_-]+
EMAIL %{LOGIN:local}@%{IPORHOST:remote}
DSN [0-9][.][0-9][.][0-9]
STAT_1 \(%{GREEDYDATA:result_message}\)
STAT_2 %{GREEDYDATA:result_message}
STAT (%{STAT_1}|%{STAT_2})

SENDMAILSMTP %{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{SYSLOGHOST:hostname2} sendmail%{COMPID:pid} %{QUEUEID:qid}: to=<%{EMAIL}>, (%{WORD}=%{DATA},)+ dsn=%{DSN:dsn}, stat=(%{DATA:result}:*|User unknown) %{STAT}
SENDMAILSMTPNODOMAIN %{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{SYSLOGHOST:hostname2} sendmail%{COMPID:pid} %{QUEUEID:qid}: to=(%{LOGIN:local},)+ (%{WORD}=%{DATA},)+ dsn=%{DSN:dsn}, stat=%{STAT:result}
SENDMAILSMTPOMS %{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{SYSLOGHOST:hostname2} sendmail%{COMPID:pid} %{QUEUEID:qid}: to=<%{EMAIL:to}>, (%{WORD}=%{DATA},)+ dsn=%{DSN:dsn}, stat=%{STAT:result}
SENDMAILSMTPMULTI %{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{SYSLOGHOST:hostname2} sendmail%{COMPID:pid} %{QUEUEID:qid}: to=(%{DATA:multiemail},) (%{WORD}=%{DATA},)+ dsn=%{DSN:dsn}, stat=(%{DATA:result}:*|User unknown) %{STAT}

然後 config 檔的部分

input {
        file {
                type => "maillog"
                path => ["/path/to/mail"]
        #       debug => "true"
        }
}
filter {

#沒有dsn的都不處理
        if ([message] !~ /dsn/){
                drop{}
        }
#處理sendmail的
        if ([message] =~ /sendmail/){
                grok {
                        patterns_dir => "/path/to/sendmailPattern"
                        match => {"message"=>"%{SENDMAILSMTP}|%{SENDMAILSMTPNODOMAIN}|%{SENDMAILSMTPOMS}|%{SENDMAILSMTPMULTI}"}
                }
        }
#處理sendmail多個收件人
        if ([multiemail] =~ /^<\S+|\S+@\S+,\S+/){
                #先換 在切
                mutate {
                        gsub => ["multiemail", "<", "", "multiemail", ">", ""]
                }
                split {
                        field => "multiemail"
                        terminator => ","
                }

                #再分割 local與remote
                if ([multiemail]=~ /\S+@\S+/){
                        grok {
                                patterns_dir => "/path/to/sendmailPattern"
                                match => {"multiemail"=>"%{EMAIL}"}
                        }
                }
}
output {
#       stdout {}
        elasticsearch {
                hosts => ["255.255.255.255:99999"]
                flush_size => 1000
                index => "maillog-%{+YYYY.MM.dd}"
                codec => "plain"
                workers => 1
                manage_template => true
                template_name => "logstash"
                template_overwrite => false
                idle_flush_time => 1
        }
}

這邊比較虐的地方是 split 會把他一分為二
然後你得再想辦法切他進 field
光想這個就破頭了
看來人笨是無極限的

---

驗證

---

# ./logstash agent --config /path/to/config

然後趁現在欣賞自己的正則一下
因為你明天就會忘掉