I was wondering if I can use the embedded tcpdump of Junos to monitor transit traffic.
I found a way to do it and this short post explains how to do that.
This tip works only on TRIO Line cards. My setup has been tested on Junos 12.3.
I used several features:
- inline port mirroring.
- loopback mode
- exception traffic (host-routed)
First of all, you need to find a free port on your chassis :) - not used - not connected - and configure it in loopback mode. This port may be down. Moreover configure on it a fake IP address with a fake next-hop (fake arp / mac entry).
set interfaces xe-8/0/0 gigether-options loopback
set interfaces xe-8/0/0 unit 0 family inet address 192.168.1.1/24 arp 192.168.1.2 mac 00:00:00:01:02:03
Then you can configure your port mirroring instance and choose the previous configured interface as the output interface for mirrored traffic. Here I configure a specific port-mirroring-instance on my MPC that receives the transit traffic.
set chassis fpc 4 port-mirror-instance LOCAL-DUMP
Then I set up my port-mirroring instance. Don't forget the no-filter-check knob
set forwarding-options port-mirroring instance LOCAL-DUMP input rate 1
set forwarding-options port-mirroring instance LOCAL-DUMP family inet output interface xe-8/0/0.0 next-hop 192.168.1.2
set forwarding-options port-mirroring instance LOCAL-DUMP family inet output no-filter-check
And finally apply mirroring (here in input direction) on interface you want to catch specific transit traffic. Here I want to punt to my local tcpdump the traffic sent to IP 60.0.3.1/32 on TCP port 80.
set firewall family inet filter MIRROR term 1 from destination-address 60.0.3.1/32
set firewall family inet filter MIRROR term 1 from protocol tcp
set firewall family inet filter MIRROR term 1 from port 80
set firewall family inet filter MIRROR term 1 then port-mirror-instance LOCAL-DUMP
set firewall family inet filter MIRROR term 1 then accept
set firewall family inet filter MIRROR term 2 then accept
set interfaces ae0 unit 0 family inet filter input MIRROR
At this step you can check that port-mirroring instance is UP.
sponge@bob> show forwarding-options port-mirroring
Instance Name: LOCAL-DUMP
Instance Id: 2
Input parameters:
Rate : 1
Run-length : 0
Maximum-packet-length : 0
Output parameters:
Family State Destination Next-hop
inet up xe-8/0/0.0 192.168.1.2
And also you can see stats traffic on the output interface xe-8/0/0 (remember this interface is in loopback mode)
interface: xe-8/0/0, Enabled, Link is Up
Encapsulation: Ethernet, Speed: 10000mbps
Traffic statistics: Current delta
Input bytes: 2594148207030 (0 bps) [0]
Output bytes: 2824068833619 (1168363304 bps) [1161604288]
Input packets: 1958284956 (0 pps) [0]
Output packets: 2411685324 (268465 pps) [2135302]
No worries! Here, the 268k pps are actually dropped at PFE level as normal discard without any impact.
sponge@bob> show pfe statistics traffic fpc 8 | match Normal
Normal discard : 1202137974
Now it's time to play with “exceptions”. The aim is to say to the PFE attached to output interface xe-8/0/0: "punt" some packets, discarded on xe-8/0/0, to the RE.
To do that I used an exception named as host-route-v4. This exception is triggered when a packet need to be routed by the RE (RIB). Actually it's never used in normal condition (or in rare cases). This exception is by default rate-limited to 2000pps. For security purposes I preferred to rate-limit this exception to 100pps for the MPC that hosts the output interface. Here the MPC in slot 8. I used a scale bandwidth of 5% of 2000pps to obtain my 100pps allowed for MPC 8.
To do that I add some configurations at ddos-protection level for this specific exception:
set system ddos-protection protocols unclassified host-route-v4 fpc 8 bandwidth-scale 5
Now to force host-routed, I created a new firewall filter with an action "next-ip" set to the local ip address of my output interface xe-8/0/0.
set firewall family inet filter to_DUMP term 1 then next-ip 192.168.1.1/32
And finally I applied this filter on the output interface xe-8/0/0:
set interfaces xe-8/0/0 unit 0 family inet filter input to_DUMP
After you have committed. You should see a DDOS-protection warning in syslog
Mar 17 11:07:01 bob jddosd[1882]: DDOS_PROTOCOL_VIOLATION_SET: Protocol Unclassified:host-route-v4 is violated at fpc 8 for 6 times
No worries again. We rate-limit this exception at 100pps (scale bandwidth of 5%).
sponge@bob> show ddos-protection protocols violations
Packet types: 190, Currently violated: 1
Protocol Packet Bandwidth Arrival Peak Policer bandwidth
group type (pps) rate(pps) rate(pps) violation detected at
uncls host-rt-v4 2000 0 0 2015-03-17 09:45:27 CET
Detected on: FPC-8
Important: if you have a firewall filter applied on your lo0 to protect your RE and if this firewall filter has a final term that discards all unauthorized traffic you should deactivate temporally this filter or term to allow punted traffic.
Now, let’s monitor traffic of interface xe-8/0/0 (our output interface) :
sponge@bob> monitor traffic interface xe-8/0/0.0 no-resolve
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is OFF.
Listening on xe-8/0/0.0, capture size 96 bytes
11:08:39.313460 In IP 161.0.0.1.60 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313586 In IP 161.0.0.1.61 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313589 In IP 161.0.0.1.64 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313592 In IP 161.0.0.1.65 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313594 In IP 161.0.0.1.62 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313596 In IP 161.0.0.1.644 > 60.0.3.1.80: . 0:504(504) win 0
11:08:39.313598 In IP 161.0.0.1.633 > 60.0.3.1.80: . 0:504(504) win 0
Sounds good! You can now use all matching criteria of the Junos tcpdump to analyze packets. Remember packets are rate-limited at 100pps, so you shouldn’t see the entire stream.
sponge@bob> monitor traffic interface xe-8/0/0.0 no-resolve matching "tcp src port 60" size 1500 detail print-ascii
Address resolution is OFF.
Listening on xe-8/0/0.0, capture size 1500 bytes
11:10:55.791814 In IP (tos 0x0, ttl 14, id 0, offset 0, flags [none], proto: TCP (6), length: 544) 161.0.0.1.60 > 60.0.3.1.80: . 0:504(504) win 0
0x0000 0200 0000 4500 0220 0000 0000 0e06 cad6 ....E...........
0x0010 a100 0001 3c00 0301 003c 0050 0000 0000 ....<....<.P....
0x0020 0000 0000 5000 0000 f666 0000 4745 5420 ....P....f..GET.
0x0030 2f64 6f77 6e6c 6f61 642e 6874 6d6c 2048 /download.html.H
0x0040 5454 502f 312e 310d 0a48 6f73 743a 2077 TTP/1.1..Host:.w
0x0050 7777 2e65 7468 6572 6561 6c2e 636f 6d0d ww.ethereal.com.
0x0060 0a55 7365 722d 4167 656e 743a 204d 6f7a .User-Agent:.Moz
0x0070 696c 6c61 2f35 2e30 2028 5769 6e64 6f77 illa/5.0.(Window
0x0080 733b 2055 3b20 5769 6e64 6f77 7320 4e54 s;.U;.Windows.NT
0x0090 2035 2e31 3b20 656e 2d55 533b 2072 763a .5.1;.en-US;.rv:
0x00a0 312e 3629 2047 6563 6b6f 2f32 3030 3430 1.6).Gecko/20040
0x00b0 3131 330d 0a41 6363 6570 743a 2074 6578 113..Accept:.tex
0x00c0 742f 786d 6c2c 6170 706c 6963 6174 696f t/xml,applicatio
Notice 1: Packets are only punted to RE when you call tcpdump command. You can check punted packets here:
sponge@bob> show pfe statistics traffic fpc 8 | match local
Packet Forwarding Engine local traffic statistics:
Local packets input : 1164 <<<<
Local packets output : 0
Notice 2: You can do the same for IPv6 traffic with exception host-route-v6.
David
