Ticket #13

Ticket #13

Network diagram

BGP diagram

The problem:

R3 was configured to send to BB1 only routes originated in AS3. So only R3 can ping BB1 from its Lo0. However, its not working, as no router can ping BB1 from it's Lo0. Find and fix the problem.

BB1's F0/0 IP address is 37.45.213.21.

The solution:


First, lets try to ping to BB1 from R3's Lo0.

On R3:

R3#ping 37.45.213.21 source loopback 0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.3 
.....
Success rate is 0 percent (0/5)


Indeed, R3 can't ping to BB1 from its loopback0.


Looking at the network diagrams, we can see that BB1 should have learned R3's Lo0 route via eBGP. Lets check R3 to see if it had formed a BGP neighbor relationship with BB1.

On R3:

R3#sh ip bgp neighbors 37.45.213.21 
BGP neighbor is 37.45.213.21,  remote AS 21, external link
  BGP version 4, remote router ID 0.0.0.0
  BGP state = Active
  Last read 02:53:01, last write 02:53:01, hold time is 180, keepalive interval is 60 seconds
  Message statistics:
    InQ depth is 0
    OutQ depth is 0
    
                         Sent       Rcvd
    Opens:                  4          4
    Notifications:          2          0
    Updates:               11          0
    Keepalives:           210        213
    Route Refresh:          0          0
    Total:                227        217
  Default minimum time between advertisement runs is 30 seconds

 For address family: IPv4 Unicast
  BGP table version 3, neighbor version 0/0
  Output queue size : 0
  Index 1, Offset 0, Mask 0x2
  1 update-group member
  Outbound path policy configured
  Outgoing update AS path filter list is 1
                                 Sent       Rcvd
  Prefix activity:               ----       ----
    Prefixes Current:               0          0
    Prefixes Total:                 0          0
    Implicit Withdraw:              0          0
    Explicit Withdraw:              0          0
    Used as bestpath:             n/a          0
    Used as multipath:            n/a          0
          
                                   Outbound    Inbound
  Local Policy Denied Prefixes:    --------    -------
    Total:                                0          0
  Number of NLRIs in the update sent: max 1, min 0
          
  Address tracking is enabled, the RIB does have a route to 37.45.213.21
  Connections established 2; dropped 2
  Last reset 02:53:08, due to User reset
  Transport(tcp) path-mtu-discovery is enabled
  TCP session must be opened passively
  No active TCP connection


We can learn several things from the command output:
  1. The connection is in "Active" state, which means that there is no BGP connection between R3 and BB1.
  2. There is no TCP connection.
  3. TCP connection is in passive mode.

We can now suspect that the connection between R3 and BB1 is configured to be passive on both R3 and BB1. Lets first check the BGP configuration of R3.

On R3:

R3#sh run | sec router bgp          
router bgp 3
 no synchronization
 bgp log-neighbor-changes
 redistribute connected route-map rmC2B
 neighbor 37.45.36.6 remote-as 6
 neighbor 37.45.213.21 remote-as 21
 neighbor 37.45.213.21 transport connection-mode passive
 neighbor 37.45.213.21 filter-list 1 out
 auto-summary

Indeed, we can see that the connection mode is passive. But how can we know that its also passive on BB1? The easiest way is to change the connection mode of R3 to BB1 and see if the connection is coming up.

On R3:

R3(config)#router bgp 3
R3(config-router)#no  neighbor 37.45.213.21 transport connection-mode passive
R3(config-router)#do clear ip bgp *
R3(config-router)#
*Jun 20 16:46:54.254: %BGP-5-ADJCHANGE: neighbor 37.45.36.6 Down User reset
R3(config-router)#
*Jun 20 16:46:55.650: %BGP-5-ADJCHANGE: neighbor 37.45.213.21 Up 
R3(config-router)#
*Jun 20 16:46:58.706: %BGP-5-ADJCHANGE: neighbor 37.45.36.6 Up 

Now the BGP session between R3 and BB1 is up and running. Lets try again to ping from R3's Lo0 to BB1.

On R3:

R3#ping 37.45.213.21 source lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.3
.....
Success rate is 0 percent (0/5)


Its not working, yet. Lets see what routes are advertised to BB1.

On R3:

R3#sh ip bgp neighbors 37.45.213.21 advertised-routes 

Total number of prefixes 0 

R3 is not sending any routes to BB1. Does R3 got any BGP routes in it own BGP table?

On R3:

R3#sh ip bgp
BGP table version is 3, local router ID is 37.46.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 37.0.0.0         0.0.0.0                  0         32768 ?
*> 37.46.0.6/32     37.45.36.6               0             0 6 i

We can see the R3's BGP table is not empty. 


We where told that R3 was configured with filtering policy to send to BB1 only routes originated in AS3. Lets examine how it was configured.


On R3:

R3#sh run | sec router bgp
router bgp 3
 no synchronization
 bgp log-neighbor-changes
 redistribute connected route-map rmC2B
 neighbor 37.45.36.6 remote-as 6
 neighbor 37.45.213.21 remote-as 21
 neighbor 37.45.213.21 filter-list 1 out
 auto-summary
R3#sh run | i as-    
ip as-path access-list 1 permit ^3$


R3 was configured with an as-path ACL. And looking at the ACL, we can see a common error. The as-path ACL is filtering entries from the BGP table to be sent to BB1. In R3's BGP table, the AS-PATH of locally originated routes is empty. Its not "3".

Lets fix the as-path ACL.

On R3:

R3(config)#no ip as-path access-list 1 permit ^3$
R3(config)#ip as-path access-list 1 permit ^$ 
R3(config)#do clear ip bgp 21 out
R3(config)#do sh ip bgp neighbors 37.45.213.21 advertised-routes
BGP table version is 3, local router ID is 37.46.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 37.0.0.0         0.0.0.0                  0         32768 ?

Total number of prefixes 1 


Now we can see a route being sent to BB1. Lets try again to ping from R3.

On R3:

R3#ping 37.45.213.21 so lo 0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.3 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms


Success! R3 can ping BB1 from it's Lo0 interface. Now lets verify that R6 can't do the same.

On R6:

R6#ping 37.45.213.21 source loopback 0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.6 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/8/8 ms

Success?? How BB1 knows about R6's Lo0 route? Does R3 sends non AS3 routes to BB1? Lets check this again.

On R3:

R3#sh ip bgp neighbors 37.45.213.21 advertised-routes
BGP table version is 3, local router ID is 37.46.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 37.0.0.0         0.0.0.0                  0         32768 ?

Total number of prefixes 1 

R3 does not send routes from AS6, but it does sends a strange route. It looks like a /8 route, which also covers R6's Lo0 route. It smells like summarized route. Let check the configuration for any summarization.

On R3:

R3#sh run  | sec router bgp
router bgp 3
 no synchronization
 bgp log-neighbor-changes
 redistribute connected route-map rmC2B
 neighbor 37.45.36.6 remote-as 6
 neighbor 37.45.213.21 remote-as 21
 neighbor 37.45.213.21 filter-list 1 out
 auto-summary

We have found a fossil, named "auto-summary", which summarizes redistributed routes when crossing major network boundaries. Lets remove it and try again to ping from R3 and from R6.

On R3:

R3(config)#router bgp 3
R3(config-router)#no auto-summary
R3(config-router)#do clear ip bgp *
R3(config-router)#
*Jun 20 19:47:43.805: %BGP-5-ADJCHANGE: neighbor 37.45.36.6 Down User reset
*Jun 20 19:47:43.813: %BGP-5-ADJCHANGE: neighbor 37.45.213.21 Down User reset
R3(config-router)#
*Jun 20 19:47:46.953: %BGP-5-ADJCHANGE: neighbor 37.45.36.6 Up 
*Jun 20 19:47:46.957: %BGP-5-ADJCHANGE: neighbor 37.45.213.21 Up 
R3(config-router)#do sh ip bgp neighbors 37.45.213.21 advertised-routes
BGP table version is 3, local router ID is 37.46.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 37.46.0.3/32     0.0.0.0                  0         32768 ?

Total number of prefixes 1 
R3(config-router)#do ping 37.45.213.21 so lo 0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.3 
!!!!.
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/5/8 ms

On R6:

R6#ping 37.45.213.21 source loopback 0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 37.45.213.21, timeout is 2 seconds:
Packet sent with a source address of 37.46.0.6 
.....
Success rate is 0 percent (0/5)

Success! We can see that R3 is sending only its Lo0 route to BB1, and that only R3 can ping BB1 from its Lo0.


                                                                   __ 

.-----.--.--.----.----.-----.-----.-----.|  |
|__ --|  |  |  __|  __|  -__|__ --|__ --||__|
|_____|_____|____|____|_____|_____|_____||__|