IP rule not installed for PBR that only match a mark tag

[asyncnomi]

Description

When declaring pbr-map with a single mark as a match, let's say:

pbr-map MY_PBR seq 5
  match mark 200
  set nexthop 1.2.3.4
int enp5s0f0
  pbr-policy MY_PBR

A new routing table is correctly installed into the kernel consisting of a single default via 1.2.3.4 dev enp5s0f0
However no ip rule are installed, making the PBR useless.

On the contrary, adding a src-ip as a supplementary match works as expected:

pbr-map MY_PBR seq 5
  match src-ip 1.2.3.4/5
  match mark 200
  set nexthop 1.2.3.4
int enp5s0f0
  pbr-policy MY_PBR

Result in a correct rule: prio XXX from 1.2.3.4/5 fwmark 0xc8 iif enp5s0f0 lookup YYYYY being created

Version

FRRouting 10.1.1 (frr) on Linux(6.8.12-3-pve).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--sbindir=/usr/lib/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--enable-rpki' '--disable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

How to reproduce

Simply write a pbr-map using a single match based on mark.

Expected behavior

Would expect the rule: prio XXX from all fwmark 0xc8 iif enp5s0f0 lookup YYYYY being created

Actual behavior

No rule are created

Additional context

No response

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.

[asyncnomi]

Hmm, I just retry this from a fresh install from DEB pkg (version frr 8.5.6) onto a new machine and it works fine.

I then retry from a fresh install from deb.frrouting.org/frr pkg with version 10.1.1 on a new machine and it failed to create the rule, as observed previously

Here is the frr.conf

pbr-map MY_PBR seq 5
  match mark 129175
  set nexthop 10.1.0.254
int eth0
  pbr-policy MY_PBR

And the pbr map dump:

frr# show pbr map
  pbr-map MY_PBR valid: yes
    Seq: 5 rule: 304
        Installed: no Reason: Valid
        MARK Match: 129175
        nexthop 10.1.0.254
          Installed: yes Tableid: 10000

And the ip rule dump:

root@frr:/etc/frr# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

The same setup on frr v8 leads to

root@frr:/etc/frr# ip rule
0:      from all lookup local
304:    from all fwmark 0x1f897 iif eth0 lookup 10000 proto zebra
32766:  from all lookup main
32767:  from all lookup default

I'm also seeing (for frr v10.1.1) ip rule leftover not removed from previous configurations, sometimes, that's weird.

[k0ste]

Do you try to set nhg instead nh?

Like:

!
pbr-map MY_PBR seq 5
  match mark 129175
  set nexthop-group MY_PBR
exit
!
int eth0
  pbr-policy MY_PBR
exit
!
nexthop-group MY_PBR
 nexthop 10.1.0.254 weight 50
exit
!

This works for us:

example.com# sh pbr map
  pbr-map alertmanager valid: yes
    Seq: 5 rule: 304
        Installed: yes Reason: Valid
        MARK Match: 211
        Nexthop-Group: alertmanager
          Installed: yes Tableid: 10000

[bodleytunes]

Hi All,

I can also confirm wierd goings on with PBR in latest 10.x FRR. Im tagging with a fwmark and the pbr route shows not installed with 10.x but 8.5.6 works fine.

pbr-map ssh seq 10                                                                 
 match dst-port 22                                                                  
 match mark 2000                                                                
 set nexthop 10.97.97.254                                                     
exit 

And yes Im using NH groups.

Cheers!

Jon

[bodleytunes]

Scratch that! I wasn't using nexthop groups, and it seems next hop groups do work on 10.x 👍🏻

[asyncnomi]

Do you try to set nhg instead nh?

Like:

!
pbr-map MY_PBR seq 5
  match mark 129175
  set nexthop-group MY_PBR
exit
!
int eth0
  pbr-policy MY_PBR
exit
!
nexthop-group MY_PBR
 nexthop 10.1.0.254 weight 50
exit
!

This works for us:

example.com# sh pbr map
  pbr-map alertmanager valid: yes
    Seq: 5 rule: 304
        Installed: yes Reason: Valid
        MARK Match: 211
        Nexthop-Group: alertmanager
          Installed: yes Tableid: 10000

Yes, using nhg instead of nh works fine, but shouldn't I be able to directly use nh ? Is that a bad practice doing so ?