div Translations are available This

Found at: gopher.blog.benjojo.co.uk:70/are-bgps-security-features-working-yet-rpki

 Are BGPs security features working yet?
Translations are available
 This post is a textual version of a talk I gave at NLNOG
2018, You can watch the talk below if that's your preferred
 BGP has had a problem for quite a while, most of the
time when we hear about this in the news outside of the
networking word it is referred to as a "BGP Hijack". Which can be
better  phrased as "someone routed someone else's addresses to
 The reason that this is a thing is because BGP wasn't
exactly built with the idea of verifying ownership of IP address
blocks in mind, so the BGP software doesn't actually know if a
route to some IP addresses is legitimate or not.



 There are currently two suggestions to fix this, BGPSec and
RPKI, both of them involve cryptography in one way or another.
The current state of adoption means that BGPSec while defined,
is science fiction in the real world. The alternative RPKI
spec though is seeing deployment!
 RPKI works somewhat in the same way that TLS does. There
are a set of root certificate authorities and downstream people
start signing what their IP ranges should be announced as, and
who they should be announced by.
 Currently speaking, the deployment of RPKI signatures on IP
addresses is getting pretty good:

Global RPKI deployment

 Global RPKI deployment

RIPE region

 Or if we look at the RIPE region alone:

RIPE Region RPKI deployment

 RIPE Region RPKI deployment
 We see a significantly higher adoption.
 However this is not the whole story, It's great if you
are signing your IP ranges (also known as prefixes), but the
process of RPKI involves networks also validating the routes that
they import from their peers and vendors.
 RPKI Validation is a little messy, to start with you
likely need to run another device on your network that your
router talks to check if prefixes should be accepted or not. This
is not helped by the fact that software to do this is not
fully mature and documented yet and routers that have not been
maintained for a while will very likely need upgrading (and for some
vendors that upgrade carries significant risk of chaos)
 But let's recap on what RPKI is supposed to be preventing

a simple network setup

 a simple network setup

Internet eXchange

 This is a "standard" setup, here we have a ISP with a
single upstream transit peer, and a Internet eXchange. Here there
is a single path to a prefix (illustrated with a book)

a simple network hijack

 a simple network hijack
 The problem comes when someone evil on the IX announces
the prefix, Because either the routing path is shorter, or that
the router has been configured to be bias to the IX (because
it's cheaper). This is bad because the router has just followed
a hijacked prefix because it knew no better, It had no way
to know it was hijacked!

a simple network hijack with RPKI

 a simple network hijack with RPKI
 This is where RPKI comes in, because the prefix is now
locked down to the netmask + AS number, the hijack over the IX
will not validate and the router will prefer the correct route
over the bad one.
 Observant readers may suddenly realise a hole in this
design, since bypassing this would only require writing the path on
the attackers side to make the hijack prefix keep the original
AS number, and just have the hijackers ASN as a upstream.
 This does work, however if a router is still operating on
"shortest BGP path = best path" this reduces the chance that the
path will be accepted as the best one.
 Let's look at another situation:

a multihomed network

 a multihomed network
 Here we have another very common setup, in this case the
ISP does not have a internet exchange connection, it just has
two upstream ISPs.
 Let's say that the attacker has managed to have their
hijack reach transit providers, this is quite common as there are
still many providers who either have bad filters, or non existent
filters on what prefixes their customers should be able to

a multihomed network with RPKI

 a multihomed network with RPKI
 With RPKI filtering, both incoming BGP feeds should be
considered invalid for that prefix, and the router should have nowhere
to route the packets back. While packets can still come in
from that hijack, almost all systems need duplex communication to
function correctly (TCP being the critical one)

a multihomed network with RPKI with a default route

 a multihomed network with RPKI with a default route

default route

 However here comes the next issue in practise, In a lot
of cases providers will send a default route to their
customers, this provides traffic a route even if the provider has not
declared it could route traffic there. It can sometimes be useful
and valid in cases.
 Because it provides a "last ditch" method of getting
packets to a place. It means that these hijacks still work,
because even though the hijack route is rejected it will still
take a default route to get back there.
 Fixing this should be easy right? Just remove the default
route or blackhole traffic in the case of there being no valid
route to get there. Well here comes the next roadblock to

RIPE RPKI rollout with the invalid highlighted

 RIPE RPKI rollout with the invalid highlighted
 The business case for rejecting what could be 0.87% of
client traffic is a hard one to argue. In a ideal world this
0.87% of IP address prefixes should be doomed anyway. However
giving that RPKI deployment is only just starting to happen, these
prefixes for the most part can still access most of the internet.
 So. With all of this in mind, how does the internet
currently do on deployment of RPKI other than just signing
validation, giving that for RPKI to work you need to both sign and
 For this, we are going need to send out a lot of

The ICMP 9000

 The ICMP 9000
 Armed with prefixes that are purposefully signed to be
invalid, and a single valid prefix. I sent out many ICMP Pings to
all IP addresses using the "green" control prefix.

The ICMP 9000 firing

 The ICMP 9000 firing
 Should a response come back in, pings are sent out on
each of the other invalid "red" prefixes to test if a response
makes it back

The buckets

 The buckets
 This leaves us with buckets of hosts! Matching hosts inside
these buckets we can tell if their ISPs are doing RPKI
 If they appear only in the green bucket, then they are
doing RPKI validation correctly! If they appear in the green
bucket, and the red ARIN bucket, then they are employing some
level of RPKI validation, however have suffered from some default
configurations in software (more on that in a moment). If they are in
all buckets, then they are not doing any RPKI validation at
 However let's look at the final counts:

The bucket results

 The bucket results

you to download it separately

Relaying Party Agreement

 The lower the number, the more enforcement there is. As
you can see there is a 0.1 million difference between ARIN and
RIPE. This is likely because of the defaults of all RPKI
validation software. See, the ARIN TAL requires you to download it
separately and with that it carries a Relaying Party Agreement that
is 3 pages of legal speak that has some alarming statements
in there. This fact means that by default, the ARIN TAL (a
critical part in verifying a ARIN RPKI validity) is not included by
default in software.
 With that in mind, I can present to you the currently
leading AS numbers in RPKI deployment:

The green sheet

 The green sheet
 Or presented nicely:
 57598   | MD | ripencc  | SHA-AS, MD
 15426   | NL | ripencc  | XENOSITE Amsterdam, NL
 34968   | NL | ripencc  | IUNXI, NL
 35470   | NL | ripencc  | XL-AS, NL
 34762   | BE | ripencc  | COMBELL-AS, BE
 28878   | NL | ripencc  | SIGNET-AS, NL
 39647   | NL | ripencc  | REDHOSTING-AS, NL
 8455    | NL | ripencc  | ATOM86-AS ATOM86, NL
 21155   | NL | ripencc  | ASN-PROSERVE Amsterdam, NL
 197902  | NL | ripencc  | HOSTNET, NL
 24679   | DE | ripencc  | SSERV-AS, DE
 20559   | NL | ripencc  | FUNDAMENTS-AS, NL
 8608    | NL | ripencc  | QINIP Esprit Telecom B.V.,
 200831  | NL | ripencc  | MIHOSNET, NL
 30870   | NL | ripencc  | TRANS-IX-AS Trans-iX, NL
 29028   | NL | ripencc  | COMPUKOS-AS, NL
 24586   | NL | ripencc  | NL-INTERMAX B.V., NL
 34756   | NL | ripencc  | ASN-GVRH, NL
 8312    | NL | ripencc  | ZYLON-AS, NL
 202955  | NL | ripencc  | IAHOSTER, NL
 201975  | NL | ripencc  | UNISCAPEB IT-Services &
Hosting, NL
 41480   | NL | ripencc  | SYSTEMEC-AS, NL
 201290  | NL | ripencc  | BLACKGATE, NL
 39637   | NL | ripencc  | NETLOGICS-AS, NL
 8587    | NL | ripencc  | INFRACOM-AS, NL
 50554   | NL | ripencc  | NCBV-BACKBONE, NL
 61349   | NL | ripencc  | MAXITEL, NL
 58075   | NL | ripencc  | X2COM, NL
 59980   | NL | ripencc  | MIJNDOMEIN, NL
 24730   | NL | ripencc  | ASN-NETHOLDING, NL
 60820   | NL | ripencc  | WIFI4ALL-AS, NL
 202916  | NL | ripencc  | IPS, NL
 28747   | BE | ripencc  | EASYHOST-COLO-AS, BE
 34215   | NL | ripencc  | ATINET, NL
 42812   | NL | ripencc  | DT-IT, NL
 48729   | NL | ripencc  | O4S-AS, NL
 199456  | GB | ripencc  | VLDTECH-ASN, GB
 60950   | NL | ripencc  | CLOUDNL-AS, NL
 202016  | NL | ripencc  | DOMINOICT, NL
 61429   | NL | ripencc  | AS-CASTOR, NL
 35027   | NL | ripencc  | ASN-SEVENP, NL
 21073   | NL | ripencc  | ZORANET-AS Amsterdam, NL
 41153   | NL | ripencc  | GNTEL-AS, NL
 49627   | NL | ripencc  | SPEAKUP, NL
 61147   | NL | ripencc  | CALLHOSTED-AS Callhosted NL
 42585   | NL | ripencc  | NETWORKING4ALL, NL
 15703   | NL | ripencc  | TRUESERVER-AS TrueServer BV,
 15879   | NL | ripencc  | KPN-INTERNEDSERVICES, NL
 35260   | NL | ripencc  | IU-NET, NL
 62353   | NL | ripencc  | ASN-DATAPLACE, NL
 202947  | NL | ripencc  | Multi ICT B.V., Almere, NL
 34141   | NL | ripencc  | IN2IP-AS, NL
 41960   | NL | ripencc  | NEXTPERTISE Nextpertise, NL
 20495   | NL | ripencc  | WEDARE wd6.NET B.V, NL
 52144   | NL | ripencc  | NOTUBIZ, NL
 42755   | NL | ripencc  | DATAFIBER, NL
 A keen eye will notice that this list is 91% Dutch! With
some Belgium ISP's leading next at 3%
 All of the alive and protected hosts could fit in roughly
a /15 of address space (128k IP addresses)
 However if you sort this by total hosts active, rather
than RPKI compliance, the image looks a lot more bleak:

The red sheet

 The red sheet
 The first 100% RPKI Invalid dropper only appears at rank
#271 on this list.
 Using the data, we can also see that the providers that
have not downloaded the ARIN TAL. Either because they were not
aware that they needed to, or could not agree to the agreement
they have with it.

The mixed sheet

 The mixed sheet
 This is frustrating to watch. Since it means that ROA
signing ARIN prefixes will be less secure than ROA signing others,
until these restrictions are abolished. Even after that it will
have a long term knock on effect as software updates take a
long time to propagate to end networks.

someone hijacked a decent chunk of Amazons Route 53

 One good place for RPKI validation to start happening would
be DNS resolvers, Since this year someone hijacked a decent
chunk of Amazons Route 53 to steal some internet (cryptographic)
money. This event promoted Amazon to start signing some of their
prefixes (they don't however block invalid RPKI traffic on their
network yet, see a pattern?)
 To test adoption of RPKI in DNS resolvers I setup DNS
servers on the IP prefixes that were used to scan for ISP's
validating, and found that no major resolver sites behind a RPKI
validated network, that includes:,,,,

RIPE Atlas

 I even tested using RIPE Atlas and found only one probe
had a DNS resolver that blocked invalid RPKI prefixes, and that
probe was on a network listed above as a 100% compliant
 The future to a cryptographic signed prefix world is a
long way from reality, but that should hopefully not stop us
from trying.
 You can find the data generated by this post+talk here:

Formatted spreadsheet

 Formatted spreadsheet

Original CSVs

 Original CSVs
 I would like to thank some groups for helping out on

Job Snijders

  * Job Snijders

Nepal Research and Education Network

  * Nepal Research and Education Network (NREN)
  * Japan Network Information Center / PPP-EXP
  * NTT Communications

LARUS Cloud Service

  * LARUS Cloud Service Ltd

the rest of the blog



 If you liked this post, you may want to check out the
rest of the blog, to stay up to date on my new writings you
can either follow me on Twitter or use RSS


Advertising? Click here to get started!