BGPKIT Blog

bgpkit-parser 0.18.0 Release

Mingwei Zhang — Thu, 02 Jul 2026 00:00:00 GMT

bgpkit-parser 0.18.0 preserves more of the original BGP wire data. Earlier versions parsed many common attributes into typed Rust structures, but element conversion could drop known attributes that did not yet have a semantic parser. That worked for many analysis workflows, but it was not enough for users who need to inspect, serialize, or re-encode less common BGP path attributes.

This release changes that behavior: the parser now retains known-but-unsupported, malformed, deprecated, and unknown BGP path attributes in AttrRaw-backed variants instead of silently dropping them.

Install

cargo add bgpkit-parser@0.18.0

For the WebAssembly package:

npm install @bgpkit/parser@0.18.0

RIS Live raw message parsing

RIS Live's JSON messages expose a convenient projection of UPDATE data, but that projection does not include every BGP path attribute. For example, attributes such as Only to Customer (OTC, code 35), BGP Prefix-SID, BFD Discriminator, and other less common path attributes may appear in the raw UPDATE while remaining absent from the projected JSON fields. Version 0.18.0 adds helpers that parse RIS Live messages containing the original raw BGP UPDATE bytes.

To request raw BGP messages from RIS Live, set socketOptions.includeRaw in the subscription message. The crate now re-exports RisSubscribe, RisSubscribeType, and RisLiveClientMessage from the crate root, so examples and downstream applications can build subscription messages directly.

use bgpkit_parser::{RisLiveClientMessage, RisSubscribe};

let subscribe = RisSubscribe::new()
    .with_host("rrc00")
    .with_include_raw(true);

let msg = RisLiveClientMessage::RisSubscribe(subscribe).to_json_string().unwrap();
// send `msg` over a WebSocket connection to RIS Live

When a RIS Live message arrives, parse the full ris_message envelope with the raw parser:

use bgpkit_parser::parse_ris_live_message_raw;

let elems = parse_ris_live_message_raw(message_text).expect("parse RIS Live message");
for elem in elems {
    println!("{} {} {:?}", elem.elem_type, elem.prefix, elem.as_path);
}

Users who prefer RIS Live's JSON-projected fields can call parse_ris_live_message_json() for the explicit JSON-path parser, or keep using the older parse_ris_live_message() function for the same JSON-projected path.

Raw BGP attribute retention

The main parser now keeps raw bytes for attributes it cannot or should not decode semantically. The new AttributeValue::Raw(AttrRaw) variant stores the wire attribute code and value bytes for known-but-unsupported attributes:

use bgpkit_parser::models::bgp::attributes::AttributeValue;

for attr in attributes.inner.iter() {
    match attr {
        AttributeValue::Raw(raw) => {
            println!("raw attribute code={} len={}", raw.code, raw.bytes.len());
        }
        other => {
            println!("typed attribute code={}", other.attr_code());
        }
    }
}

This matters for round-trip workflows. If a route contains a known attribute that bgpkit-parser does not yet decode into a typed model, 0.18.0 preserves and re-encodes the original attribute bytes.

Malformed attributes that target typed parsers now fall back to Raw after recording a validation warning, rather than disappearing from the parsed attribute list.

Deprecated attribute code handling

The IANA registry lists attribute code 13 as deprecated: RCID_PATH / CLUSTER_ID. Version 0.18.0 removes AttrType::CLUSTER_ID and retains code 13 as deprecated raw data instead.

If downstream code exhaustively matches AttrType or AttributeValue, this is one of the release's breaking changes. Use AttributeValue::attr_code() when the raw code point matters, and AttrRaw::attr_type() when you want to map the raw wire code back to an AttrType value.

New typed BGP path attribute parsers

This release adds typed parsers and round-trip encoding support for several RFC-defined path attributes:

RFC 7311 — AIGP, attribute code 26
RFC 9015 — SFP, attribute code 37
RFC 9026 — BFD Discriminator, attribute code 38
RFC 8669 — BGP Prefix-SID, attribute code 40
RFC 9793 — BIER, attribute code 41

Each parser preserves unknown TLVs, so applications can inspect known fields without losing the rest of the attribute payload.

For example, AIGP exposes an accumulated_metric() helper:

use bgpkit_parser::models::bgp::attributes::AttributeValue;

for attr in attributes.inner.iter() {
    if let AttributeValue::Aigp(aigp) = attr {
        if let Some(metric) = aigp.accumulated_metric() {
            println!("AIGP accumulated metric: {metric}");
        }
    }
}

API changes to note

The main breaking changes affect raw attribute representation and exhaustive matches:

AttrRaw now stores code: u8 instead of attr_type: AttrType.
AttrRaw.bytes now uses bytes::Bytes instead of Vec<u8>.
AigpTlv.value now uses Bytes.
AttributeValue has new variants: Raw, BfdDiscriminator, BgpPrefixSid, Bier, and Sfp.
The release removes AttrType::CLUSTER_ID.

For most users who iterate over BgpElem, the visible behavior should stay the same except that the parser preserves more attributes internally. Code that destructures attributes or performs exhaustive matching will need updates.

WebAssembly package

The @bgpkit/parser npm package also ships 0.18.0. The WASM package continues to expose parsers for BGP UPDATE, BMP/OpenBMP, and MRT data in Node.js, bundler, browser, and Worker environments.

import { parseBgpUpdate } from '@bgpkit/parser';

const elems = parseBgpUpdate(updateBytes);
for (const elem of elems) {
  console.log(elem.type, elem.prefix, elem.as_path);
}

The Rust crate includes RIS Live parsing in this release. We track WASM-level RIS Live parser exports and WebSocket helpers separately.

Finding Well-Known BGP Communities with monocle

Mingwei Zhang — Thu, 25 Jun 2026 00:00:00 GMT

BGP communities are easy to parse and hard to interpret. A route can carry a list of 32-bit values, but the meaning of each value depends on a mix of RFCs, IANA registries, operator conventions, and local routing policy.

This post walks through a small ad-hoc monocle analysis for turning raw MRT updates into a protocol-aware summary:

search public BGP updates for communities in the 65535:* range,
summarize where each value appears,
join observed values with the IANA BGP Well-known Communities registry,
check specific values, such as BLACKHOLE and NOPEER, against the relevant RFC text.

The goal is not to infer operator intent from route collector data. This is a one-off research note, not production automation. The goal is narrower: show how BGPKIT tooling can make this kind of evidence gathering easier to repeat and audit.

Why `65535:*`?

RFC 1997 introduced the BGP COMMUNITIES path attribute. In the "Well-known Communities" part of the document, it defines several globally significant values and says:

The following communities have global significance and their operations shall be implemented in any community-attribute-aware BGP speaker.

The current IANA registry is the BGP Well-known Communities registry.

The registry divides the community space into ranges, including:

Range	Registration rule
`0x00000000-0x0000FFFF`	Reserved
`0x00010000-0xFFFEFFFF`	Private Use
`0xFFFF0000-0xFFFF8000`	First Come First Served
`0xFFFF8001-0xFFFFFFFF`	Standards Action

That makes values with high-order 16 bits set to 65535 interesting: they sit in the part of the 32-bit community space where many globally significant values live.

Setup

On macOS, install monocle with Homebrew:

brew install monocle

If you prefer building from Rust crates, or are not using Homebrew:

cargo install monocle

For reproducibility, record the version before running the scan:

monocle --version

This run used:

monocle 1.3.0

This analysis used the public MRT index available to monocle at run time. To make the collector/file set auditable, first write the matching MRT file list:

START=1782324000   # 2026-06-24T18:00:00Z
END=$((START+900)) # 15 minutes
OUT=updates-20260624T1800Z-15m-allcollectors-wk

monocle search \
  --dump-type updates \
  --start-ts $START \
  --end-ts $END \
  --community '65535:*' \
  --broker-files \
  > "$OUT-files.txt"

wc -l "$OUT-files.txt"

For this run, monocle found 125 matching MRT update files.

The exact file count can vary if the public collector index changes or if late files appear. If you need a fixed rerun, save this file list with the analysis artifacts and use the same monocle version.

Searching MRT updates with `monocle`

For this run, I scanned the same 15-minute update window:

START=1782324000   # 2026-06-24T18:00:00Z
END=$((START+900)) # 15 minutes
OUT=updates-20260624T1800Z-15m-allcollectors-wk

monocle search \
  --dump-type updates \
  --start-ts $START \
  --end-ts $END \
  --community '65535:*' \
  --fields timestamp,collector,prefix,peer_asn,origin_asns,as_path,communities \
  --format json-line \
  > "$OUT.jsonl"

A representative JSONL row looks like this:

{
  "timestamp": 1782324014.12,
  "collector": "rrc07",
  "prefix": "83.171.244.0/24",
  "peer_asn": 3399,
  "origin_asns": ["197761"],
  "as_path": "3399 5511 6204 51202 51202 197761",
  "communities": ["3399:100", "3399:107", "5511:500", "6204:1000", "65535:65284"]
}

The important part is the community filter:

--community '65535:*'

monocle handles the collector/file discovery and MRT parsing. The output is JSON Lines, so the next step can be a short post-processing script rather than a custom MRT parser.

For this post, I used small Python scripts rather than building a reusable tool. The main script, summarize_existing_well_known.py, groups rows by observed 65535:* value and counts rows, distinct prefixes, collectors, peer ASNs, origin ASNs, address families, and prefix lengths. It also normalizes symbolic community names that monocle may emit, such as no-export, back to their numeric well-known-community values before doing the 65535:* grouping.

python3 summarize_existing_well_known.py "$OUT.jsonl" "$OUT"

The second script, enrich_iana_status.py, joins the per-community CSV against a local transcription of the IANA BGP Well-known Communities registry, so each observed value can be labeled as BLACKHOLE, NOPEER, GRACEFUL_SHUTDOWN, Unassigned, and so on. The registry source is the IANA BGP Well-known Communities registry, checked for this write-up on 2026-06-25.

python3 enrich_iana_status.py \
  "$OUT-by-community.csv" \
  "$OUT-by-community-iana.csv"

What the scan found

In this 15-minute window:

Metric	Value
Matching update rows	769,537
Distinct numeric `65535:*` values	109
Values not assigned in the IANA registry snapshot	102

The highest-volume values were:

Community	IANA status	Rows	Distinct prefixes	Collectors
`65535:6939`	Unassigned	553,352	20	27
`65535:1200`	Unassigned	63,194	10,455	2
`65535:34`	Unassigned	41,584	6,022	58
`65535:30`	Unassigned	37,436	2,896	54
`65535:65284`	NOPEER, RFC 3765	27,766	1,033	51
`65535:9957`	Unassigned	13,245	1,125	1
`65535:0`	GRACEFUL_SHUTDOWN, RFC 8326	2,176	210	51
`65535:666`	BLACKHOLE, RFC 7999	1,270	499	18

This table is a good example of why registry-aware scripting helps. A raw list of communities is just numbers. Once joined with IANA, the same data separates into named RFC-defined values and values that need local/operator-specific interpretation.

Importantly, "unassigned" does not mean "bad". Public BGP data alone does not tell us whether a value is a local convention, a route-server policy signal, a legacy convention, an implementation artifact, or something else. It only tells us where the value appeared and gives us concrete rows to inspect next.

Checking BLACKHOLE behavior against RFC 7999

BLACKHOLE is assigned as 65535:666 by RFC 7999.

Two parts of the RFC are useful for a mechanical check.

RFC 7999 Section 3.2 says:

A BGP speaker receiving an announcement tagged with the BLACKHOLE community SHOULD add the NO_ADVERTISE or NO_EXPORT community as defined in RFC 1997, or a similar community, to prevent propagation of the prefix outside the local AS.

RFC 7999 Section 3.3 says:

However, blackhole prefix length should be as long as possible in order to limit the impact of discarding traffic for adjacent IP space that is not under DDoS duress. The blackhole prefix length is typically as specific as possible, /32 for IPv4 or /128 for IPv6.

Those sentences translate into simple checks:

is the prefix /32 for IPv4 or /128 for IPv6?
does the route also carry NO_EXPORT or NO_ADVERTISE?

In the 15-minute scan, 65535:666 appeared in 1,270 rows:

Observed property	Rows with property	Rows without property
`/32` or `/128`	474 (37.3%)	796 (62.7%)
Carries `NO_EXPORT`	460 (36.2%)	810 (63.8%)
Carries `NO_ADVERTISE`	24 (1.9%)	1,246 (98.1%)

Address-family split: 463 IPv4 rows (36.5%) and 807 IPv6 rows (63.5%).

The resulting rows include examples that match the RFC 7999 profile:

{
  "collector": "decix.fra",
  "peer_asn": 6695,
  "prefix": "193.56.63.113/32",
  "origin_asns": ["215514"],
  "as_path": "201053 50607 215514",
  "communities": ["65535:666", "no-export"]
}

It also found examples that lack those observed properties. This is not a compliance judgment: RFC 7999 Section 3.2 describes behavior for a receiving BGP speaker, and collector data may show the route before or after other policy actions. Still, rows like this are useful starting points for follow-up:

{
  "collector": "iix.cgk",
  "peer_asn": 7597,
  "prefix": "2001:4:112::/48",
  "origin_asns": ["112"],
  "as_path": "136106 35280 50628 44097 112",
  "communities": ["65535:666", "7597:1000:1"]
}

That is as far as the data should be taken without more context. The collector peer may be a route server. The community may have been attached earlier in the AS path. The value may reflect a local convention. The useful part is that this ad-hoc monocle analysis can separate these cases quickly and produce concrete rows for follow-up.

Checking NOPEER visibility against RFC 3765

NOPEER is assigned as 65535:65284 by RFC 3765.

RFC 3765 Section 2 describes it as:

an advisory qualification to readvertisement of a route prefix, permitting an AS not to readvertise the route prefix to all external bilateral peer neighbour AS's.

In the same 15-minute update window, NOPEER was broadly visible:

Metric	Value
Rows	27,766
Distinct prefixes	1,033
Collectors	51
Peer ASNs	103
Origin ASNs	146
IPv4 rows	25,387
IPv6 rows	2,379

Example row:

{
  "collector": "rrc07",
  "peer_asn": 3399,
  "prefix": "83.171.244.0/24",
  "origin_asns": ["197761"],
  "as_path": "3399 5511 6204 51202 51202 197761",
  "communities": [
    "3399:100",
    "3399:107",
    "5511:500",
    "6204:1000",
    "65535:65284"
  ]
}

Route data alone does not prove whether any downstream AS enforces the advisory signal. But it does show that the value is operationally present across many collectors, peer ASNs, and origin ASNs in this window.

A note on policy behavior

It is worth being careful when interpreting well-known communities. RFC 8642 exists because well-known community handling has not always been consistent across implementations and policies. Its introduction says:

In hindsight, RFC 1997 did not prescribe as fully as it should have how well-known communities may be manipulated by policies applied by operators. Currently, implementations differ in this regard, and these differences can result in inconsistent behaviors that operators find difficult to identify and resolve.

That is another reason to keep the analysis evidence-focused. The data can say which routes carried which values at which collectors; it should not assign intent by itself.

Takeaways

A small ad-hoc monocle analysis can turn public MRT updates into useful, RFC-aware evidence. The first step is the MRT query; the second step is ordinary JSONL processing that groups rows, counts distinct routing dimensions, and annotates observed values with registry metadata.

The interesting parts are not hidden inside the MRT files anymore:

which 65535:* values are visible,
whether those values are assigned in the IANA registry,
where the values appear,
whether a route matches a simple RFC-derived profile,
which rows are worth deeper operator-policy investigation.

For this particular window, the main observations were:

many observed 65535:* values were not assigned in the current IANA well-known-community registry snapshot;
BLACKHOLE appeared both on highly specific prefixes with NO_EXPORT and on broader prefixes without NO_EXPORT/NO_ADVERTISE;
NOPEER was broadly visible across collectors, peers, origins, IPv4, and IPv6.

The broader lesson is the monocle research pattern: query a narrow slice of MRT data, summarize it with simple scripts, join with source-of-truth registries, then inspect specific rows with the RFC text next to the data.

The scripts used for this note are available here:

summarize_existing_well_known.py — summarize an existing JSONL file;
enrich_iana_status.py — add IANA status labels to the per-community CSV;
filter_well_known.py — filter generic JSONL output to rows carrying well-known communities;
summarize_well_known.py — stream-oriented variant that reads JSONL from stdin.

Investigating the India–Telegram BGP Hijack with monocle

Mingwei Zhang — Sat, 20 Jun 2026 00:00:00 GMT

On June 16, 2026, Indian carrier Reliance Communications (AS18101) began originating Telegram's IP prefixes in BGP, disrupting access for users inside and briefly outside India. The event has been covered from a policy angle (Anurag Bhatia's analysis, the IPv6 blackholing follow-up) and a traffic-impact angle (Kentik's post). This post does something different: it shows how to investigate the forensic evidence yourself using monocle and the BGPKIT API - open-source tools, real commands, real output. No screenshots of a vendor portal, no proprietary telemetry. Everything here you can run on your own machine.

The goal is forensics methodology, not opinion. We won't speculate about intent or mitigation policy. We'll establish ground truth, find the hijack in the data, measure how it propagated, and check the RPKI timeline - the same steps you'd follow for any BGP origin-conflict incident.

Prerequisites

brew install monocle  # monocle 1.3.0 used in this post
# cargo install monocle          # also works, but takes longer to build from source

All timestamps below are UTC. monocle accepts RFC 3339 strings and converts them internally:

$ monocle time "2026-06-16T16:14:19Z" -s
2026-06-16T16:14:19+00:00

Step 1: Establish ground truth - who should originate this prefix?

Before touching any MRT data, establish the expected state of the prefix. This is free and instant, and it gives you the baseline to compare against.

Telegram operates five ASNs: AS62041, AS62014, AS59930, AS44907, and AS211157. A single monocle inspect call pulls all of them:

monocle inspect 62041 62014 59930 44907 211157

ASN | Name | Country | Org
--- | --- | --- | ---
AS62041 | Telegram Messenger EUR | VG | Telegram Messenger Inc
AS62014 | Telegram Messenger APAC | VG | Telegram Messenger Inc
AS59930 | Telegram Messenger NA  | VG | Telegram Messenger Inc
AS44907 | Telegram Messenger Inc | VG | Telegram Messenger Inc
AS211157| Telegram Messenger Inc | VG | Telegram Messenger Inc

Now check who currently announces the prefix that was reported hijacked, 91.105.192.0/23:

monocle pfx2as 91.105.192.0/23 --show-name

The legitimate origin is AS211157 (Telegram). The suspect, AS18101 (Reliance Communications), should not appear here in a clean table.

The single most important check: RPKI validation

$ monocle rpki validate 91.105.192.0/23 18101 --format json

{
  "prefix": "91.105.192.0/23",
  "asn": 18101,
  "state": "invalid",
  "reason": "ASN 18101 not authorized; authorized ASNs: 211157",
  "covering_roas": [
    {"prefix": "91.105.192.0/23", "max_length": 23, "origin_asn": 211157, "ta": "RIPENCC"}
  ]
}

The prefix is covered by a ROA authorizing AS211157 only. AS18101 originating it is RPKI-invalid by definition. Any network enforcing Route Origin Validation (ROV) should have rejected this announcement. Confirm the legitimate origin validates clean:

$ monocle rpki validate 91.105.192.0/23 211157 --format json
{"prefix":"91.105.192.0/23","asn":211157,"state":"valid",...}

You now know: the prefix is signed, the legitimate origin is valid, and the suspect origin is invalid - before downloading a single MRT file.

List Telegram's signed prefixes

To understand what address space was protected by RPKI, pull every Telegram ROA across all five ASNs:

for asn in 62041 62014 59930 44907 211157; do
  monocle rpki roas $asn --format json-line
done | grep '^{' \
  | jq -r '"\(.prefix) AS\(.origin_asn)"' \
  | sort -u

This returns 55 ROAs - Telegram's signed prefix set, from 91.105.192.0/23 up through 149.154.160.0/22 and the 91.108.x.0/22 blocks. Every one of these prefixes, if originated by AS18101, would be RPKI-invalid.

Understand the suspect's connectivity

monocle as2rel 18101 --is-downstream --show-name  # upstreams
monocle as2rel 18101 15412 --format json | jq '.results[0]'  # relationship with FLAG

Reliance's primary upstreams are AS15412 (FLAG Telecom) and AS4755 (Tata Communications). Keep these in mind - they're the transits that would carry (or filter) any announcement Reliance originates.

Step 2: Find the hijack in the data

Now look for the actual BGP announcements. The reported time is 16:14:19 UTC on June 16. Start with a narrow 5-minute window and a single small collector.

Why a small collector?

rrc01 (London) and route-views2 are large collectors with hundreds of peers. Searching them downloads and parses a lot of data. For a first confirmatory search, a small collector is much faster - and if the event is real, a small collector that has a peer seeing the path will confirm it just as well.

Use the BGPKIT API to find collector peer counts:

$ curl -s "https://api.bgpkit.com/v3/broker/peers?collector=rrc14" | jq '.data | length'
21

rrc14 (Palo Alto, 21 peers) is small. Let's search there:

monocle search -p 91.105.192.0/23 -s -S -o 18101 \
  -t 1781626459 -d 5m -c rrc14 \
  --cache-dir ~/.cache/monocle \
  --format json-line \
  | jq -c '.timestamp |= todateiso8601'

{"timestamp":"2026-06-16T16:14:49Z","type":"ANNOUNCE","prefix":"91.105.192.0/23",
 "as_path":"19151 15412 18101","peer_asn":19151,"collector":"rrc14",...}

Confirmed. At 16:14:49 UTC, rrc14's peer AS19151 received 91.105.192.0/23 originated by AS18101, via the path 19151 15412 18101 - meaning FLAG (AS15412) was the transit carrying the hijack to the global table.

Why `-s -S`?

The -p prefix filter is exact-only by default. Hijacks routinely involve more-specifics - the victim announces /24s to fight back, the hijacker follows with even more specific routes. A bare -p 91.105.192.0/23 misses both the covering aggregate and any more-specifics. -s (include-super) and -S (include-sub) expand the match. Get in the habit of adding both to any hijack search.

The prefix arms race

Widen the search slightly (still one collector, still cached) to see the full set of prefixes Reliance originated in the escalation wave:

monocle search -p "91.108.4.0/22,149.154.160.0/22,91.105.192.0/23,95.161.64.0/20" \
  -s -S -o 18101 -t 1781594400 -T 1781630400 -c rrc14 \
  --cache-dir ~/.cache/monocle --format json-line \
  | jq -r '"\(.timestamp | todateiso8601) \(.prefix)"' | sort -u

2026-06-16T16:14:49Z  149.154.160.0/24
2026-06-16T16:14:49Z  149.154.161.0/24
2026-06-16T16:14:49Z  149.154.162.0/24
2026-06-16T16:14:49Z  149.154.163.0/24
2026-06-16T16:14:49Z  91.105.192.0/23
2026-06-16T16:14:49Z  91.108.4.0/23
2026-06-16T16:14:49Z  91.108.6.0/23
2026-06-16T16:14:49Z  95.161.64.0/21
2026-06-16T16:14:49Z  95.161.72.0/21

Nine prefixes, all announced simultaneously at 16:14:49. Note the specificity: Reliance originated /23s under Telegram's legitimate /22s, and /24s under the legitimate /23s. In BGP, a more-specific route wins. This is the prefix arms race Kentik described visually - and here it is in the raw update stream, with exact prefixes and timestamps.

Verify every hijacked prefix was RPKI-invalid

Validate each against AS18101:

for pf in 91.108.4.0/23 149.154.160.0/24 91.105.192.0/23; do
  monocle rpki validate $pf 18101 --format json-line 2>/dev/null \
    | grep '^{' \
    | jq -r --arg pf "$pf" '"\($pf) → \(.state): \(.reason)"'
done

91.108.4.0/23 → invalid: ASN 18101 not authorized; authorized ASNs: 62041
149.154.160.0/24 → invalid: ASN 18101 not authorized; authorized ASNs: 62041
91.105.192.0/23 → invalid: ASN 18101 not authorized; authorized ASNs: 211157

Every hijacked prefix was RPKI-invalid. This is machine-verified evidence, not an inference. Any transit that carried these was, by definition, not enforcing ROV.

Step 3: Was the hijacked route actually selected in the routing table?

Announcements in BGP updates tell you what was advertised. To see what was actually installed in a router's RIB at a given moment, use monocle rib. This is more expensive than search - always pass a collector filter.

$ monocle rib 1781627400 -p "91.105.192.0/23,91.108.4.0/22,149.154.160.0/22" \
    -s -S -c rrc14 --format json-line \
    | jq -r '"\(.prefix) origin=AS\(.origin_asns[0]) via \(.as_path)"' | sort -u

149.154.160.0/24 origin=AS18101 via 19151 15412 18101
149.154.161.0/24 origin=AS18101 via 19151 15412 18101
149.154.162.0/24 origin=AS18101 via 19151 15412 18101
149.154.163.0/24 origin=AS18101 via 19151 15412 18101
91.105.192.0/23  origin=AS18101 via 19151 15412 18101
91.108.4.0/23   origin=AS18101 via 19151 15412 18101
91.108.6.0/23   origin=AS18101 via 19151 15412 18101

At 16:30 UTC - sixteen minutes after the hijack began - rrc14's peer AS19151 had AS18101 installed as the origin for all seven prefixes, all via FLAG. The hijacked route wasn't just announced; it was the selected route in the RIB.

When did it end?

Check a later RIB snapshot:

$ monocle rib 1781683200 -p "91.105.192.0/23,91.108.4.0/22,149.154.160.0/22" \
    -s -S -c rrc14 --format json-line \
    | jq -r '"\(.prefix) origin=AS\(.origin_asns[0]) via \(.as_path)"' | sort -u

91.105.192.0/23 origin=AS211157 via 19151 9002 211157 211157

By 2026-06-17 08:00 UTC, the prefix is back to origin AS211157 (Telegram) - and notice the path changed from ... 15412 18101 to ... 9002 211157. The prefix is now reaching this vantage point via AS9002, not FLAG. The hijack was withdrawn sometime between 20:00 UTC on June 16 and 08:00 UTC on June 17. (Anurag's update noted the prefix fading behind FLAG around 01:47 IST / 20:17 UTC on June 16.)

Step 4: Who propagated the invalid routes?

From the RIB and update data, the propagation path at rrc14 is consistently 19151 15412 18101 - FLAG (AS15412) was the transit. Confirm the relationship:

$ monocle as2rel 18101 15412 --format json | jq '.results[0]'
{"asn1":18101,"asn2":15412,"connected":"86.5%","peer":"61.6%","as2_upstream":"24.9%"}

FLAG is both a peer and an upstream of Reliance at high visibility. Since the announcement was RPKI-invalid (Step 1) yet FLAG carried it into the global table, FLAG was not enforcing ROV on this session. The same logic applies to Tata (AS4755), Reliance's other transit, which appeared in paths at other collectors.

Hurricane Electric (AS6939) also appeared briefly in observed AS paths. In the UPDATE stream, HE propagated the invalid route for about 40 seconds and then withdrew it, consistent with an RPKI-invalid rejection path that was slower than the initial propagation. That is useful timing evidence, but it should not be read as a statement about traffic volume or broad customer impact.

Step 5: The RPKI timeline - did Telegram have ROAs before the incident?

A common pattern in BGP hijack history is reactive ROA creation: Twitter created ROAs for its routes after the 2021 Myanmar hijack, and that protection limited damage during the 2022 Russia incident. Was Telegram in the same reactive position, or did it already have ROAs?

The BGPKIT Alpha API provides historical ROA data with date_ranges arrays showing when each ROA was active:

$ curl -s "https://alpha.api.bgpkit.com/roas?asn=211157&date=2026-06-16" \
    | jq '.data[] | {prefix, date_ranges}'

{
  "prefix": "91.105.192.0/23",
  "date_ranges": [
    ["2023-09-23", "2025-05-04"],
    ["2025-05-06", "2026-06-12"],
    ["2026-06-14", "2026-06-20"]
  ]
}

Telegram's ROA for 91.105.192.0/23 dates back to September 2023 - nearly three years before this incident. Unlike Twitter's reactive adoption, Telegram had protection in place well ahead of time. This is why RPKI-RoV was able to limit the hijack's global impact (Kentik reported only ~1.6% of their BGP sources saw the hijacked route). The protection was already there; the question was whether transits enforced it.

Step 6: What about the IPv6 blackholing story?

Anurag's second post describes a separate phenomenon: Indian ISPs (TTSL AS45820, Lightstorm AS152144/AS135709) originating Telegram's IPv6 prefix 2a0a:f280::/48 and its aggregate 2a0a:f280::/32, which he hypothesizes is a misapplied remote-triggered blackhole configuration.

For this post, the important question is not whether the RFC 7999 BLACKHOLE community (65535:666) appears. A standards-compliant blackhole route should be tightly scoped: section 3.2 says receivers should add NO_ADVERTISE or NO_EXPORT, and section 3.3 says the neighboring network must be authorized to advertise the covering prefix. TTSL and Lightstorm are not authorized for Telegram's IPv6 space, so the observable BGP fact remains the same either way: these are unauthorized origins.

Bryton Herdes' False immunity: long prefixes that bypass ROV talk is useful background on how TE and RTBH exceptions can create loose handling for long prefixes. But the public BGP evidence here cannot prove the operators' internal mechanism; it can only show the invalid origins.

What is observable is that these ISPs are originating prefixes they do not own - RPKI-invalid for those origins. This is the same routing anomaly pattern as the Reliance Communications hijack, regardless of the mechanism. You can verify it the same way:

monocle pfx2as 2a0a:f280::/48 --show-name          # who announces it?
monocle rpki validate 2a0a:f280::/48 45820 --format json  # is TTSL authorized?

A secondary puzzle remains: if this is not standard RFC 7999 RTBH, what mechanism caused these ISPs to originate Telegram's IPv6 prefixes? That question is worth investigating, but it is not something BGP UPDATE data alone can answer - it requires off-band information about the operators' configuration practices.

Summary

Here is the complete evidence chain, reproducible from the commands above:

Ground truth: 91.105.192.0/23 is RPKI-signed for AS211157 (Telegram). AS18101 (Reliance Communications) originating it is RPKI-invalid. (monocle rpki validate)
The hijack: At 16:14:49 UTC on June 16, Reliance originated 9 Telegram prefixes (/23s and /24s under the legitimate aggregates), all via FLAG (AS15412). (monocle search -s -S -c rrc14)
RIB confirmation: At 16:30 UTC, the hijacked routes were installed in rrc14's RIB with AS18101 as origin. By 08:00 UTC on June 17, the prefix was back to AS211157. (monocle rib -c rrc14)
ROV failure: FLAG (AS15412) and Tata (AS4755) carried RPKI-invalid announcements, so neither enforced ROV on those sessions. (monocle as2rel + propagation data)
ROA timeline: Telegram's ROAs date to September 2023 - protection was proactive, not reactive. (BGPKIT Alpha API)
IPv6 anomaly: TTSL and Lightstorm originated Telegram's IPv6 prefixes (2a0a:f280::/48 and aggregate 2a0a:f280::/32), RPKI-invalid for those origins. RFC 7999 section 3.3 describes blackhole prefixes as "typically as specific as possible, /32 for IPv4 or /128 for IPv6," and section 3.3's MUST conditions require the origin to be authorized - neither fits aggregates originated by an unauthorized AS.

Every command in this post has been run against live public data. That's the point: BGP forensics doesn't require a proprietary platform. monocle, the BGPKIT API, and public MRT archives are sufficient to investigate - and verify - any routing incident.

References

monocle v1.3.0: Reconstruct BGP RIB State at Any Timestamp

Mingwei Zhang — Wed, 27 May 2026 00:00:00 GMT

We are happy to announce the release of monocle v1.3.0, now available on GitHub and crates.io, also available on Homebrew via brew install monocle.

This release introduces a major new command, monocle rib, which reconstructs BGP Routing Information Base (RIB) state at arbitrary timestamps.

New Feature: `monocle rib`

Most BGP tools show you what happened during an update file. monocle rib shows you the routing table as it existed at any moment in time.

Given a target timestamp, the command:

Finds the latest RIB dump at or before that time
Downloads and replays all overlapping BGP update files up to the exact target timestamp
Materializes the final route state per peer and prefix

The result is the reconstructed RIB — not raw BGP messages, but the actual route state.

Basic Usage

Reconstruct the RIB for a single timestamp and print to stdout:

monocle rib 2025-09-01T12:00:00Z -c route-views6

Write multiple snapshots to a single SQLite file:

monocle rib \
  2025-09-01T08:00:00Z \
  2025-09-01T12:00:00Z \
  --sqlite-path /tmp/rib-snapshots.sqlite3 \
  -c route-views6

Filtering

Apply filters at reconstruction time to narrow the result set:

# Only routes originated by AS13335 (Cloudflare)
monocle rib 2025-09-01T12:00:00Z -c route-views6 -o 13335

# Routes for a specific prefix, including more-specifics
monocle rib 2025-09-01T12:00:00Z -c route-views6 -p 2606:4700::/32 --include-sub

# Routes from full-feed peers only
monocle rib 2025-09-01T12:00:00Z -c route-views6 --full-feed-only

# Routes where AS path contains a specific ASN
monocle rib 2025-09-01T12:00:00Z -c route-views6 -a "13335"

Performance Notes

The rib command keeps the working RIB state in memory during reconstruction, avoiding SQLite lookups and writes on the hot path. The updates query window was tightened from +2 hours lookahead to the exact target timestamp, reducing update file downloads by roughly one third for typical requests.

Installation

cargo install monocle

Or download pre-built binaries for Linux (x86_64, aarch64) and macOS (universal) from the GitHub release page.

monocle v1.1.0

Mingwei Zhang — Sat, 14 Feb 2026 00:00:00 GMT

monocle v1.1.0 focuses on interface consistency and day-to-day usability. This release simplifies feature gates, standardizes data refresh APIs, and adds quality-of-life improvements across parsing, search, and configuration workflows.

TL;DR

Feature flags are now simplified to lib, server, and cli.
Config and update flows are more consistent (config update, config backup, config sources, and --no-update).
Data refresh APIs are standardized across ASInfo, AS2Rel, RPKI, and Pfx2as.
Cache TTL defaults are unified at 7 days, with clearer staleness reporting.
Parse/search workflows gain multi-value filters, negation filters, field selection, ordering, timestamp format control, and local cache support.

What's New

Simpler feature flags

The crate feature model is now reduced to three options:

lib: complete library functionality (database + lenses + display)
server: WebSocket server support (implies lib)
cli: full command-line binary (implies lib and server)

This replaces the previous multi-tier setup and makes dependency selection easier for downstream users.

Standardized data refresh APIs

Database refresh behavior is now more uniform across ASInfo, AS2Rel, RPKI, and Pfx2as:

Consistent needs_*_refresh(ttl) checks
A shared RefreshResult shape with source and load details
Standardized naming (refresh_*) with compatibility aliases where needed
URL and local-path loading paths available across repositories

This update reduces API drift and makes maintenance code paths more predictable.

Config command updates

Configuration and maintenance commands now use clearer naming:

monocle config db-refresh -> monocle config update
monocle config db-backup -> monocle config backup
monocle config db-sources -> monocle config sources

The global no-refresh toggle was also renamed for consistency:

--no-refresh -> --no-update

The following command shows the active configuration, cache TTL settings, database status, and server defaults:

monocle config

Example output:

monocle Configuration
=====================

General:
  Config file:    /home/user/.monocle/monocle.toml
  Data dir:       /home/user/.monocle/

Cache TTL:
  ASInfo:         7 days
  AS2Rel:         7 days
  RPKI:           7 days
  Pfx2as:         7 days

Database:
  Path:           /home/user/.monocle/monocle-data.sqlite3
  Status:         exists
  Size:           512.47 MB
  Schema:         initialized (v3)
  ASInfo:         120953 records (updated: 2026-02-02 19:54:01 UTC)
  AS2Rel:         877937 records (updated: 2026-02-02 14:25:34 UTC)
  RPKI:           796899 ROAs, 962 ASPAs (updated: 2026-02-10 19:53:50 UTC)
  Pfx2as:         1580626 records (updated: 2026-02-02 20:02:10 UTC)

Better cache control defaults

All major data sources now support configurable cache TTL with a 7-day default. This applies to ASInfo, AS2Rel, RPKI, and Pfx2as.

monocle config sources now reports staleness based on TTL, so it is easier to see what needs updating.

The following command shows per-source status, staleness, and last update recency:

monocle config sources

Example output:

Data Sources:

  Name         Status          Stale      Last Updated
  ------------------------------------------------------------
  asinfo       120953 records  yes        a week ago
  as2rel       877937 records  yes        a week ago
  rpki         797861 records  no         2 hours ago
  pfx2as       1580626 records yes        a week ago

Configuration:
  ASInfo cache TTL: 7 days
  AS2Rel cache TTL: 7 days
  RPKI cache TTL:   7 days
  Pfx2as cache TTL: 7 days

RPKI improvements

monocle now supports fetching ROAs via RTR (RPKI-to-Router), including endpoint override support and fallback behavior.

The following command refreshes only RPKI data and uses the provided RTR endpoint for this run instead of the default configured source.

monocle config update --rpki --rtr-endpoint rtr.rpki.cloudflare.com:8282

Parse and search enhancements

parse and search gained several output and filtering improvements:

Multi-value filters with OR semantics
Negation filters using !
Validation for ASN/prefix filter inputs
--fields for column selection
--order-by and --order for sorted output
--time-format for unix or RFC3339 display
search --cache-dir local file + broker query caching

The following command searches one hour of updates starting at 2024-01-01, filters for prefix 1.1.1.0/24, and caches downloaded MRT files plus broker query results under /tmp/mrt-cache for faster repeat runs.

monocle search -t 2024-01-01 -d 1h -p 1.1.1.0/24 --cache-dir /tmp/mrt-cache

The following command uses multi-value filters with negation to exclude two origin ASNs while also matching either of two peer ASNs:

monocle search -t 2024-01-01 -d 1h -o '!13335,!15169' -J 174,2914

Negation and positive values cannot be mixed within the same filter field.

Breaking Changes and Migration Notes

1) Feature flag migration

If you previously used feature tiers like database, lens-core, lens-bgpkit, lens-full, or display, switch to:

lib for library use
server for WebSocket API use
cli for full command-line use

2) CLI and subcommand renames

Update scripts and automation:

--no-refresh -> --no-update
config db-refresh -> config update
config db-backup -> config backup
config db-sources -> config sources

3) Parse/search filter type updates (library API)

ParseFilters moved from scalar optional fields to vector-based values for multi-value and negation support. Library consumers should update filter construction accordingly.

Additional Improvements

AS name rendering now prefers PeeringDB naming fields before falling back to AS2Org/core names
Data refresh logging now shows specific reasons (empty vs outdated)
Example layout was reorganized to one example per lens

Full Change List

See the v1.1.0 section in CHANGELOG.md for the complete list of changes.

BGPKIT Parser v0.14.0 Release and v0.13.0 Highlights

Mingwei Zhang — Thu, 25 Dec 2025 00:00:00 GMT

We are pleased to announce the release of BGPKIT Parser v0.14.0. This update introduces support for negative filters and the RPKI-to-Router (RTR) protocol. We also want to highlight key features from the recent v0.13.0 release, including enhanced debugging tools.

v0.14.0 Features

Negative Filter Support

A frequent request has been the ability to filter out specific data points. We have added support for negative filters across most filter types, allowing exclusion of specific origins, prefixes, peers, or communities.

In the CLI, use the != operator. For example, to process all records except those originating from AS 13335:

bgpkit-parser https://spaces.bgpkit.org/parser/update-example.gz --filter "origin_asn!=13335"

In Rust code, use the ! prefix:

let parser = BgpkitParser::new("...")
    .add_filter("!origin_asn", "13335")
    .add_filter("!peer_ip", "192.168.1.1");

Supported negative filters include !origin_asn, !prefix, !peer_ip, !peer_asn, !type, !as_path, !community, and !ip_version.

RPKI RTR Protocol Support

We have added support for the RPKI-to-Router (RTR) protocol, covering both version 0 (RFC 6810) and version 1 (RFC 8210).

The new models::rpki::rtr and parser::rpki::rtr modules allow developers to build custom RTR clients or servers.

Here is how you can use the library to connect to an RTR server and request data:

use bgpkit_parser::models::rpki::rtr::*;
use bgpkit_parser::parser::rpki::rtr::{read_rtr_pdu, RtrEncode, RtrError};
use std::net::TcpStream;
use std::io::Write;

// 1. Connect to the RTR server
let mut stream = TcpStream::connect("rtr.rpki.cloudflare.com:8282")?;

// 2. Send a Reset Query to request the full database
let reset_query = RtrResetQuery::new_v1();
stream.write_all(&reset_query.encode())?;

// 3. Read the response PDUs
loop {
    match read_rtr_pdu(&mut stream)? {
        RtrPdu::IPv4Prefix(p) => {
            println!("Received IPv4 ROA: {}/{}-{} -> AS{}", 
                p.prefix, p.prefix_length, p.max_length, p.asn);
        }
        RtrPdu::EndOfData(_) => break,
        _ => {}
    }
}

We have included a fully functional RTR client example that connects to a server, fetches ROAs, and performs route validation.

You can run the example yourself:

cargo run --example rtr_client -- rtr.rpki.cloudflare.com 8282

In Case You Missed It: v0.13.0

The v0.13.0 release introduced several improvements for debugging and analyzing MRT data.

Record-Level Output

The CLI supports inspecting individual MRT records rather than just parsed BGP elements. This aids in debugging parser issues or analyzing raw MRT files.

Switch to record-level output with --level records and choose a format (e.g., JSON):

bgpkit-parser https://spaces.bgpkit.org/parser/update-example.gz --level records --format json

Raw Bytes Access

For developers, RawMrtRecord now includes a header_bytes field, and the raw_bytes field has been renamed to message_bytes. This provides access to the exact bytes of the MRT header and the message body as they appeared on the wire, enabling byte-for-byte export and debugging without re-encoding.

Other Improvements

Testing & Fuzzing: Added a cargo-fuzz harness and initial fuzz targets.
Performance: Continued optimizations for faster processing.

Check out the full CHANGELOG for more details.

Happy Parsing!

BGPKIT Broker v0.9.0: Better Pagination and New Collectors

Mingwei Zhang — Sat, 01 Nov 2025 00:00:00 GMT

We are excited to announce the release of BGPKIT Broker v0.9.0. This release introduces total count support for efficient pagination, making it easier to build data exploration interfaces and manage large result sets. We also expand our collector coverage with two new Internet Exchange points.

Previously on BGPKIT Broker

Since the major v0.7 release that unified the architecture into a single SQLite-backed CLI application, BGPKIT Broker has been serving the community with stable uptime and continuous data coverage. The v0.8 series brought several enhancements including automated backup systems, configuration validation, and convenient query shortcuts.

However, one common challenge remained for developers building interfaces on top of Broker: pagination. When querying large time ranges, users needed to fetch all results to know the total count, making it inefficient to build proper pagination controls or display result statistics.

V0.9: Efficient Pagination Support

Version 0.9.0 addresses pagination needs with total count support at both the SDK and API levels. This allows applications to fetch result counts independently from the data itself, enabling responsive user interfaces without over-fetching data.

SDK: `query_total_count()` Method

The SDK now provides a dedicated query_total_count() method that returns the total number of matching items without retrieving the actual data. This is useful when you need to know how many results exist before deciding how to paginate through them.

use bgpkit_broker::BgpkitBroker;

#[tokio::main]
async fn main() {
    let broker = BgpkitBroker::new()
        .ts_start("2024-10-01")
        .ts_end("2024-10-02")
        .project("route-views");

    // Get total count without fetching items
    let total = broker.query_total_count().await.unwrap();
    println!("Total matching items: {}", total);

    // Now fetch paginated results
    let result = broker
        .page_size(100)
        .page(1)
        .query()
        .await
        .unwrap();

    println!("Retrieved {} items out of {} total", 
             result.data.len(), 
             result.total.unwrap_or(0));
}

This method executes a COUNT(*) query against the database with the same filters as your main query, returning just the number rather than all the data. For large time ranges with thousands of results, this can save significant bandwidth and processing time.

API: Total Field in Search Results

The /v3/search endpoint now includes a total field in every response, providing the total count of matching items alongside the paginated results. This enhancement makes it straightforward to build pagination controls in web interfaces or CLI tools.

curl "https://api.bgpkit.com/v3/broker/search?ts_start=2024-10-01&ts_end=2024-10-02&page_size=10&page=1"

Response:

{
  "total": 1234,
  "data": [
    {
      "ts_start": 1727740800,
      "ts_end": 1727740800,
      "collector": "route-views2",
      "project": "route-views",
      "url": "...",
      ...
    }
  ]
}

With the total field, client applications can:

Display "showing X of Y results" to users
Calculate the number of pages needed for pagination
Decide whether to fetch all results or paginate
Provide accurate progress indicators for data processing

New RouteViews Collectors

This release adds support for two new RouteViews collectors:

hkix.hkg: Hong Kong Internet Exchange (HKIX) collector in Hong Kong
ix-br.gru: IX.br (PTT.br) in São Paulo

These additions enhance BGPKIT Broker's global coverage, providing more diverse vantage points into internet routing behavior. Users who update to v0.9.0 can run bgpkit-broker update to automatically bootstrap data for these new collectors.

Behind the Scenes Improvements

Beyond the user-facing features, we also made several code improvements:

Refactored bootstrap download logging to centralize progress reporting and eliminate redundant code paths
Updated the oneio dependency to v0.20.0 with optimized feature flags for better handling rustls providers
Enhanced test coverage overall

Upgrading to v0.9.0

For SDK Users

Update your Cargo.toml to use the latest v0.9 release:

[dependencies]
bgpkit-broker = "0.9"

Then run:

cargo update

The new query_total_count() method is immediately available for use. The total field in BrokerQueryResult is backward compatible as an optional field.

For Self-Hosted Instances

If you run your own BGPKIT Broker instance, update to the latest version:

cargo install --force bgpkit-broker --version 0.9.0 --features cli

Or pull the latest Docker image:

docker pull bgpkit/bgpkit-broker:latest

After upgrading, run the update command to add the new collectors:

bgpkit-broker update --db-path your-database.sqlite3

The update process will automatically detect and bootstrap historical data for the two new collectors.

Looking Forward

BGPKIT Broker continues to serve as a fundamental component for BGP data processing pipelines. With v0.9.0, we focused on making the developer experience better for building applications that need pagination and result statistics. We continue to maintain the public Broker instance at https://api.bgpkit.com/v3/broker with 99.996% uptime, and encourage users to deploy on-premise instances for production pipelines to reduce external dependencies.

For the full v0.9.0 release notes, please check out our GitHub release page. If you have any comments, please drop us a message on Twitter, Mastodon, Bluesky or email.

Get Started

Ready to try out the new features? Here are some resources:

GitHub: github.com/bgpkit/bgpkit-broker
Examples: Browse example code demonstrating common usage patterns
Public API: Try queries at https://api.bgpkit.com

Have questions or feedback? Open an issue on GitHub or join discussions in our Discord Channel.

💖

If you find our libraries and services useful, we would highly appreciate if you consider sponsoring us on GitHub.

Run BGPKIT on Cloudflare Containers

Mingwei Zhang — Mon, 30 Jun 2025 00:00:00 GMT

For the longest time, I’ve been using Cloudflare exclusively for web/API hosting and relatively lightweight tasks. For the most of my work on BGP, there is really not much that I can accomplish with just JavaScript/TypeScript (except maybe working with RIS Live WebSocket). The computationally intensive nature of the most BGP data processing doesn't naturally fit within the typical Cloudflare developer platform.

This changes with the recent announcement of Cloudflare Containers. In short, it allows developers to build and run custom containers on the Cloudflare’s platform, enabling the heavy workload to mix with other primitives and unify the deployment.

In this blog post, I will show you how to build a BGP data search API with BGPKIT and deploy on Cloudflare Containers. The source code is available on GitHub.

BGP Data Search API

For this example, I will show you how to build a very straightforward HTTP API to allow accepting search parameters and let BGPKIT to fetch and parse BGP archives and return the parsed messages.

The API accepts four parameters: collector, prefix, ts_start, and ts_end, to filter and parse BGP archives efficiently.

collector: the BGP route collector ID to use (e.g. rrc00 or route-views2). We want to limit the search to a single collector.
prefix: the prefix of the BGP relevant updates. Open-ended search will burn up resources quick, but you can opt-out this requirement.
ts_start and ts_end: the starting and ending timestamps. The goal is to limit the search to end up parsing a very small amount of MRT files (e.g. give them the same value will do). We should probably leave large-scale data crunching to a environment with more CPU powers.

The parameters are defined in a struct to be able to passed in to a axum “GET” route:

#[derive(Deserialize, Serialize)]
struct Params {
    collector: String,
    prefix: String,
    ts_start: String,
    ts_end: String,
}

With the given parameters, we will first find the relevant MRT files by setting the filters of timestamps and collector to BgpkitBroker instance:

        let files = match bgpkit_broker::BgpkitBroker::new()
            .ts_end(ts_end.clone())
            .ts_start(ts_start.clone())
            .collector_id(collector.clone())
            .query(){
            Ok(items) => items,
            Err(e) => {
                return Json(Result {
                    error: Some(e.to_string()),
                    data: vec![],
                    meta: None,
                });
            }
        };

For each file, we will parse the whole MRT file and collect BGP updates that is relevant to the target prefix:

        for file in files {
            let mut parser = match bgpkit_parser::BgpkitParser::new(file.url.as_str()){
                Ok(parser) => parser,
                Err(e) => {
                    return Json(Result {
                        error: Some(e.to_string()),
                        data: vec![],
                        meta: None,
                    });
                }
            };

            parser = match parser.add_filter("prefix", prefix.as_str()){
                Ok(parser) => parser,
                Err(e) => {
                    return Json(Result {
                        error: Some(e.to_string()),
                        data: vec![],
                        meta: None,
                    });
                }
            };
            items.extend(parser.into_elem_iter());
        }

Because the BGPKIT parser and broker code are implemented in the sync environment, we will need to wrap the code above in a blocking thread in order to use it in async web frameworks like axum.

let result = tokio::task::spawn_blocking(move || {
   // THE BLOCKING CODE PIECES
}).await.unwrap();

Please see the full source code here for more details:

https://github.com/bgpkit/bgpkit-cf-containers/blob/main/container-src/src/main.rs

Cloudflare Containers Deployment

Now that we have an Rust-based API code working, we will need to 1. containerize the code, 2. put a Cloudflare Container wrapper around it for deployment.

The container definition is a typical two-stage build definition, with a builder stage to build the binary application, and a minimum deployment stage to call the executable. The two-stage build is almost necessary as Cloudflare Containers has limits on the size of each image and the total storage per account. The smaller the image the better.

# ---- Build Stage ----
FROM rust:1.86 AS builder

WORKDIR /app

# Install build dependencies
RUN apt-get update && apt-get install -y pkg-config libssl-dev

# Build application
COPY container-src/Cargo.lock container-src/Cargo.toml ./
COPY container-src/src ./src
RUN cargo build --release

# ---- Runtime Stage ----
FROM debian:bookworm-slim

# Install minimal runtime dependencies
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Copy the statically linked binary from the builder
COPY --from=builder /app/target/release/bgpkit-cf-container /app/bgpkit-cf-container

EXPOSE 3000

CMD ["/app/bgpkit-cf-container"]

The rest of the task is to build a API app for Workers and configure Containers. The following config is pretty much all it needs to configure the Workers script to build and push the container image, and create a Durable Object to coordinate and run Containers.

	"containers": [
		{
			"class_name": "BgpkitContainer",
			"image": "./Dockerfile",
			"max_instances": 5
		}
	],
	"durable_objects": {
		"bindings": [
			{
				"class_name": "BgpkitContainer",
				"name": "BGPKIT_CONTAINER"
			}
		]
	},
	"migrations": [
		{
			"new_sqlite_classes": [
				"BgpkitContainer"
			],
			"tag": "v1"
		}
	]

The main Workers script is only 22 lines long:

import { Container, getContainer } from '@cloudflare/containers';
import { Hono } from "hono";

export class BgpkitContainer extends Container {
	defaultPort = 3000;
	sleepAfter = '5m';
}

// Create Hono app with proper typing for Cloudflare Workers
const app = new Hono<{
	Bindings: { BGPKIT_CONTAINER: DurableObjectNamespace<BgpkitContainer> };
}>();

app.get("/search", async (c) => {
	if (!c.req.query('collector') || !c.req.query('prefix') || !c.req.query('ts_start') || !c.req.query('ts_end')) {
		return c.json({ error: "Missing required query parameters: collector, prefix, ts_start, ts_end" }, 400);
	}
	const container = getContainer(c.env.BGPKIT_CONTAINER);
	return await container.fetch(c.req.raw);
});

export default app;

The important pieces are:

class BgpkitContainer extends Container block defines the port to use and configures how long the container should run after the last query. In this example, containers will be killed after 5 minutes of inactivity. It is crucial to realize that Cloudflare Containers are not a drop-in replacement of some other container deployment platforms like fly.io or Railway, as the workload for Containers are intended to be short-lived (ping me if this changes) and scales horizontally depending on the requests amount.
the getContainer function here tries to reach a container. If the intended container is overloaded, it may create a new container on demand. You may choose to use the getRandom function to round-robin containers. See the docs for more.
the container.fetch(c.req.raw) forwards the query to the container, including the query parameters, which will then be handled by the running container.

Example Queries

The following example will reach the Workers script (handled by Hono), and then reach the container to run the actual BGP data crunching task. (This URL won’t actually work as we don’t have budget to provide such service openly. Feel free to deploy it on your own account to try it out.)
https://EXAMPLE.bgpkit.workers.dev/search?collector=rrc00&prefix=1.1.1.0/24&ts_start=1751231488&ts_end=1751231488

BGPKIT Broker 0.7 Release

Mingwei Zhang — Sat, 22 Jun 2024 00:00:00 GMT

BGPKIT Broker is a fundamental component to our design of a all-purpose BGP data processing pipeline. In short, it is a BGP data file meta information "broker" that tells the data consumers what MRT files from RouteViews and RIPE RIS are available for any given time range in question. It commonly serves as a data input entry point for data pipelines.

For instance, here is a simple diagram for a system that creates a semi-real-time BGP data stream with BGPKIT Broker and Parser (a very common use case for these two libraries).

BGPKIT Broker periodically crawls the websites of RIPE RIS and RouteViews MRT data pages of their collectors and index meta information into a database. Downstream consumers can ask and retrieve new files and process the files into BGP messages.

Previously on BGPKIT Broker

In the BGPKIT Broker version 0.1 to 0.6, a working broker instance consists of three individual components: a crawler, a Postgres database, and an API.

Each of the three component runs independently, and requires independent configuration, cronjobs, deployment, and all these goodies. For example, to run BGPKIT Broker v0.6, a user will need to configure and run

a PostgreSQL database with proper credentials and schema set up;
a cronjob instance that periodically crawls the data sources, with optional locks to prevent overlapping executions in case some crawl became slow;
a API application likely sitting behind a configured reverse proxy like Caddy to serve the data.

It's fun and exciting to set up all these for the first time, but quickly became tiring and too complex for repeated set up or bootstrapping for new users.

V0.7: one CLI app that does everything

We completely revamped the architecture for BGPKIT Broker in V0.7 to merge every functionality needed for a running Broker instance into one single command-line application: bgpkit-broker. V0.7 provides a single application to configure, run, debug, query everything on BGPKIT Broker.

To achieve this redesign, we made some major changes to our architecture.

SQLite instead of PostgreSQL

There are two major topics to concern when choosing a backend database for BGPKIT Broker: performance and portability.

SQLite is more than fast enough

BGPKIT Broker indexes metadata for all collectors from RouteViews and RIPE RIS, which includes time, URL, type, size of every RIB dump and updates MRT files from these two public archives. Dating all the way back to 1999, we have indexed roughly 48 million MRT files' metadata.

With a single index on the timestamp of files, we are able to search data files in less than 0.5s for any queries, which is more than fast enough for our use cases. We admit that we have spent our time for "early optimization" and in the end, the simple schema out-weighs the small performance gains.

Backup and bootstrap with just one file

Now in terms of portability, we can appreciate enough the beauty of single-file database like SQLite. In our current production setup, we periodically backup the database, and it literally involves just copying a single file to another directory (well, we also upload it to Cloudflare R2 for safekeeping).

Portability also means users can move their instance anywhere they want with ease. This is definitely the case for V0.7 where new users can bootstrap by simply download a SQLite file (our CLI provides all that functionality), and move to new locations by scp it to anywhere they desire.

Here is a video demonstrating bootstrapping a local BGPKIT Broker SQLite database with the new bgpkit-broker bootstrap command.

%[https://youtu.be/SsCyfQ5q0G0]

New file notification via NATS

Before V0.7, pipelines that needs to continuously processing new MRT files will need to "pull" data from BGPKIT Broker instance periodically and keep track of the latest files processed. We consider this a hassle that developers should not be dealing with and thus introduced a new NATS-based message channel allowing data consumers to subscribe to the public/private NATS channel where a Broker instance may publish new file notification to.

We dedicated nats.broker.bgpkit.com as the public endpoint for any NATS consumers to connect to. Whenever a new file becomes available in Broker, it will publish a new file notification with all metadata as in the database entry to the public channel. Consumers (e.g. data pipelines) can use the NatsNotifier::new(None).start_subscription() to start waiting for new files. The following snippet below shows how a simple pipeline can use this feature in a loop.

let mut notifier = match NatsNotifier::new(url).await {
    Ok(n) => n,
    Err(e) => {
        error!("{}", e);
        return;
    }
};
if let Err(e) = notifier.start_subscription(subject).await {
    error!("{}", e);
    return;
}
while let Some(item) = notifier.next().await {
    if pretty {
        println!("{}", serde_json::to_string_pretty(&item).unwrap());
    } else {
        println!("{}", item);
    }
}

We also implemented a simple new file watcher in the app as bgpkit-broker live subcommand. It will start a subscription to the public BGPKIT NATS endpoint and print out new file data as they come to the channel.

One command to serve and update

As mentioned previously, the new bgpkit-broker application includes everything one needs to start a instance. Once one bootstrapped the database to a local sqlite file (via bgpkit-broker bootstrap <FILENAME> command), all they need to start a auto-updating API is to run bgpkit-broker serve <FILENAME> .

bgpkit-broker serve --help
Serve the Broker content via RESTful API

Usage: bgpkit-broker serve [OPTIONS] <DB_PATH>

Arguments:
  <DB_PATH>  broker db file location

Options:
  -i, --update-interval <UPDATE_INTERVAL>  update interval in seconds [default: 300]
      --no-log                             disable logging
  -b, --bootstrap                          bootstrap the database if it does not exist
      --env <ENV>                          
  -s, --silent                             disable bootstrap progress bar
  -h, --host <HOST>                        host address [default: 0.0.0.0]
  -p, --port <PORT>                        port number [default: 40064]
  -r, --root <ROOT>                        root path, useful for configuring docs UI [default: /]
      --no-update                          disable updater service
      --no-api                             disable API service
  -h, --help                               Print help
  -V, --version                            Print version

The serve subcommand will also start a thread that periodically crawl and update the SQLite database to make sure the API always serve the up-to-date data.

Noticed that error message? It's by design as it tries to connect to notification channel for new files as a default behavior for a service, but not NATS URL is configured. We use the BGPKIT_BROKER_NATS_URL environment variable to configure the NATS channel to use.

We also allow users to optionally configure a heartbeat URL to monitor the data updating status. After every success data crawling run, Broker will try to execute a HTTP GET to a URL if BGPKIT_BROKER_HEARTBEAT_URL is set in the environment. This is useful to monitor the running status of the Broker instance without the need of setting up a cronjob.

We use Better Stack's Uptime monitoring service for page and heartbeat monitoring, and the public Broker instance is running V0.7 with the heartbeat URL set to this service. All status information can be found at https://status.bgpkit.com/

Production-ready, on-prem deployment

Although BGPKIT Broker has not yet reached V1.0, we consider it to be feature-complete and production-ready. Ever since V0.2, we have made our better efforts on not introducing any breaking changes and the service has been serving the community with a stable uptime ever since. We believe all libraries running in production should at least be 1.0, and thus we will release V1.0 soon this summer.

We also made significant efforts in V0.7 release to make sure BGPKIT Broker is as portable as possible. New users can spin up a fully functioning Broker instance with just two commands: bgpkit-broker bootstrap and bgpkit-broker serve, all within 5 minutes. With V0.7 released, we encourage all data pipeline designers to deploy a Broker instance on-premise, ensuring data pipelines are self-containing and reduce external dependencies as much as possible. We also will continue maintain our public instance to our best efforts (we are currently at 99.996% uptime). Thanks to our sponsors, we are able to keep the services up as we do, and we plan to continue serving the community the same way in the foreseeable future.

For the full V0.7 release notes, please check out our GitHub release page. If you have any comments, please drop us a message at Twitter, Mastodon, or email.

💖

If you find our libraries and services useful, we would highly appreciate if you consider sponsor us on GitHub.

Command-line Routing Stats with monocle and Cloudflare Radar API

Mingwei Zhang — Sun, 21 Apr 2024 00:00:00 GMT

BGPKIT monocle is a command-line utility program that helps users quickly pull Internet routing-related information from publicly available sources.

https://github.com/bgpkit/monocle

In BGPKIT monocle version V0.5, we add support for querying Cloudflare Radar's new BGP routing statistics and prefix-to-origin mapping APIs, the same APIs that power the Cloudflare Radar routing section. monocle users can now quickly glance overview of routing stats for any given ASN, country, or the whole Internet. Users can also quickly look up prefix origins and examine their RPKI validation status as well as prefix visibility on the global routing tables.

Using `monocle radar`

We added a new monocle radar command group in V0.5, which contains the following to subcommands:

monocle radar stats [QUERY]: get routing stats (like prefix count, rpki invalid count) for a given country or ASN.
monocle radar pfx2as [QUERY] [--rpki-status valid|invalid|unknown]: get prefix to origin mapping for a given prefix or ASN

mingwei@terrier ~ % monocle radar
Cloudflare Radar API lookup (set CF_API_TOKEN to enable)

Usage: monocle radar <COMMAND>

Commands:
  stats   get routing stats
  pfx2as  look up prefix to origin mapping on the most recent global routing table snapshot
  help    Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Cloudflare API token needed

Since the monocle radar command relies on querying data using Cloudflare Radar public API, we also need to specify a user API token as the environment variable CF_API_TOKEN. Obtaining an API token is free and only needs a Cloudflare account. Interested users can follow their official tutorial to obtain a token. The environment variable can be set in a .env file in the current directory, or set in ~/.bashrc or ~/.profile etc.

`monocle radar stats`

Users can query the routing statistics for a given country or ASN. For example, monocle radar stats us returns the routing stats for the United States, while monocle radar stats 174 returns the stats for Cogent (AS174).

The displayed table is further divided into three rows, one for overall counting, and one for IPv4 and IPv6-specific counting. For each row, we show the following fields:

origins: the number of origins ASes registered in the given country
prefixes: the number of prefixes originated by the given ASN or ASes registered in the given country
rpki_valid/invalid/unknown: the number of RPKI valid/invalid/unknown prefix routes (prefix-origin mapping) on the global routing table and their percentage of the overall routes.

`monocle radar pfx2as`

Users can query the prefix-to-origin API to get the mapping of origin ASes and their originated prefixes on the global routing table.

In the following example, monocle radar pfx2as 174 --rpki-status invalid, we ask for all the prefixes originated by AS174 with the RPKI validation status to be invalid. This command returns us the list of RPKI invalid prefixes originated by AS174 at the time of generating the dataset.

Questions it can answer now (more in the future)

Here is a selected list of questions that monocle radar command can answer you:

How many ASes are there on the Internet that announce at least one prefix? (81,770)
How many of these ASes announce only IPv6 prefixes? (6,853)
How many prefixes are there on the global routing table? (1,205,218)
How many prefixes do AS400644 announce? (1)
Which AS(es) originates 1.1.1.0/24? (AS13335)
How many prefixes originated by AS174 are NOT covered by some RPKI ROA? (a lot, 94%+)
How about the RPKI valid ratio for the Philippines? (77%, nice!)

Powered by Cloudflare Radar free API

Cloudflare Radar is a hub that showcases global Internet traffic, attack, and technology trends and insights.

What Cloudflare Radar shines is its data openness. Everything you see on the Cloudflare Radar website is powered by their free publicly available APIs. It's a treasure trove there, and all users need is a free API token to access everything.

At BGPKIT, we think we can further improve the usability of the API by exposing them as a proper Rust SDK: radar-rs. This is our (unofficial) effort on bringing the Cloudflare Radar's rich data to Rust developers. For example, monocle radar is powered by this SDK.

ASPA Path Verification Explained

Mingwei Zhang — Sun, 25 Jun 2023 00:00:00 GMT

Originally published on Hashnode.

Autonomous System Provider Authorization (ASPA) is a recently proposed solution designed to detect and mitigate route leaks and improbable AS paths. ASPA utilizes the existing RPKI infrastructure to allow storing and verifying cryptographically protected AS-level relationship information. This post focuses on the AS path verification process used by ASPA.

Note: This post is based on this IETF draft from June 2023. The draft text, including address-family handling, may differ from the final RFC.

Route leak

In overview, the definition of route leak considered in ASPA is based on RFC 7908: Problem Definition and Classification of BGP Route Leaks. The route leaks defined in this RFC is mostly based on the "valley-free" principle.

A previous Cloudflare Radar post on route-leak detection summarized four common route leak types as follows:

Routes from	To provider	To peer	To customer
From provider	Type 1	Type 3	Normal
From peer	Type 4	Type 2	Normal
From customer	Normal	Normal	Normal

In essence, route leaks defined in RFC 7908 can be summarized into one principle: routes obtained from a non-customer (peer, customer, route-server) AS can only be propagated to customers.

In ASPA, the authors inherited the definition of BGP roles (e.g. customer, provider) from RFC 9234. See the RFC9234 post for more detail on RFC9234 and the OTC BGP attribute.

Local AS Role	Remote AS Role
Provider	Customer
Customer	Provider
RS	RS-Client
RS-Client	RS
Peer	Peer

Table: allowed pairs of BGP role capabilities.

ASPA TL;DR

At a high level, the ASPA mechanism has the following properties:

ASPA stores customer-to-provider information in RPKI
- One customer AS (CAS) associated with a set of provider ASes (SPAS)
ASPA is useful for verifying BGP AS_PATH attribute content
- AS_PATH that violates the "valley-free" routing principle can be detected with ASPA objects information
ASPA is incrementally deployable, benefiting early adopters

ASPA Verification Explained

Before describing how ASPA detects route leaks, it is useful to review the shape of a typical valley-free route.

A typical long BGP route that travels from one edge network to a different edge network would look like this:

The route travels three different phases:

upward: a consecutive customer-to-provider segments
peering: a single peer-to-peer segment
downward: a consecutive provider-to-customer segments

Note that in many cases, the routes may only have one or two phases, but they must be in this order. For example, one legit route could have an upward then downward phase, without a peering phase. A route leak route could be any route with reverse order of the phases, e.g. downward then upward.

In ASPA, the phases are defined into two categories:

upstream: path received from a customer or a peer or a route server
downstream: path received from a provider or sibling

ASPA verification uses different procedures for upstream and downstream paths. Each category is described below.

Upstream paths

ASPA verification for upstream paths (received from a customer or peer or route server) is relatively simple. The key principle is that a normal upstream path should only contain customer-to-provider relationships for the consecutive AS path hops.

A path is

Valid: if all hops in the AS paths are verifiable customer-to-provider, i.e. every hop on the path registers their providers on ASPA and the providers match the next hop on the path;
Unknown: one of the hops is missing ASPA information;
Invalid: one of AS has a set of provider AS (SPAS) but the next hop is not in it.

Downstream paths

The downstream paths are considerably more complex as it may have legit paths segments that travel all three phases (up, peer, down) before reaching the receiving AS. Interested readers should refer to the original IETF draft document's section on downstream path verification for a more definitive definition. The summary below simplifies the wording to give a high-level view of the verification process.

Based on the original documentation, a downstream path is Invalid if there exist hops indexed as u and v such that

u<=v
hop(AS(u-1), AS(u))="Not Provider+"
hop(AS(v+1),AS(v))="Not Provider+"

In practical terms, this condition can be read as follows:

two hops at the location u and v and u is before v
hop u is verified to be NOT a customer-to-provider hop, i.e. peer-to-peer or provider-to-customer, i.e. a peering or downward phase
hop v is verified to be NOT a provider-to-customer hop, i.e. peer-to-peer or customer-to-provider, i.e. a peering or upward phase

In summary, the path would be ASPA Invalid, if it contains a downward or peering then later a upward or peering. The AS relationship would look like the following figure, where u and v could be the same ASN. As long as this pattern appears somewhere in the path, it violates the "valley-free" routing and thus is considered as route leak.

graph LR
  UProvider["u-1"] -->|"provider-to-customer"| U["u"]
  UPeer["u-1"] -->|"peer-to-peer"| U
  U -.-> V["v"]
  V -->|"customer-to-provider"| VProvider["v+1"]
  V -->|"peer-to-peer"| VPeer["v+1"]

The Unknown and Valid definitions are then straightforward:

Unknown: the path is NOT Invalid and there are hops that miss ASPA information
Valid: the path is NOT Invalid and NOT Unknown

Note: The ASPA document allows different verification algorithms, as long as they produce results equivalent to the specified process.

ASPA vs RFC9234

ASPA and RFC 9234 both aim to prevent or mitigate route leaks, but they use different mechanisms. The table below summarizes the main differences.

	ASPA	RFC9234
By whom	Customer AS (owner) only	BGP sessions, providers
Content security	Cryptographically signed	Secured by BGP session; OTC attributes can be forged
Where to detect/filter	Any AS	Neighbors or ASes that receive OTC attribute
Prevent forged path?	Yes	No
Deployment timing	Initial objects had already appeared; general deployment was expected in 2025 or later (Job Snijders / MANRS)	Initial deployment had appeared; general deployment remained unclear

How to view ASPA objects on RPKI today?

Although still in the draft phase, ASPA support has already been added to popular routing software like routinator and OpenBGPd. BGPKIT monocle also supports reading ASPA objects from RPKI repositories through the monocle rpki read-aspa command. The following steps show how to inspect currently published ASPA objects.

Install BGPKIT monocle

BGPKIT monocle is a Rust-based BGP data processing command-line tool. Install the Rust toolchain using the instructions at https://rustup.rs/.

Then install monocle:

cargo install monocle

Download the recent RPKI archive

The RPKI repository archive used here comes from Job Snijders' RPKIViews project. The example below downloads one archive from josephine.sobornost.net (about 260 MB).

cd /tmp
wget http://josephine.sobornost.net/josephine.sobornost.net/rpkidata/2023/06/25/rpki-20230625T173535Z.tgz
tar xzf rpki-20230625T173535Z.tgz

Find and view ASPA objects

The ASPA objects are stored as files with the suffix .asa in the RPKI repository. You can use find command to search across the untarred repo file from the previous step:

➜  rpki-20230625T173535Z find . -name "*.asa"
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148808.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148809.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148810.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148804.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148805.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148807.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148806.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148802.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148803.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148801.asa
./data/rpki.sub.apnic.net/repository/A914BC7A0000/0/AS148800.asa
./data/rpki.cc/repo/MythicalKitten/9/AS60310.asa
./data/rpki.cc/repo/MythicalKitten/0/AS203635.asa
...

Use monocle rpki read-aspa to inspect an ASPA object. For example:

➜  rpki-20230625T173535Z monocle rpki read-aspa ./data/rpki.cc/repo/MythicalKitten/9/AS60310.asa

| asn   | afi_limit | allowed_upstream |
|-------|-----------|------------------|
| 60310 | none      | 924              |
|       |           | 6939             |
|       |           | 48581            |
|       |           | 50224            |
|       |           | 52210            |
|       |           | 204857           |
➜  rpki-20230625T173535Z monocle rpki read-aspa ./data/rpki.co/repo/Mlgt/0/AS204508.asa

| asn    | afi_limit | allowed_upstream |
|--------|-----------|------------------|
| 204508 | none      | 6939             |
|        |           | 7721             |
|        |           | 29632            |
|        |           | 34872            |
|        |           | 34927            |
|        |           | 38230            |
|        |           | 41051            |
|        |           | 50058            |
|        |           | 60326            |
|        |           | 200105           |
|        |           | 208753           |
|        |           | 210633           |
|        |           | 211411           |
|        |           | 212271           |
|        |           | 212895           |

BGPKIT is experimenting with library support for ASPA to support measurement and research use cases. Feedback and feature requests are welcome at data@bgpkit.com.

RFCs for IP Address Usage in Network Documentation and Examples

Mingwei Zhang — Wed, 22 Mar 2023 00:00:00 GMT

Originally published on Hashnode.

Network documentation and topology examples should use address ranges reserved for documentation, rather than globally routed addresses such as 1.1.1.1 or 1.2.3.4.

Two RFCs define standardized IP address ranges reserved specifically for documentation and examples: RFC3849 and RFC5737. RFC3849 outlines the use of the IPv6 address prefix 2001:DB8::/32, while RFC5737 outlines the use of several IPv4 address blocks, including 192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24. These addresses are not intended for use in a production network but instead serve as placeholders to ensure accurate and clear documentation.

RFC3849: IPv6 Address Prefix Reserved for Documentation

RFC3849 specifies the IPv6 address prefix 2001:DB8::/32 for use in documentation and example code, not in a production network. For example, a network administrator might use the IPv6 address 2001:DB8::1 in their documentation or example code to represent a specific device or network location. This address would not be used for actual network traffic, but rather as a reference point for documentation purposes.

RFC5737: IPv4 Address Blocks Reserved for Documentation

RFC5737 outlines the use of several IPv4 address blocks for use in documentation and example code. These address blocks are similar to the IPv4 addresses outlined in RFC5735 (which contains more special blocks such as 0.0.0.0/8), but are specifically for use in documentation and examples.

The address blocks outlined in RFC5737 include:

192.0.2.0/24
198.51.100.0/24
203.0.113.0/24

Similarly, these address blocks are not intended for use in a production network, but rather as placeholders for documentation and examples.

Proper Use of Reserved IP Addresses

Using reserved IP addresses for documentation and examples is essential for a number of reasons. First, it ensures that these addresses are not used for general Internet routing, which can help to prevent routing issues and other network problems. Second, it helps to prevent confusion and technical issues that can arise when non-reserved addresses are used in documentation or example code.

In recent years, there have been cases where non-reserved IP addresses have been used in documentation and examples, leading to confusion and technical issues. In a recent YouTube video, Cloudflare's Tom Strickx highlighted this problem and encouraged operators to avoid addresses such as 1.1.1.1 and 1.2.3.4, using the ranges specified in RFC5735, RFC5737, and RFC3849 instead.

https://www.youtube.com/watch?v=SIb_wxO1wt0

Using reserved IP addresses in documentation and examples avoids unintended traffic to real networks and improves clarity for readers. RFC3849 and RFC5737 provide the standard address ranges for these use cases, and network operators and researchers should use them consistently in examples and documentation.

RFC9234 Observed in the Wild

Mingwei Zhang — Thu, 16 Mar 2023 00:00:00 GMT

Originally published on Hashnode.

BGP Route Leaks

Route leaks occur when BGP prefixes are propagated in a way that goes against the expected topology relationships of BGP. For example, this can happen when a route learned from one transit provider is announced to another transit provider or a lateral peer (peer-peer-peer), or when a route learned from one lateral peer is announced to another lateral peer or a transit provider (see RFC7908). These leaks often result from misconfiguration or the absence of BGP route filtering, or from inadequate coordination between autonomous systems (ASes).

Cloudflare Radar includes a public route-leak detection system. The system detects potential route leaks by first inferring inter-AS relationships on a per-prefix basis, then examining each announced AS path for valley-free violations. Detection is useful for visibility, but prevention is the stronger operational goal.

RFC9234

RFC9234 documents an active route-leak prevention approach where it defines new BGP capacities (BGP Roles) exchanged during the eBGP session open time and allows the BGP routers to understand AS relationships between local and remote ASes, and thus prevent the propagation of route leaks. RFC9234 also defines a new BGP attributes type, only-to-customer, which tells the receiving BGP routers whether some routes should never be announced to another provider.

When a pair of eBGP routers both implement RFC9234, they will first confirm the BGP role of each other.

With roles defined, it handles the only-to-customer attributes as follows.

In essence, RFC9234 uses BGP roles and OTC attributes to make sure routes received from a provider or a peer can only be propagated to customers.

Only-to-customer Attribute in the Wild

RFC9234 was published in May 2022. Public discussion of RFC9234 deployment has been limited, and public measurements of deployment on the Internet remain relatively scarce.

To measure RFC9234 deployment, RFC9234 support was added to BGPKIT Parser. The repository also includes example code demonstrating how to parse a RIB file and identify messages that contain Only-to-Customer attributes.

The following output comes from parsing a single route-views2 RIB dump (file link).

https://gist.github.com/digizeph/756bd6b4d6627bf1e2a0a4f87a1c8290

This output shows that at least four ASes (AS6939, AS15562, AS20555, AS212068) were associated with RFC9234 OTC attributes in messages that propagated to the route collector and were preserved in MRT files.

The code that generates this result iterates over parsed BGP messages and checks whether the only_to_customer attribute is present.

use bgpkit_parser::BgpkitParser;

fn main() {
    for elem in BgpkitParser::new(
        "http://archive.routeviews.org/bgpdata/2023.03/RIBS/rib.20230316.0200.bz2",
    )
    .unwrap()
    {
        if let Some(otc) = elem.only_to_customer {
            println!(
                "OTC found: {} for path {}\n{}\n",
                &otc,
                &elem.as_path.as_ref().unwrap(),
                &elem
            );
        }
    }
}

To reproduce this example, install the Rust toolchain and run:

git clone https://github.com/bgpkit/bgpkit-parser
cd bgpkit-parser
cargo run --release --example only-to-customer

Update, Mar. 16, 2023: Job Snijders provided additional context on the source of these OTC attributes: YYCIX.

https://twitter.com/JobSnijders/status/1636291640519282688

2022 Year in Review

Mingwei Zhang — Tue, 31 Jan 2023 00:00:00 GMT

In 2022, BGPKIT as an open-source organization made significant progresses. As the founder, I am grateful for all the opportunities would like to take this time to appreciate all the milestones we achieved. In this post, I will go through some notable changes we made in 2022, and take a look at what we are excited about for the year of 2023.

BGPKIT Parser

There were a number of major features added to BGPKIT Parser in 2022:

v0.7.0 added support for filtering messages by many fields, and allowing reading from uncompressed files.
v0.7.1 added better examples of parallel MRT files processing with rayon.
v0.7.2 added filtering by multiple peer_ips.
v0.8.0 includes many internal refactoring and brought in the new oneio library to improve developer experience.

BGPKIT Broker

In 2022, we have revised the BGPKIT Broker backend to support crawling for estimated file sizes in addition to other fields like timestamps and URLs. This allows us to keep track of MRT file size changes and help users to pick suitable collectors to use, especially at the age where RIB dumps from a single collector could reach over 1 GB in size.

Figure of RIB sizes over time. Blue line is for rrc00, while yellow line is for route-views2.

We also made major revisions to our Rust SDK for more features like .latest() to get the lastest MRT files from each collector.

https://github.com/bgpkit/bgpkit-broker/releases/tag/v0.5.0

Plus, if you would like to deploy a broker instance yourself, we also have made some significant efforts to improve our documentation on self-hosting guide.

https://github.com/bgpkit/bgpkit-broker-backend/blob/main/deployment/README.md

Python Bindings

In addition to core Rust code base for parser and broker, we have also added support for Python bindings for various Rust SDKs. Users can easily parse MRT files directly using pybgpkit Python library. It is also proven to be usable on cloud-based Jupyter notebooks like Google Colab (examples).

https://github.com/bgpkit/pybgpkit

monocle

To ties things together, we have also developed our first investigative tool, monocle , to help users to quick find relevant BGP announcements with a suite of easy-to-use utilities. Users with Rust toolchain installed can run cargo install monocle to install the tool.

https://github.com/bgpkit/monocle

Users can use the following subcommands

parse: parse single MRT files, remotely or locally
search: find and filter BGP messages accross multiple public collectors
time: convert between local time string and Unix timestamps
whois: find out AS names, ASN, registration countries, and organizations.

Web/API and Infrastructure

In 2022, we started to experiment new cloud-based infrastructure, especially cloud-based databases for better API stability and developer experiences. We ended up selecting Supabase as our PostgreSQL production host and a self-hosted instance for backup. We are happy with the performance and cost provided by Supabase and more exicted about the potential capability it brings such as user authentication, cloud storage, local dev schema changes, etc.

Based on the new infrastructure, we have started to test a new integrated API system (still in alpha). This allows us to put all of our data access and processing end-points into one location. Based on the new API, we have also developed a newer version of the BGPKIT Broker statistics page: https://alpha.stats.bgpkit.com/

New Datasets

Apart from provide SDKs and data APIs, we have also started providing free access to historical archives of some data that we find interesting. We blogged about one dataset previous, the peer-stats dataset:

/blog/introducing-peer-stats-dataset

Here is a list of our currently available datasets at https://data.bgpkit.com/

[peer-stats](https://data.bgpkit.com/peer-stats/): route collector peers statistics (IP, ASN, v4/v6 prefixes counts)
[as2rel](https://data.bgpkit.com/as2rel/): AS-level relationship, using all available collectors
[pfx2as](https://data.bgpkit.com/pfx2as/): prefix-to-AS mapping, using all available collectors
[ihr-hegemony](https://data.bgpkit.com/ihr/hegemony/ipv4/global/): mirror of IIJ-IHR's global hegemony score dataset (big shout out to Romain and Internet Health Report for producing this data)

All the above datasets are free to use for research or commercial usages, and here is the acceptable usage agreement.

More Public Repositories

There are more open code repositories set available, some experimental and some for data analysis. You can check out the full list here:

https://github.com/orgs/bgpkit/repositories

Founder's Notes

In later 2022, I have joined Cloudflare to continue working on routing security for public benefits. During the first few months, we have built and shipped our new route-leak detection system under the public Radar platform. BGPKIT suite is now used in production for the BGP data anaysis pipeline at Cloudflare. While working full-time now, I am still committed to maintain the software suite and bringing new features to BGPKIT. For example, this year, I ported back our Kafka support used in Cloudflare to BGPKIT Broker backend. Folks at Cloudflare are doing great things in the open-source realm, and BGPKIT software suite will continue to become more useful to BGP enthuesastics and remain completely open-source.

Quote from Cloudflare's route-leak detection system blog.

Looking at 2023, here are a few things that I am really excited to work on for BGPKIT suite:

continue improve the infrastructure of the system and adding new data processing pipelines
continue improve the parser's performance and reliability
adding new RFC supports to parser (e.g. RFC9234 for route-leak prevention)
productionizing the new API and stats website
write more examples and documentations (don't we all love that)

And yeah, we will continue to be open-source first!

If you like what we do here, please consider subscribe to our blog. For all code repositories, check out our GitHub page.

https://github.com/bgpkit

Introducing Peer-Stats Dataset

Mingwei Zhang — Mon, 16 May 2022 00:00:00 GMT

Public BGP data collector projects like RouteViews and RIPE RIS provide valuable research and operational information for understanding BGP and detecting Internet routing anomalies.

There are many BGP routers involved in BGP collection projects.

A project includes many "collectors," and each serves as a collection of messages from several active BGP peers from different networks. Some bigger collectors collect BGP data from more than one hundred BGP routers. For example, RIPE RIS rrc00 has 112 active BGP peers at writing. (To learn about the complete list of BGP peers from all collectors, try theexperimental toolswe developed.)

Sometimes, too many BGP peers may become problematic.

Not all peers present the same amount of data. Some peers are so-called "full-feed" peers, which are the ones that provide the full routing tables to the collector. In a routing table dump file from the collectors, we can observe the full table of these peers. Some peers, however, only provide a limited number of routing entries to the collectors, not representing the whole routing status from these peers. In a project that tries to rebuild full routing tables, e.g., some BGP hijack detection or anomaly detectors, people prefer to use the full-feed peers as their data source.

At times, we are only interested in data from certain peers. For example, when studying the routing data from a particular network, if the network connects to BGP data collectors, we can directly pull data from the collectors' data. However, it can be troublesome to learn about what collectors have data from certain peers. RIPE RIS provides a nice API for querying such info, but we couldn't find one for RouteViews.

Historical data for such information is also missing. Unfortunately, for the researchers who want to study the evolution of the data collectors, even RIPE RIS's peers API could not help with that.

Introducing BGPKIT Peer-Stats Dataset

Peer-Stats dataset is a publicly available, free-to-use dataset that aims to provide daily collector peer information for all RouteViews and RIPE RIS collectors for ten years.

https://data.bgpkit.com/peer-stats/

The data includes the following fields for each peer of a BGP collector:

asn: Autonomous System Number of the collector peer
ip: the IP address of the collector peer
num_v4_pfxs: the number of IPv4 prefixes propagated from the collector peer
num_v6_pfxs: the number of IPv6 prefixes propagated from the collector peer
num_connected_asns: the number of connected (immediate next hop) ASes from the collector peer

The dataset is organized by the following structure.

- collector
	- year
		- month
			- data files

Screenshots of the dataset file listing site.

Each data file is in JSON format (see the section below) and compressed with bzip2. Users can easily use tools like bzcat and jq to view the data files. For example, you can run the following command to quickly view any of the peer-stats data for the collector rrc00 on 2022-05-01.

curl "https://data.bgpkit.com/peer-stats/rrc00/2022/05/rrc00-2022-05-01-1651363200.bz2" --silent | bzcat | jq

{
  "collector": "rrc00",
  "peers": {
    "102.67.56.1": {
      "asn": 328474,
      "ip": "102.67.56.1",
      "num_connected_asns": 330,
      "num_v4_pfxs": 919443,
      "num_v6_pfxs": 0
    },
    "103.102.5.1": {
      "asn": 131477,
      "ip": "103.102.5.1",
      "num_connected_asns": 184,
      "num_v4_pfxs": 895482,
      "num_v6_pfxs": 0
    },
...

Because all the data files are generated against the midnight UTC RIB dump of the day, you can also easily construct a URL to a data file for any particular date using the following template.

https://data.bgpkit.com/peer-stats/{COLLECTOR}/{YEAR}/{MONTH}/{COLLECTOR}-{YEAR}-{MONTH}-{DAY}-{MIDNIGHT_TIMESTAMP}.bz2

Open-source

We also open-sourced the data collection command-line tool source code on GitHub. Feel free to check it out and run it on your infrastructure if needed.

https://github.com/bgpkit/peer-stats

Credits and Sponsorship

The original idea for this work came from our extensive discussion with Romain Fontugne (follow him on Twitter at @romain_fontugne) from IIJ. This work is made possible by IIJ's generous sponsorship.

Please consider sponsoring us on GitHub if you find our work valuable and would like to see more open-source code and datasets on BGP.

https://github.com/sponsors/bgpkit

KhersonTelecom Outage and Connectivity Change

Mingwei Zhang — Tue, 03 May 2022 00:00:00 GMT

Internet service in Russian-occupied Kherson, Ukraine was disabled at 16:12 UTC (6:12pm local) on Saturday, 30 April. #UkraineRussiaWar

Khersontelecom service was restored ~24hrs later via Russian transit from nearby Crimea. pic.twitter.com/uN31jLrzEc

— Doug Madory (@DougMadory) May 2, 2022

The AS47598 experienced an outage shortly after 2022-04-30T16:10:00 and then resumed connectivity 2022-05-01T16:15:00 (both UTC time). After the outage, the AS47598 is then connected via a different upstream provider, AS201776.

The upstream provide change can also be seen on IIJ's Internet Health Report.

Prefixes

AS47598 announces only one prefix 91.206.110.0/23 (data from Hurricane Electric). Most of the BGP announcements are for this prefix. However, after the provider change happened, there were also a IPv6 prefix announcements for this V6 prefix as well 5bce:6e00::/23. It is possible that this prefix is a V4-translated prefix propagated to a V6 collector peer, but we do not know for sure.

We can also confirm the outage of the prefix with RIPEstat’s Routing History data widget. (uncheck the No low visibility box to reveal the outage).

BGP Messages

We can visualize the overall BGP announcements volume with Cloudflare Radar’s AS-level page.

The following are the UTC timestamps for the corresponding BGP message spikes.

2022-04-30T16:10:00: announcements with old provider 12883 in the paths.
2022-05-01T16:00:00: announcements with the new provider 201776 in the paths
2022-05-03T10:45:00: similar announcements with a new provider in the paths.

During the first gap time (2022-04-30T16:15:00 to 2022-05-01T16:00:00), there were 0 BGP updates for the prefix or from the ASN.

The old provider paths look like this where AS12883 is the next hop for AS47598. See the full list of messages from rrc00 here: https://gist.github.com/digizeph/c58b77f755d7fec8a7969807fb17d5ba.

207564 56655 3257 12883 47598

The new provider paths look like this where AS12389 and AS201776 are the next hops for AS47598. See the full list of messages from rrc00 here: https://gist.github.com/digizeph/896a4a7e4de23082b496b92ab5bdab5b

207564 28824 28824 1299 12389 201776 47598

BGP Data Tooling

The analysis is done using a privately hosted open-source BGPKIT parser web API. You can host it on your infrastructure, and the source code is freely available at https://github.com/bgpkit/pybgpkit-api. Comments and feedback are welcome!

Update on 2022-05-04T09:20:00 Pacific

#Kherson — Internet connectivity is returning to the occupied city in South of #Ukraine, after an outage since Saturday. @Cloudflare data shows growth in requests since 04:15 UTC, and telecom connection was confirmed by the Ukrainian Vice PM @FedorovMykhailo. pic.twitter.com/JxT3kcM234

— Cloudflare Radar (@CloudflareRadar) May 4, 2022

The upstreams for AS47598 has reverted back to the original ASes, and the traffic has started coming back to normal.

Parallel MRT Files Parsing with BGPKIT

Mingwei Zhang — Wed, 16 Mar 2022 00:00:00 GMT

In this post, we will talk about how to implement a Rust workflow that can process a large number of BGP data files as fast as we can. We will use BGPKIT Parser and Broker for data collection and parsing, and Rayon crate for parallelization of the code.

Task Overview

Before we begin to talk about the code design, we first need to introduce the data we are dealing with. We want to process the BGP data collected by various collectors, saved in compressed binary MRT format, and archived to files with a fixed interval. In this post, we are using RouteViews archive data as an example. The average data file size ranges from 2MB to 10MB by different collectors (AMSIX collector for example has pretty large files).

For processing, we can use the simplest task possible to do: sum the number of MRT records in all the files. We want to download and process all the updates files for one hour from all the collectors in RouteViews project. Here is the estimated amount of data we are dealing with:

35 collectors
5-minute interval — 12 files per collector
420 total number of files to download and process
840MB to 4.2GB total download size (it’s somewhere in between)

Ok! Now that we know what we want to do and have a sense of the estimated workload of the overall task, let’s coding!

Photo by Glenn Carstens-Peters on Unsplash

1. Sequential Parsing

Our first attempt to achieve the goal is to design and implement a naive sequential workflow as described below

find all BGP updates files within the hour of interest
iterate through each file, parse the MRT data and count the number of records
sum all record counts and print out the result

For this sequential workflow, we will need to pull in two dependencies into Cargo.toml:

[dependencies]
bgpkit-parser = "0.7.2"
bgpkit-broker = "0.3.2"

The bgpkit-broker handles looking for updates files within the hour, while bgpkit-parser handles parsing each individual file.

Finding files

BGPKIT Broker indexes all available BGP MRT data archive files from both RouteViews and RIPE RIS in close-to-real-time. For each data file, it saves the following information:

project: route-views or riperis
collector: the collector ID, e.g. rrc00 or route-views2
url: the URL to the corresponding MRT file
timestamp: the UNIX time of the start time of the MRT data file.

With all this information indexed, we can then query the backend and retrieve files information as we want. BGPKIT Broker provides both RESTful API, Rust API, as well as a Python API. Here we use the Rust API to pull in the information we need:

let broker = BgpkitBroker::new_with_params(
    "https://api.broker.bgpkit.com/v1",
    QueryParams {
        start_ts: Some(1640995200),
        end_ts: Some(1640998799),
        project: Some("route-views".to_string()),
        data_type: Some("update".to_string()),
        ..Default::default()
    });

for item in &broker {
    println!("processing {:?}...", &item);
}

The above block queries the broker and prints out all information of the retrieved files' metadata. The BgpkitBroker::new_with_params call accepts two parameters, one for the endpoint of the broker instance, and the other specifies the filtering criteria. In this example, we search for all BGP updates files from RouteViews with timestamps between 2022-01-01T00:00:00 and 2022-01-01T00:59:59 UTC. It prints out the output as the following:

processing BrokerItem { collector_id: "route-views.telxatl", timestamp: 1640997000, data_type: "update", url: "https://archive.routeviews.org/route-views.telxatl/bgpdata/2022.01/UPDATES/updates.20220101.0030.bz2" }...
processing BrokerItem { collector_id: "route-views.uaeix", timestamp: 1640997000, data_type: "update", url: "https://archive.routeviews.org/route-views.uaeix/bgpdata/2022.01/UPDATES/updates.20220101.0030.bz2" }...
processing BrokerItem { collector_id: "route-views.wide", timestamp: 1640997000, data_type: "update", url: "https://archive.routeviews.org/route-views.wide/bgpdata/2022.01/UPDATES/updates.20220101.0030.bz2" }...
processing BrokerItem { collector_id: "route-views2", timestamp: 1640997900, data_type: "update", url: "https://archive.routeviews.org/bgpdata/2022.01/UPDATES/updates.20220101.0045.bz2" }...

Parse each MRT file

Previously, in the for loop, we only print out the retrieved meta information of the MRT files. Now let's add the actual parsing of the files into the loop. The code is very simple, as designed:

let mut sum: usize = 0;
for item in &broker {
    println!("processing {}...", &item.url);
    let parser = BgpkitParser::new(&item.url).unwrap();
    let count = parser.into_record_iter().count();
    sum += count;
}

We first define a mutable variable sum outside the loop. Then for each file, we create a new Parser instance by BgpkitParser::_new_(&item.url). Here, as our goal is to count the number of records, we call the parser's .into_record_iter() function to create a iterator over the records of the file, and then .count() to get the count of the records. Lastly, we add the count to the overall sum variable.

Run and timing

For testing, I use a fairly powerful VM on a host with AMD 3950x CPU (32 threads), then build the release build and time the release run. The runtime includes downloading the MRT files to my machine with 1Gpbs down link in Southern California.

cargo build --release
time cargo run --release --bin sequential

It ended up taking about 1 minute and 23 seconds to sequentially parse 144 MRT files from RouteViews for all available ones within the first hour of 2022 (UTC).

total number of records for 144 files is 10554212

real    1m23.081s
user    0m39.535s
sys     0m1.006s

2. Parallel Parsing

Since the parsing of each file is completely independent of each other, we can parse the files in parallel and then sum up the count for each thread at the end. In Rust with Rayon, this conversion is very simple.

Let's first add the dependency of Rayon first:

[dependencies]
bgpkit-parser = "0.7.2"
bgpkit-broker = "0.3.2"
rayon = "1.5.1"

Then we change the broker code one tiny bit to collect all meta information for files into a vector first.

let items = broker.into_iter().collect::<Vec<BrokerItem>>();

This would enable us to fully utilize rayon's great syntax sugar to turn our sequential code into a parallel one.

let sum: usize = items.par_iter().map(|item| {
    println!("processing {}...", &item.url);
    let parser = BgpkitParser::new(&item.url).unwrap();
    let count = parser.into_record_iter().count();
    count
}).sum();

The key difference here is the calling of .par_iter(). It turns a sequential iterator into a parallel iterator, and by default going to utilize all available cores on the host machine for scheduling. Then we call .map() to define the parsing steps for each file, and then call .sum() at the end to add all results up.

The final result is approximately 10x faster than the sequential version, and it took only 8 seconds to parse all MRT files and get the record counts.

total number of records is 10554212

real    0m8.086s
user    0m42.569s
sys     0m1.068s

The full code for this example is as follows:

The source code of the two examples is available on GitHub. Feel free to poke around and tweak it as you wish.

%[https://gist.github.com/digizeph/c23ba39968c6cb4e1ad323520540010f#file-parallel-parsing-rs]

https://github.com/bgpkit/bgpkit-tutorials/tree/main/parallel-parsing

Real-time RIS Live Data with BGPKIT Parser

Mingwei Zhang — Fri, 12 Nov 2021 00:00:00 GMT

In terms of real-time BGP data processing, RIPE NCC provides a great data source: Routing Information Service Live (RIS Live).

To begin with, here is what RIS Live by the creators:

RIS Live is a feed that offers BGP messages in real-time. It collects information from the RIS Route Collectors (RRCs) and uses a WebSocket JSON API to monitor and detect routing events around the world. A non-interactive full stream (“firehose”) is also available.

In essence, RIS Live provides:

a WebSocket interface to stream BGP messages in real-time
ability to subscribe to “sub-streams” with custom filtering messages
JSON-encoded BGP messages as the stream payload
“firehose” HTTPS stream interface as well, without needing to work with websocket.

In this post, we will discuss how to use the RIS Live stream in practice.

RIS Live Message Format

RIS Live has client messages and server messages.

The client messages is used to setup or dismantle “subscriptions”, which essentially tell the server what kind of BGP messages a client would like to receive, and allow the server to send only the interested messages to the client.

A server acknowledges the requests from the client and afterwards start streaming requested data back to the client. At a high-level, a server sends either ris_message or ris_error messages. The ris_message is the main payload that we are interested in, while the ris_error message provides debugging messages for the scenarios where stream or subscription fails.

Subscribe to a WebSocket Stream

RIS Live provides great flexibility for the clients to specify/narrowdown the interested messages, allowing both the server and client process less messages during a streaming session.

host: only messages collected from a particular RRC (e.g. rrc21)
type: only messages of a given type, e.g. UPDATE , OPEN
require : only messages containing a given key, e.g. withdrawals will return only message that contains any withdrawn prefixes
peer: messages from a particular BGP peer
path : ASN or pattern to match the AS Path attribute in BGP update messages
prefix: only messages containing information for a given prefix
moreSpecific and lessSpecific: only messages that are the subprefix or super-prefix of the specified prefix
includeRaw: whether to include the Base64-encoded RAW BGP messages

As an example, let’s take a look at the following message from the official manual:

{
  "host": "rrc01",
  "type": "UPDATE",
  "require": "announcements",
  "path": "64496,64497$"
}

Example subscription message composer on RIS Live official site.

As an example, let’s take a look at the following message from the official manual:

{
  "host": "rrc01",
  "type": "UPDATE",
  "require": "announcements",
  "path": "64496,64497$"
}

collected by rrc01
BGP UPDATE messages
have at least one announced prefix
the last two hops of the AS Path is 66496 and 64497 (the origin)

The ris_message consists of “common header” fields and “data” fields (although they’re on the same level).

The “common header” fields are present for all types of sub-type messages, including timestamp, peer, peer_asn, id, host, type . The rest of the fields are the data fields that are dependent on the type of the messages. For most people, the UPDATE message is what they need. The following JSON block is an example message pulled directly from the demo site.

Example JSON formatted RIS message:

This example shows a BGP announcement of AS132354 originating two prefixes 103.249.208.0/23 and 103.14.184.0/24 , with the next hop to be 37.49.237.228. At this point, the information we see here is pretty similar to what we can see from other BGP MRT reader’s output (e.g. from bgpdump or bgpreader), just in JSON format.

WebSocket or Firehose?

Provided that RIS Live provides both WebSocket and HTTP Firehose, one would naturally wonder which one is the right choice for their application. Here we have a brief comparison between the two in the context of RIS Live.

WebSocket

Good:

easy to customize stream by composing a simple JSON subscribe message
work with various toolings in languages like Python and JavaScript

Bad:

requires extra library dependencies to work with WebSocket
need to write somewhat lengthy to get started (comparing to firehose)

Firehose

Good:

easy to consume by simply calling GET request on the URL
simple single-liner commandline program can start the stream (e.g. a simple curl call), no need complex script

Bad:

customizing stream is doable with XRIS-SUBSCRIBE HTTP request header, but feels clunky and limited
in my personal tests, the stream get disconnected often due to the stream cannot keep up with the data producer. this did not happen with websocket tests.

Summary

If your application could afford additional dependencies or writing extra scripts, WebSocket is the better choice. RIS Live official manual also makes implication that the WebSocket format is the current formally-supported streaming method.

RIS Live Coding Example with BGPKIT Parser

Now that we have a basic idea of what is RIS Live and the basic message format, we can get started working on some code that will actually use RIS Live to do something useful.

In the following example, we will build a short monitoring service that alerts us when Facebook operators announces their DNS IP prefix (see what happened before here). We are going to build the service in Rust with BGPKIT Parser , WebSocket library Tungstenite.

First, lets collect some basic information about what we are going to monitor here:

Facebook’s autonomous system number is 32934. So we will watch for all messages that was originated from AS32934.
Facebook’s DNS server IP prefixes involved in the previous incidence are 129.134.30.0/23 and 185.89.218.0/23 . So we want to carefully watch these two prefixes in our monitoring system.
We want to use one of the RIPE RIS collectors’ data for monitoring, rrc21 is a good choice since it’s being used by RIS Live’s demonstration. You can easily extend this service by tweaking the subscription message later.

OK, we are good to go. Let’s do it!

Setting up the stream

We picked the Tungstenite library as our WebSocket library of choice, partly because it has a very straightforward API design.

Let’s first connect to the websocket server by calling connect function given a websocket URL. One thing to notice is that the URL protocol section here is ws as opposed to the wss mentioned in the RIS Live documentation. For some reason, Tungstenite does not work with wss protocol (with SSL).

use tungstenite::{connect, Message}; 
const RIS_LIVE_URL: &str = "ws://ris-live.ripe.net/v1/ws/?client=rust-bgpkit-parser";
let (mut socket, _response) =
    connect(Url::parse(RIS_LIVE_URL).unwrap())
    .expect("Can't connect to RIS Live websocket server");

Now, with a socket ready, we will first send a subscription message to let server know that we want some messages and we are ready to receive.

let msg = json!({"type": "ris_subscribe", "data": {"host": "rrc21"}}).to_string();
socket.write_message(Message::Text(msg)).unwrap();

Here we composed a simple subscription message that limits the stream to have messages only from rrc21 collector.

Parsing JSON messages

At this point, we have a WebSocket connection to RIS Live server, and have sent out a subscription message to the server. The server should be sending back messages anytime now, and we are ready to consume the stream.

We would like to code the following behavior:

continuously reading the websocket messages;
parse JSON string into internal BGP structs;
check each message if it contains origins (withdraw-only messages does not contain AS paths, and thus no origins either;
if the origin AS is AS32934, and the announced prefix is 129.134.30.0/23 or 185.89.218.0/23 , then we print out the message to output.

loop {
    let msg = socket.read_message().expect("Error reading message").to_string();
    if letOk(elems) = parse_ris_live_message(msg.as_str()) {
        for elem in elems {
            if letSome(origins) = elem.origin_asns.as_ref() {
                if origins.contains(&32934) &&
                    ( elem.prefix.to_string() ==  "129.134.30.0/23".to_string() ||
                        elem.prefix.to_string() ==  "185.89.218.0/23".to_string() )
                {
                    println!("{}", elem);
                }
            }
        }
    }
}

The full example code can be found here:

%[https://gist.github.com/digizeph/fcac3027555c0b744ea0b3a11197b694#file-bgpkit-parser-kafka-bmp-example-rs]

Building More with BGPKIT Tools

As introduced in our previous blog post, we added support of real-time BMP stream to BGPKIT Parser as well. Combining with RIPE RIS Live, and RouteViews BMP stream, we can build a powerful real-time BGP monitoring service directly within BGPKIT Parser. We also offer indexing and processing of historical BGP data as well with BGPKIT Broker.

Our goal at BGPKIT is to design, develop, and deploy the most developer-friendly BGP data processing toolkit. To learn more about our offerings, please check out our website and official Twitter account.

Real-time BMP with BGPKIT Parser

Mingwei Zhang — Wed, 10 Nov 2021 00:00:00 GMT

Real-time BGP data processing is very critical on building monitoring services that can detect BGP issues quickly with minimum delay and react to anomalies quickly and mitigate potential issues.

We are creating a new series of posts describing how we design our software to work with real-time BGP data streams. As an opening, we will describe how we handle data streams with BMP protocol and OpenBMP messages.

BMP

The BGP Monitoring Protocol (BMP) is a protocol that allows monitoring of BGP devices.

The RFC7854 describes the purpose of BMP as:

Many researchers and network operators wish to have access to the contents of routers’ BGP Routing Information Bases (RIBs) as well as a view of protocol updates the router is receiving. This monitoring task cannot be realized by standard protocol mechanisms. Prior to the introduction of BMP, this data could only be obtained through screen scraping.

BMP provides access to the Adj-RIB-In of a peer on an ongoing basis and a periodic dump of certain statistics the monitoring station can use for further analysis. From a high level, BMP can be thought of as the result of multiplexing together the messages received on the various monitored BGP sessions.

There are multiple types of BMP messages, each serving different purposes.

Peer up and down notification: notification about the status of peering sessions to a monitored router;
Initiation message: inform the monitoring station of the routers vendor, software version, and so on;
Termination message: provides information on why a monitored router is terminating a session;
Route monitoring:initial synchronization of the routing table;
Route mirroring: verbatim duplication of messages as received.

For real-time BGP data processing, we are specifically interested in the route monitoring and route-mirroring messages, as we provide the routing information encoded as actual BGP messages.

OpenBMP

OpenBMP is a software implementation of the BMP protocol. It is an open-source project created by Cisco and currently maintained by nice folks from CAIDA/UCSD and RouteViews. It is implemented in C++, can be used with any compliant BMP sender (e.g., router).

Architecture graph of OpenBMP

OpenBMP provides multiple formats for outputting the BMP messages collected from the connected routers, one of which is the raw_bmp format, which is a thin wrapper of the raw BMP messages. The raw_bmp format provides the best performance and allows use to handle the BMP messages directly without having to write a different parser for the plaintext messages.

RouteViews currently provides a OpenBMP Kafka stream that streams BMP messages from their collectors.

BGPKIT Parser with BMP/OpenBMP Support

We develop BGPKIT Parser to provide a one-stop solution for handling all parsing tasks regarding BGP data. Supporting real-time data like BMP is a very important milestone for us.

We have recently developed the full support for BMP messages, and partial support for OpenBMP messages (for raw_bmp type only). This enables us to start working with real-time BMP streams like RouteViews’ Kafka stream.

Below is an example code that takes RouteViews’ Kafka OpenBMP stream and parse the messages into internal data structures:

let mut reader = Cursor::new(Vec::from(kafka_payload));
let header = parse_openbmp_header(&mut reader).unwrap();
if let Ok(msg) = parse_bmp_msg(&mut reader) {
    info!("Parsing OK: {:?}", msg.common_header.msg_type);
    match msg.message_body {
        MessageBody::RouteMonitoring(m) => {
            dbg!(m.bgp_update);
        }
        _ => {}
    }
}

Here is a break down of what it does:

it first creates a bytes reader from the raw Kafka message payload;
then parse OpenBMP message header, which contains some basic information about the BMP session;
then it calls the parse_bmp_msg function to parse the embedded raw BMP messages and print out the BGP update messages if the parsing is successful.

Here is a full code example:

%[https://gist.github.com/digizeph/fcac3027555c0b744ea0b3a11197b694#file-bgpkit-parser-kafka-bmp-example-rs]

We have published the SDK on crates.io and GitHub. Feel free the check out the example code at examples/routeviews-kafka.rs if you are interested.

Introducing BGPKIT Parser

Mingwei Zhang — Mon, 01 Nov 2021 00:00:00 GMT

BGPKIT Parser is an open-source Rust-based MRT/BGP data parser that takes a MRT formatted binary file and turns it into BGP messages. It is one of the most important building block software that enables BGP data processing and analysis tasks.

Design and Features

As mentioned in our previous post introducing BGPKIT Broker, the most used BGP data collection projects, RouteViews and RIPE RIS, both publish their collected BGP data in MRT format on their data platform. The BGPKIT Parser is designed to handle parsing tasks for these data sources.

We design our parser to strictly follow the industry standard, e.g. RFC4271 and RFC6396. BGPKIT Parser is also designed with the following goals:

performant: comparable to C-based implementations like bgpdump or bgpreader.
actively maintained: we consistently introduce feature updates and bug fixes, and support most of the relevant BGP RFCs.
ergonomic API: a three-line for loop can already get you started.
battery-included: ready to handle remote or local, bzip2 or gz data files out of the box.
open-source: we want people to use our parser freely and we can continue develop and improve it based on community feedbacks.

To demonstrate how easy it is to get started using BGPKIT Parser, check out the example below where we print out all BGP messages from a remote MRT file on RouteViews:

Example of reading a remote MRT file from RouteViews and print out BGP messages. Code available at https://gist.github.com/digizeph/9977371653f39a459ff3ae507dc3636c

There are a number of things happens when we call for elem in BgpkitParser::new(url) :

it creates a new BgpkitParser struct instance with the provided URL to the data file;
it tries to retrieve the content of the remote file and download the raw compressed bytes into memory;
it determines the compression type by file suffix and calls corresponding decompression library to create a buffered reader;
it then creates an iterator (used by the for loop) that continuous return new parsed items until it reaches the end of the data stream.

We determined that it is worth the extra binary file size to bring in the network and compression libraries into the project so that the library users will never have to worry about handling data downloading and decompression by themselves again.

The Future

The future of the BGPKIT Parser lies on continuous performance and stability improvements, as well as some exciting features that we are currently planning. Some of the coming features include

adding capability of handling real-time data streams coming from RIPE RIS Live and RotueViews’s Kafka BMP stream;
supporting data serialization back to MRT files (reverse-parsing), which allows users to produce customized MRT files after data processing;
adding WASM support to allow BGP data parsing directly on the web with JavaScript.

Because we are building our software in Rust, we can effortlessly tapping into Rust’s great software ecosystem and continue introducing new features and improvements. The future of BGPKIT Parser is exciting and we can’t wait to bring more features for you to try out!

For more details about the BGPKIT Parser, check out our GitHub repo and our website. Feedback is highly appreciated!

https://github.com/bgpkit/bgpkit-parser

Introducing BGPKIT Broker

Mingwei Zhang — Sun, 31 Oct 2021 00:00:00 GMT

BGPKIT Broker is a data API service that focuses on building a BGP data file index to enable searching for public/private BGP data files with custom filters. It is one of the building block components designed by BGPKIT to facilitate BGP data processing with ease.

The first step to investigate a BGP event.

Imagine this scenario: a malicious player just attempted to hijack a IP prefix using BGP announcements, and you are interested in learning what exactly happened during that half-hour down time of the victim network, how would you start investigating?

Collecting evidences. Luckily for us, the actors on the Internet, good and bad, always leave traces behind if they use BGP as their method. There are a number of reputable public BGP route collectors operating for years collecting all BGP messages received from their connected router peers and dump them into regular dump files. The most used projects are RouteViews and RIPE RIS Data.

Files are all over the places. There are more than 60 different data collectors from the two projects alone, and each publishing their data under separate sites. The two projects also have different data file structure and compression algorithms for their data dump files. It is not hard to go find one data file during the interested event time from one collector, but it will be a real hassle to gather URLs toward all data files that include information within a specified time range.

Collector’s data is published as compressed MRT files at regular frequency.

BGPKIT Broker — A BGP Data File Index API Service

We designed the BGPKIT Broker to resolve one and only one problem: quickly collect links to the BGP data files that matches a filtering criteria.

You can filter BGP data using multiple criteria:

start_ts : UNIX timestamp that all files must be dumped after
end_ts : UNIX timestamp that all files must be dumped before
data_type : the type of the data file, can be update or rib
collector: the collector ID that the files are generated from
project: the data collection project, can be route-views or riperis
page and page_size: the pagination control for collecting a large number of files

Here is an example REST API call: https://api.broker.bgpkit.com/v1/search?data_type=update&start_ts=1633046400&end_ts=1633132800&collector=rrc00&project=riperis&page%20=2&page_size=3

It asks for all data updates files dumped between 1633046400 and 1633132800, from RIPE RIS’s collector rrc00. It also requested for the second page of the results and each page contains 3 items.

DEPRECATED: BGPKIT Broker API service is freely available to use, hosted at https://api.broker.bgpkit.com/v1/. The documentation is available at https://docs.broker.bgpkit.com/

BGPKIT Broker API and SDK has been upgraded to V2 now. The V1 examples are left up for legacy services that depends on it. Please check out the current API documentation at for more:https://api.broker.bgpkit.com/v2/

BGPKIT Broker Rust API

The BGPKIT Broker API service is built entirely using Rust, and of course we also developed native Rust API to access the broker data with ease using Rust.

Example BGPKIT Broker Rust API call

The Rust API is open source under MIT license, and with the free API, you can already build your own workflow today!

https://github.com/bgpkit/bgpkit-broker

Easy On-premise Deployment

For API service like this, especially that it also collects and indexes data of over 10 years span, one might imagine the deployment could be complex and slow.

For BGPKIT Broker, we spent extra efforts to make the API deployment process as quickly as possible, and also efficient on resource consumption. For references, we bootstrapped the entirety of the database in under 5 minutes, and cost less than 1 GB storage to store the information in PostgreSQL database. The whole database and API run fluently with less than 500MB of RAM usage. This allows use to deployment extra instances when under heavy load without costing a fortune.

Users who need dedicated resource allocation for query performance can contact us for private API hosting that are not shared by others. Enterprise option is also available for on-premise deployment with customization consultation available. If you are interested in testing it out, feel free to shoot us an email at contact@bgpkit.com.

For more information, checkout our website!

https://bgpkit.com/broker

BGPKIT Journey Started

Mingwei Zhang — Tue, 26 Oct 2021 00:00:00 GMT

BGPKIT is a small-team start-up that aims to provide comprehensive tool suite to facilitate companies building on-premise BGP data monitoring services. We started our journey of building the best BGP data toolkit for developers in October, 2021. Here is a brief glance on what we are working on and what values we strive to provide.

BGP

Border Gateway Protocol (BGP) is the de facto inter-domain routing protocols being used by every major companies on the Internet. The main purpose of BGP is to allow companies exchange IP prefix reachability information. In other words, companies tell other companies what IP blocks they have, and how to reach these IP blocks. This the key functionality that enables the Internet.

Because the purpose of BGP is to exchange information and allow everyone to know how to reach certain IP blocks, the BGP messages must be propagated globally and publicly. There are a number of data sources available out there that provides BGP data archives (e.g. RouteViews and RIPE RIS), or real-time data like BGP looking glasses or live BGP data stream.

The aforementioned data sources contains a wealth of information if you know what to look for. For example, by looking at BGP information, researchers can infer companies relationships, monitor security incidences. Having handy toolkit in hand enables people to build successful businesses around BGP data.

BGPKIT

At BGPKIT, we build software tools to process BGP data and reveal insights from BGP messages. We aims to provide the best developer experience, and enable customers to build their own BGP data processing pipeline and monitoring services on-premise.

Complete Tool Suite

Our goal is to build complete tool suite for BGP data processing: data collection, parsing, analysis, programmable and visual interface, and data warehousing. Everything you need to handle BGP data.

Rust Implementation

To achieve the best performance and security, we choose to focus on building our tools using the Rust programming language.

We believe that Rust’s ecosystem is now mature enough that building modern features like data streaming, async data workflow, parallel processing, web API, or even porting the entire codebase onto WASM and running it on browsers is a archivable task with reasonable efforts.

Powerful Extensibility

At BGPKIT, we design our libraries to provide powerful API and assist customers to further customize workflow to meet individual needs. We strive to provide the most ergonomic interfaces that allow library consumers to easily integrate our library into theirs.

Embrace Open-Source

We also believe that good tools empowers people, so we open-sourced our building block libraries to the public with very permissive license so that any interested parties can explore and build their own ideas free of charge and limitation.

We are excited to start this journey of building, and we hope to have the chance to work with more people and building more dreams together! Follow us here, on Twitter, or visit our website to learn more!

BGPKIT Blog

bgpkit-parser 0.18.0 Release

Install

RIS Live raw message parsing

Raw BGP attribute retention

Deprecated attribute code handling

New typed BGP path attribute parsers

API changes to note

WebAssembly package

Links

Finding Well-Known BGP Communities with monocle

Why 65535:*?

Setup

Searching MRT updates with monocle

What the scan found

Checking BLACKHOLE behavior against RFC 7999

Checking NOPEER visibility against RFC 3765

A note on policy behavior

Takeaways

Investigating the India–Telegram BGP Hijack with monocle

Prerequisites

Step 1: Establish ground truth - who should originate this prefix?

The single most important check: RPKI validation

List Telegram's signed prefixes

Understand the suspect's connectivity

Step 2: Find the hijack in the data

Why a small collector?

Why -s -S?

The prefix arms race

Verify every hijacked prefix was RPKI-invalid

Step 3: Was the hijacked route actually selected in the routing table?

When did it end?

Step 4: Who propagated the invalid routes?

Step 5: The RPKI timeline - did Telegram have ROAs before the incident?

Step 6: What about the IPv6 blackholing story?

Summary

References

monocle v1.3.0: Reconstruct BGP RIB State at Any Timestamp

New Feature: monocle rib

Basic Usage

Filtering

Performance Notes

Installation

monocle v1.1.0

TL;DR

What's New

Simpler feature flags

Standardized data refresh APIs

Config command updates

Better cache control defaults

RPKI improvements

Parse and search enhancements

Breaking Changes and Migration Notes

1) Feature flag migration

2) CLI and subcommand renames

3) Parse/search filter type updates (library API)

Additional Improvements

Full Change List

BGPKIT Parser v0.14.0 Release and v0.13.0 Highlights

v0.14.0 Features

Negative Filter Support

RPKI RTR Protocol Support

In Case You Missed It: v0.13.0

Record-Level Output

Raw Bytes Access

Other Improvements

BGPKIT Broker v0.9.0: Better Pagination and New Collectors

Previously on BGPKIT Broker

V0.9: Efficient Pagination Support

SDK: query_total_count() Method

API: Total Field in Search Results

New RouteViews Collectors

Behind the Scenes Improvements

Upgrading to v0.9.0

For SDK Users

For Self-Hosted Instances

Looking Forward

Get Started

💖

Run BGPKIT on Cloudflare Containers

Why `65535:*`?

Searching MRT updates with `monocle`

Why `-s -S`?

New Feature: `monocle rib`

SDK: `query_total_count()` Method

Using `monocle radar`

`monocle radar stats`

`monocle radar pfx2as`