after measure

Comparison

Same stream, same subscriber, two paths. Charts and tables below are from a live Measure run in ap-southeast-6.

Next: Teardown Back: Measure

Two paths, one stream

Two paths to one stream: PrivateLink versus TGW with inspection Path A, in teal: the consumer EC2 host resolves an interface VPC endpoint in the consumer VPC, crosses PrivateLink into the provider account, reaches an internal Network Load Balancer on TCP 9000, then the relay instance, which ingests the Wikimedia stream from the public internet. Path B, in amber: the same consumer host routes to the provider private IP through the hub Transit Gateway, into the inspection VPC for Network Firewall inspection, back through the same Transit Gateway, then to the same relay instance on TCP 9000. Path A - PrivateLink 2 accounts - 1 hop - no shared CIDR Consumer VPC Consumer EC2 sub.py Interface endpoint ENI in consumer VPC PrivateLink allowed principal Provider VPC Internal NLB TCP 9000 Relay EC2 relay.py :9000 Public internet Wikimedia SSE endpoint DNS endpoint service target group SSE ingest Path B - TGW + inspection 3 accounts - 2 TGW hops - firewall Consumer VPC Consumer EC2 sub.py Transit Gateway spoke route table Inspection VPC Network Firewall stateful inspection same TGW appliance mode Provider VPC same Relay EC2 private IP :9000 provider CIDR inspection RT no-inspection RT TCP 9000

Teal = PrivateLink (consumer ENI → provider NLB). Amber = TGW hairpin through Network Firewall. Same relay.py, same TCP 9000.

  PrivateLink TGW + inspection
TCP peer from the subscriber Interface ENI in the consumer VPC Relay private IP in the provider VPC
Hops on the data path Endpoint → PrivateLink → NLB → relay Spoke → TGW → NFW → TGW → relay
Hub dependency None for the data path TGW + inspection VPC + stateful PASS
Shared CIDR required? No Yes (routable spoke CIDRs)

PrivateLink wins on in-path RTT because the subscriber’s peer is local. TGW pays for two Transit Gateway crossings and a stateful firewall. At firehose rates both paths still deliver the same upstream bytes — latency diverges; throughput does not.

Lab run (ap-southeast-6)

One 60 s capture-series.py run per path, started together from the same consumer host, then a warm sub.py summary and nping probes (Measure). Both sockets sit on the same relay process, so they receive the identical broadcast — that is why the rates match rather than merely being close.

Signal How it is measured
In-stream RTT PING <seq> / PONG <seq> on the established stream socket, one per second, timed on the consumer clock only
Series RTT point Median of that second’s samples — with one PING per second, effectively a single sample
connect socket.create_connection, so it includes DNS resolution and interpreter overhead
first_byte Measured from the start of connect, not from connect completion
Sustained rate Stream bytes over the run window, KiB/s
nping SYN → SYN/ACK handshake timing, 20 probes, no stream content

Single run, one Region, one AZ pair, ~60 RTT samples per path — so p95 carries real uncertainty and the ratios are directional, not a benchmark. connect and nping disagree by a couple of milliseconds because they measure different things (see the table above); the ratio between paths is what to read.

Lab summary — latency and rate Warm connect and 60s in-stream RTT p50 plus sustained rate for PrivateLink and TGW plus inspection. Lab summary — same Region, same relay ap-southeast-6 · warm + 60s series In-stream RTT p50 0.43 ms PL 1.07 ms TGW (~2.5×) nping TCP avg 0.66 ms PL 1.07 ms TGW (~1.6×) Sustained rate (60s) 55.1 KiB/s PrivateLink 55.1 KiB/s TGW + inspection Warm connect / first byte PL 2.7 / 3.4 ms · TGW 3.4 / 4.5 ms Same order — path tax shows up in-stream, not only on handshake

~2.5× in-stream RTT on TGW, ~1.6× on nping handshake probes, nearly identical KiB/s.

Side-by-side RTT

In-stream RTT over 60s Per-second median RTT from application PING/PONG. Parallel 60s capture on both paths from the same consumer host. In-stream RTT over 60s PING/PONG · same host · parallel PrivateLink TGW + inspection 00.40.81.21.6 0s15s30s45s60s Elapsed time (s) RTT (ms)

One PING/PONG sample per second on the established stream. PrivateLink ~0.4 ms; TGW ~1.1 ms.

Side-by-side rate

Sustained stream rate over 60s Per-second received bitrate on the subscriber. Parallel 60s capture on both paths from the same consumer host. Sustained stream rate over 60s same Wikimedia firehose · parallel PrivateLink TGW + inspection 0306090120 0s15s30s45s60s Elapsed time (s) KiB/s

Same Wikimedia firehose through one relay — rates overlap; path tax is delay, not bandwidth.

Results

Signal PrivateLink TGW + inspection Insight
In-stream RTT p50 / p95 (60 s) 0.43 / 0.71 ms 1.07 / 1.31 ms ~2.5× p50 — hairpin + NFW
Mean series RTT 0.45 ms 1.09 ms Gap holds for the full minute
Sustained rate (60 s) 55.1 KiB/s 55.1 KiB/s Same upstream
Warm connect / first_byte 2.7 / 3.4 ms 3.4 / 4.5 ms Handshake same order
nping TCP avg (20 probes) 0.66 ms 1.07 ms ~1.6×

The durable signal is in-stream RTT, not warm connect. Both handshakes land in a few milliseconds; the hairpin shows up on the established flow.

Why the gap exists

Factor PrivateLink TGW + inspection
Peer location ENI next to the subscriber Relay IP across accounts
Path structure Endpoint ENI → PrivateLink → NLB Two TGW attachment crossings (hairpin)
Middlebox None on the data path Network Firewall stateful inspection
Failure mode Endpoint / NLB / SG Route without hub PASS → handshake completes, then the stream stalls (drop_established default)

Streaming differences

Topic PrivateLink TGW + inspection
Idle timeout NLB TCP idle (default 350 s, 60–6000 s configurable) Firewall / TGW flow idle timeouts
Client attribution NLB node address; Proxy Protocol v2 adds the consumer address and endpoint ID Real consumer private IP at the relay
Flow stickiness Endpoint / NLB AZ affinity Appliance mode pins both directions to one firewall endpoint
Access model Allowed principals on the endpoint service Routes + SGs + hub firewall PASS
Overlapping CIDRs Supported Not supported without NAT

Cost shape

On-Demand list prices for ap-southeast-6 (Asia Pacific — New Zealand), from the AWS Price List API in September 2026. Re-check before you quote them (PrivateLink, Transit Gateway, Network Firewall, ELB).

Meter PrivateLink TGW + inspection
Fixed (hourly) Interface endpoint $0.01365/hr per AZ + NLB $0.02646/hr (+ NLCU) TGW VPC attachment $0.07/hr + Network Firewall $0.705/hr first endpoint, $0.282/hr each secondary AZ endpoint
Data processing VPC endpoint $0.01/GB (0–1 PB) TGW $0.02/GB on every pass into the gateway + NFW $0.065/GB

The hairpin sends the stream into the TGW twice — spoke → inspection, then inspection → provider — so ~$0.04/GB TGW + $0.065/GB NFW ≈ $0.105/GB.

Path New for this lab Already running in the hub
PrivateLink Endpoint hours + NLB hours + endpoint GB —
TGW + inspection Data processing only Attachment hours + firewall endpoint hours

New PrivateLink fixed cost is two AZ endpoints plus one NLB; the crossover is that fixed bill divided by the per-GB gap:

privatelink_fixed_monthly = (2 × 0.01365 + 0.02646) × 730  ≈ $39
break_even_GB_per_month   = 39 / (0.105 − 0.01)            ≈ 410 GB

Both lines exclude NLB capacity units. NLCUs bill at $0.0063/NLCU-hour here and bandwidth converts at roughly 1 GB/hour per NLCU, so if the bandwidth dimension binds, PrivateLink costs nearer $0.016/GB and the crossover moves to about 440 GB/month. Treat 410 GB as the floor.

Volume posture Likely outcome Why
Lab / low continuous KiB/s Hub cheaper if already sunk New PrivateLink fixed hours (endpoint + NLB) dominate; per-GB barely matters
High sustained GB through one service PrivateLink often cheaper ~$0.01–0.016/GB vs ~$0.105/GB hairpin+NFW once volume clears the ~410–440 GB break-even
Many ports / whole CIDR estate Hub (capability, not per-GB) TGW+NFW per-GB stays higher; PrivateLink does not replace L3 estate access without many endpoint services (+ NLB each)

When each wins

Measure — same host, two targets On the consumer subscriber EC2, open two concurrent TCP 9000 streams: left path uses the PrivateLink interface endpoint DNS; right path uses the provider relay private IP through Transit Gateway and Network Firewall. Watch both with tmux, nc, and pv. Measure — same host, two targets tmux · nc | pv · TCP 9000 Consumer VPC plvtgw-subscriber left + right panes Left — PrivateLink Interface EP $PL_DNS PrivateLink NLB :9000 Right — TGW + inspection TGW NFW PASS TGW Provider VPC plvtgw-relay $RELAY_PRIVATE_IP :9000

The run behind the numbers: one subscriber host, two targets. The decision is which peer the subscriber talks to — an endpoint you publish, or an address you route to.

Choose When
PrivateLink One (or a few) services; client-initiated; IAM principals instead of CIDR routes; lower latency; overlapping CIDRs OK
TGW + inspection Many hosts/ports or bidirectional estate access; mandatory east–west inspection; hub already exists below the PrivateLink break-even

Decision

Service product with a clear consumer boundary → PrivateLink. Network platform with mandatory inspection → stay on the hub and pay the hairpin.

Teardown Delete harness resources Measure Re-run interactive compare