How API Gateways handle raw TCP packets

How these gateways actually perform their roles at the HTTP packet level?

Let’s break it down into something more concrete with examples of how these gateways perform their “unique entrance” function by requiring clients to follow specific protocols, often via custom HTTP headers or query parameters.

---

🔐 1. Unique Entrance via HTTP Headers (or Tokens)

Gateways often serve as the single entry point into your microservice architecture. This is where they inspect incoming requests, enforce rules, and route traffic.

✅ Typical Header-Based Pattern

Clients are expected to add custom headers to each request, like:

GET /api/orders HTTP/1.1
Host: api.example.com
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR...
X-Client-ID: mobile-app
X-Trace-ID: 93f8de12-312f-4561-abb8-9fe9123345cd

🔧 Gateways Check These Headers:

• Authorization: For OAuth2/JWT verification.
• X-Client-ID: For client identity (mobile, web, internal).
• X-Trace-ID: For distributed tracing (e.g., with Zipkin).
• X-Version or X-Gray-Group: For gray (canary) releases.

If a header is missing or invalid, the gateway can:

• Return 401 Unauthorized
• Route to a fallback service
• Log and terminate the request

---

🚦 2. What Do Gateways Actually Do? (Packet-Level Breakdown)

🛣️ Dynamic Routing

Client:
GET /user-service/profile?id=123 HTTP/1.1
Host: api.example.com

Gateway:
- Inspects path `/user-service/`
- Routes to internal service: `http://user-service.local/profile?id=123`

⚖️ Load Balancing

Gateways maintain a list of backend instances:

"user-service": [
  "http://10.0.0.2:8080",
  "http://10.0.0.3:8080"
]

And randomly or round-robin routes requests.

🚫 Authentication (JWT)

Gateway verifies the Authorization: Bearer ... token.
If token invalid:

HTTP/1.1 401 Unauthorized
Content-Type: application/json
{"error": "invalid_token"}

🧯 Circuit Breaker / Degrade

If backend service is down:

GET /product HTTP/1.1

→ Circuit breaker detects repeated 5xx
→ Response:
HTTP/1.1 503 Service Unavailable
{"error": "Service temporarily unavailable"}

🧪 Gray (Canary) Release

Clients marked with a special header get routed to new version:

X-Gray-Group: test-users

→ Route to v2 instance
Others → Route to stable v1

---

🛠️ How This Is Implemented in Practice

🔹 Spring Cloud Gateway (Java)

Uses filters + predicates:

routes:
  - id: user_route
    uri: lb://user-service
    predicates:
      - Path=/user-service/**
      - Header=X-Client-ID, mobile-app
    filters:
      - AddRequestHeader=X-Trace-ID, #{UUID}
      - RewritePath=/user-service/(?<segment>.*), /${segment}

🔹 Netflix Zuul (Deprecated)

Java filters (Pre, Post) to intercept requests and do routing.

🔹 Kong (Lua / NGINX)

Can enforce plugin policies:

-- Custom plugin example
if ngx.req.get_headers()["X-Client-ID"] == nil then
  return kong.response.exit(400, { message = "Client ID required" })
end

🔹 OpenResty (Lua + NGINX)

Lua scripts inspect headers and perform routing:

local client_id = ngx.req.get_headers()["X-Client-ID"]
if not client_id then
    ngx.status = ngx.HTTP_BAD_REQUEST
    ngx.say("Missing Client ID")
    return ngx.exit(ngx.HTTP_BAD_REQUEST)
end

---

🧪 Real-World Scenario Packet Example

# Client Request:
GET /orders HTTP/1.1
Host: api.myapp.com
Authorization: Bearer eyJhbGciOiJIUzI1Ni...
X-Client-ID: ios-app
X-Gray-Group: test

# Gateway Inspects:
- Valid JWT? → Yes
- X-Client-ID present? → Yes
- X-Gray-Group: test → Send to beta backend

# Routes to:
GET http://orders-v2.internal/orders
Authorization, X-Client-ID, etc. are preserved or rewritten

---

🧩 Summary

• Gateways do expect clients to include specific headers, tokens, or query params.
• These headers are inspected by filters/plugins/scripts.
• Most functionality like auth, circuit breaking, gray release, logging is driven by what’s in these headers.
• Tutorials miss real packet examples because they focus on “what it can do” instead of “how it does it.”

Network service gateways like Spring Cloud Gateway, Netflix Zuul, Kong, and OpenResty are built to handle HTTP(S)-based traffic, not raw TCP packets or non-HTTP protocols (like custom binary protocols or EBCDIC-encoded streams).

---

✅ What Gateways Can Handle

These gateways operate at Layer 7 (Application Layer) and expect:

• HTTP/1.1 or HTTP/2 protocols
• Readable headers and bodies encoded in UTF-8 or ASCII
• JSON, XML, or form data in body
• Sometimes custom headers or cookies

For example:

GET /serviceA/api HTTP/1.1
Host: gateway.company.com
Authorization: Bearer ...
X-Trace-ID: ...

---

❌ What They Can’t Handle Directly

They cannot natively handle:

• Raw TCP sockets (e.g. telnet-style sessions, legacy protocols)
• Custom binary protocols (e.g. ISO8583, COBOL-style, or EBCDIC)
• Length-prefixed binary streams (where first 8 bytes indicate packet length)
• Protocols requiring byte-level parsing before decoding

For example, this kind of payload:

[00 00 00 2E] [C1 D7 D6 E2 40 D4 D6 D9 E2 C5]...
   ^            ^--- EBCDIC encoded payload
  Length = 46

is completely invisible and meaningless to an HTTP-based gateway.

---

🧱 What Handles This Instead?

You’d need an L4 (Transport Layer) or custom TCP server before the gateway to:

1. Accept TCP connection
2. Parse the custom binary format (e.g. read first 8 bytes as length, decode EBCDIC)
3. Translate it to a valid HTTP request
4. Forward it to the gateway

✅ Common Tools for This Purpose:

Tool	Purpose
Custom Java TCP Server	Use java.net.ServerSocket to accept binary TCP streams
Netty	Build high-performance custom TCP → HTTP proxies
Nginx (stream block)	Works at Layer 4, but still can’t decode binary
HAProxy (TCP mode)	Load balancing TCP traffic, but no payload parsing
Envoy + Wasm filter	Can parse TCP streams if extended carefully
Framing Proxy	Some banks write one that converts ISO8583 → JSON

---

🧪 Real-World Example (Banking Context)

A mainframe system sends:

• EBCDIC-encoded binary stream
• First 4 or 8 bytes are a length prefix
• Payload contains financial transaction data (ISO8583)

A custom TCP parser is built in Java or C++:

1. Listens on TCP port
2. Parses the length-prefixed binary stream
3. Converts EBCDIC to UTF-8
4. Maps payload to JSON:

{
  "cardNumber": "12345678",
  "amount": 500,
  "currency": "USD"
}

5. Sends it as HTTP POST to:

POST /processTransaction HTTP/1.1
Content-Type: application/json
Content-Length: ...

{...}

Then a Spring Cloud Gateway or Kong receives this as a normal HTTP request and can:

• Log
• Route
• Authenticate
• Forward to microservices

---

🔚 Summary

• ✅ API gateways like Spring Cloud Gateway, Kong, Zuul, and OpenResty only handle HTTP.
• ❌ They do not support raw TCP or binary packet processing, such as reading EBCDIC or length-prefixed binary data.
• 🛠️ You need an intermediary TCP service that converts raw streams into HTTP requests if you’re dealing with legacy systems or custom protocols.

---

✅ HTTP = Standard Gateway-Friendly Protocol

• Modern gateways like Spring Cloud Gateway, Netflix Zuul, Kong, OpenResty are designed to handle HTTP/HTTPS traffic only.

• HTTP includes:

  • A request line (e.g. GET /path HTTP/1.1)
  • Headers (e.g. Host, Authorization, X-Custom-Header)
  • A body (optional, usually JSON, form data, XML, etc.)

Because of this, HTTP is the universal entry format for almost all cloud-native microservice infrastructures.

---

❌ Raw TCP = Incompatible with Application-Layer Gateways

• Raw TCP packets (like those used in legacy systems, COBOL backends, mainframes, binary protocols) do not have HTTP structure:

  • No headers
  • No standard request line
  • Often have custom formats like length-prefix + binary body

• Therefore, HTTP gateways can’t understand or route them.

---

🔁 TCP-to-HTTP Conversion Pipeline

If your legacy client speaks TCP, and your target services are HTTP-based (behind a gateway), the traffic must go through a conversion layer.

🎯 You need a middle-layer that does:

[RAW TCP Packet] → [Wrap in HTTP format] → [HTTP Gateway] → [Destination Service]

or reverse:

[Modern HTTP Client] → [Gateway adds header] → [Custom Handler unwraps + sends raw TCP] → [Legacy System]

---

🧭 Two Common Scenarios

🔹 Scenario 1: Legacy system sends TCP

You want to send legacy packets through a modern API gateway.

You need:

• TCP server (bridge) to read the raw packet
• Add HTTP headers and body (as JSON or binary blob)
• Forward via HTTP to gateway
• Gateway routes to microservice

[Legacy TCP Client] 
     ↓
[TCP-to-HTTP Bridge Server]
     ↓ (HTTP POST)
[API Gateway (Spring Cloud Gateway, Kong, etc.)]
     ↓
[Modern HTTP Microservice]

🔹 Scenario 2: Modern service needs to call a legacy TCP backend

You want to access legacy TCP-based systems from modern HTTP services.

You need:

• A microservice or sidecar that:

  • Receives an HTTP request (via gateway)
  • Strips headers and parses JSON
  • Converts to a binary TCP format (e.g. EBCDIC, ISO8583)
  • Opens a socket to the legacy system

[Modern HTTP Client]
     ↓
[API Gateway]
     ↓
[HTTP-to-TCP Adapter Service]
     ↓
[Legacy Backend (TCP)]

---

✳️ Think of HTTP as a “Protocol Adapter Format”

HTTP isn’t just a web protocol. It’s become the standard envelope that:

• Lets services be routed
• Carries metadata in headers
• Enables observability (tracing, logging)
• Integrates with API management, firewalls, and security tools

But it’s just a wrapper. The real payload can still be:

• Raw bytes (Base64 or binary)
• Encoded legacy formats
• Anything your adapter logic knows how to parse

---

🔍 The term “protocol detection” is often misleading.

When API gateways (like Spring Cloud Gateway, Kong, Envoy, etc.) talk about “protocol detection”, they usually mean:

Detecting between different application-layer HTTP protocols, like:

• HTTP/1.1 vs HTTP/2
• gRPC (which runs over HTTP/2)
• WebSocket upgrade
• Possibly TLS sniffing (SNI) if used for routing

But…

---

❌ They do not mean:

• Detecting or handling raw non-HTTP binary protocols, like:

  • EBCDIC packets
  • ISO8583 (banking)
  • FIX (finance)
  • MQTT, Redis, Telnet
  • Custom socket protocols

---

✅ In practice, all traffic handled by these gateways must already:

• Start as a valid HTTP request

• Include all expected parts:

  • Method: GET, POST, etc.
  • Headers (especially Host, Content-Type)
  • Body (optional)

Any “detection” happens after the gateway has confirmed it’s dealing with HTTP.

---

🔁 Real “protocol recognition” (at a raw TCP level) only happens in:

• L4 proxies like:

  • Envoy (L4 sniffing mode)
  • NGINX stream module
  • HAProxy in TCP mode

• Custom TCP servers or sidecars you write

Even then, they must read bytes manually to:

• Identify “magic bytes” (e.g., 0x16 for TLS)
• Check headers (e.g., GET or PRI * HTTP/2.0)
• Do content-based routing

---

🔄 So, if you hear:

“Our gateway does automatic protocol detection”

You can mentally translate that to:

✅ “It auto-detects HTTP/1 vs HTTP/2 vs gRPC (via headers or ALPN)”
❌ “It does not understand your legacy TCP protocol unless you wrap it in HTTP”