Uniform Resource Locators (URLs)

A Uniform Resource Identifier (URI) is a compact character sequence that identifies an abstract or physical resource. The generic syntax is standardized in RFC 3986 (January 2005, also published as Internet Standard STD 66). A Uniform Resource Locator (URL) is the subset of URIs that, in addition to identifying a resource, describe its primary access mechanism — i.e., its network location. A Uniform Resource Name (URN) is the subset required to remain globally unique and persistent even when the resource ceases to exist.

URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

scheme: // user:pass @ host:port        /some/path   ?a=multi&part=query #fragment
  http: // foo:P4s$  @ example.com:8080 /also/a/path ?id=1&task=2        #some_ref

Anatomy and parsing traps

• Scheme — ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ), case-insensitive, terminated by :. Determines which scheme-specific specification governs the rest of the identifier.
• Hierarchical vs. non-hierarchical — If the // is absent, the URI is non-hierarchical (e.g. mailto:user@example.com, urn:oasis:...), and interpretation of everything after scheme: is left to the handling client. In practice most clients follow hierarchical-URI conventions anyway.
• Authority — [ userinfo "@" ] host [ ":" port ]. The @ separates userinfo from host. This is a classic phishing/SSRF trap: http://example.com&foo=bar@167772161/ actually points to 10.0.0.1 (a decimal-format IPv4 address), with example.com&foo=bar interpreted as the username for authentication.
• Host — Per RFC 3986, must be an IP-literal (bracketed IPv6/IPvFuture), an IPv4 address in dotted-decimal, or a registered name. Most real-world parsing libraries also accept decimal (167772161), octal (012.0.0.1), hexadecimal (0xA000001), and mixed-format IP addresses — a discrepancy that matters for SSRF filter bypasses, since a blocklist matching only dotted-decimal can be evaded by encoding the same address differently.
• Path — A sequence of /-delimited segments. Dot-segments (., ..) are removed during reference resolution (RFC 3986 §5.2.4).
• Query — An arbitrary string delimited from the path by ?. Conventionally &-separated key/value pairs (application/x-www-form-urlencoded), but the RFC leaves interpretation entirely to the server.
• Fragment — An arbitrary string after #, interpreted by the client only. The fragment is not sent to the server in an HTTP request — a fact that matters when reasoning about what a server-side log can see.

Percent-encoding

Characters outside the unreserved set (A–Z a–z 0–9 - . _ ~) must be percent-encoded as % followed by two hex digits. Reserved characters (: / ? # [ ] @ ! $ & ' ( ) * + , ; =) act as delimiters and must be encoded when used as data within a component. Decoding percent-encoded octets that correspond to unreserved characters is safe; decoding reserved characters changes semantics.

Security relevance

• Parser differentials — Two components that parse the same URL differently (filter vs. fetcher, proxy vs. backend) enable SSRF and cache-poisoning attacks. Alternate IP encodings and embedded credentials in the authority component are the canonical vectors.
• Credential leakage — user:pass@host URLs end up in browser history, logs, and Referer headers; RFC 3986 deprecates the password form for this reason.
• Open redirects and phishing — Visually plausible authority components (http://trusted.com@evil.example/) exploit the @ separator.

Related attack surfaces: xss-attacks (javascript: pseudo-scheme payloads), and any page dealing with request smuggling or proxying such as chisel.

Sources

• ietf-rfc3986-uri-syntax
• Uniform Resource Identifier - Wikipedia

See also

• xss-attacks
• chisel