Escape and Encode Strings with jq
jq is a JSON processor, but its @format strings make it a general-purpose string escaper: URL-encoding, Base64, CSV/TSV, HTML, and POSIX shell quoting are all built in. On any box with jq installed, printf %s "$STRING" | jq -sRr @uri percent-encodes a string with no other tooling required.
URL-encode a string
printf %s "$STRING" | jq -sRr @uriEach flag does one thing: -R reads input as raw text instead of parsing it as JSON, -s slurps the whole input into a single string, @uri applies percent-encoding, and -r prints the result without JSON quotes. Everything outside the RFC 3986 unreserved set (A-Za-z0-9-._~) becomes a %XX sequence.
Watch the trailing newline. echo appends one, and with -s it is part of the string being encoded — echo 'a b' | jq -sRr @uri yields a%20b%0A, not a%20b. Use printf %s (no newline), or drop -s and let jq process input line by line.
The format strings
| Format | Effect |
|---|---|
@text | tostring — render values as plain strings |
@json | serialize the input as JSON |
@html | escape <>&'" as XML entities |
@uri | percent-encode (everything outside the unreserved set) |
@urid | percent-decode — the inverse of @uri (jq 1.8+) |
@csv, @tsv | render an array as CSV or TSV |
@sh | escape for use in a POSIX shell command line |
@base64, @base64d | Base64 encode/decode per RFC 4648 |
Base64 in the shell without leaving jq:
printf %s "$STRING" | jq -sRr @base64 # encode
printf %s "$B64" | jq -Rr @base64d # decodeEscape only the interpolated parts
A format filter can take a string literal. Literal text passes through unescaped; only \(...) interpolations get escaped. This builds URLs where the structure stays readable and only the values are encoded:
echo '{"q":"what is jq?"}' | jq -r '@uri "https://example.com/search?q=\(.q)"'
# https://example.com/search?q=what%20is%20jq%3FShell quoting with @sh
@sh produces POSIX-shell-safe quoting — handy for turning untrusted strings into command arguments:
printf %s "O'Hara Ale" | jq -sRr '@sh "echo \(.)"'
# echo 'O'\''Hara Ale'Passing shell variables into jq
Don’t build JSON by string concatenation — hand the values over with --arg (string) or --argjson (parsed JSON) and let jq handle the escaping:
jq -n --arg user "$USER" '{user: $user}'Decoding URLs
Before jq 1.8 there is no inverse for @uri; on older jq, decode with Python’s urllib.parse.unquote, or use the bulk recipes in url-encode-file-python.
Related
- curl-jq-web-apis — curl + jq as a web-API toolkit
- url-encode-file-python — percent-encoding whole wordlists with Python
- uniform-resource-locators — the reserved/unreserved characters behind percent-encoding
- openssl-base64-encoding — Base64 via openssl instead of jq