Distinguish Missing and Empty URL Components in Python 3.15

Distinguish Missing and Empty URL Components in Python 3.15

A URL that ends with ? is not the same as a URL without a query string. The first one supplies an empty query, the second supplies none at all. Servers, caches, and request signatures can treat the two differently, so a program that parses a URL and builds it again has to keep them apart.

The following code splits two such URLs and puts each one back together:

from urllib.parse import urlsplit, urlunsplit

urls = ['https://example.com/search', 'https://example.com/search?']

for url in urls:
    parts = urlsplit(url)
    print(repr(parts.query), urlunsplit(parts))

Output:

'' https://example.com/search
'' https://example.com/search

The query is printed with repr so that an empty string stays visible in the output. Both URLs give query='', which means the trailing ? left no trace, and urlunsplit drops it when rebuilding. The second URL produces a different URL than the one originally provided.

Since Python 3.15, urlsplit accepts the missing_as_none argument. When it is enabled, an absent component is reported as None and a component that is present but empty stays an empty string. The keep_empty argument of urlunsplit then writes those empty components back into the URL.

from urllib.parse import urlsplit, urlunsplit

urls = ['https://example.com/search', 'https://example.com/search?']

for url in urls:
    parts = urlsplit(url, missing_as_none=True)
    print(repr(parts.query), urlunsplit(parts, keep_empty=True))

Output:

None https://example.com/search
'' https://example.com/search?

The first URL has no query string, so parts.query is None. The second has an empty one, so parts.query is an empty string. Testing parts.query is None now answers whether the query string was supplied, and each URL is rebuilt exactly as it was written.

The two arguments work together. Passing keep_empty a result that came from a plain urlsplit raises ValueError, because that result cannot tell which components were absent.

The missing_as_none argument also applies to the scheme, netloc, and fragment components, and it is available in urlparse and urldefrag as well.

Leave a Comment

Cancel reply

Your email address will not be published.