Force python mechanize/urllib2 to only use A requests?

Question

Here is a related question but I could not figure out how to apply the answer to mechanize/urllib2: how to force python httplib library to use only A requests

Basica

Answer 1

No answer, but a few datapoints. The DNS resolution appears to be originating from httplib.py in HTTPConnection.connect() (line 670 on my python 2.5.4 stdlib)

The code flow is roughly:

for res in socket.getaddrinfo(self.host, self.port, 0, socket.SOCK_STREAM):
    af, socktype, proto, canonname, sa = res
    self.sock = socket.socket(af, socktype, proto)
    try:
        self.sock.connect(sa)
    except socket.error, msg: 
        continue
    break

A few comments on what's going on:

• the third argument to socket.getaddrinfo() limits the socket families -- i.e., IPv4 vs. IPv6. Passing zero returns all families. Zero is hardcoded into the stdlib.

• passing a hostname into getaddrinfo() will cause name resolution -- on my OS X box with IPv6 enabled, both A and AAAA records go out, both answers come right back and both are returned.

• the rest of the connect loop tries each returned address until one succeeds

For example:

>>> socket.getaddrinfo("python.org", 80, 0, socket.SOCK_STREAM)
[
 (30, 1, 6, '', ('2001:888:2000:d::a2', 80, 0, 0)), 
 ( 2, 1, 6, '', ('82.94.164.162', 80))
]
>>> help(socket.getaddrinfo)
getaddrinfo(...)
    getaddrinfo(host, port [, family, socktype, proto, flags])
        -> list of (family, socktype, proto, canonname, sockaddr)

Some guesses:

• Since the socket family in getaddrinfo() is hardcoded to zero, you won't be able to override the A vs. AAAA records through some supported API interface in urllib. Unless mechanize does their own name resolution for some other reason, mechanize can't either. From the construct of the connect loop, this is By Design.

• python's socket module is a thin wrapper around the POSIX socket APIs; I expect they're resolving every family available & configured on the system. Double-check Gentoo's IPv6 configuration.