Repository navigation
Allow custom authentication (in particular NTLM) to proxies #1582
Description
Requests can do many kinds of authentication—basic, digest, OAuth1. Even better, the great API allows users to specify a custom authentication class
import requests
from requests_ntlm import HttpNtlmAuth
requests.get("http://ntlm_protected_site.com",auth=HttpNtlmAuth('domain\\username','password'))
However for proxies, the only authentication presently supported is Basic. The API doesn't allow you to specify other or custom classes. http://docs.python-requests.org/en/latest/user/advanced/#proxies
To use HTTP Basic Auth with your proxy, use the http://user:password@host/ syntax:
Please could you expand the API to allow custom authentication for proxies too?
This would be mega useful. NTLM proxies are common in Windows corporate networks. Python development can't get off the ground in my office because the package manager (and everything else) falls over at the proxy. Examples
Activity
hickford commentedon Sep 7, 2013
It's not obvious what the syntax should look like. This is the current API (string based)
Will want to preserve that, but also allow user to specify an HtmlAuth object if they want to.
How about, define a new Proxy class, which is permissible as a value in the
proxiesdictionaryhickford commentedon Sep 7, 2013
Does proxying happen in
requestsor does this need a fix upstream in https://github.com/shazow/urllib3 first?Edit: Urllib3 only does Basic auth. Asked there if there's any plan to support custom auth. I don't know if it would be possible since AuthBase is in Requests. urllib3/urllib3#242
Lukasa commentedon Sep 8, 2013
This is a good idea @matt-hickford. I see three ways of doing this:
Proxyclass in theProxiesdictionary. Definitely do-able, but adds quite a bit of complication into the API. Possible. +0I'm generally in favour of TAs. In particular, Transport Adapters are really the only thing that knows anything about HTTPS over proxy (the CONNECT verb), which needs to be differently handled from other kinds of messages. To that end, it seems more natural to provide proxy authentication solutions at the Transport Adapter level.
Potentially TAs could take pluggable auth modules, just like individual requests? /cc @sigmavirus24
sigmavirus24 commentedon Sep 8, 2013
I would think Transport Adapters would take Proxy Adapters which handle using proxies in much the same way Transport Adapters handle the protocols (even though by default we only handle two). That said, Proxy Adapters could contain all the logic for pluggable auth "modules". This would then factor out the Proxy logic from the Transport Adapters and make things far more testable as well and allow people to configure crazy proxies however they like.
Lukasa commentedon Sep 8, 2013
I don't think we want to be adding a further Transport Adapter layer. Best to just say 'a Transport Adapter knows what to do'.
Lukasa commentedon Sep 21, 2013
Until urllib3 has this as an option there's not much we can do, so I'll close this until that time.
hickford commentedon Mar 26, 2015
Cool I've offered a bounty on the upstream issue urllib3/urllib3#242
You could label this 'blocked' or something
[-]Allow custom authentication (eg. NTLM) to proxies [/-][+]Allow custom authentication (in particular NTLM) to proxies [/+]ryandebruyn commentedon Mar 10, 2016
Hello gents. I apologize to wake the sleeping giant. However Ive been trying to track down a viable workaround to request https: with NTLM proxy authentication and have come up short. @Lukasa et al I appreciate all the work to date. As of what is currently available it doesn't look like this is supported in urllib3, requests-ntlm or request. What does one do? Any suggestions?
Lukasa commentedon Mar 10, 2016
I believe requests-ntlm does have support for this in the code: have you tried it?
ryandebruyn commentedon Mar 10, 2016
Thanks for the reply. I have tried requests-ntlm if i have my creds correct this is what I get.
ConnectionError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /get (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 407 Proxy Authentication Required',)))
ryandebruyn commentedon Mar 11, 2016
@Lukasa. From what I can tell requests-ntlm will let you do NTLM auth with a proxy when using HTTP, but not with HTTPS. Am I missing something? Thanks again for your time.
Lukasa commentedon Mar 11, 2016
Ah, yes, @ryandebruyn that's correct. Unfortunately, httplib makes it very difficult to authenticate to proxies when setting up a TLS tunnel because for any non-200 response to the CONNECT it will throw an exception and lose the response data. We can in principle work around that but it's extremely difficult to do and potentially breaks quite a few behaviours.
Sadly, this is just something that is very, very difficult to do.