Use Ubuntu behind a Microsoft ForeFront TMG proxy with cntlm
Published: 27-10-2018 | Author: Remy van Elst | Text only version of this article
❗ This post is over three years old. It may no longer be up to date. Opinions may have changed.
Table of Contents
Recently I had to deploy a few machines in a network where outgoing network
access was forced through a Microsoft Forefront TMG proxy. For all the Windows
clients this went automatically due to domain policies, for Linux this has to be
set up manually. Defining the proxy in
/etc/environment was not enough since
NTML authentication is required, which is not supported by default. I found
cntlm, a piece of software which acts as a local proxy, translating all
requests to authenticated NTLM requests to your upstream proxy. This guide
covers the (offline) installation, setup, getting the correct password hash and
system-wide configuration. It should work on a desktop as well, but I did not
I'm developing a desktop monitoring app, Leaf Node Monitoring, open source, but paid. For Windows, Linux & Android, go check it out.
Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.
You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days.
This guide was tested on both Ubuntu 16.04 and 18.04. You need a user account with the correct permissions for the proxy. The account will be locked, so make sure you have access the the Active Directory to unlock it when needed.
If you can get a temporary exeption for the machine, you can use your favorite package manager to install cntlm:
apt-get install cntlm
If you have no network access, download the package from the ubuntu packages
dpkg file has no dependencies other than
Place it on the server and install it:
dpkg -i cntlm*.deb
Make sure the service is not started yet:
systemctl stop cntlm
The configuration file lives in
/etc/cntlm.conf and is very simple. You can
setup cntlm as a proxy for other servers, but that is not in the scope of this
guide. For me, I used Ansible to configure these few servers.
You can put the password as plaintext in the configuration file, but we are not going to do that since the software supports placing the ntlm hash directly.
First we use the commandline to figure out which type of hash is used. Use the
-M (magic) parameter with a username and password to autodetect the correct
cntlm -u $USERNAME@$ADDOMAIN -M http://raymii.org proxy.$ADDOMAIN.EXT 8080 Password:
Config profile 1/4... OK (HTTP code: 200) ----------------------------[ Profile 0 ]------ Auth NTLMv2 PassNTLMv2 AAAAAAABBBBBBBBBXXXXXXX99999AAAA ------------------------------------------------
The username format is
$USERNAME@$domain. The password is your domain
account's password. The last two parameters are the proxy hostname/ip and port.
In this case we have the NTLMv2 hash and the output format. In other cases, there might be an NTLM hash, an NT hash, an LM hash or any combination.
In the configuration file, abide to the following rules:
- Auth is NT: use only PassNT
- Auth is LM: use only PassLM
- Auth is NTLM: use both PassLM and PassNT
- Auth is NTLMv2: use only PassNTLMv2
If you cannot connect right away or need to generate the hash offline, cntlm can do that as well:
echo "P@ssw0rd" | cntlm -H -u $USERNAME -d $ADDOMAIN
Password: PassLM xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx PassNT yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy PassNTLMv2 AAAAAAABBBBBBBBBXXXXXXX99999AAAA # Only for user '$USERNAME', domain '$ADDOMAIN'
Use your favorite editor to place these values in the configuration file.
The file is self-explanatory, but read through it if you want to setup a gateway. Here is the config we need for the above setup:
Username $USERNAME Domain $ADDOMAIN PassNTLMv2 AAAAAAABBBBBBBBBXXXXXXX99999AAAA Auth NTLMv2 Workstation $SERVER_HOSTNAME Proxy proxy.$ADDOMAIN.EXT:8080 NoProxy localhost, 127.0.0.*, 10.*, 192.168.* Listen 3128
WorkStation is optional, by default the system hostname is used. The other
Listen are self-explanatory as well.
When your configuration file is done, make sure only root can read it:
chmod 600 /etc/cntlm.conf
Start the service:
systemctl start cntlm
Use either cntlm itself with debug on, or a tool like curl with the proxy configured to test the local proxy.
Testing with curl
curl has the
-x option to provide a proxy. Since
cntlm is configured and
127.0.0.1:3128 we can use it to test with curl:
curl -v -x http://127.0.0.1:3128/ http://raymii.org
* Rebuilt URL to: raymii.org/ * Trying 127.0.0.1... * Connected to 127.0.0.1 (127.0.0.1) port 3128 (#0) > GET http://raymii.org/ HTTP/1.1 > Host: raymii.org > User-Agent: curl/7.47.0 > Accept: */* > Proxy-Connection: Keep-Alive [...] < HTTP/1.1 200 OK < Via: 1.1 proxy < Connection: Keep-Alive < Proxy-Connection: Keep-Alive < Content-Length: 376 [...] <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2//EN"> <html> <head> <meta name="generator" content="newspeak.py"> <title>Raymii.org</title> <meta http-equiv="REFRESH" content="0; url=https://raymii.org/s/"> </head> <body> You should be redirected to <a href="https://raymii.org/s/">https://raymii.org/s/. If that is not the case, please click here to continue.</a> </body> * Connection #0 to host 127.0.0.1 left intact
As you can see, the proxy is used and working. If it is not, the output will include something like below:
# lots of html 407 Proxy Authentication Required. Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. (12209).
Your password hash could be wrong, the account might be locked out or the other authentication credentials are wrong.
Testing with cntlm
-v option, for verbose with the
-M option as before, we can test
cntlm -fv -c /etc/cntlm.conf -M http://raymii.org
section: global, Username = '$USERNAME' section: global, Domain = '$ADDOMAIN' section: global, PassNTLMv2 = 'AAAAAAABBBBBBBBBXXXXXXX99999AAAA' section: global, Auth = 'NTLMv2' section: global, Workstation = '$SERVER_HOSTNAME' section: global, Proxy = 'proxy.$ADDOMAIN.EXT:8080' section: global, NoProxy = 'localhost, 127.0.0.*, 10.*, 192.168.*' section: global, Listen = '3128' cntlm: Proxy listening on 127.0.0.1:3128 Adding no-proxy for: 'localhost' Adding no-proxy for: '127.0.0.*' Adding no-proxy for: '10.*' Adding no-proxy for: '192.168.*' cntlm: Using proxy proxy.$ADDOMAIN.EXT:8080 cntlm: Resolving proxy proxy.$ADDOMAIN.EXT... Config profile 1/4... Resolve proxy.$ADDOMAIN.EXT: -> 192.0.2.10 NTLM Request: Domain: $ADDOMAIN Hostname: $SERVER_HOSTNAME Flags: 0xA208B205 Sending PROXY auth request... Proxy-Connection => keep-alive Host => raymii.org Proxy-Authorization => NTLM xxxx Content-Length => 0 Reading PROXY auth response... HEAD: HTTP/1.1 407 Proxy Authentication Required ( Access is denied. ) Via => 1.1 proxy Proxy-Authenticate => NTLM xxxx Connection => Keep-Alive Proxy-Connection => Keep-Alive Pragma => no-cache Cache-Control => no-cache Content-Type => text/html Content-Length => 0 NTLM Challenge: Challenge: xxx (len: 208) Flags: 0xA2898205 NT domain: $ADDOMAIN Server: proxy Domain: $ADDOMAIN.EXT FQDN: proxy.$ADDOMAIN.EXT TLD: $ADDOMAIN.EXT 7: TBofs: 66 TBlen: 142 ttype: 0 NTLMv2: Nonce: xxxxxxxxxxxxxxxxxx Timestamp: 131850205990000000 NTLM Response: Hostname: '$SERVER_HOSTNAME' Domain: '$ADDOMAIN' Username: '$USERNAME' Response: 'xxxx' (190) Response: 'xxxx' (24) HEAD: HTTP/1.1 200 OK OK (HTTP code: 200) ----------------------------[ Profile 0 ]------ Auth NTLMv2 PassNTLMv2 AAAAAAABBBBBBBBBXXXXXXX99999AAAA ------------------------------------------------ cntlm: Terminating with 0 active threads
200 OK is what we're looking for. Stuff like below is wrong, just as
above, check your credentials and config:
HEAD: HTTP/1.1 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) Credentials rejected Wrong credentials, invalid URL or proxy doesn't support NTLM nor BASIC.
The account will be locked, so make sure you have access the the Active Directory to unlock it.
Systemwide proxy configuration
When the proxy is working, you make it available for the entire system. Most software will understand this, but make sure to check the specific manpages if software is not working for you.
Edit the following file:
Append the following:
http_proxy="http://127.0.0.1:3128/" https_proxy="http://127.0.0.1:3128/" ftp_proxy="http://127.0.0.1:3128/" no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com" HTTP_PROXY="http://127.0.0.1:3128/" HTTPS_PROXY="http://127.0.0.1:3128/" FTP_PROXY="http://127.0.0.1:3128/" NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"
Save it and logout. Log back in to make it active.
apt-get, you need to edit the following file:
Append the following:
Acquire::http::proxy "http://127.0.0.1:3128/"; Acquire::ftp::proxy "ftp://127.0.0.1:3128/"; Acquire::https::proxy "https://127.0.0.1:3128/";
After logging out and in, test it with curl once again, but now without the
option (so no proxy is specified, but the system proxy is used):
curl -v raymii.org
Tags: cntlm , microsoft , ntlm , proxy , server , tutorials , ubuntu , windows
* Rebuilt URL to: raymii.org/ * Trying 127.0.0.1... * Connected to 127.0.0.1 (127.0.0.1) port 3128 (#0) > GET http://raymii.org/ HTTP/1.1 > Host: raymii.org > User-Agent: curl/7.47.0 > Accept: */* > Proxy-Connection: Keep-Alive > < HTTP/1.1 200 OK < Via: 1.1 proxy < Connection: Keep-Alive