Skip to main content

Raymii.org Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Use Ubuntu behind a Microsoft ForeFront TMG proxy with cntlm

Published: 27-10-2018 | Author: Remy van Elst | Text only version of this article


Table of Contents


Recently I had to deploy a few machines in a network where outgoing networkaccess was forced through a Microsoft Forefront TMG proxy. For all the Windowsclients this went automatically due to domain policies, for Linux this has to beset up manually. Defining the proxy in /etc/environment was not enough sinceNTML authentication is required, which is not supported by default. I foundcntlm, a piece of software which acts as a local proxy, translating allrequests to authenticated NTLM requests to your upstream proxy. This guidecovers the (offline) installation, setup, getting the correct password hash andsystem-wide configuration. It should work on a desktop as well, but I did nottest that.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

This guide was tested on both Ubuntu 16.04 and 18.04. You need a user accountwith the correct permissions for the proxy. The account will be locked, so makesure you have access the the Active Directory to unlock it when needed.

Installing cntlm

If you can get a temporary exeption for the machine, you can use your favoritepackage manager to install cntlm:

apt-get install cntlm

If you have no network access, download the package from the ubuntu packagessite. The dpkg file has no dependencies other than libc.

Place it on the server and install it:

dpkg -i cntlm*.deb

Make sure the service is not started yet:

systemctl stop cntlm

Configuring cntlm

The configuration file lives in /etc/cntlm.conf and is very simple. You cansetup cntlm as a proxy for other servers, but that is not in the scope of thisguide. For me, I used Ansible to configure these few servers.

You can put the password as plaintext in the configuration file, but we are notgoing to do that since the software supports placing the ntlm hash directly.

First we use the commandline to figure out which type of hash is used. Use the-M (magic) parameter with a username and password to autodetect the correctsettings:

cntlm -u $USERNAME@$ADDOMAIN -M http://raymii.org proxy.$ADDOMAIN.EXT 8080Password: 

Example output:

Config profile  1/4... OK (HTTP code: 200)----------------------------[ Profile  0 ]------Auth            NTLMv2PassNTLMv2      AAAAAAABBBBBBBBBXXXXXXX99999AAAA------------------------------------------------

The username format is $USERNAME@$domain. The password is your domainaccount's password. The last two parameters are the proxy hostname/ip and port.

In this case we have the NTLMv2 hash and the output format. In other cases,there might be an NTLM hash, an NT hash, an LM hash or any combination.

In the configuration file, abide to the following rules:

If you cannot connect right away or need to generate the hash offline, cntlm cando that as well:

echo "P@ssw0rd" | cntlm -H -u $USERNAME -d $ADDOMAIN 

Output:

Password: PassLM          xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPassNT          yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyPassNTLMv2      AAAAAAABBBBBBBBBXXXXXXX99999AAAA    # Only for user '$USERNAME', domain '$ADDOMAIN'

Use your favorite editor to place these values in the configuration file.

vim /etc/cntlm.conf

The file is self-explanatory, but read through it if you want to setup agateway. Here is the config we need for the above setup:

Username    $USERNAMEDomain      $ADDOMAINPassNTLMv2  AAAAAAABBBBBBBBBXXXXXXX99999AAAAAuth        NTLMv2Workstation $SERVER_HOSTNAMEProxy       proxy.$ADDOMAIN.EXT:8080NoProxy     localhost, 127.0.0.*, 10.*, 192.168.*Listen      3128

WorkStation is optional, by default the system hostname is used. The othervalues like Proxy and Listen are self-explanatory as well.

When your configuration file is done, make sure only root can read it:

chmod 600 /etc/cntlm.conf

Start the service:

systemctl start cntlm

Testing cntlm

Use either cntlm itself with debug on, or a tool like curl with the proxyconfigured to test the local proxy.

Testing with curl

curl has the -x option to provide a proxy. Since cntlm is configured andlistening on 127.0.0.1:3128 we can use it to test with curl:

curl -v -x http://127.0.0.1:3128/ http://raymii.org

Example output:

* Rebuilt URL to: raymii.org/*   Trying 127.0.0.1...* Connected to 127.0.0.1 (127.0.0.1) port 3128 (#0)> GET http://raymii.org/ HTTP/1.1> Host: raymii.org> User-Agent: curl/7.47.0> Accept: */*> Proxy-Connection: Keep-Alive[...]< HTTP/1.1 200 OK< Via: 1.1 proxy< Connection: Keep-Alive< Proxy-Connection: Keep-Alive< Content-Length: 376[...]<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2//EN"><html><head><meta name="generator" content="newspeak.py"><title>Raymii.org</title><meta http-equiv="REFRESH" content="0; url=https://raymii.org/s/"></head><body>You should be redirected to <a href="https://raymii.org/s/">https://raymii.org/s/. If that is not the case, please click here to continue.</a></body>* Connection #0 to host 127.0.0.1 left intact

As you can see, the proxy is used and working. If it is not, the output willinclude something like below:

# lots of html 407 Proxy Authentication Required. Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. (12209).

Your password hash could be wrong, the account might be locked out or the otherauthentication credentials are wrong.

Testing with cntlm

Using the -v option, for verbose with the -M option as before, we can testthe connection.

cntlm -fv -c /etc/cntlm.conf -M http://raymii.org

Output:

section: global, Username = '$USERNAME'section: global, Domain = '$ADDOMAIN'section: global, PassNTLMv2 = 'AAAAAAABBBBBBBBBXXXXXXX99999AAAA'section: global, Auth = 'NTLMv2'section: global, Workstation = '$SERVER_HOSTNAME'section: global, Proxy = 'proxy.$ADDOMAIN.EXT:8080'section: global, NoProxy = 'localhost, 127.0.0.*, 10.*, 192.168.*'section: global, Listen = '3128'cntlm: Proxy listening on 127.0.0.1:3128Adding no-proxy for: 'localhost'Adding no-proxy for: '127.0.0.*'Adding no-proxy for: '10.*'Adding no-proxy for: '192.168.*'cntlm: Using proxy proxy.$ADDOMAIN.EXT:8080cntlm: Resolving proxy proxy.$ADDOMAIN.EXT...Config profile  1/4... Resolve proxy.$ADDOMAIN.EXT:  -> 192.0.2.10NTLM Request:       Domain: $ADDOMAIN     Hostname: $SERVER_HOSTNAME        Flags: 0xA208B205Sending PROXY auth request...Proxy-Connection               => keep-aliveHost                           => raymii.orgProxy-Authorization            => NTLM xxxxContent-Length                 => 0Reading PROXY auth response...HEAD: HTTP/1.1 407 Proxy Authentication Required ( Access is denied.  )Via                            => 1.1 proxyProxy-Authenticate             => NTLM xxxxConnection                     => Keep-AliveProxy-Connection               => Keep-AlivePragma                         => no-cacheCache-Control                  => no-cacheContent-Type                   => text/htmlContent-Length                 => 0NTLM Challenge:    Challenge: xxx (len: 208)        Flags: 0xA2898205    NT domain: $ADDOMAIN       Server: proxy       Domain: $ADDOMAIN.EXT         FQDN: proxy.$ADDOMAIN.EXT          TLD: $ADDOMAIN.EXT            7:         TBofs: 66        TBlen: 142        ttype: 0NTLMv2:        Nonce: xxxxxxxxxxxxxxxxxx    Timestamp: 131850205990000000NTLM Response:     Hostname: '$SERVER_HOSTNAME'       Domain: '$ADDOMAIN'     Username: '$USERNAME'     Response: 'xxxx' (190)     Response: 'xxxx' (24)HEAD: HTTP/1.1 200 OKOK (HTTP code: 200)----------------------------[ Profile  0 ]------Auth            NTLMv2PassNTLMv2      AAAAAAABBBBBBBBBXXXXXXX99999AAAA------------------------------------------------cntlm: Terminating with 0 active threads

The 200 OK is what we're looking for. Stuff like below is wrong, just asabove, check your credentials and config:

HEAD: HTTP/1.1 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )Credentials rejectedWrong credentials, invalid URL or proxy doesn't support NTLM nor BASIC.

The account will be locked, so make sure you have access the the ActiveDirectory to unlock it.

Systemwide proxy configuration

When the proxy is working, you make it available for the entire system. Mostsoftware will understand this, but make sure to check the specific manpages ifsoftware is not working for you.

Edit the following file:

vim /etc/environment

Append the following:

http_proxy="http://127.0.0.1:3128/"https_proxy="http://127.0.0.1:3128/"ftp_proxy="http://127.0.0.1:3128/"no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"HTTP_PROXY="http://127.0.0.1:3128/"HTTPS_PROXY="http://127.0.0.1:3128/"FTP_PROXY="http://127.0.0.1:3128/"NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"

Save it and logout. Log back in to make it active.

For apt-get, you need to edit the following file:

vim /etc/apt/apt.conf

Append the following:

Acquire::http::proxy "http://127.0.0.1:3128/";Acquire::ftp::proxy "ftp://127.0.0.1:3128/";Acquire::https::proxy "https://127.0.0.1:3128/";

After logging out and in, test it with curl once again, but now without the -xoption (so no proxy is specified, but the system proxy is used):

curl -v raymii.org

Output:

* Rebuilt URL to: raymii.org/*   Trying 127.0.0.1...* Connected to 127.0.0.1 (127.0.0.1) port 3128 (#0)> GET http://raymii.org/ HTTP/1.1> Host: raymii.org> User-Agent: curl/7.47.0> Accept: */*> Proxy-Connection: Keep-Alive> < HTTP/1.1 200 OK< Via: 1.1 proxy< Connection: Keep-Alive
Tags: cntlm, microsoft, ntlm, proxy, server, tutorials, ubuntu, windows