Republished with permission from WatchGuard Technologies, Inc.


Basic and Advanced HTTP Proxy Uses

By David M. Piscitello, President, Core Competence, and ICANN Fellow

Application proxies are the most seasoned veterans of network security services. Despite this venerable pedigree, application proxies have been generally underutilized. Sadly, proxies suffer from over-hyped and under-informed claims that they perform badly in comparison to packet filtering. These claims often scare security novices away from using proxies. The good news is that in reality, packet filtering and application proxies are complementary solutions.

In addition to being underutilized, application proxies, especially HTTP proxies, are not well understood. In this article, we'll go back to basics, examine what proxies are, what they do, and how you can use your firewall's HTTP Proxy to add countermeasures for a broad range of threats.

What is an Application Proxy?

An application proxy is a server that performs two security functions.

First, it blocks all traffic originating from and delivered to a protected network except for traffic related to a specific application.

Second, the proxy inspects messages of the permitted application to ensure that they conform to the configured security policy. Depending on the application, a proxy can be configured to allow or block request methods (e.g., block HTTP PUT, described in "HTTP Proxy Blocks the Madness in Methods"), query types (e.g., block DNS full zone transfers), and commands (e.g., FTP PASV). Proxies can also be used to allow, block, or strip message content and attachment types and to scan allowed types for viruses and spyware.

Finally, proxies can be used to normalize headers to hide topology and personal information. For example, an SMTP proxy can rewrite email addresses such as and so that all outbound email appears to originate from the domain

Applications that can be proxied effectively include HTTP, FTP, email, DNS, and certain VoIP protocols (e.g., Session Initiation Protocol, or SIP). In early application proxy firewall implementations, there were no direct network connections between the networks the proxy protects and "untrusted" networks. Today, however, security administrators can use application proxies to protect some applications and dynamic packet filtering to protect others. (Remember, dynamic packet filtering primarily inspects IP and TCP/UDP headers, while proxies examine application headers plus the packet's actual payload.)

Generally speaking, security administrators should consider the following factors when deciding whether to packet filter or proxy a particular application:

  • Performance. Does policy processing by the proxy introduce more delay and jitter than a real time application can tolerate?
  • Feature set. Does the packet filter provide the application content inspection, content blocking, and header field substitution your organization's security policy requires?
  • Compatibility with other security systems. Does the proxy support your authentication protocol?
  • "Proxy redundancy." Are you already proxying this service behind the firewall?

Because of the numerous ways that it is used and abused, HTTP is an excellent candidate for proxying. In most deployment scenarios, HTTP proxies measurably improve security without adversely impacting performance.

Basic HTTP Proxy Services

Used in the most basic manner, the Firebox X HTTP Proxy checks HTTP response and request messages to ensure that they conform to protocol standards and do not contain any extraneous, suspicious or malformed fields. For example, the default configuration of the HTTP Proxy only allows HEAD, GET, POST, OPTIONS, PUT and DELETE request methods as defined in RFC 2616. It provides simple browser anonymity for traffic leaving your network by stripping FROM: fields it encounters in HTTP requests (these contain the browser user's email address). The proxy also strips VIA: fields which can be used to trace chained HTTP requests back to their origin (i.e., when the firewall strips the Via: field, the origin appears to be the HTTP proxy.

The HTTP proxy provides several HTTP response checks as well. Allowed HTTP response field types are white listed by default. Any field not included in this list is stripped before the proxy forwards the response to the client. This table gives some examples of header fields the HTTP Proxy will strip (and their contents):

Header field







XRE response from Origin Server \x0d\x0a


HIT from\x0d\x0a


MISS from\x0d\x0a


X-INKT-URI: \x0d\x0a


response from IC \x0d\x0a












It's difficult to find documentation for many of these fields. My philosophy is that whether such fields are used for good or for evil, it's generally a good practice to block unknowns.

Advanced HTTP Proxy Services

Content inspection at the HTTP proxy is one of the most effective security measures you can implement. HTTP uses a Content-type response header field to identify the MIME-media types included in a message. The Firebox X HTTP Proxy begins with a white list of media types commonly encountered by Web users (text and HTML, image, PDF, Javascript, Shockwave flash, XML). All other types are blocked by default; more importantly, any media data that are not identified using Content-Type are stripped. Security proxies typically interpret RFCs strictly: if it's not specifically permitted, it's prohibited.

An HTTP Proxy can also provide an effective and uniform defense against spyware and other malicious code. Numerous file types are known to harbor spyware and viruses. By blocking ZIP and Windows cabinet archives, Windows and Java executables, and dynamic link libraries (DLLs), you greatly reduce the number of malware vectors into your trusted network. To block unwanted body contents, the HTTP proxy examines the initial or magic bytes in a file's header. For example, to block Windows executables (.EXE), the proxy opens a file, examines the header, and if the header begins with the hex pattern 0x4d5a90000300000004000000ffff0000, blocks the download.

You can add content-types to the HTTP Proxy's default black list. For example, certain viruses (e.g., Win32.Beagle) have used Windows Control Panel Applets to install and infect a PC. To protect your organization from repeat outbreaks of W32.Beagle and possible variants (you know they're coming!), you could add the body content type for Windows CPL files, which begin with 0x4D5A. Note that by adding this magic byte pattern, you will also block all these windows file types:


Windows/DOS executable file


Library cache file


Control panel application


Font file


Graphic filter file


ActiveX or OLE Custom Control


OLE object library


Screen saver


VisualBASIC application

VXD, 386

Windows virtual device

If you are trying to block potential malware, this is a pretty effective list to block with a single magic byte pattern.

If you want to learn more about which file types might contain malicious code, and are candidates for body content type blocking using magic bytes, read "Potentially Malicious Windows Files."

Implementing content inspection is beneficial in other ways as well. Your help desk staff will spend less time dealing with problems associated with unauthorized software installation, and your legal department may worry less about copyright infringements.

Beyond Proxy Stripping and Blocking Actions: IPS

Content stripping and blocking by file type can ebb the tide of malicious downloaders and autoinstalling software, but will not stem the tide entirely. Mobile users may infect their laptops when they venture beyond the protective umbrella of the HTTP proxy. When they return to the office and once again connect, they're already infected. Fortunately, for spyware to "spy," it must communicate the information it collects back to an ad server, identity thief, or eavesdropper.

If you complement HTTP proxy stripping and blocking with intrusion detection techniques, your proxy can observe the patterns or signatures of many ad- and spyware communications and block known spyware back-channel activities. By enabling Intrusion Prevention Services (IPS) in the HTTP Proxy, and selecting server signature sets, you can detect and block back channels of trackware, keyloggers, and ad spyware. [Editor's Note: This feature requires the purchase of a Fireware Pro option, called Gateway AntiVirus/Intrusion Prevention Service.] IPS definitions are updated constantly. To view the current installed IPS signatures, open the Firebox System Manager, select the Security Services tab and click on the Show Signatures button.

The Power is in the Proxy: Use it!

Nearly every organization is struggling with drive-by spyware downloads, viruses, unauthorized software installation and illegal use of copyrighted software. If you own a Firebox X Core or Peak and are not using the HTTP Proxy, you owe it to your organization to try it.

Begin by defining a small user group in the Firebox Internal Database. Create an HTTP proxy policy for this group, so your experimentation doesn't interfere with other users' Web activities. Initially, use only the defaults, and enable logging. Experiment with different log levels ( Setup => Logging => Logging Setup => Advanced Diagnostics) until you are comfortable with the level of detail you're logging. Allow your test group to surf for awhile, then review your logs and assess the results of your experiment. Share your results with management, human resources and legal counsel. Use the opportunity to update your organization's acceptable use policy, revise the HTTP proxy settings accordingly, and apply the proxy policy to all employees.

If your results are anything like mine, you won't regret the time you invested. ##

      Copyrightę 2006, WatchGuard Technologies, Inc. All rights reserved. WatchGuard, LiveSecurity, Firebox and ServerLock are trademarks or registered trademarks of WatchGuard Technologies, Inc. in the United States and other countries.