Five UTMs Attacked

Wednesday, September 05, 2007

We have tested a lot of enterprise network equipment in our Labs. From Servers to NAS boxes to Workgroup printers and so on. However, testing a UTM device is amongst one of the most time consuming tasks. The reason for this is very simple.

In just one device or box we have to test at least five or ten different components. So testing a single UTM device will include testing an anti-virus, spam filter, content filter, firewall, IDS/IPS, DMZ, etc.

For our shootout we created a test bed for testing four of the most important components-anti-virus, anti-spam, IDP and firewall. In the following sections we unfold our experiences with these UTMs and describe in detail how we went about testing them. You can use a similar setup to test not only UTMs but also other security devices. However, a word of caution! These tests are potentially damaging for your network as they involve attack these devices with real life viruses, spam and hacking tools. So, never carry them out on a production network.

Anti-virus tests
Testing for anti-virus capability is the easiest amongst all tests. We simply need to create a Web, FTP and SMB server, and a set of different types of viruses on top of it.

We used a Linux machine to host these viruses so that the hosting machine itself doesn't get affected by them. The viruses that we used had old 16-bit viruses to the latest Trojans and malware. We used a set of viruses with around 1000 virus files. This set was kept constant for all UTM devices.
Once the host machine was ready with all viruses hosted on top of it, we connected it to the public port of the UTM devices one after the other and tried downloading all viruses from the private network. Once done, we counted the number of viruses which bypassed the UTM and got downloaded on the public network.

Anti-spam tests
These tests are pretty much similar to the anti-virus tests, but more categorized. We setup a machine with a POP3 Mail server running on it and dumped around 2000 different spam mails on it. However, before dumping the spam, we categorized it into text, image and PDF. Then we connected the machine to the Internet and gave it a public IP address which is mapped with the MX record of a domain. We took the UTM devices one by one and connected their WAN port to the Internet.

We then connected a few machines to its private network and started downloading the spam using Outlook Express. Once done we checked how many spam the devices had missed; to either tag or block, and counted the number for all devices. Again, to compare the performance of all devices we kept the set of spam identical for all devices.

Page(s)   1  2