As firewall engineers, we all have an intuitive sense for how complex a given firewall configuration is. We see it in the size of the ruleset. We see it in the errors that we know are there but don’t have time to track down. We see it in the amount of hair-pulling in involved in making sure a simple change is correct. (From my photo in the sidebar, you can tell I’ve seen my share of complex firewalls!) But what exactly is it about a given firewall configuration can make it so complex and difficult to manage?

Here at Athena Security, we felt there must be a way to identify the factors that really contribute to firewall complexity and quantify them.  We have accumulated a large database of firewall configurations from companies in a wide variety of industries, including financial services, health care, construction, human resources, manufacturing, IT services, and network security. These configurations were manually assessed for a firewall administrator’s “intuitive” sense of complexity and then analyzed using Athena FirePAC to identify configuration errors and policy risks. These results were subjected to intensive statistical analysis to find correlations between different aspects of the configurations that might contribute to complexity.

After evaluating over 100 firewall configurations, we found twelve factors that correlate strongly with our intuitive sense of complexity. These factors include obvious items like the number of ACLs and NATs in the ruleset.  Large rulesets are clearly more difficult to understand and maintain, and thus more complex.  But we also found that certain structural elements correlate strongly with complexity, including the number of discrete address elements (individual IP addresses, subnets, or address ranges), rules that have a wildcard match for source, destination, or service, and rules with a deny action.  These factors all cause complex interactions between the rules, magnifying the complexity of even small rulesets.

Based on the results of this study, we have developed a tool, called Firewall Grader, to measure these factors in Cisco, Checkpoint, or Juniper Netscreen firewall configurations.  It derives a complexity metric for the firewall based on the assessment of all twelve factors and generates a report of the findings.  We have found the metric useful in identifying those firewalls that are most difficult to manage or to audit.  Tracking the complexity metric over time can help you ensure that your firewall configurations don’t get out of control.   It will also recommend ways to restructure the configuration to reduce overall complexity.

Firewall Grader is available as a free download from our web site. Try it out!  Let us know what you think.

A study published in this month’s ISSA Journal presents an analysis of firewall management practices. The authors surveyed 260 firewall administrators and supervisors. The results show that corporate firewall rulebases are unmanageably large and getting bigger, the administrators responsible for them know they’re riddled with errors, and they can’t fix the problems because they lack adequate tools.

Based upon the results of this study, we draw three conclusions:

Firewall rulebase complexity greatly exceeds that discovered in prior research and that administrators feel this complexity is a major contributing factor to rulebase configuration errors.

Evidence exists that administrators make errors on a routine basis and most consider it likely that their rulebases contain undetected errors that expose their organization to risk.

In general, firewall administrators are not following recognized best practices for firewall administration on a regular basis.

[...]

Seven years prior to our study, Wool conducted an analysis of Checkpoint firewall rulebases and discovered an average rulebase size of 144. Our results showed a dramatically higher average of 793 rules per firewall. Our sample included seven rulebases containing a significantly higher number of rules than the maximum size discovered in Wool’s study (our maximum size was 17,000, while Wool’s was 2,671).

[...]

In addition to measuring the size of rulebases, we measured the rate of change in rulebases as an indicator of the stability of the ruleset. We used rulebase churn to evaluate the change rate (ψ) relative to the overall rulebase size…. The average churn rate of 9.9% indicates that the
average firewall administrator modifies one-tenth of his or her firewall rulebase on a monthly basis. This large degree of turnover in the system introduces a high likelihood of error.

Coincidentally, Secure Passage today published the results of a survey of 253 IT network, firewall and security executives from Fortune 1000 companies.

The survey revealed that poor firewall management practices lead to security gaps, compliance violations, substandard firewall performance, and premature device purchases. The survey also revealed that although organizations are experiencing more compliance and security challenges due to increasing firewall policy complexity, few know about solutions or have access to resources that can address these challenges.

Among the top administration problems identified by the firewall administrators were rules allowing more ports or IPs than required by best practices, redundant rules, failure to identify and clean up unneeded rules, and absence of any review of the rulesets. Using a firewall analysis tool like Athena FirePAC can solve many of these issues. Finding redundant rules or shadowed rules is a snap with its Rule Conflicts report and the Culprit Rules report will call out problematic rules failing to meet best practices. In the face of today’s mind-bogglingly complex rulesets, FirePAC really helps the firewall administrator understand what’s going on inside the firewall.