Image may be NSFW.
Clik here to view.

Useless Services

Like a vestigial tail, there are often applications running on our machines that no longer serve any useful purpose. These services may be part of an earlier set of libraries that the programmers built on and never bothered to take out. This is one of the downsides of ever-increasing processing power and memory capacity. Programmers used to carefully ration every byte they used and would never allow unnecessary lines in their code. However, in this age of bloatware and gigabyte-sized operating systems, it is often easier to leave legacy services in rather than risk breaking some other program that depends on them. The incredible thing is that these services are often turned on by default. Here is a list of some of those services:

Useless Services in Linux

If you are running Windows, there are other services you will probably want to disable. Here is a partial listing of those useless services:

Useless Windows in Windows

If you disable at least some of these services, your system should be harder for hackers and bots to attack, and your system will be more secure.

External Links:

Remove useless services/apps at linuxquestions.org

Useless services in CentOS VDS/VPS at nixcraft.com

Turn off Unnecessary Windows Services at www.marksanborn.net

Disable Unneeded Services in Windows at www.winhelp.us

The post Useless Services appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Introducing the Nessus Vulnerability Scanner

Modern computer networks have multiple potential areas of insecurity. How do you protect all these avenues of attack? You might feel that protecting your network is an impossible situation. You could spend all day, every day, just checking for these security holes manually. Even if you tried to automate it with scripts, would would seem to take dozens of programs. Fortunately, there are packages out there called vulnerability scanners that will automatically check all these areas and more.

Nessus is an excellent program. It is a great example of how well open source projects can work, although the project has been since been changed to a proprietary (closed source) license. [The Nessus 3 engine is still free of charge, though Tenable Network Security, the company founded by Nessus creator Renaud Deraison, charges $100/month per scanner for the ability to perform configuration audits for PCI, CIS, FDCC and other configuration standards, technical support, SCADA vulnerability audits, the latest network checks and patch audits, the ability to audit anti-virus configurations and the ability for Nessus to perform sensitive data searches to look for credit card, social security number and many other types of corporate data.] It is robust, well documented, well maintained, and the premiere vulnerability scanner. Nessus has consistently rated at the top of all vulnerability scanners, commercial or noncommercial. This is amazing when you consider its competitors cost thousands of dollars and are created by large companies. It continues to impress and improve, and most importantly, to protect thousands of companies’ networks. There are some design features that make Nessus unique and superior to other vulnerability scanners.

Even as Nessus 3 and subsequent versions went closed source, the Nessus 2 engine and a minority of the plugins are still GPL, leading to forked open source projects based on Nessus. Tenable Network Security has still maintained the Nessus 2 engine and has updated it several times since the release of Nessus 3. The current stable version of Nessus is 5.2.1 (released May 7, 2013).

Nessus currently offers over 2000 individual vulnerability tests that cover practically every area of potential weakness in systems. Very few scanners out there can compete with this level of testing, and new tests are being added daily by a worldwide network of developers. The speed of release of new tests for emerging vulnerabilities is usually measured in days if not hours. Its plug-in based architecture allows new tests to be added easily.

You can turn off whole categories of tests if they do not apply or if you are worried they could be dangerous to your systems, or you can deactivate individual tests if you have it concern about a specific one. For example, you may prefer to disable the untested category, which contains tests that haven’t been fully tested yet.

Nessus uses a client-server architecture to run its security checks. The server runs the tests and the client configures and controls the sessions. The fact that the client and server can be separated offers some unique advantages. This means that you can have your scanning server outside your network, yet access it from inside your network via the client. this also allows other operating systems to be supported via different clients. There are currently UNIX and Window clients available, with projects to create additional ones ongoing. These is also now a web client interface available, which makes Nessus truly platform independent (at least on the client end).

In the next article, we will continue our discussion of Nessus and its features.

External Links:

Nessus home page on www.tenable.com

Nessus on Wikipedia

The post Nessus Vulnerability Scanner: An Introduction appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.In the previous article, we introduced the Nessus vulnerability scanner. In this article, we will discuss some of the additional Nessus features.

Nessus Features: Scripting Language, Integration with Other Tools, Smart Testing

To supplement the plug-in architecture, Nessus has its own scripting language called Nessus Attack Scripting Language (NASL), one of the more important Nessus features. This easy-to-learn utility language allows you to quickly and easily write your own custom security plug-ins without having to know C or all of the internal workings of the main program.

Nessus can be used by itself or with several other open source security tools. You can use Nmap, the best port scanner in the world, or the port scanning part of the job, rather than the built-in one. the Nesseus port scanner is faster and a little more efficient with memory, but Nmap allows for a lot more options and settings. Almost all the Nmap settings are configurable from within the Nessus client. Nessus also works with Nikto and Whisker, tools that run more complex tests on web servers; CGI programs; and Hydra, a tool for running brute-force password attacks against common services. The functionality of these tools is written right into Nessus, so you can make configuration changes from a single interface.

Another ability which you may find is one of the more useful Nessus features is that it can be set up so that it does not automatically run all of the vulnerability tests on every host. Based on the results of a port’s scan or other input such as past vulnerability tests, Nessus will run only tests appropriate to that machine. So if the server is not running a web server, it won’t run web server-related tests. Nessus is also smart in that it does not automatically assume that web servers will run on port 80, but rather checks all the possible ports for signs of a web server. Nessus will even find multiple instances of services running on different ports. This is especially important if you are inadvertently running a web server or other public service on an unusual port.

Nessus Features: Knowledge Base and Reporting

Another one of the more useful Nessus features is that it can save all scan results in a database called the Knowledge Base. This allows it to use the results of past scans to intelligently figure out what tests to run. You can use this to avoid doing a port scan every time you run Nessus, because it will remember what ports it found open last time on each host and test only those. It can also remember what hosts it saw last time and test only new hosts. You probably shouldn’t do this every time, because you may miss new ports that open up on machines or new vulnerabilities that show up on previously scanned boxes. However, it can allow you to run scans more often with less bandwidth and processor power as long as you do a complete scan on a regular basis.

Nessus has some of the best reporting capabilities in the open source field. Although it is not perfect, it can output your scan data in just about any format. Basic HTML and HTML with pie charts and graphs are two of the more popular formats. These reports include summary data and are suitable for posting to an internal web site with little or no editing. Other report formats supported include XML, LaTeX, and plain text. The Windows client offers additional report formats. There are additional tools available that allow you to do further manipulation of the data.

Mailing Lists

Nessus has an extensive support network for getting help, both on basic installation and use as well as more complex programming and customization. there are no fewer than five Nessus mailing lists, each dedicated to a different area. There is an archive of all the past posts so you can check to see if your question has ever been answered. The following are the main Nessus mailing lists:

• nessus: A general discussion list about Nessus.
• nessus-devel: Talks about the development of the upcoming versions.
• nessus-cvs: Shows the CVS commits made on the Nessus tree.
• nessus-announce: A low-traffic moderated list that is dedicated to the announcements of the availability of new releases.
• plug-ins-writers: A list dedicated to the writing of new Nessus plug-ins. If you want to write your own security checks, you should subscribe to it.

Of the discussion lists, nessus is the most active, broadest in scope, and probably most useful to the average reader. Much of the traffic consists, of questions and answers rather than actual discussions, and topics include Nessus and NessusWX, the plugins, vulnerabilities themselves, third-party add-ons, and so forth. Nessus-devel tends to be more discussion oriented, with the focus on revisions to Nessus and the NASL language. Plugins-writers leans more toward questions and answers, generally how to accomplish something in NASL or whether plugins are properly testing for vulnerabilties. None of these lists has an actual charter, though, and in practice there’s a fair amount of overlap among them.

To subscribe to any of the above lists, send an e-mail to majordomo@list.nessus.org with the following text in the body of the e-mail:

Subscribe listname

Replace listname with the name of the list to which you want to subscribe. To unsubscribe, do the name but write Unsubscribe listname in the body.

Nessus has quite a bit of documentation on its web site, including detailed instructions on installation, basic operation and use of Nessus features, and tutorials on how to write your own security checks in NASL.

Now the we have covered some of the more important Nessus features, in the next article, we will cover installing Nessus in Linux.

External Links:

Nessus home page on www.tenable.com – Additional information on Nessus features can be found here

Nessus on Wikipedia

The post Nessus Features and Capabilties appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Installing Nessus using the Debian package manager in Mint Linux.

Nessus Installation and Setup

In the previous article, we discussed some of the features and capabilities of Nessus 5. Here we will discuss downloading and installing the program.

There are two prerequisites you must have before you begin Nessus installation, and two others that are nice to have installed beforehand to take full advantage of the add-on capabilities.

1. The two prerequisites are the Gimp Tool Kit (GTK) and libpcap. If you installed Nmap, you should already have these programs installed. If not you can download GTK from:

   ftp.gimp.org/pub/gtk/3.12/gtk+-3.12.0.tar.xz

   and libpcap from:

   http://www.tcpdump.org/release/libpcap-1.5.3.tar.gz

2. The two programs that are optional for Nessus installation but recommended are OpenSSL and Nmap. Nessus can use Nmap as its port scanner and OpenSSL for secure communications between the server and client.

Image may be NSFW.
Clik here to view.

Entering the registration code during the Nessus installation process.

It is fairly easy to download and install Nessus from Tenable’s official site at http://www.tenable.com/products/nessus/select-your-operating-system. Nessus is available for several operating systems, including:

• Windows (XP, 2003 Server, Vista, 7 and 8)
• Mac OS X
• Linux
• FreeBSD
• Solaris

In Mint Linux, I began Nessus installation by downloading Nessus 5.2.6 from the Tenable web site (I downloaded the version for Ubuntu 9.10). When you download Nessus, you will be asked to register by providing your name and an e-mail address. Once I downloaded Nessus, I chdir-ed to the downloads directory, and at the command line, I typed:

sudo dkpg -i Nessus-5.2.6-ubuntu910_i386.deb

Within moments, the Debian package installer had unpacked and installed Nessus. Once it was done, I typed

sudo /etc/init.d/nessusd start

to start Nessus. Now the Nessus installation process can be completed by accessing the web interface via port 8834 on a web browser (using the HTTPS protocol). The first screen you’ll see upon accessing the web interface for the first time is the “Welcome to Nessus 5″ screen. Click on the “Get Started” button to continue.

Next is the “Initial Account Setup” screen, where you will be asked to created an admin user and password. Fill in the relevant fields and click on the “Next” button.

The next screen is “Plugin Feed Registration”. Here, you need to enter the activation code that was e-mailed to you when you first registered. There is also a section at the bottom for “Optional Proxy Settings”. Here you can enter the proxy hostname, proxy username, and proxy password if you want to configure a proxy. Enter the activation code, configure the proxy settings (if desired), and click on the “Next” button to register your scanner.

After registration, Nessus must download the plugins from Tenable. This process may take several minutes. The plugin setup process involves transferring a considerable amount of data to the machine, verifying file integrity, and compiling them into an internal database. Once the plugins have been downloaded and compiled, the Nessus GUI will initialize and the Nessus server will start. Nessus installation is complete and you are now ready to use Nessus.

In the next article, we will go through the process of using Nessus to secure your network.

External Links:

<a href=”http://www.tenable.com/products/nessus”>Nessus home page on www.tenable.com</a>

<a href=”http://en.wikipedia.org/wiki/Nessus_(software)”>Nessus on Wikipedia</a>

The post Nessus Installation: A Guide appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Advanced settings in the Nessus web GUI.

Nessus Configuration: Proxy and Advanced Settings

The first thing you will see when you access Nessus is the login page. You must first enter the login name and password you created when you set up Nessus. Since Nessus uses a web-based front end, you can access the Nessus server from any computer on the local network that has a web browser, making Nessus configuration much easier. This increases the scalability of Nessus for larger organizations. You can configure scan options and make other changes, all from the web interface.

Once you are logged in, there are several settings you can change. If you click on your username (in this example, “nessusadmin”) on the right side of the page, there is a drop-down menu with several options; select “Settings”. On the left sidebar, there is an option called “Proxy”. If you did not set up a proxy during the setup process and need to do so now (for example, if your organization requires that all web traffic be directed through a corporate proxy), you can input the settings here.

Proxy Setting Options

Different Nessus configuration options can be set by clicking on the “Advanced” link in the Settings menu. Each option can be configured by editing the corresponding field and clicking the “Save” button at the bottom of the screen. In addition, the option can be removed completely by clicking on the “X” button.

By default, the Nessus GUI operates on port 8834. To change this port, edit the xmlrpc_listen_port to the desired port. The Nessus server will process the change within a few minutes.

If additional preferences are required, click on the “New Setting” button, input the name and the value, and press “Save”. Once a preference has been updated and saved, Nessus will process the changes within a couple of minutes.

Image may be NSFW.
Clik here to view.

Adding a new user in Nessus.

Nessus Configuration: Adding/Editing Users

During the initial setup, one administrative user is created. Using the credentials specified during the setup, you can log into the Nessus GUI. Once authenticated, click on the “Users” heading at the top. To create a new user, click “New User” on the upper right. This will open a dialog box asking for required details. Input the username and password, verify the password, and determine if the user should have administrator privileges. If a user account needs to be modified, double-click on the user.

You cannot rename a user. If you want to change the name of a user, delete the user and create a new user with the appropriate login name. To remove a user, select the check box next to the account on the list, select “Options” on the upper right, and then click “Delete User” and confirm.

A non-admin user cannot upload plugins to Nessus, cannot restart it remotely, and cannot override the max_hosts/max_checks setting in the configuration section. If the user is intended to be used by SecurityCenter, it must be an admin user. SecurityCenter maintains its own user list and sets permissions for its users.

In the next article, we will cover more Nessus configuration options.

External Links:

Nessus home page on www.tenable.com

Nessus on Wikipedia

The post Nessus Configuration: Part One appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.The Nessus GUI configuration menu contains several configurable options. For example, this is where the maximum number of checks and hosts being scanned at one time, the resource you want nessusd to use and the speed at which data should be read are all specified, as well as many other options. These settings should be reviewed and modified appropriately based on your scanning environment.

Nessus Configuration: max_hosts and max_checks

In particular, the max_hosts and max_checks values can have a great impact on your Nessus system’s ability to perform scans, as well as those systems being scanned for vulnerabilities on your network. Pay particular attention to these two settings.

Default Values for max_hosts/max_checks

Note that these settings will be overridden on a per-scan basis if you are using Tenable’s SecurityCenter or write a custom policy in the Nessus user interface. To view or modify these options for a scan template in SecurityCenter, edit a Scan Template’s “Scan Options”. in the Nessus user interface, edit the scan policy and then click on the “Options” tab.

8834983181171783"; /* pfSense middle of page ad #2 */ google_ad_slot = "8926342897"; google_ad_width = 728; google_ad_height = 90; //-->

It should be noted that the max_checks parameter has a hardcoded limit of 15. Any value over 5 will frequently lead to adverse effects as most servers cannot handle that many intrusive requests at once.

As the name implies, max_hosts is the maximum number of target systems that will be scanned at any one time. The greater the number of simultaneously scanned systems by an individual Nessus scanner, the more taxing it is on that scanner system’s RAM, processor, and network bandwidth. Take into consideration the hardware configuration of the scanner system and other applications running on it when setting the max_hosts value.

As a number of other factors that are unique to your scanning environment will also affect your Nessus scans, experimentation will provide you with the optimal setting for max_hosts.

max_checks is the number of simultaneous checks or plugins that will run against a single target host during a scan. Note that setting this number too high can potentially overwhelm the systems you are scanning sepending on which plugins you are using in the scan.

Multiply max_checks by max_hosts to find the number of concurrent checks that can potentially be running at any given time during a scan. Because max_checks and max_hosts are used in concert, setting max_checks too high can also cause resource constraints on a Nessus scanner. As with max_hosts, experimentation will provide you with the optimal setting for max_checks, but it is recommended that this always be set relatively low.

Here is a selective list of some other Nessus configuration options:

External Links:

Nessus Documentation at www.tenable.com

The post Nessus Configuration: Part Two appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.Before you start vulnerability scanning, you should take into consideration some issues. Port scanning is a fairly innocuous activity, althouh it is annoying when you see the activity showing up in your logs. Vulnerability testing, however, can be quite a bit more disruptive, crashing servers, taking down Internet connections, or even deleting data. Many of the Nessus tests, for example, are specifically designed to cause a denial-of-service attack. Even with the safe checks option turned on, the tests can cause problems with some systems. With this in mind, here are some guidelines.

Scan Only with Permission

You should never scan a network that is not under your direct control or if you do not have explicit permission from the owner. Some of the activity initiated by Nessus could be legally considered hacking (especially with the DoS checks turned on). Unless you want to take the chance of being criminally and civilly charged, or having a complaint lodged against you by your ISP, you should always scan with permission. Non-company outsiders such as consultants should make sure to obtain written permission with all the legal disclaimers necessary. Internal personnel should make sure they have authority to scan all the machines in the range they are scanning. Coordinate with other departmental personnel as necessary, such as firewall administrators and security staff.

Modern vulnerability scanners are easy to install and use, but they are generally installed with generic settings. Using such settings may not be a good idea if you have legacy hardware. Approaching legacy hardware with a scan calls for caution, as a scan may cause problems with the legacy machine’s approaches to port management and binding. In such cases, port scanning can cause connection hang-ups and even system crashes. As a result, if your network includes such hardware, you want to be aware of the potential issues of running an untested scan and plan accordingly. Segmenting your risk by testing only a few servers at a time may be the way to go.

You should always make sure your backups are current anyway, but it is doubly important when vulnerability scanning, just in case the scan causes a problem with a server. Doing a Nessus scan right after you run backups will ensure that you can restore the most current version. But also make sure you aren’t running your scan during a backup. Not only could you cause a corruption of your backup data, but both processes will slow to a crawl.

Scheduling Your Scans

Along the lines of the last comment, make sure you coordinate your scan to get the results you want with minimal impact on other employees. Scanning the mail server at 8:00 AM when everyone is getting their e-mail will probably not make you very popular with the staff. Schedule scans on always-up servers for off-hours, and be sure to avoid overlapping with other system administration and general activity levels. If you are scanning internal machines, you will probably want to do it during the day unless you can arrange for everyone to leave their machines on at the end of the day. The best time to do it during business hours is generally around the lunch hour, as a minimal number of people will be using the network.

Schedule your scans as often as you feel is necessary, but don’t automatically assume that nightly scans are going to make your network more secure. If you cannot interpret and respond to scan reports on a daily basis, then don’t do the scan; all it will do is put additional traffic on your network. Base your frequency on the capability of your staff to deal with the results. You should do it at least once a month, but if you have a particularly busy network, you may want to do it weekly. Similarly, if you have a very small external network, you may feel comfortable with quarterly scans. Daily scans are probably excessive unless you have dedicated staff to handle the remediation work. If you have that much need for up-to-the minute protection, then use an intrusion detection system to supplement your vulnerability testing.

If you want a true of your external vulnerability (for the Internet), you should make sure your Nessus server is located outside your firewall. This can be on a home Internet connection, at a data center that is outside your company network, or at another company (perhaps you can negotiate a trade to use another company’s facilities for scanning and let them user yours for the same). Remember, because of the Nessus client-server architecture, you can still control your scans from inside your firewall. Just make sure you enable the SSL support so communications between your client and the server are encrypted.

If you are scanning your internal network, your internal network, your server will have to be located inside your firewall. Loading Nessus on a laptop can facilitate doing scans from both inside and outside your network without requiring multiple machines.

External Links:

Vulnerability Scanning Do’s And Don’ts at www.darkreading.com

The post Vulnerability Scanning Tips appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.Security Issues That Won’t Be Fixed By Vulnerability Scanning

While vulnerability testing is a valuable tool in your security arsenal, you should not think of it as a silver bullet. There are still situations and areas that a vulnerability testing program won’t help you with. You have to develop additional systems and procedures to lessen your exposure in these areas. The following include security issues that won’t be found by vulnerability testing.

Logic errors are security holes that involve faulty programming logic inside a program. these are generally undiscovered or unpatched bugs where the program does not perform as it was supposed to. An example would be a web login page that does not authenticate properly or one that allows users to get more privileges than they should have. Well-known logic errors in major programs might be included in the Nessus vulnerability tests, but most of them are too obscure to be noticed except by a dedicated hacker.

Vulnerability testers rely on published reports of vulnerabilities. Usually, once a vulnerability is announced, an add-on or plug-on for the system is written. With open source programs, this may only take a few days. However, during that time there may be a window of vulnerability because your scanner won’t be finding that security hole if it exists. Of course, you could quickly write your own tests using NASL while you wait for the official one to come out.

Vulnerability testing programs typically only address published commercial and open source programs. If you have a program that was developed for internal use only, a vulnerability tester probably won’t test anything on it. If it uses standard protocols or subprograms such as HTTP, FTP, or SQL, then some of the tests may apply. There are additional programs specially designed to test code for its security that you should run on these applications. The good news is that with an open source vulnerability tester like Nessus, you can write tests custom designed for your in-house application.

All the testing in the world won’t help you if you have poor or nonexistent security policies for your employees. Hackers denied technical means to gain access to your network can revert to social engineering – in other words, trying to talk someone into giving them access. This can be surprisingly easy, because the hacker takes advantage of the basic human nature of people generally wanting to help others, especially people perceived as fellow employees. There is only one way to combat this kind of hacking, and it does not involve any technical systems. Having good security policies, educating employees about them, and enforcing them will lessen your exposure to these kinds of attacks.

Vulnerability testing only shows you potential security holes in your system; it won’t tell if those holes have been exploited or alert you if an attack is taking place. Programs like Nessus are purely preventative in nature, and they are effective only if you take action to fix problems when they are found. Vulnerability scanners won’t fix them for you, although Nessus is very helpful in giving you detailed information on how to fix any issues found.

External Links:

Vulnerability scanner on Wikipedia

The post Vulnerability Scanning: What It Won’t Fix appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Arping in action under pfSense 2.1.3.

Arping is a computer software tool that is used to discover hosts on a computer network, and is available as a package for pfSense. The program tests whether a given IP address is in use on the local network, and it can get additional information about the device using that address. The utility is similar to the ping utility, which has been discussed on this site in an earlier posting. Whereas ping probes hosts using the Internet Control Message Protocol (a routable protocol that operates on the network layer of the OSI model), arping operates entirely on the data link layer.

There are two popular arping implementations. One of them, part of the Linux iputils suite, cannot resolve MAC addresses to IP addresses. However, the version of this utility that is available as a package for pfSense was written by Thomas Habets and can ping hosts by MAC address as well as by IP address.

Installing Arping

Installing this utility is easy. In the pfSense web GUI, navigate to System -> Packages and click on the “Available Packages” tab. Arping should be on the list. Scroll down to arping and click on the “plus” button on the right side to install arping. The pfSense package installer will ask you to confirm that you want to install arping; press the “Confirm” button. The package installer status window will provide information about the installation and let you know when installation is complete. once it is, arping should appear on the “Installed Packages” tab.

Using Arping

Once arping is installed, you can access arping by navigating to Services -> Arping. From there, you can enter a host ip or MAc address and press the “ARPing” button to ARP ping.

What is it good for, given that the utility essentially replicates the functionality of ping? One case where arping is helpful is when the host you want to ping is firewalled and will not respond to a ping request. Even a firewalled host will respond to ARP.

Another case is when you do not have network layer (layer 3) connectivity to the host you wish to ping (possibly because you want to find out if an IP is taken), but you have data link layer (layer 2) connectivity. Without network layer connectivity, you won’t be able to ping a host, but you can use ARP (since ARP is a data link layer protocol), albeit only for hosts on the local subnet. One note of caution is that on networks employing repeaters that use proxy ARP, the ARP response may be coming from a proxy host and not from the probed target.

External Links:

Arping website for Thomas Habets’ arping

Arping on Wikipedia

The post Arping with pfSense: Installation and Use appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Installing Squid under pfSense 2.1.3.

Squid is a proxy server and web cache daemon. It was originally developed as part of the Harvest project at the University of Colorado Boulder. Further work on the program was completed at the University of California, San Diego (UCSD) and funded via two grants from the National Science Foundation. Duane Wessels forked the last pre-commercial version of Harvest and renamed it Squid, and Squid version 1.0.0 was released in July 1996. It has a number of uses. Under pfSense, it can be used to cache repeated requests.

Squid Proxy Installation

Installing and configuring a Squid proxy server under pfSense is relatively easy. From the pfSense web GUI, go to the top menu and navigate to System -> Preferences. Scroll down to “squid” on the list of packages, and click on the “plus” button on the right to install Squid. On the next screen, press the “Confirm” button to confirm that you want to install Squid. It will take a few minutes for the package installer to unpack and install Squid.

Squid Proxy Configuration

Once installation is complete, “Proxy Server” will show up as an option under Services. Navigate to Services -> Proxy Server to configure Squid. Most users will find the default settings to be acceptable, but there are several settings worth noting. There are 7 tabs in the Squid proxy settings: General, Upstream Proxy, Cache Mgmt, Access Control, Traffic Mgmt, Auth Settings, and Local Users.

Image may be NSFW.
Clik here to view.

The General settings tab in Squid proxy configuration.

General Settings: This covers the most commonly configured Squid proxy settings. The first setting, “Proxy Interface“, determines which interface or interfaces the proxy will bind to. You probably want to leave this set to “LAN” so that the proxy server is accessible to hosts connected. You probably want to leave “Allow users on interface” checked, to allow users connected to the interface selected in the Proxy Interface field to use the proxy. You probably also want to check the “Transparent proxy” check box so all requests for destination port 80 will be forwarded to the proxy server.

Upstream Proxy: If you want Squid to forward requests to an upstream proxy server, you can enable forwarding here. There are also settings to specify the IP address/hostname of the proxy, TCP and ICP port, and username and password, if the upstream proxy requires them.

Cache Mgmt: This controls a number of settings relating to the cache size. “Hard disk cache size” sets the total amount of hard disk space Squid will use to cache objects. If you have a large hard drive, you can increase this setting to cache more objects; otherwise you can probably leave it at the default value (100 MB).

“Memory cache size” is the amount of physical RAM to be used for negative cache and in-transit objects. Objects that squid cannot store in memory end up getting swapped to disk which is much slower than RAM. Squid recommends that this setting should be 50% or less of the installed RAM.

“Maximum object size” sets the maximum size of objects saved on disk. The default value is 4 KB. You can increase this parameter to save bandwidth, or lower it to improve speed. Most cache hits tend to take place on small files, although you probably want to increase this parameter from the default of 4 KB.

Access Control: This controls a number of settings regarding who is allowed to access the proxy server. In “Allowed subnets“, you can enter each subnet that is allowed to use the proxy (the proxy interface subnet specified in “General” is already an allowed subnet, so you don’t have to specify it here). “Unrestricted IPs” allows you to specify IP addresses that will not be filtered out by the other access control directives set out in this section. “Banned host addresses” allows you to specify IP addresses that are not allowed to use the proxy. “Whitelist” allows you to specify domains that will be accessible to users that are allowed to use the proxy. “Blacklist” allows you to specify domains that will be blocked to users that are allowed to use the proxy.

Traffic Mgmt: This controls a number of traffic settings. “Maximum download size” limits the maximum total download size to the size specified here (the default is no limit). “Maximum upload size” limits the maximum total upload size to the size specified here (the default is no limit). “Overall bandwidth throttling” specifies the bandwidth throttle for downloads; users will gradually have their download speed increased to this value (default is no throttling). “Per host throttling” specifies the download throttling per host (again, the default is no throttling).

Image may be NSFW.
Clik here to view.

The Authentication settings tab.

Auth Settings: Here, you can enable authentication of Squid proxy users. Squid supports local authentication, as well as authentication through an external LDAP, RADIUS or NT server. The default setting is “None” for no authentication.

Local Users: This allows you to set usernames and passwords for individual users. Assuming authentication is enabled and you chose the local authentication option, users will then be able to use the credentials set here to log in to the Squid proxy server.

For basic Squid proxy usage, the above may be all the information you need. If you really want to understand some of the more advanced options, though, you probably should read the Squid man page. You can use these command-line options by [1] executing a command from the Command prompt option in the Diagnostics menu; [2] SSH-ing into your pfSense box; or [3] accessing pfSense directly from the console.

To shut down Squid, issue this command:

squid -k shutdown

To restart squid, issue this command:

/usr/local/sbin/squid -D

However, it should be noted that pfSense seems to start Squid on its own if it notices Squid is not running. Some of the more interesting options are -u port (to specify a different ICP port than 3130; this can also be done from the pfSense GUI), and -z to create swap directories (useful if you have just deleted the cache and want to recreate the swap directories). There is also a loader.conf.local file in the /boot directory with settings that can be configured. The “Squid Package Tuning” document on doc.pfsense.org suggests changing the kern.ipc.nmbclusters parameter from “0″ to “32768″. This increases the amount of memory used for socket buffers, and may improve performance.

External Links:

Squid on Wikipedia

How to Set Up a Transparent Squid Proxy Server Using pfSense at hubpages.com

Squid Package Tuning at doc.pfsense.org

Proxy Servers at pfsensesolution.blogspot.com

The post Squid Proxy Configuration in pfSense appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The “Settings” tab of LightSquid in pfSense 2.1.3.

In the previous article, we covered installation and configuration of Squid under pfSense. In this article, we will cover how to monitor Internet usage with Squid and by installing and using the Squid log file analyzer LightSquid.

LightSquid Installation

Like other packages, LightSquid can easily be installed through the pfSense package manager. To access the package manager, navigate to System -> Packages. Scroll down to LightSquid and click on the plus symbol on the right side of the package to start the installation. Click the “Confirm” button on the next screen to confirm the installation, and LightSquid will be installed within a few minutes. When the installation is complete, there will be a new entry on the “Status” menu called “Proxy Report“.

LightSquid Settings for Log File Analysis

Here are some of the settings available (configurable by navigating to Status -> Proxy report and clicking on the “Settings” tab):

Language: Here you can select the language in which LightSquid reports are displayed in.

Bar color: This setting lets you control the color of the parts in the reports.

Report scheme: Lets you choose the theme for the appearance of the reports. Unless you have a preference for one of the alternate themes, you can leave it as the default of “Base”.

IP resolve method: When parsing the log files, LightSquid attempts to resolve the IP address into domain names. You can change the method it uses to resolve the IPs with this setting. The choices are: IP (just return the IP address); Demo (return AUTHNAME; if AUTHNAME not available return DNSNAME; if DNSNAME not available, return IP address); DNS (return DNSNAME); Simple (return AUTHNAME; if not available return IP); SMB (return smb name of PC); and Squidauth (return AUTHNAME; if not available, return IP address, and allow cryllic names).

Image may be NSFW.
Clik here to view.

A LightSquid user access report.

Refresh sheduler: This setting affects how often the Squid log files are analyzed. Decreasing the value will make the reports stay more up to date but will consume more system resources. Be careful not to set the refresh cycle to occur to frequently. If the system cannot finish one update before another one is requested, you will eventually crash the system.

Skip url: If there are any URLs you do not want to show up in the reports, you can list them here.

To view the reports, click on the “Lightsquid Report” tab. If you get an error message when you click on this tab, you may have to run lightparser.pl (to parse the log files) from the command line on your pfSense box. [To do this, SSH into your pfSense box or access the console directly, drop to the shell, and type in /usr/local/www/lightsquid/lightparser.pl.] Once reports have been generated, you should be able to navigate them. After you select a day you will see a list of clients that accessed the proxy on that day. Once you select a host from this list, you will see a list of all the URLs accessed by that client. Clicking the clock icon at the top of the page will show you the time of the day that each URL was accessed.

External Links:

LightSquid official site

Monitoring Internet Usage with LightSquid and pfSense at hubpages.com

The post Log File Analysis with LightSquid appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The General settings tab for SquidGuard under pfSense 2.1.3.

Now that we’ve covered both Squid and LightSquid, I thought it might be useful to cover some other Squid plugins. In this article, we will cover how to implement web filtering with SquidGuard.

SquidGuard is a URL redirector software, which can be used for web filtering of sites users can access. It uses blacklists to define sites for which access is redirected. Here, we are concerned mainly with SquidGuard installation under pfSense, but it can also be installed under Unix or GNU/Linux. The software’s filtering extends to all computers in an organization, including Windows, Macintosh, UNIX and Linux computers. It was originally developed by Pål Baltzersen and Lars Erik Håland, and was implemented and extended by Lars Erik Håland in the 1990s. The current version is 1.4, released in 2009.

As with other packages, installation of SquidGuard is easy. Just navigate to System -> Packages, scroll down to SquidGuard on the package list, click on the plus button on the right side of the listing, and on the next screen, click the “Confirm” button to confirm installation. The installation will take a few minutes. Once installation is complete, you will have a new menu item under Services called Proxy Filter, with which web filtering can be implemented.

The basic configuration of SquidGuard can be gleaned from the documentation on the official SquidGuard site. The simplest web filtering configuration has a single list of blocked sites and the URL of the page to show when the user tries to access a blocked site. The administrator, though, may choose to define more than one list, each representing a category to block. Finally, sometimes there is a demand to allow specific URLs and domains although they are part of the blacklists for a good reason. In this case, you want to whitelist these domains and URLs. This is generally accomplish in SquidGuard by editing the squidGuard.conf file, but if SquidGuard is installed under pfSense, the basic configuration can be done from the pfSense web GUI.

Web Filtering with SquidGuard: Configuration

You can configure SquidGuard in pfSense by navigating to Services -> Proxy Filter and clicking on the General settings tab. There is a check box at the top; check this box to enable the blacklist. Also on the General settings tab (towards the bottom of the page), you can specify the URL of the blacklist. You can use one of your own blacklists, or you can use one of the publicly available lists on the web. The official SquidGuard site has one at http://www.squidguard.org/blacklists.html. Enter the URL of the blacklist you want to use for web filtering in the appropriate edit box. Once you have done this, press the “Save” button at the bottom of the page.

Image may be NSFW.
Clik here to view.

The Blacklist tab in SquidGuard; here you can download blacklists.

Next, click on the blacklist tab and press the “Download” button. Once the download is finished, the status box will display “Blacklist update complete” (it may take several minutes to download).

Once you have uploaded your blacklist, you will need to configure which categories of sites on the blacklist should be either allowed, blocked, or whitelisted. The simplest way to configure it is with the common ACL tab. The common access list settings will apply to all users of the proxy. The next tab is the “Groups ACL” tab, and if you want to apply different rules to ther source networks you should use this tab. This way, you wan have different policies for different networks. When you modify a target rule, you need to click on “Apply” in the General settings tab in order for the changes made to take effect.

In the dropdown box next to each of the target rules, you can select one of the following three web filtering actions:

• Allow: Grant access to the target category unless blocked by another rule or exception
• Deny: Block access to sites in the target category
• Whitelist: Always allow access to the target category (even if blocked by another rule or exception.

At the bottom of the page, there is a check box to block IP addresses in the URL. This will prevent users from bypassing the filter by using the IP of the web site instead of the URL. They can still use the IP addresses of whitelisted sites in the URL, though.

By default, when a user attempts to visit a blocked page, they will see an internal error page indicating that the page was blocked and under which target category it falls. However, you can change the redirect page to a blank page, or any other internal or external URL.

In the next article, we will continue our look at SquidGuard configuration.

External Links:

The official SquidGuard site

URL Filtering – How To Configure SquidGuard in pfSense on hubpages.com

SquidGuard on Wikipedia

The post Web Filtering with SquidGuard: Part One appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The General settings tab in SquidGuard in pfSense 2.1.3.

In the previous article, we discussed how to install SquidGuard and began to look at configuration options, focusing on blacklists and access control lists. In this article, we continue our look at SquidGuard configuration.

Filtering Sites By Domain Name, URL, or Regular Expression

We will begin by considering sites that you need to allow your users to access. To prevent these sites from being blocked, you could create a new target category and add a list of domains or URLs that should not be blocked. To do this, click on the “Target categories” tab. From here, click on the plus symbol to add a new category. Each category must be assigned a name (no spaces allowed). The new target category can filter by domain name, URL, or by an expression. Filtering by domain will grant access to the main site and any sub pages on it. Entering a URL will allow access to that exact web page and nothing more. Expressions allow the administrator to grant access based on certain keywords. When you have created all the categories you want to create, press the “Save” button. Then go back to either the “Common ACL” or “Group ACL” tab (wherever you created the rule) and select the option of “Whitelist” for your new category. [You can just as easily select the "Deny" option and blacklist all sites in the category.]

In addition to domain and URL filtering, administrators can create filters using regular expressions in SquidGuard. These types of filters are useful if you want to search for certain strings of text in a URL to decide what rule to apply. We won’t go through all the rules of regular expressions, but I should mention that regular expressions typically consist of a series of characters and metacharacters. The metacharacters have a special meaning unless preceeded by an escape sequence (usually a backslash). Here are some of the more important metacharacters:

• . : Matches any single character – for example, a.c matches aac, abc, etc. Putting brackets around it causes the dot to be interpreted as a literal dot – [a.c] matches a, ., or c
• [ ] : A bracket expression; matches a single character or a range contained within the brackets. [abc] matches a, b, or c; [a-z] matches any lowercase letter. – is interpreted literally if it’s the first or last character.
• [^ ] : Matches any single character that is not contained within the brackets. [^abc] matches any character other than a, b, or c.
• ^ ; Matches the starting position of any line.
• * : Mathches the preceding element zero or more times. ab*c matches “ac”, “abc”, “abbbc”, etc.

To create a filter that uses an expression click on the target categories tab and either create a new category or edit an existing one. Enter the expression you want to filter on the expression box, and then press the “Save” button. Then go back to the common or group ACL tab and select Deny, Allow, or Whitelist for your target category.

Here are a few examples of filters in action:

#block some video sites
(.*(metacafe\.|dailymotion\.|/videosearch|video.|myvideo\.|youtube\.com))

#block all .gov sites
(.gov)

#block all .gov and .mil sites
(.gov|.mil)

Squidguard also allows the admin to apply URL filtering based on schdules, which are useful for applying rules at different times during the day, or only on certain days of the week. One way this could be used is for applying strict URL filtering rules during business hours and automatically disable the rules after 5 PM.

To create a time-based rule, click on the “Times” tab. Then click the “plus” sign to create a new schedule. Schedules can be applied using the “Groups” ACL tab. You can create a new group ACL tab (or edit an existing one) and in the “time” dropdown box select the schedule you created. You need to press the “Apply” button on the general tab for the settings to take effect.

External Links:

The official SquidGuard site

URL Filtering – How To Configure SquidGuard in pfSense on hubpages.com

The post Web Filtering with SquidGuard: Part Two appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The Pentium P-233 that served as my m0n0wall firewall/router

When I started using pfSense as my primary firewall, it replaced my previous firewall solution: a Pentium P-233 running m0n0-wall. I eventually switched to a Neoware thin client running pfSense, which I ultimately upgraded to version 2.1.3. The Neoware thin client meets the pfSense hardware requirements for running pfSense on an embedded system, and offered pretty good value for the money – one would be hard-pressed to put together a system more cheaply than these pfSense appliances which has the same features and functionality. Yet while running pfSense from a thin client may be the best option for some users, if you have an old computer that meets the pfSense hardware requirements, this may be the better option. For that reason, I thought it would be an interesting exercise to see how easy (or how hard) it is to turn an old PC into a pfSense firewall.

Indeed, the system I used to run m0n0wall had been scrounged from spare parts. The case and power supply had come from an old barebones system I had bought in the late 1990s. The motherboard/CPU was one of a lot of three I had bought on eBay a few years later, and the CD-ROM was from a group of spare CD-ROM drives I had, as was the floppy drive. I only had 32 MB of RAM initially. I found that with only 32 MB of RAM installed, m0n0wall’s web-based configurator would eventually crash (although the firewall itself would continue to function). I found another 32 MB of RAM on eBay for a few dollars, and my system was complete. The NICs had also been taken from old computers, although I eventually bought a lot of 10 Intel Pro 100 cards for $35. As underpowered as this system might seem, it served ably as my firewall for several years. Thus, I began to wonder if I had any old hardware that could run pfSense, and decided that for my next mini-project, I would take an old computer and turn it into a serviceable pfSense router.

pfSense Hardware: The Guidelines

For this project, I set out some basic guidelines:

1. The hardware had to meet the general requirements for pfSense hardware. These requirements are listed on the official pfSense web site. For any installation, a Pentium II or better with at least 256 MB of RAM is recommended. For hard drive installations, a 1 GB hard drive is required (and a CD-ROM drive for installation).
2. When possible, I would scrounge from existing resources to put together a system that would serve as my new pfSense box. If necessary, I would buy new hardware, but only as a last resort.
3. I was not completely sure what the final system would have installed on it, but I knew at a minimum I wanted to have the most recent pfSense version (2.1.3 at this writing), and probably Squid, SquidGuard, and probably some other packages.
4. To the fullest extent possible, I would document the process, so I would have a record of what worked (and what didn’t work).

These guidelines should provide a rough road map for this project. In the next article, I will cover the selection of hardware, putting together my pfSense box, and installing pfSense onto it.

External Links:

Hardware for pfSense at pfsense.org – pfSense hardware requirements guide

The post pfSense Hardware: A Scrounger’s Guide (Part One) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The computer I used as my new pfSense box.

In the last article, I discussed my project to turn an old computer into a pfSense firewall and set some guidelines for the project. In this article, I get to configuration of the pfSense box and pfSense installation.

pfSense Installation: Selecting the Hardware

As you recall from part one of this series, the base system requirements for a pfSense installation are:

• Pentium II or better
• 256 MB RAM
• 1 GB of disk space for a standard installation; 512 MB of disk space for embedded systems

I immediately realized the system I used for m0n0wall would not make the grade (too slow and not enough memory). However, I had another old system that might work. I had a Pentium III (733 MHz) with 256 MB RAM. The motherboard for this system died a few months ago; I found a replacement on eBay (for $15), but the system has been running slow ever since. It seemed like an ideal candidate for conversion to a pfSense firewall.

Since I did not want to erase the contents of the original hard drive, I had to find another one to install into the system. I went through a box of old hard drives and found a Western Digital Caviar 22000. With 2 GB of disk space, it had more than enough space for pfSense. I swapped out the original hard drive with the Western Digital.

The next consideration was what network cards to install on the system. You need at least two NICs: one for the WAN and one for the LAN. Installing a third NIC allows you to have an OPT1 interface for a DMZ. Fortunately, there was already one Intel Pro 100 NIC in the computer, and I had a spare two. The Intel Pro 100s are PCI cards, and there are three PCI slots on this motherboard, so I used up all the available PCI slots, but that shouldn’t be a problem. If you need to buy NICs, the folks at pfsense.org recommend purchasing Intel cards (or systems with built-in Intel NICs) up to 1 Gbps. It would behoove you to by Intel PRO 1000s, at least for the LAN and OPT1 interfaces (on the WAN side, using a 100 Mbps NIc will not create a bottleneck for most residential broadband customers). A quick eBay search revealed than PRO 1000s are available for less than $10 (for both PCI and PCI-X interfaces). My Neoware thin client has a 1 Gbps 2-port NIC for the LAN and OPT1 interfaces, and a 100 Mbps NIC for the WAN interface. An upgrade to Intel PRO 1000s on this system is definitely something I will consider in the near future.

Image may be NSFW.
Clik here to view.

The Compaq Deskpro motherboard recognizes the Samsung drive, so we can proceed.

With the hard drive and NICs installed, I was ready to move the computer over to the test bench and begin pfSense installation. After running setup to make sure the BIOS recognized the Western Digital drive, I put the pfSense CD in and booted the system. When prompted whether to boot pfSense from the CD or run the installer, I hit “I” and invoked the installer. This is where I had my first real setback: although the motherboard’s BIOS recognized the Caviar, pfSense did not, and I therefore could not install pfSense onto it. Fortunately, I had a Samsung sW0434A (total capacity: 4.3 GB) I could install (again courtesy of the box of old hard drives), so I powered down the system and replaced the Western Digital with the Samsung.

pfSense Installation: Options

Once the hard drive had been replaced, I was able to boot pfSense from the CD and begin pfSense installation. When the installer starts, you have a chance to change the video font, change the screenmap, change the keymap, or accept the settings. Since I had no reason to change the defaults, I chose “Accept These Settings“.

On the next screen, you have a choice between quick/easy install and custom install (there are also options to rescue config.xml and reboot). In most cases you can opt for the quick/easy install, but if you do not want to reformat the hard drive, or if you want to partition the hard drive onto which pfSense is installed, or specify a different hard drive geometry than what was detected by pfSense, you want to opt for the custom install. I just wanted to reformat the hard drive and install pfSense onto it, so I opted for “Quick/Easy Install“.

Next, the pfSense installer will give you a choice between installing the standard pfSense kernel, or the embedded kernel (which has no vGA console or keybaord available). I selected “Standard Kernel” and continued. After a few minutes, pfSense was installed, and I was prompted to reboot the system. With pfSense installation complete, I rebooted the system and was ready to run pfSense on this computer for the first time.

When pfSense runs for the first time, it will ask you to assign interfaces. I assigned fxp0 for the WAN and fxp1 for the LAN. [I opted to set up OPT1 from the web configurator, later on]. I also assigned the IP address for the LAN interface.

By now, pfSense installation and configuration was complete, and I had a fully functional pfSense box, but I hadn’t connected it to my network. That’s no fun, so in the next article, I will talk about what happened when I used the new system as my firewall.

External Links:

pfSense Hardware at www.pfsense.org

Intel PRO/1000 GT Desktop Adapter – Overview at www.intel.com

The post pfSense Installation: A Scrounger’s Guide (Part Two) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

The General Information page in the pfSense setup wizard.

In the last article, we covered selection and installation of hardware for our new pfSense system, as well as the installation and initial pfSense configuration. In this article, we will continue with pfSense configuration.

At this point, I connected the WAN interface to the cable modem. On the LAN side, I decided to connect the LAN interface to an Asus RT-N12 router running in WAP (wireless access point) mode. pfSense would act as a DHCP server and would be responsible for routing, while the RT-N12′s job would be to enable wireless devices to connect to the network.

pfSense Configuration: The Setup Wizard

Now I could complete pfSense configuration by accessing the web configurator via the pfSense box’s LAN IP address (in this case, 192.168.2.1). I typed the address into a web browser tab, specified the default admin password (pfsense), and began. When accessing the web GUI for the first time, you will be presented with a wizard which will guide you through the initial configuration.

The first pfSense configuration screen (after clicking on the first “Next” button) is the “General Information” settings. Here, you can specify a hostname and domain name. You can also specify the primary and secondary DNS servers; you can use your ISP’s DNS servers or other DNS servers. The last setting on this page is the “Allow DNS servers to be overridden by DHCP/PPP on WAN”, which causes pfSense to use the DNS servers specified by your ISP rather than the ones you specify here. You can usually leave this checked, but if you have a setup that requires pfSense to use a specific DNS server, then you want to make sure it is not checked. I could not think of any reason it would be a problem, so I left it checked.

On the next screen, you can enter the time server hostname and the timezone. I left the time server hostname set to the default, and set the timezone to US/Eastern, and clicked on “Next“. The next screen allows you to set the admin password. You will want to change the password from the default, so here we enter a new password (twice).

Image may be NSFW.
Clik here to view.

The pfSense dashboard.

That brings us to the end of the setup wizard. Clicking on “Next” will bring up the pfSense dashboard for the first time. As you can see, we now have a system running the current (as of this writing) version of pfSense on an Intel Pentium III. My current network activity doesn’t come close to taxing the system. Moreover, I have plenty of disk space, so if I want to install additional packages like Squid, SquidGuard or nmap, I should be able to do so.

By now, all I needed to do to have a more or less functional pfSense system was to add NAT entries for ports I need to have forwarded. Fortunately, pfSense makes it easy to set up port forwarding (if you leave “Filter rule association” set to the default of “Add associated filter rule”, each NAT entry you add will have a corresponding rule). It took about ten minutes to add my NAT entries (I only had six), and I was done.

In this series, we have demonstrated that the process of repurposing an obsolete computer into an epic pfSense firewall is not that difficult or even time-consuming. Hopefully, if you have been considering undertaking such a project, this will inspire you to do so.

What’s next? First, I want to install some additional packages, such as Squid and SquidGuard. In addition, I still have pfSense on a Neoware thin client, so I am considering setting up a CARP (Common Address Redundancy Protocol) redundancy group in which the Neoware box would act as a backup firewall. This would prevent me from setting up a DMZ on the OPT1 interface, however, since CARP requires a dedicated SYNC interface on each system. This seems like a worthwhile project, however, and if I undertake it, I will be sure to blog about it.

External Links:

Installing pfSense at doc.pfsense.org

The post pfSense Configuration: A Scrounger’s Guide (Part Three) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Backend server settings page for Varnish under pfSense 2.1.3.

I recently ran a series of articles on Squid, a proxy server which began life as a client-side cache and can be used under pfSense as such. In contrast, Varnish is a reverse proxy HTTP accelerator designed for content-heavy dynamic web sites. It is focused exclusively on HTTP, unlike other reverse proxy servers that support other network protocols.

The Varnish Reverse Proxy: Architecture and Performance

Varnish works by storing data in virtual memory and leaving the task of deciding what is stored in memory and what gets paged out to disk to the operating system. This helps avoid the situation where the operating system starts caching data while it is moved to disk by the application. In addition, Varnish is heavily threaded, with each client connection being handled by a separate worker thread, and an overflow queue to handle incoming connections once the configured limit on the number of active worker threads is reached. When the overflow queue limit is reached, incoming connections will be rejected.

Varnish uses the Varnish Configuration Language (VCL), which is used to write hooks that are called at critical points in the handling of each request. When a VCL script is loaded, it is translated to C, compiled into a shared object by the system compiler, and loaded directly into the accelerator.

The creators of Varnish claim that it speeds up delivery by a factor of 300-1000 times, depending on your architecture. When the Varnish cache is enabled, performance is bound by the speed of the network, thus turning web server performance into a non-issue. It is free software licensed under a two-clause BSD license, also known as the FreeBSD license.

The Varnish Reverse Proxy: Installation and Configuration

Image may be NSFW.
Clik here to view.

The Settings tab in Varnish.

Installation of Varnish under pfSense is similar to installation of other packages. From the pfSense web GUI, navigate to System -> Packages and scroll down. You will see both “Varnish” and “Varnish 3” on the packages list. Click on the “plus” sign to the right of the listing for Varnish 3 to install the Varnish reverse proxy. On the next screen, press the “Confirm” button to confirm that you want the package installed. It should take about 3-4 minutes for installation to complete.

Once Varnish is installed, you will have a new item on the “Services” menu called “Varnish“. If you navigate there, you will see seven tabs covering different settings for the Varnish reverse proxy server: “Backends“, “Settings“, “Custom VCL“, “LB Directors“, ‘XMLRPC Sync“, “View Configuration“, and “VarnishSTAT“. Before you can enable Varnish, you need to configure at least one backend server, so click on the “Backends” tab.

Varnish has a concept of “backend” or “origin” servers. A backend server is the server providing the content Varnish will accelerate. To add a backend server, click on the “plus” button on the “Backends” tab. There are four sections on this page. “Backend settings” covers the most basic settings. “Backend name” is the name of the backend web server, and “IP address” is the IP address (on your local network) of the backend web server. At “Port“, you enter the port of the web server (usually 80), and for “Description“, you can enter a description.

The next section is “Performance metrics“. “First byte timeout” represents the time to wait for the first byte for the backend and the time to wait between each received byte. “Connect timeout” is simply the time to wait for a backend connection.

Next is “Probe settings“. If this is configured properly, the Varnish reverse proxy will check the health of each backend with a probe. “Probe URL” is the URL that Varnish will use to ensure that the backend is healthy. “Probe interval” specifies how often we should poll. “Timeout” simply specifies the timeout of the probe; this is the amount of time (in seconds) that Varnish will wait for a backend probe. “Probe Window” specifies how many probes Varnish will retain when considering backend health. “Threshold” specifies how many of the probes specified by “Window” must have succeeded for us to consider the backend healthy. Finally, checking the “Disable Probe” check box disables probing for this backend. In the last section, “Backend Mappings“, you can map either a hostname or a URL to this server. It can either match a string or a regular expression.

The second tab is the “Settings” tab. Under “Daemon options” you can enable Varnish by checking the “Enable Varnish” check box. At “Listening port” you can set the port Varnish will listen on. The “Management interface” specifies the IP address and port for the management interface, and “Advanced startup options” specifies any startup options to include in the rc.d configuration file.

If you specify settings for these options and check the “Enable Varnish” check box, you should have Varnish up and running and working with any web servers you specified as a backend. There are several other settings that you might find useful, and we will cover those in the next article.

External Links:

The official Varnish web site

Varnish at Wikipedia

The post Reverse Proxy Services with Varnish (Part One) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Worker thread settings and VCL settings in Varnish under pfSense 2.1.3.

In the last article, we introduced Varnish, a powerful reverse proxy server, and covered installation and basic configuration. In this article, we will continue to look at some of the configuration options.

We will begin by looking at the settings for the second tab, “Settings“. We already covered “Daemon options” in the previous article. Next is “Storage type“. Here, you can select which storage type Varnish uses. By default, the cache is stored in memory (which makes it faster), but you can switch to disk storage instead. You can also specify the cache size in megabytes here (obviously, you do not want to exceed your RAM/storage space).

Next is “Worker thread configuration“. Here you can set the minimum number of Varnish worker threads, the maximum number of Varnish worker threads, and the worker thread timeout. The next subheading is “General VCL Settings“. The first item is the “Client identity method“. This setting is relevant if you have load balancing configured and you want to load balance based on something the client provides: an IP address, a requested URL, or a user agent. The “Don’t cache posts” check box causes Varnish to stop caching POST requests. The “streaming support” check box enables streaming support. “Session Cache” defines Varnish’s behavior with respect to cookies. By default, Varnish does not cache content with cookies in it. This is the recommended behavior, but should you want to cache even when there are cookies involved (and changing the web application is not possible), you can set up a session cache on a per-user basis here (which makes for a fairly low cache hit ratio). It requires your system to not change the cookie on each page hit. “Cache static content” specifies when Varnish should cache static content. If set to “When possible“, Varnish will cache only static content and per user objects when the session cache is set to it. “Always” will cause Varnish to cache all static content; when cookies are present, Varnish will unset it from the object before caching. “Never” will cause Varnish to never cache static content.

The “Fix gzip compression” check box causes Varnish to ignore compression for image files and unknown compression algorithms if checked. The “Be rFC2616 compliant” check box will cause Varnish to ignore requests that do not comply with RFC 2616. “Forward client IP” lets you select how to log the client’s IP address: “set X-Forwarded For“, “append X-Forwarded For“, “set X-Forwarded-Varnish“, and “unset” to leave it unlogged. “Fetch Grace” allows you to set how long Varnish will keep cached objects.

The last section on this page is “Error Settings“. “Retries” determines how many times Varnish will try before it sends an error message. “Saintmode” determines how long Varnish will send cached objects from a backend which has gone offline. Finally, “Custom HTML error message” lets you paste a custom html error page code.

That covers the settings in the Varnish “Settings” tab. In the next article, we will continue our look at Varnish settings.

External Links:

Caching, even when cookies are present at www.varnish-cache.org

The post Reverse Proxy Services with Varnish (Part Two) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

LB Directors tab in Varnish settings under pfSense 2.1.3.

In the last two articles, we introduced the Varnish reverse proxy and covered installation and basic configuration. In this article, we will cover the remaining configuration options.

The third tab on the Varnish settings page is “Custom VCL“. The first two fields are “vcl_recv_early” and “vcl_recv_late“. Code pasted into “vcl_recv_early” will be executed at the beginning of the vcl_recv function and code pasted into “vcl_recv_late” will be executed at the end of the vcl_recv function. vcl_recv is called at the beginning of a request for a document, after the complete request has been received and parsed. Its purpose is to decide whether or not to serve the request, how to do it, and if applicable, which backend to use. You can use these fields to alter the request, if necessary. Typically you can alter the cookies and add and remove request headers.

The next two fields are “vcl_fetch_early” and “vcl_fetch_late“. Code pasted into “vcl_fetch_early” will be included at the beginning of the vcl_fetch function, and code pasted into “vcl_fetch_late” will be included at the end of the vcl_fetch function. vcl_fetch is called after a document has been successfully retrieved from the backend. Normal tasks included here are to alter the response headers, trigger ESI processing, and try alternate backend servers in case the request failed. The last two fields are for “vcl_pipe_early” and “vcl_pipe_late“. Code pasted into “vcl_pipe_early” will be included at the beginning of the vcl_pipe function, and code pasted into “vcl_pipe_late” will be included at the end of the vcl_pipe function. vcl_pipe is called upon entering pipe mode. In this mode, the request is passed on to the backend, and any further data from either client or backend is passed on unaltered until either end closes the connection.

The next tab is “LB Directors“. You can group several backends into a group of backends; these groups are called directors. this will give you increased performance and resilience. Press the “plus” button to add a new director. The first heading is “Director Settings“. The “Director name” is where you can specify a name for the group. In “Match type“, you can select the field type that you would like to use in matching the host or URL, and in the “Host” and “URL” fields, you can specify both of these. The “Rewrite Host” and “Rewrite URL” fields allow you to specify an alternate host and URL that will be redirected to the host and URL specified above. “Req Grace” specifies how long Varnish will keep cached objects for the director. “Additions options” allows you to paste custom Varnish code for this host.

The next subsection is “Backend Settings“. The “Algorithms” dropdown box allows you to select how Varnish will balance clients. “Round-robin” selects a backend in round-robin fashion. “Random selects a backend randomly. “Client” picks a backend based on the client’s identity. You can set the VCL variable client.identity to identify the client by picking up the value of a session cookie or something similar. “Hash” causes Varnish to select a backend based on the URL hash value. This is useful if you are using Varnish to load balance in front of other Varnish caches or other web accelerators as objects won’t be duplicated across caches. “Backend” is where you specify the backends for this director (from the dropdown list) and a weight for each one. The last section, “Failover Settings“, allows you to select a director for failover.

The next tab, “XMLRPC Sync“, allows you to sync Varnish configuration changes. In the first dropdown box, you can choose “Sync to configured system backup server” to sync to a server specified in “High Avail. Sync” under the System menu. “Sync to host(s) defined below” allows you to sync to one or more servers specified at “Remote Server“. “Do not sync this package configuration” disables synchronization. “Sync timeout” allows you to specify a maximum sync wait time.

There are two more tabs. “View Configuration” allows you to view (but not edit) the underlying default.vcl file. “VarnishSTAT” allows you to view the VarnishSTAT server logs.

External Links:

The official Varnish web site

The post Reverse Proxy Services with Varnish (Part Three) appeared first on pfSense Setup HQ.

Image may be NSFW.
Clik here to view.

Anyterm settings page under pfSense 2.1.3.

Sometimes, you may want to access Secure Shell (SSH) servers but the access point you are using blocks port 22 or whatever port your SSH server is using. Fortunately, there is a solution: web-based SSH makes it possible to access SSH servers through standard web browsers. Respective clients are based on JavaScript/Ajax or JavaScript/WebSockets and can be used to access SSH servers from behind a firewall or proxy.

Anyterm is one such web-based SSH program. It is written in C++ for the server side and JavaScript for the client side, and uses server-side terminal emulation. It also utilizes long polling for client/server communication. The server-side implementation is a stand-alone daemon which is typically used with a reverse proxy, such as Apache’s mod_proxy. Anyterm is licensed under the terms of the GPL.

Anyterm consists of some JavaScript on a web page, an XmlHttpRequest channel on standard ports back to the server, an HTTP proxy and the Anyterm daemon. The daemon uses a pseudo-terminal to communicate with a shell or other application, and includes terminal emulation. Key presses are picked up by the JavaScript, which sends them to the daemon. Changes to the emulated screen are sent from the daemon to the JavaScript which updates its display. SSL can be used to secure the connection, and is recommended.

Web-Based SSH with Anyterm: Installation and Configuration

To install Anyterm in pfSense, navigate to System -> Packages; Anyterm should be at the top or near the top of the list. Press the “plus” button to the right of the listing for Anyterm; on the next screen, click the “Confirm” button to confirm installation. It will take a few minutes for installation to complete, after which there will be a new menu item on the Diagnostics menu (Anyterm).

When you navigate to Diagnostics -> Anyterm, there are two tabs. The first tab, “Settings“, has four options. The first two fields are “Username” and “Password“, in which you can specify the username and password for accessing Anyterm. The third field is “Port“, where you can enter the port that Anyterm will use (default is 8080). The last field is “STunnel Port” where you can enter the STunnel port if you have an STunnel forward. Press the “Save” button at the bottom of the page to save the settings.

You probably want to set up port forwarding for the Anyterm port so you can use Anyterm from other networks. Navigate to Firewall -> NAT, and press the “plus” button on the bottom right to add a new entry. For “Destination port range”, enter the Anyterm port, and for “Redirect target IP“, type the IP of your pfSense system. For “Redirect target port“, enter the Anyterm port again. At “Description”, add a brief description, and leave “Filter rule association” at the default of “Add associated filter rule“. Press the “Save” button to save the entry. Press “Apply changes” on the next page to apply the changes.

Image may be NSFW.
Clik here to view.

Anyterm in action, used to access the pfSense shell.

You should be able to use Anyterm to gain shell access to pfSense now and thus take full advantage of its web-based SSH capabilities, but you probably want to install STunnel as well so you have an SSL encryption tunnel between you and pfSense. STunnel can be installed from System -> Packages, and installation should only take a minute. Once you install stunnel, it will be an option on the “Services” menu.

Under “STunnel”, there are two tabs: “Tunnels” and “Certificates“. “Listen on IP” and “Listen on port” specifies the listening socket IP address and port. “Certificate” specifies the certificate to use for the listening socket. “Redirects to IP” and “Redirects to Port” specifies the target IP address and port. The “Outgoing source IP” is the IP address to bind to when connecting to the target.

Certificates are managed by requiring the user to provide an RSA key and certificates/chains in PEM format. The Certificates tab will list the configured certificates along with status information, indicating whether the certificate is valid, will expire soon, or is already expired. A check is also performed to make sure the key and certificate matches. Once you finish setting up stunnel, remember to go back to the Anyterm “Settings” tab to enter the STunnel forward port.

By now, you should be able to use web-based SSH to access the shell of your pfSense system from outside your local network by entering your WAN IP address and the Anyterm port. If you installed STunnel, you will have an SSL encryption tunnel, so your web-based SSH session should be relatively secure.

External Links:

The official Anyterm web site

The post Web-Based SSH with Anyterm appeared first on pfSense Setup HQ.