Posts tagged freebsd

Installing Apache and Its Ecosystem: Part 2

Jul13th
2012 avatar Written by Rob

Back here in Apache Land it is time to continue with the install. In this post I’ll be installing PHP. Part 3 will cover Subversion, and Part 4 I’ll be running through my standard configuration and website setup.

Installing PHP 5.4.4

At the time of writing PHP 5.4.4 is the current stable release of PHP, so we’ll be installing that and a dozen or so modules. I used to be a PHP developer in a past life (or so it feels, sometimes) so there are a lot of modules I used to make regular use of.

Additionally, I run a lot of the usual software like WordPress, Roundcube,  Gallery2, WebSVN, etc.

A quick look at the modules installed on Tyrande (the existing server):

Read on for full installation instructions..
READ MORE »

Server Build    , , , ,

Installing Apache and its Ecosystem: Part 1

Jul10th
2012 avatar Written by Rob

I guess it is time for the big boy. Apache can be one of the most complicated things you install, mostly for the additional languages and modules. Like many, I have my own way of configuring it.

I’ll lay this out up front: I don’t like the FreeBSD layout for Apache. Everything is scattered throughout /usr/local and buried deep in the system. I got used to using the OpenBSD layout during my stint with that OS, and so I continue to use that on FreeBSD. It puts everything in /var/www/ and keeps it nice and clean together.

That said, the binaries and other bits and pieces should remain in /usr/local/.

Building from Scratch

For once we’re not going to install Apache from ports. Why? Because it’s too damn hard to adjust the layout without building a completely custom port. What we can do though is use the ports tree to do a big chunk of the work for us.

Configure your options

Run make config in the port and select the options you want installed. I go with the defaults here usually.

Install the dependencies

Now that we’ve selected our options, we can also use it to build and install the dependencies, much the same as we would do if we were installing Apache through ports.

So that is everything we should need.

Generating the configure command

Now we let the port install get as far as configuring the build, that will generate the configure command that was used, and we can copy that command and use it to build our own setup.

Setting up our own build environment

Now we’ve had our fun with ports it is time to build our own copy of Apache. I do my compilation in a directory called /archive/compile, but you can do it anywhere you like.
READ MORE »

Server Build    , , , ,

Firewalling: The OpenBSD Packet Filter

Jul10th
2012 avatar Written by Rob

It is time to get security conscious! Installing and using pf for your firewall is a breeze, so this post will mostly be describing my firewall configuration and some of the awesome things you can do.

Warning: This is another long one.

I use the OpenBSD Packet Filter, or pf, for my server-level firewall. Why? Based originally on ipf, I find it to be much simpler and more powerful than the usual tools like iptables. How powerful? One line will normalise all incoming traffic. Normalisation is the process of cleaning up bad or invalid packets that can wreak havoc in poorly written server software.

It has the usual port filtering and Network Address Translation (NAT) features, plus traffic redirection (port forwarding),  Operating System-based filtering, packet queuing, prioritisation, address pools, and load balancing. Yep, when paired with CARP (the Common Address Redundancy Protocol), pf can bring you the same level of redundancy and load balancing as enterprise level Cisco gear. I had a series of four servers setup in an old workplace, each was redundant for the other using CARP + pf. You could even control the order of failover across all four servers.

Anyway, pf is installed by default, we just need to setup our rules and enable it. The first thing you need to know though, is don’t turn it on with untested rules. If you do you’re more than likely to lock yourself out of the server, at which point you’d better hope your console access is working.

Before we dive into my configuration, an overview of the configuration file is probably in order. A pf.conf file is generally broken into several sections.

Macros Variables that you can set to be re-used later on.
Tables Variables that can hold a list of data, like a list of IP Addresses
Options Options that control how pf functions.
Queuing Rules and options to support queuing and prioritisation
Filter Rules The bulk of your firewall ruleset, providing port filtering, NAT and traffic redirection.

Now, while my general configuration has a few interesting things you can do with pf, it is actually rather boring compared to some setups I’ve done in the past. I don’t use traffic redirection, queuing, prioritisation or load balancing. It’s just a server firewall.

The most useful resource you will find for pf is the OpenBSD FAQ page on the topic.

Read on for my configuration and the step by step instructions to set it up.
READ MORE »

Server Build    , , , , , , , , , ,

DNS Setup: The Pitfall of Bind

Jul9th
2012 avatar Written by Rob

DNS can be a vastly complicated thing, particularly setting up Bind the most common DNS server product in use. However, it doesn’t have to be scary.

In this post I’ll go through setting up Bind and configuring a test domain. If you’re not sure what DNS is or how it works, hit up Google and find out. This article by How Stuff Works isn’t too bad an overview either.

For my DNS setup, I run Bind as the master server, and have an account with EasyDNS for additional DNS servers that synchronise changes from the master server to their copy. This is more commonly known as a Secondary DNS.

Picture time! (I like pictures).

Overview of my DNS Setup

What happens when a DNS request comes in (say you want to load this blog), is that your ISP’s DNS servers will lookup the name servers for my domain, which would return one of the four options above. At that point your ISP’s DNS server will pick one at random and ask it for the IP address of my blog (known as an A record). Your computer can then connect directly to the server and open the blog.

Each of the EasyDNS servers listed above is actually an anycast constellation, which means it has four or five DNS servers spread across the globe. For example, at the time of writing, the EasyDNS 1 constellation had servers in Ashburn, Chicago, San Jose, Amsterdam and Tokyo.

The EasyDNS servers stay in sync through a method called a zone transfer. When I update the records on Shana, she will send a message to the EasyDNS transfer server that there is an update available for a specific domain. EasyDNS then requests a zone transfer to grab the most recent copy of the DNS records for that domain and updates their local server records.

Installing Bind

That’s probably enough of an overview of my setup for now, lets get cracking on installing and configuring Bind. Fortunately, we don’t have to do anything here, Bind is already installed by default on FreeBSD.

It is worth noting here that the service/daemon that runs is actually called named. From here on out I’ll refer to everything using named as the product name “Bind” is rarely mentioned.

Configuring Named

As it is already installed, you can imagine that a good chunk of the setup is already done for you too. This is true, all of the named configuration files can be found in /etc/namedb/. Lets check what we have in there.

dynamic/ A subdirectory where dynamic zones live. Dynamic zones are typically updatable by client systems.
master/ A subdirectory where master zones live. Master zones are ones where we (this server) are considered authoritative for.
named.conf The main Named configuration file.
named.root A list of the root name servers that make up the core of DNS.
slave/ A subdirectory where slave/secondary zones live. Slave zones are ones where we retrieve config via zone transfer from somewhere else.
working/ A subdirectory named keeps its working files.

You might notice too that the dynamic, slave and working directories are owned and writable by the bind user. That is because named will write to these directories itself.

For this post, we’re only really interested in the named.conf file and the master subdirectory.
READ MORE »

Server Build    , , , , ,

Mail Server Setup: Part 2 – Dovecot and PostgreSQL

Jul6th
2012 avatar Written by Rob

This is the second part in the two part posting about setting up my mail server. In this one we tackle Dovecot and PostgreSQL.

This post will be remarkably short compared to the first one. A bit of a refresher on the setup I’m aiming for:

We’re looking at the red arrows in this one, so Dovecot, as well as setting up the Virtual Mailboxes and Sieve support. Because we’re using dovecot-lda to handle delivery of mail into the actual “mailbox” on the file system we need to have completed configuring Dovecot before our setup of Postfix from Part One will be functional.

Installation of Dovecot

Dovecot itself was installed as a dependency of Postfix back in Part One. That was because we selected Dovecot2 SASL Auth option. If you want to install Dovecot manually head on over to ports and run through the usual process, selecting the following options on the config screen:

  • KQUEUE (default)
  • SSL (default)
  • PGSQL

Under Dovecot2 the Sieve plugin support is provided by Pigeonhole, which can be easily installed via ports:

Configuration

Lets separate our configuration into several bits to make it a bit easier to follow.

Virtual Mailboxes, Databases and Delivery

We need to start with the delivery of mail into the virtual mailboxes, so that we can finally test the Postfix setup and delivery of mail into the mailboxes.

From Post One, we setup dovecot-lda using the following line in /usr/local/etc/postfix/master.cf.

dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${recipient}

So it calls /usr/local/libexec/dovecot/deliver with the sender’s email address passed in via the -f option, and the recipient’s email via -d. We’re also running it with a user called vmail. Lets setup the user and group first.

Now that that is done, clear out all of the skeleton files that were copied into /var/vmail and we can update the config file for LDA.

Almost all of Dovecot’s configuration can be found in /usr/local/etc/dovecot/, including the LDA settings that we’re after now. If that directory is empty for you then the README file should tell you where to find the example ones, copy the example files we need into the target directory.

There are some things we need to setup in order to get the LDA working, including Sieve, the Passdb, Userdb and authentication. Lets go in that order.

To enable sieve, edit /usr/local/etc/dovecot/conf.d/15-lda.conf and set mail_plugins to sieve.

protocol lda {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = sieve
}

Because we’re delivering mail as vmail, you may need to adjust the first_valid_uid and first_valid_gid settings in /usr/local/etc/dovecot/conf.d/10-mail.conf.

Set the value to whatever UID and GID your vmail user has, unless the ID is greater than 500, in which case the default is fine. Even better, set the last_valid_uid and last_valid_gid to your vmail UID and GID. That means only vmail can deliver mail, no other user.

first_valid_uid = 145
last_valid_uid = 145

first_valid_gid = 145
last_valid_gid = 145

To setup the authentication properly, open up /usr/local/etc/dovecot/conf.d/10-auth.conf, comment out the existing include for auth-system, and uncomment the auth-sql.conf.ext line, it should look like this:

#!include auth-system.conf.ext
!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext

Then we can edit /usr/local/etc/dovecot/conf.d/auth-sql.conf.ext to point to the correct SQL configuration file, this happens under passdb and userdb.

passdb {
driver = sql

# Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}
userdb {
driver = sql
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
}

Then we want to open up /usr/local/etc/dovecot/dovecot-sql.conf.ext and make it look like the following. I’ll step through it after the paste.

# This file is opened as root, so it should be owned by root and mode 0600.
#

# Database driver: mysql, pgsql, sqlite
driver = pgsql

# Database connection string. This is driver-specific setting.
connect = host=/tmp dbname=mail user=mail password=xxx

# Query to retrieve the password.
password_query = SELECT DISTINCT ON (mbox) mbox as user, password, '/var/vmail/' || mbox || '/home/' as home, 'maildir:/var/vmail/' || mbox || '/' as mail, 145 as uid, 145 as gid FROM COALESCE((SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE (mailboxes.mbox = '%u' AND mailboxes.disabled = false) OR (aliases.alias = '%u' AND mailboxes.disabled = false)), (SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE (aliases.alias = '@%d' AND mailboxes.disabled = false)), (SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE (aliases.alias = '%n@' AND mailboxes.disabled = false))) as mbox JOIN mailboxes USING (mbox);

# Query to retrieve the user information.
user_query = SELECT '/var/vmail/' || mbox || '/home/' as home, 'maildir:/var/vmail/' || mbox || '/' as mail, 145 as uid, 145 as gid FROM COALESCE((SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE mailboxes.mbox = '%u' OR aliases.alias = '%u'), (SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE aliases.alias = '@%d'), (SELECT DISTINCT ON (mbox) mbox FROM mailboxes LEFT OUTER JOIN aliases USING (mbox) WHERE aliases.alias = '%n@')) as mbox;
  • driver = pgsql - Tells Dovecot that we want to connect to a PostgreSQL server
  • connect – The connection string. Set host to where your UNIX socket lives (usually /tmp), and dbname, user and password to the same credentials you used configuring Postfix in Post One. Alternatively, you can use different credentials, make sure you grant the appropriate permissions to the database.
  • password_query – This is the query that Dovecot will execute to retrieve the hashed password from the database. It then compares it to the hash it creates of the password given to it by the mail client to see if they match.
  • user_query – This query tells Dovecot where to find information on the users and mailboxes as configured.

These queries should look similar to the ones from Part One. Lets break them down a bit for better understanding.

So it executes three subqueries that are almost identical to the ones used by Postfix, except the substitutions are different this time around.

  • %u = the full email address being looked up
  • %d = only the domain (the bit after the @)
  • %n = only the username (the bit before the @)

It returns a lot more information though from the coalesced queries, and it checks if the email address in question is an alias too. Whereas Postfix was only looking for validation before handing it to dovecot-lda, Dovecot needs to know where and how to deliver the message to. The columns returned are as follows.

Column Description
user The email address of the user in question. Used as a username of sorts.
password The hashed password.
home The full path on the filesystem to the user’s home directory. In this case it is in a subfolder of their virtual mailbox: /var/vmail/email/home/. This is where the Sieve files live. The trailing slash is important.
mail The full path on the filesystem to the user’s mailbox. In this case it is in a subfolder of their virtual mailbox: /var/vmail/email/. The trailing slash is important.
kid The user ID of the local account dovecot should use when accessing the mailbox. This should be the ID vmail user we setup earlier, and you need to make sure it has read and write access to the folders above.
kid The group ID of the local group dovecot should use when accessing the mailbox. This should be the ID vmail group we setup earlier.

Now that we have that out of the way, its time to test mail delivery! Oh, but we need a user account.

Creating User Accounts

There are two things you need to do to create a user account:

  1. Add the user in the mailboxes table in the database.
  2. Create the user’s mailbox on the filesystem.

To add the user, connect to your database using psql and run the following. Change <password> to the password you generated using doveadm pw -s ssha512.

Then create the directory on the filesystem to match.

Then we can test!

Testing the mail delivery

If you haven’t already, we’ll need to configure Dovecot to start, add this line to your /etc/rc.conf file.

dovecot_enable="YES"

Now lets start Postfix first.

If everything went well there shouldn’t be any errors there, or any in /var/log/maillog.

Jul 5 22:49:05 shana postfix/postfix-script[86436]: starting the Postfix mail system
Jul 5 22:49:05 shana postfix/master[86437]: daemon started -- version 2.9.3, configuration /usr/local/etc/postfix

We don’t need to start Dovecot to test mail delivery as dovecot-lda is called automatically by Postfix. Lets connect and see what happens (the comments are server responses, the rest are commands I’m typing).

Accepted! Lets see where that went by checking /var/log/maillog.

Jul 5 23:44:03 shana postfix/smtpd[87478]: connect from tyrande.odynia.org[202.62.159.130]
Jul 5 23:44:23 shana postfix/policy-spf[87495]: : SPF softfail (Mechanism '~all' matched): Envelope-from: test@test.com
Jul 5 23:44:23 shana postfix/policy-spf[87495]: handler sender_policy_framework: is decisive.
Jul 5 23:44:23 shana postfix/policy-spf[87495]: : Policy action=PREPEND Received-SPF: softfail (test.com: Sender is not authorized by default to use 'test@test.com' in 'mfrom' identity, however domain is not currently prepared for false failures (mechanism '~all' matched)) receiver=unknown; identity=mailfrom; envelope-from="test@test.com"; helo=tyrande.odynia.org; client-ip=202.62.159.130
Jul 5 23:44:23 shana postfix/smtpd[87478]: 26E051E3: client=tyrande.odynia.org[202.62.159.130]
Jul 5 23:44:29 shana postfix/cleanup[87497]: 26E051E3: message-id=&lt;&gt;
Jul 5 23:44:29 shana postfix/qmgr[87474]: 26E051E3: from=, size=519, nrcpt=1 (queue active)
Jul 5 23:44:29 shana dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: No such file or directory
Jul 5 23:44:29 shana dovecot: lda: Fatal: Internal error occurred. Refer to server log for more information.
Jul 5 23:44:29 shana postfix/pipe[87498]: 26E051E3: to=, relay=dovecot, delay=10, delays=9.9/0.01/0/0.04, dsn=4.3.0, status=deferred (temporary failure)
Jul 5 23:44:35 shana postfix/smtpd[87478]: disconnect from tyrande.odynia.org[202.62.159.130]

Hmm, a temporary lookup failure trying to use the wrong userdb. Lets see what we missed.

Oh!

So when you’re using the SQL userdb and passdb setup you do need to have Dovecot running so it creates the authentication sockets that dovecot-lda needs. Lets start it up now then.

Uh oh! We missed a step in setting up Dovecot. We forgot to generate the SSL certificate that is configured by default in /usr/local/etc/dovecot/conf.d/10-ssl.conf. You have a few choices here.

  • Go buy a certificate from a trusted vendor and copy your public and private key files (PEM files) into /etc/ssl/certs/dovecot.pem and /etc/ssl/private/dovecot.pem respectively. This is the recommended and most secure option.
  • Generate a self-signed certificate using the instructions on the Dovecot Wiki. (Note that the files mentioned in doc/ are actually in /usr/local/share/examples/dovecot/ under FreeBSD).
  • Disable SSL (not recommended at all)

Once you’ve done one of those we can start Dovecot again.

Better. Nothing in the logs?

Jul 6 00:08:01 shana dovecot: master: Dovecot v2.1.7 starting up

Right, lets try that delivery again, that previous message was still in the queue, so lets flush it.

So, was that successful?

Jul 6 00:05:54 shana postfix/qmgr[87474]: 26E051E3: from=, size=519, nrcpt=1 (queue active)
Jul 6 00:05:54 shana dovecot: auth: pgsql(/tmp): Connected to database mail
Jul 6 00:05:54 shana dovecot: lda(test@odynia.org): msgid=unspecified: saved mail to INBOX
Jul 6 00:05:54 shana postfix/pipe[87829]: 26E051E3: to=, relay=dovecot, delay=1295, delays=1295/0.01/0/0.1, dsn=2.0.0, status=sent (delivered via dovecot service)
Jul 6 00:05:54 shana postfix/qmgr[87474]: 26E051E3: removed

Yay! It was delivered to the folder, lets check the contents.

One complete message, and the softfail there shows SPF is working correctly also (test.com’s SPF record is set to softfail).

Configuring IMAP and POP3

All that is left now is configuring IMAP and POP3 access, including accessing both via SSL.

But wait! It is already working. You can configure your favourite mail client to connect via POP3 or IMAP and it will already be configured and working. There is nothing additional you need to do.

Jul 6 00:15:50 shana dovecot: imap-login: Login: user=, method=PLAIN, rip=172.16.0.127, lip=172.16.0.4, mpid=87945, TLS, session=
Jul 6 00:15:50 shana dovecot: imap(test@odynia.org): Connection closed in=17 out=352

I configured Mail on my Macbook to connect as test@odynia.org, and its already coming in over SSL for IMAP too. But what about sieve?

Configuring and Testing Sieve

Lets try a simple test, create a Folder in your Mail client and we’ll create a sieve file to redirect matching mail to that folder.

We’ll need to create the home folder inside the mailbox, like so:

Then we can create a .dovecot.sieve file in there. So in this example it is /var/vmail/test@odynia.org/home/.dovecot.sieve. Make sure it is owned by vmail:vmail also.

require ["fileinto"];
if anyof (header :contains "Subject" "[test]")
{
fileinto "Test Folder";
stop;
}

So if we send an email again with a subject that includes “[test]” it should get redirected automatically to that folder.

Lets check the logs.

Jul 6 00:26:50 shana postfix/smtpd[87988]: connect from tyrande.odynia.org[202.62.159.130]
Jul 6 00:27:01 shana postfix/policy-spf[87994]: : SPF softfail (Mechanism '~all' matched): Envelope-from: test@test.com
Jul 6 00:27:01 shana postfix/policy-spf[87994]: handler sender_policy_framework: is decisive.
Jul 6 00:27:01 shana postfix/policy-spf[87994]: : Policy action=PREPEND Received-SPF: softfail (test.com: Sender is not authorized by default to use 'test@test.com' in 'mfrom' identity, however domain is not currently prepared for false failures (mechanism '~all' matched)) receiver=unknown; identity=mailfrom; envelope-from="test@test.com"; helo=tyrande.odynia.org; client-ip=202.62.159.130
Jul 6 00:27:01 shana postfix/smtpd[87988]: E3F212D0: client=tyrande.odynia.org[202.62.159.130]
Jul 6 00:27:17 shana postfix/cleanup[87996]: E3F212D0: message-id=&lt;&gt;
Jul 6 00:27:17 shana postfix/qmgr[87474]: E3F212D0: from=, size=586, nrcpt=1 (queue active)
Jul 6 00:27:17 shana dovecot: lda(test@odynia.org): sieve: msgid=unspecified: stored mail into mailbox 'Test Folder'
Jul 6 00:27:17 shana postfix/pipe[88010]: E3F212D0: to=, relay=dovecot, delay=19, delays=19/0.01/0/0.04, dsn=2.0.0, status=sent (delivered via dovecot service)
Jul 6 00:27:17 shana postfix/qmgr[87474]: E3F212D0: removed
Jul 6 00:27:19 shana postfix/smtpd[87988]: disconnect from tyrande.odynia.org[202.62.159.130]

Yep. Sieve also requires no additional setup.

I sure do like your test.

ManageSieve

Now, because of the length of this post I’m going to leave setting up ManageSieve to another day. I only use it for one service anyway, which is RoundCube webmail, so we’ll setup ManageSieve when we do RoundCube.

That’s it for another day. I hope this has been helpful.

Server Build    , , , , , , , , , , , ,
← Older Entries