Dan Benamy

Projects | Blog | Guides | Evidence Based Opinions

Work in progress!!

Introduction

Most basic use case. Way too hard to set up. TODO

OS Installation

Install Ubuntu 10.04 server edition on the server. These directions were originally written for 8.04 and may not be totally updated yet. Only install the ssh package in installer. You can use the desktop edition if you want a gui. I don’t recommend using it as a desktop machine though. I like easy to remember and hard to mess up names, so it’s named server1.

When this guide was written, Ubuntu 9.10 was the latest and greatest so I used that for the clients. I set up one client and then copied its drive to the others but you could set them all up at once. They’re named client1, client2…

Lots depends on the ips not changing, so the dhcp server needs to be set up for static dhcp for these machines or they need static ip themselves. I use static dhcp. NFS traffic is not encrypted so the machines should be firewalled off from public machines and the internet. A little router from Linksys or similar, with the server and clients plugged into the LAN side will do the trick.

Basic Machine Setup

This applies to all machines unless otherwise noted.

Set up SSH key login

Update everything

Set up a firewall

Ubuntu has the nice ufw command for configuring the firewall. If running

ufw

says “command not found”, install it with:

aptitude install ufw

Then:

ufw allow ssh
ufw enable

If at any point you’re having trouble with client-server communication and want to eliminate the firewall as a source of trouble, you can turn it off with

ufw disable

When you’re done troubleshooting, run

ufw enable

You can check the firewall settings with

ufw status

Keep Track of Configuration Changes

Etckeeper is a tool which keeps track of all changes to the system configuration (the files in /etc). It allows you to see what you’ve changed and go back to an earlier version if something gets messed up.

aptitude install etckeeper mercurial

I like switching to mercurial. That’s just personal preference. If you want to stay with the default setup of bazaar, you can skip the following steps.

etckeeper uninit
nano /etc/etckeeper/etckeeper.conf
# comment "bzr" and uncomment "hg"
# uncomment and add hg_option "-u etckeeper@localhost"
etckeeper init
aptitude remove bzr bzrtools

Denyhosts

Denyhosts is a tool to block people who are trying to break into your computer by connecting to ssh and guessing passwords.

aptitude install denyhosts
nano /etc/denyhosts.conf

Change DENY_THRESHOLD_ROOT to something like 3 to allow yourself a couple of mess-ups.

I don’t like to get emails when deny hosts blocks someone so I also commented out the ADMIN_EMAIL line.

If someone gets their password wrong too many times and denyhosts blocks their computer, you’ll see their ip address show up in /etc/hosts.deny. You can’t fix this by removing that line because denyhosts will add it back. Instead, add a line to /etc/hosts.allow that says “ALL: 1.2.3.4” (use the blocked ip address). Now would be a good time to add your desktop to /etc/hosts.allow to prevent it from being blocked if you do something dumb.

Disable password logins, at least for root

nano /etc/ssh/sshd_config

Change PermitRootLogin from yes to without-password. To disable password logins for all accounts change PasswordAuthentication to no.

Install security updates automatically

aptitude install unattended-upgrades
nano /etc/apt/apt.conf.d/10periodic

Change the settings to: APT::Periodic::Update-Package-Lists “1”; APT::Periodic::Download-Upgradeable-Packages “1”; APT::Periodic::AutocleanInterval “7”; APT::Periodic::Unattended-Upgrade “1”;

Utilities

clusterssh is great for runnning the same commands on multiple machines. If you’re working from a linux machine with a graphical interface, you can do this with clusterssh: aptitude install clusterssh Run it with clusterssh -l USERNAME machine1 machine2 ...

nmap can be handy for troubleshooting firewall issues so

aptitude install nmap

Host Resolution

Add all machines to /etc/hosts on each machine.

TODO explain better

LDAP

LDAP is a system for sharing user information across computers. This will allow the same logins to work on all the machines.

LDAP uses some particular terms that you may want to know: dn - distinguished name The full “path” that refers to a piece of information stored in LDAP dc - domain component LDAP stores information in a tree. All information will be within one or more dc nodes in the tree. I suggest using a root node/suffix of “dc=workgroup” to keep things simple. ou - organizational unit / organization name In our case, “ou"s will be used for entry types. So we’ll have an ou for users and an ou for groups. cn - common name For users, this will be their username.

Server

aptitude install slapd ldap-utils

Follow this guide but ignore the part on replication and security certificates. It’s tough enough to get this all right without them, so start as simple as possible. https://help.ubuntu.com/10.04/serverguide/C/openldap-server.html

Install web-based LDAP admin tool:

aptitude install phpldapadmin

Tell it about the ldap server:

nano /etc/phpldapadmin/config.php

Change

$servers->setValue('server','base',array('dc=example,dc=com'));

to

$servers->setValue('server','base',array('dc=workgroup'));

Make the same change (dc=workgroup) to the line

$servers->setValue('login','bind_id','cn=admin,dc=example,dc=com');

Uncomment

$servers->SetValue($i,'auto_number','enable',true);
$servers->SetValue($i,'auto_number','mechanism',search); and
$servers->setValue('auto_number','search_base','ou=People,dc=example,dc=com');

and change the dc part of that last line.

Uncomment the min search number line and change it from 1000 to 2000.

Now you should be able to log in to the web interface. Assuming you’re working from a machine that’s not the server, set up an ssh tunnel to port 80 with a command like

ssh -L 8000:localhost:80 server1

and then (on your machine) go to http://localhost:8000/phpldapadmin

TODO explain why tunnel, how it works

Log in with “cn=admin,dc=workgroup”

Client

aptitude install libnss-ldap ldap-utils
dpkg-reconfigure ldap-auth-config

For the ldap account for root enter

cn=admin,dc=workgroup

For the password enter your ldap admin password.

On Ubuntu 10.04 run

auth-client-config -t nss -p lac_ldap

On 9.10 run

auth-client-config -a -p lac_ldap

TODO missing steps

To allow users to change their passwords with the passwd command, edit /etc/pam.d/common-password to remove use_authtok from the line it’s on. The passwd command doesn’t work on the server.

Install nscd to cache ldap info, so clients don’t need to make constant queries of the server. This will speed things up.

aptitude install nscd

Sometimes this caching can mess things up if you’re changing setting, so if you change an ldap setting and it doesn’t appear to be working, run

/etc/init.d/nscd restart

Securing LDAP Communication

Once LDAP is working with unsecured communication, you’ll want to set it up with SSL so that traffic between the clients and server is encrypted. Follow the section on setting up SSL certificates from the Ubuntu LDAP guide linked to above. Then copy the file /etc/ssl/certs/cacert.pem to the clients as /etc/ssl/certs/server1_cacert.pem. In /etc/ldap.conf, change the uri to from ldap:// to ldaps://. In /etc/ldap/ldap.conf, make the same change, set TLS_REQCERT to demand, and set TLS_CACERT to /etc/ssl/certs/server1_cacert.pem.

The most recent time I had to set this up, I jotted down the following terse notes. When I get a chance, I’ll flush them out: Use ubuntu server cert guide, make a ca, also change default days to 36500 days, make a csr, sign it, cp server.key to /etc/ssl/private/hostname.key, cp ca cert to clients

TODO Link to page with lots of curses. They’ll be needed.

NFS

Server

aptitude install nfs-kernel-server
nano /etc/exports

Add a line for each client that looks like

/home CLIENT(rw,sync,no_subtree_check)

where CLIENT is a name for the client that the server knows, probably from /etc/hosts. You should be able to ping that client from the server and have it work.

Allow all connections from your local network to the server by running

ufw allow proto both from 192.168.1.0/16

(Yeah, the 16 should be an 8 or 24, but I’m not sure which, and 16 is ok.) Client Mount the home directories (/home) over nfs by adding a line to fstab.

nano /etc/fstab

Add the line

server1:/home /home nfs rsize=8192,wsize=8192,timeo=14,intr 0 2

Then

mount -a # Mount everything in fstab.
mount # See what's mounted. Should include the server at /home.
ls /home # You should see the home directories from the server.

Tightening the Firewall

If you only have a few clients, it’s easiest to allow connections from only those machines. On the server, run:

ufw allow from 192.168.0.100
ufw allow from 192.168.0.101

etc. Obviously, use your clients’ ip addresses. Then you can skip the rest of this section.

If you have many machines or they’ll be coming and going or changing ips a lot, you may want to set up NFS to run on specific ports and then open only those ports in the firewall. The following is all done on the server. (This info is from http://ubuntuforums.org/showthread.php?t=352486.)

Edit /etc/default/nfs-common and set

STATDOPTS="--port 4000"

Then

/etc/init.d/nfs-common restart # to apply the change

Edit /etc/default/nfs-kernel-server and set

RPCMOUNTDOPTS="--p 4002"

Then

/etc/init.d/nfs-kernel-server restart # to apply the change

Edit /etc/modprobe.d/options and add

options lockd nlm_udpport=4001 nlm_tcpport=4001

Then reboot. If you can unload and reload the nfs and lockd kernel modules this should take effect without rebooting, but there was a chain of dependencies that made that difficult for me so I just rebooted.

Now run

rpcinfo -p

and you’ll see that all the nfs services are running on well-defined ports. portmapper on 111, nfs on 2049, status on 4000, nlockmgr on 4001, and mountd on 4002.

Finally, update the firewall rules:

ufw delete allow proto any from 192.168.1.0/16 to any
ufw allow proto any from 192.168.1.0/16 to any port 111
ufw allow proto any from 192.168.1.0/16 to any port 2049
ufw allow proto any from 192.168.1.0/16 to any port 4000
ufw allow proto any from 192.168.1.0/16 to any port 4001
ufw allow proto any from 192.168.1.0/16 to any port 4002

Use

ufw status

to verify the rules are set up properly. Then for a final test, go to a client machine and run

nmap server1

and it should show the ports opened so far.

Imaging Clients

TODO

Clonezilla

After loading client image, update:

Backup

All user data gets stored on the server, so that’s the only machine that really needs a backup. You’ll be able to reinstall clients using the image (if you’ve made one). At a minimum, I recommend having a second drive with a periodic backup job that copies all data there. Optimally you’ll also have a copy off-site. I do this by having two backup disks set up identically which I swap periodically. I keep the one that’s not in use, somewhere else.

You’ll want to set up email so the backup job can tell you if anything goes wrong.

aptitude install postfix # accept the default settings

Test your mail set up by sending a email from the command line:

sendmail foo@example.com # use ctrl + d to end the message

If the mail doesn’t go through within a minute or two, check the log:

tail -50 /var/log/mail.log

Some ISPs don’t allow outgoing mail in an effort to block spam. If that’s the case, you’ll see messages about the connections to the gmail servers timing out. You can work around this by relaying mail through gmail if you have a gmail account. There are instructions at http://ubuntu-tutorials.com/2008/11/11/relaying-postfix-smtp-via-smtpgmailcom/. Set things up so that any mail that would go to the root account gets forwarded to you:

echo "root: you@gmail.com" >> /etc/aliases # adds a line to the file
newaliases

Test it with:

sendmail root # use ctrl + d to end the message

Now set up the backup scripts:

Create /usr/local/bin/backup and backup-logger based on mine at https://bitbucket.org/dbenamy/misc/src. You can remove the file extensions I have if you want them to look like typical commands. Edit backup-logger to point to the backup command. Customize the backup command. It’s set up for two ext3 partitions and an ntfs partition, but if you’re using the basic scheme the installer sets up, you’ll only have one partition to back up. Change the first mount to use the option -L “Backup”. If mounting by label (-L) doesn’t work, get the partition’s UUID (TODO) and use -U UUID. Get rid of the second and third sections. Run

/usr/local/bin/backup

and verify the results.

You’ll need to create a folder for the backup logs:

mkdir /var/log/backup # make sure you do this as root
chmod 700 /var/log/backup/

You’ll want to set up cron to run it automatically. Edit the backup script and add some gibberish to the second line so it causes an error when it runs. Make sure you’re root and run

crontab -e

Add a line

*/5 * * * * /usr/local/bin/backup-logger

which will run a backup every 5 minutes. Make sure that within 5 minutes you get an email about the error. Once you’re satisfied, run crontab -e again and change it to

0 4 * * * /usr/local/bin/backup-logger

for a run at 4am every day, and get rid of the gibberish in the backup script. Add some test file somewhere like:

echo "test" > /root/test.txt

The next day, mount the backup disk and make sure that file is there! For even more confidence that the backup system is working, install another ubuntu machine and try restoring all your data there. It’s easy to think a backup system is working and then when you actually need it, you realize something important is broken or missing.

Benchmarking

Disk:

hdparm -tT /dev/sda

Network- BROKEN TODO

aptitude install pv

On one machine run

nc -ulp 5000 > /dev/null

and on another

pv < /dev/zero | nc -u 158.130.10.46 5000

Creating User Accounts

On the server:

aptitude install ldapscripts
echo -n 'your ldap admin password' > /etc/ldapscripts/ldapscripts.conf
chmod 600 /etc/ldapscripts/ldapscripts.conf
nano /etc/ldapscripts/ldapscripts.conf

I made the following changes (some of which are carried over from an old version and may not be necessary any more):

BINDDN='cn=admin,dc=workgroup'
BINDPWDFILE="/etc/ldapscripts/ldapscripts.passwd"
SUFFIX='dc=workgroup'
GSUFFIX='ou=groups'
USUFFIX='ou=people'
MSUFFIX='ou=hosts'
LOGFILE="/var/log/ldapscripts.log"
Removed TMPDIR="/tmp"
USHELL=/bin/bash
UHOMES="/home/%u"
ASKGECOS="yes"
CREATEHOMES="yes"
HOMESKEL="/etc/skel"
HOMEPERMS="700"
PASSWORDGEN="cat /dev/random | LC_ALL=C tr -dc 'a-zA-Z0-9' | head -c8"
RECORDPASSWORDS="yes"
PASSWORDFILE="/var/log/ldapscripts_passwd.log"

Now you can create new users with

ldapadduser someuser users

It’ll create the user in ldap, create their home directory, and write their initial password to a log file. You can see it with:

tail /var/log/ldapscripts_passwd.log

There are a series of ldapscripts commands which can be used to manage users. See http://sourceforge.net/projects/ldapscripts/ and “man ldapscripts” for more info.

Onwards and Forwards

Comments and corrections are welcome at “FIRST at LAST dot info”. FIRST is daniel, LAST is benamy.

Thanks

Made with help from Ben Cohen and Nico Viennot. Any errors are mine.