My personal notes on all things technical and factual....

Fri, 23 Mar 2012

/Admin/backups/dirvish: Simple, Elegant, Small-Enterprise Backups with Dirvish

I do not like Bacula, Backuppc has some issues, rdiff-backup is too crude, where to turn for rock-solid backups? Rsync-based Dirvish[1] is making me very happy at the moment:

simple configuration file setup, only a couple of hours necessary to get off the ground
backups are de-duped (but not compressed)
backup sets are easy to find and available unobfuscated on the backup server file system, simply copy to restore
dirvish is another layer of abstraction on top of the rock-solid foundation of rsync and rdiff-backup
Documentation[2][3] is not copious, but not deficient either

As a layer on top of rdiff-backup, Dirvish's basic operation is the same: the first backup rsyncs all non-excluded files and directories. (Obviously, this may be quite a chunk of time and bandwidth if we are talking about a remote server with a large file system.) Thereafter, each succeeding backup is an rsync of just the changes since the last backup. Adjacent backups appear as directories with the entire contents of the backup within, BUT files that are the same are hard linked, ie. only the delta is taking up actual storage space. This means that pruning old backups simply consists of deleting the directories they are contained in, with hard-linking automatically taking care of the house-keeping for remaining adjacent backups. And it means that maintaining a complete backup of a remote server is reasonable, since after the first backup only incremental changes go over the network on subsequent backups.

What Dirvish adds on top of rdiff-backup is:

configuration via a simple set of configuration files
cron framework that automatically performs backups, and prunes old backups according to rules
displays all past backup sets as visible directories (rdiff-backup only does this for the most recent backup, and obfuscates older backup sets).
pre- and post-backup hooks for those who want them (MySQL dumps, for instance).

On Debian-based systems the framework is already setup with a cronjob that will, by default, fire at 2200 and prune/refresh all backups. Configuration involves creating an /etc/dirvish/master.conf that orchestrates the whole process, and creating a set of directories to house your Dirvish "banks" (groupings of backups) and "vaults" (each "vault" is a directory tree on a specific machine). And then inside each "vault" adding a dirvish/default.conf file that gives specific direction for that particular backup.

For instance, /etc/dirvish/master.conf:

bank:   
        /home/backups/dirvish-local
        /home/backups/dirvish-officeServer
        /home/backups/dirvish-ibmProductionServer
exclude:
        lost+found/
        core
        *~
        .nfs*
        tmp
        /proc
        /sys
        /dev
        /var/cache/apt/archives/*.deb
Runall: 
        local-root      22:00
        ibmFull         22:00
        officeEtc       22:00
        officeWWW       22:00
        officeMySQL     22:00

expire-default: +15 days
expire-rule:
#       MIN HR    DOM MON       DOW  STRFTIME_FMT
        *   *     *   *         1    +3 months
#       *   *     1-7 *         1    +1 year
        *   *     1-7 1,4,7,10  1    +1 year
        *   10-20 *   *         *    +4 days
#       *   *     *   *         2-7  +30 days

In the above, for instance, officeEtc / officeWWW / officeMySQL are all "vaults" for which I have created subdirectories under /home/backups/dirvish-officeServer. This file defines what is backed up, and a set of defaults for all "vaults". For instance, the above expire rules will keep 15 days of daily backups, three months of weekly backups, and one year of quarterly backups for all vaults, unless there are local vault rules that over-ride this behavior.

An example vault config file, /home/backups/dirvish-ibmProductionServer/ibmFull/dirvish/default.conf:

client: prodServer
tree: /
xdev: 0
index: gzip
log: gzip
image-default: %Y%m%d
speed-limit: 100
exclude:

"prodServer" must appear in /etc/hosts or resolve via DNS. "tree: /" tells us we are backing up the whole server. "speed-limit: 100" uses the same syntax as rsync, and limits bandwidth to 100 kB/s.

Normally after setting up all the configuration files, one tests the configuration by setting the "tree" parameter to a relatively small subdirectory in each vault in turn, and running:

for each vault. This causes that tree to be backed up immediately, and you can see and correct any errors. After all is working, reset "tree" to the desired value, and check a couple days later to see if everything is working as expected.

[1] http://www.dirvish.org/
[2] http://www.dirvish.org/docs.html
[3] http://wiki.edseek.com/howto:dirvish

posted at: 02:31 | path: /Admin/backups/dirvish | permanent link to this entry

Fri, 27 Jan 2012

/Admin/SSH-SSL: Passwordless Authentication with SSH

How to use SSH keys to login to your server without giving a password -- perhaps contra-inuitively, this kind of passwordless login is usually more, not less, secure then a password login. (Not to mention convenient and time-efficient....)

Say we want to login to server.com from our desktop without a password. The rdiff-backup wiki provides a somewhat obtuse and hard-to-read article[1] on the subject. For the basics, I prefer to start with this article[2].

On your desktop, run:

ssh-keygen -b 4096 -t rsa -f /home/username/.ssh/id_rsa
Do not enter a pass-phrase!! Leave it blank

(Note: we are creating a 4096 bit key here as recommended by nearlyfreespeech.net[3]. It is possible that some situations will require a 1024 bit key, and this key will not be useable in that situation. It is possible to have multiple keys, which may be invoked by the "ssh -i" option, for instance.)

Now copy the public key "id_rsa.pub" to root@server.com:

scp /home/username/.ssh/id_rsa.pub root@server.com:

Go to server.com and append the new key to the authorized_keys:

ssh server.com
cd /root/.ssh/
cat ../id_rsa.pub >> authorized_keys

Restrict access to these keys on both your desktop and your root@server:

chmod -R go-rwx ~/.ssh

Test to verify you are not prompted for a password. In a terminal on your desktop, try a verbose ssh to server.com:

ssh -v root@server.com

If there are problems and you are prompted for a password (you should not be) the -v output should give you some clues.

In some situations, one can make SSH key logins even more secure. For instance, on server.com, add some security directives to a particular key in /root/.ssh/authorized_keys by pre-pending the following:

command="rdiff-backup --server --restrict-read-only/",no-port-forwarding,no-X11-forwarding,no-pty

ie. in /root/.ssh/authorized_keys the key in qestion now contains the following, ALL ON ONE LINE, and note the single space before "ssh-rsa":

command="rdiff-backup --server --restrict-read-only /",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AA ... uqdswe= user@desktop

"no-pty" explicitly forbids terminal priveleges. "command" here restricts the session to running one and only one command: "rdiff-backup --server --restrict-read-only".

Now if you try to ssh to root@server.com from a terminal, your terminal will just lock-up and stop responding. If you really do want to allow this (a terminal ssh to root@server.com without a password) just remove the "command" and "no-pty" directives from the server.com /root/.ssh/authorized_keys file.

Note that this will only work from the account "username" on your desktop machine where you have generated a /home/username/.ssh/id_rsa file and then passed the public key to server.com. Trying to ssh from any other desktop account to root@server.com will result in a password prompt.

[1] http://wiki.rdiff-backup.org/wiki/index.php/UnattendedRdiff
[2] http://linuxgazette.net/104/odonovan.html
[3] https://members.nearlyfreespeech.net/ckoen/support/faq?q=SSHKeys#SSHKeys

posted at: 00:51 | path: /Admin/SSH-SSL | permanent link to this entry

Fri, 20 Jan 2012

/Admin/OpenVPN: Basic OpenVPN as an Internet Gateway

Aka. How to bore through the Great Firewall if you do not want to use an SSH tunnel.

This[1] is the OpenVPN documentation, but it is not altogether straight-forward to read, and it is missing some necessary detail.

This[2] will get OpenVPN basically working and connected for you. First create your keys:

Here I would add a warning from [1] that when creating your certificates above you must enter something at the "Common Name" prompt. My first time I just accepted all the defaults and got a connection error when I tried to start OpenVPN on my client. The second time, with a "Common Name" on the server certificates, everything just worked.

Then distribute the keys to server and client(s) per the references.

This will get the VPN client to the VPN server. However, we want to route *all* network traffic on the client through VPN and out to the internet. There are two components to this: some additional OpenVPN configuration, and some routing configuration on the server. On my VPN server, this is my /etc/openvpn/server.conf:

port 1348
proto udp
dev tap
ca ca.crt
cert server.crt
key server.key # This file should be kept secret
dh dh1024.pem
server 10.10.10.0 255.255.255.0 # vpn subnet
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
user nobody
; group nobody
persist-key
persist-tun
verb 10
mute 20
client-to-client
client-config-dir ccd "route 134.33.0.0 255.255.0.0"

; push "route 192.168.1.0 255.255.255.0" # home subnet
push "redirect-gateway def1"
push "dhcp-option DNS 10.10.10.1"

This should be the same as in [2] except that "group nobody" line did not work on my Ubuntu Lucid server for some reason, and the last two "push" lines are what is needed on the server end to tell connecting clients to redirect all network traffic to the VPN. (Though I am not convinced that last dhcp line is having any effect on my Debian box at the moment....) Also, per [1], note that if your network interface is DHCP, it may die periodically because it is unable to communicate with your DHCP server.

On my VPN client, this is my /etc/openvpn/client.conf:

client
dev tap
proto udp
resolv-retry infinite # this is necessary for DynDNS
nobind
user nobody
; group nobody
persist-key
persist-tun
ca ca.crt
cert x60s.crt
key x60s.key
comp-lzo
verb 4
mute 20

which is the same as in [2], except for commenting out "group nobody", which did not work on my Debian Testing client machine.

Now for routing on the server. What a PITA. I tried at some length to get raw iptables to do the job, but in the end turned to my old faithful, firehol. Here is my /etc/firehol/firehol.conf on my server, which enables the routing of traffic between tap0 (VPN) and eth0 (internet):

version 5

# interface eth0 internet
interface eth0 internet
   protection strong 10/sec 10
   server "https http icmp ssh"  accept
   server openvpn accept
   server ident reject with tcp-reset
   client all   accept

interface tap0 vpn
   server all   accept
   client all   accept

router internet2vpn inface eth0 outface tap0
   masquerade reverse
   client all      accept
   server ident    reject with tcp-reset

(After the fact, I ran into this[3] very interesting post....)

[1] http://www.openvpn.net/index.php/open-source/documentation/howto.html
[2] https://www.debian-administration.org/article/Connecting_to_office_network_using_OpenVPN_tunnel
[3] http://www.hermann-uwe.de/blog/howto-using-openvpn-on-debian-gnu-linux

posted at: 03:58 | path: /Admin/OpenVPN | permanent link to this entry

Sun, 15 Jan 2012

/Admin/virtualization/virtualBox: Debian Host and Guest

At least on Debian, there is one thing that is constantly breaking in my VirtualBox (VB) installation: missing kernel headers. The symptom is, that after a new kernel comes in, VB suddenly will not work any longer. It seems particulary insidious on the guest Debian OS, because "all" that stops working are certain conveniences, like (what I was just sorely missing) the ability to copy and paste between host and guest OS. I found it very easy to assume that something was just broken, when in fact all that was missing was the kernel headers package.

So once and for all, here is the drill for getting VB working properly on both host and guest....

On the host, this is the list of modules I have installed:

virtualbox
virtualbox-dkms
virtualbox-guest-additions
virtualbox-guest-additions-iso
virtualbox-ose
virtualbox-ose-dkms
virtualbox-ose-qt
virtualbox-ose-source
virtualbox-qt
virtualbox-source

On the host, if VB is not working and

modprobe vboxdrv

does not find your VB kernel driver, ie.

FATAL: Module vboxdrv not found.

you are almost certainly missing headers, ie. there should be a headers package installed that exactly matches your current kernel:

linux-image-3.1.0-1-686-pae
linux-headers-3.1.0-1-686-pae

Install it and all should be well. On the Debian guest OS, things are only slightly different. Here are my current VB modules on the guest:

virtualbox-guest-dkms
virtualbox-guest-source
virtualbox-guest-utils
virtualbox-guest-x11
virtualbox-ose-guest-source
virtualbox-ose-guest-utils
virtualbox-ose-guest-x11

if you cannot copy and paste between host and guest (my recent symptom) and the vboxguest kernel module is missing, same drill: install your headers. Then:

/etc/init.d/virtualbox-guest-utils start

and now

# ps -ef | grep -i vbox
clayton     1641     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --clipboard
clayton     1650     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --display
clayton     1655     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --seamless

you can see all the things that were not working perfectly before. ("Clipboard" being the shared clipboard whose absence meant my copying and pasting was not working.)

One final tweak to get sound working on the guest OS:

apt-get install pulseaudio pavucontrol

Then start pavucontrol, and in the output section, turn off the mute button. (I did try installing a couple of simpler mixers to see what was going on, but they would not even start for some reason. And in any case, in my experience, pulseaudio works well and I do not mind having it around.)

posted at: 01:10 | path: /Admin/virtualization/virtualBox | permanent link to this entry

Fri, 30 Dec 2011

/Admin/Monitoring/munin: Turn on the Apache Munin Plugins

You need to enable extended status on Apache. Assuming the status module is enabled, create another file /etc/apache2/conf.d/extendedStatus containing the following:

(Note that /etc/apache2/mods-enabled/status.conf contains some of the above, but may not be enough.)

Restart Apache, and now on the server itself, if you use a terminal-based web browser like w3m, for instance, this

w3m localhost/server-status

should show you Apache's status page. And now these plugins should work:

apache_accesses -> /usr/share/munin/plugins/apache_accesses
apache_processes -> /usr/share/munin/plugins/apache_processes
apache_volume -> /usr/share/munin/plugins/apache_volume

And finally, if due to some insoluble weirdness in your Apache configuration you cannot get it working, I have found (thanks to [1]) that putting the status stuff on another port can work, ie. /etc/apache2/mods-enabled/status.conf:

Then in the tree /usr/share/munin/plugins/apache_* files, replace

my @PORTS = exists $ENV{'ports'} ? split(' ', $ENV{'ports'}) : (80);

with

my @PORTS = (8001);

And hopefully you are good to go.

[1] http://serverfault.com/questions/200320/apache-server-status-403-at-non-standard-port

posted at: 03:04 | path: /Admin/Monitoring/munin | permanent link to this entry

Thu, 03 Nov 2011

/Admin/databases/MSSQL: A MS-SQL Server GUI for Linux

As it turns out there seem to be a number of candidates out there, none of them apparently packaged for Debian, most or all of them Java-based. I picked one that seemed to have a bit of history and advertising more then one developer: SQuirrel SQL[1]

Squirrel SQL is kind enough to at least provide an RPM for download, which is convertable to a .deb using the alien package, and thence installable with "dpkg -i", ie. (as root):

alien squirrel-sql-3.2.1-1.noarch.rpm
dpkg -i squirrel-sql_3.2.1-2_all.deb
/opt/SQuirreLSQLClient/squirrel-sql.sh

will install Squirrel, and finally start it up. Unfortunately this RPM seems to be missing all of its database drivers. I found one[2] on the MicroSoft site, where I downloaded sqljdbc_3.0.1301.101_enu.tar.gz and unpacked it in /opt/SQuirreLSQLClient/plugins/.

After this you will find two .jar files in /opt/SQuirreLSQLClient/plugins/sqljdbc_3.0/enu. In the Squirrel SQL GUI, go to Windows --> View Drivers, then right-click on "Microsoft MSSQL Server JDBC Driver" and select "Modify Driver". Then click on the "Extra Class Path" tab, and "Add" those two sqljdbc .jar files.

Click on OK, and "Microsoft MSSQL Server JDBC Driver" should now have a check mark beside it. Click on Windows --> View Aliases then the + button to add your server connection information, which should be quite straight-forward with the possible exception of the URL field: there you need to put real values in fields demarked by <...>.

[1] http://www.squirrelsql.org/
[2] http://www.microsoft.com/downloads/details.aspx?FamilyID=A737000D-68D0-4531-B65D-DA0F2A735707&displaylang=pt-br&displaylang=en

posted at: 02:23 | path: /Admin/databases/MSSQL | permanent link to this entry

Wed, 02 Nov 2011

/Admin/databases/MSSQL: Connecting to MS SQL Server from Linux

This reference was MOST helpful:[1]

apt-get install sqsh freetds-bin freetds-common

Edit /etc/freetds/freetds.conf and add something like this to the end:

[MServer]
        host = msuckserver
        port = 1433
        tds version = 8.0

Edit ~/.sqshrc to contain this:

\set username=msqlloginname
\set password=msqlloginpassword
\set database=msqldbname
\set style=vert

For the case of only one database, it seems to be sufficient to delete the above database=msqldbname line from ~/.sqshrc. And then connect as follows:

sqsh -SMServer

Figuring out what to do next is a bit tricky. This will get the list of tables:

sp_tables;
go

And this will get the table contents:

select * from tablename;
go

[1] http://www.foscode.com/connect-microsoftsql-server-from-linux/

posted at: 21:56 | path: /Admin/databases/MSSQL | permanent link to this entry

Wed, 26 Oct 2011

/Admin/commandLine: Find & process files & directories

Find files ending in ".db", in and below the current directory:

find . -name "*.db" -print | xargs /bin/ls -al
find . -name "*.db" -print | xargs /bin/rm -f

Files last modified more then thirty days ago, in a specified directory:

find /home/userid/trash_* -mtime +30 -type f -exec rm -rf {} \;

Remove empty directories:

find /path/to/base/directory -type d -empty -delete

Find files with a particular name:

find . -type f -name "*unison.tmp-bad" -exec ls -alht {} \; | less

Calculate the size of files found by "find":

find . -type f -name "*unison.tmp-bad" -exec ls -l {} \; | awk '{ s+=$5 } END { print s }'

posted at: 07:22 | path: /Admin/commandLine | permanent link to this entry

Sat, 01 Oct 2011

/Admin/databases/MySQL: Cheat Sheet for the MySQL Command Line

phpmyadmin is a very handy GUI front-end for MySQL administration, but if there is no web server installed on your MySQL server, it is non-trivial to get phpmyadmin talking to it from a remote host. Especially if your remote host is behind a dynamic IP. And of course, the command line is just plain good for you. ;-) Here are some other cheat sheets for reference[1][2][3]. And here is mine:

mysql -h hostname -u root -p

Create a database:

mysql> create database [databasename];

Delete a database:

mysql> drop database [databasename];

List all databases:

mysql> show databases;

Switch to a database:

mysql> use [db name];

List the tables in the db:

mysql> show tables;

See a table's field formats:

mysql> describe [table name];
mysql> show columns from [table name];

Delete a db:

mysql> drop database [database name];

Delete a table:

mysql> drop table [table name];

Show data in a table:

mysql> SELECT * FROM [table name];
mysql> SELECT * FROM [table name] limit 10;
mysql> SELECT * FROM [table name] limit 100,10;
mysql> SELECT * FROM [table name] order by [col name] limit 10;
mysql> SELECT * FROM [table name] order by [col name] desc limit 10;

Counting the number of rows in a table:

mysql> SELECT COUNT(*) FROM pet;
mysql> SELECT owner, COUNT(*) FROM pet GROUP BY owner;

Show rows where "field name" has the value "whatever":

mysql> SELECT * FROM [table name] WHERE [field name] = "whatever";
mysql> SELECT * FROM [table name] WHERE [field1] = "what1" AND [field2] = "what2";

Pattern Matching (% is wildcard for an arbitrary # of chars, _ is any single character):

mysql> SELECT * FROM table WHERE rec LIKE "blah%";
mysql> SELECT * FROM table WHERE rec like "_____";

To insert records into a table:

INSERT into backup_run_log (start_time) VALUE (NOW());

To get the index of the most recently inserted row:

SELECT MAX(backup_run_log_id) FROM backup_run_log;

To change the contents of a table:

mysql> UPDATE table_name SET field1 = 'x',field2 = 'y',field3 = 'z' where fieldname = 'user';

mysql> UPDATE tablename SET tablefield = replace(tablefield,"findstring","replacestring");

mysql> UPDATE `mos2_content` SET introtext = replace(introtext,"{p}","") WHERE `title` REGEXP '-0';

To delete a row, for eg. to delete a user record from MySQL:

mysql> DELETE FROM mysql.user WHERE user='username' and host='localhost';

Column operations:

Dumping databases / tables to a (backup) file:

mysqldump -u root -ppassword --opt >/tmp/alldatabases.sql

mysqldump -u username -ppassword --databases databasename >/tmp/databasename.sql

mysqldump -c -u username -ppassword databasename tablename > /tmp/databasename.tablename.sql

Restoring a database from a dump:

mysql> create database databasename;
mysql> quit;
mysql -ppassword databasename < dump_databasename.sql

[1] http://www.tech-geeks.org/contrib/mdrone/mysql-stuff/mysql-cheatsheet.html
[2] http://www.pantz.org/software/mysql/mysqlcommands.html
[3] http://www.nparikh.org/unix/mysql.php

posted at: 04:50 | path: /Admin/databases/MySQL | permanent link to this entry

Wed, 07 Sep 2011

/Admin/commandLine: Logging to the System Log

This is actually more of a scripting thing, but what is a bash script except a collection of command lines....

To send a bit of text to /var/log/syslog:

logger Hello logging world!!

If you wish the output of a particular line in the script to be logged:

mysql -V | logger

will send the MySQL version to syslog. If you wish to also capture errors (STDERR) output, this will do it:

mysql < /path/to/a/sql/script.sql 2>&1 | logger

will send both script and error output to syslog.

posted at: 04:55 | path: /Admin/commandLine | permanent link to this entry

Fri, 02 Sep 2011

/Admin/email/deliverability: You Want Your E-mail Delivered: Simple Steps

You want the e-mail your mail server sends out delivered to an inbox, not to a SPAM folder. And certainly not bounced. Here are the simple things that you can do:

FQDN: Make sure your e-mail server, when communicating with other e-mail servers, identifies itself with a Fully Qualified Domain Name (FQDN). For a postfix server, an easy way to do this is add a line like the following to /etc/postfix/main.cf:
myhostname = hostname.domain.com
Reverse DNS: You cannot set this in the DNS settings for your domain names. This is something special that must be done at a higher level. If available, your website host / server provider will probably offer this setting, not your DNS provider. Set the reverse DNS for the IP address of your mail server to the most important domain name whose e-mail your mail server is responsible for.
SPF: Sender Policy Framework (SPF) is a DNS TXT record that tells receiving e-mail servers what IP addresses to accept e-mail from for a given domain. Here[1] is some online documentation that gives a pretty good idea of what the SFP TXT DNS record should look like. This DNS record:
domain.com TXT "v=spf1 mx -all"
says that a server should only accept e-mail for domain.com from a server with the same IP address as domain.com's MX record. This DNS record:
domain.com TXT "v=spf1 a -all"
says that a server should only accept e-mail for domain.com from a server with the same IP address as domain.com's A record. This
domain.com TXT "v=spf1 mx a ip4:122.248.255.249 include:_spf.google.com -all"
directs a server to accept e-mail from this domain from several sources, including Google's mail servers[3].
dig +short domain.com txt
will display domain.com's current TXT record(s).

port25.com[2] provides an excellant facility for testing your setup. Send an e-mail from your e-mail server to check-auth@verifier.port25.com or check-auth2@verifier.port25.com and they will send you an e-mail back showing your e-mail's hostname and SPF status, among other things.

[1] http://www.openspf.org/SPF_Record_Syntax
[2] http://www.port25.com/domainkeys/
[3] https://www.google.com/support/a/bin/answer.py?hl=en&answer=178723

posted at: 01:58 | path: /Admin/email/deliverability | permanent link to this entry

Fri, 26 Aug 2011

/Admin/SSH-SSL: Some SSH Tricks

You can re-use existing SSH connections by adding this to ~/.ssh/config file:

Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r@%h:%p

Thereafter, after you login and open a connection to an SSH server, if you open another connection from another terminal, it will re-use the first connection. No need to login again, no need to wait for the connection to be re-negotiated. Over a slow, long-distance SSH connection, this can slow the wait time from tens of seconds to two seconds. This applies to SCP transfers as well. Much saved time.

If you have a long list of servers you log into regularly, particularly on non-standard ports, these connections can also be aliased in the ~/.ssh/config file:

Host my-chosen-alias
  Hostname server.com
  IdentityFile /path/to/id_rsa
  User root
  Port 10122

Thereafter, "ssh my-chosen-alias" will connect to root@server.com on port 10122. Unfortunately SCP does not seem to respect these aliases, or at least I have not found out how to make that work. But this is a way, for instance, to transparently get backuppc[1] to connect to a backup client using a non-standard port.

[1] http://blog.langex.net/index.cgi/Admin/backups/backuppc/

posted at: 05:22 | path: /Admin/SSH-SSL | permanent link to this entry

Fri, 29 Jul 2011

/Admin: Apt-Cacher-NG: Caching Downloaded .deb Packages

This is a polite thing to do if you have multiple machines running the same distribution on your network since it takes some of the stress off of the distribution's mirrors. It is also the right thing to do, as you do not waste bandwidth, and in places where bandwidth sucks, you also save a lot of time waiting for downloads.

I used to use apt-move for this, but it has recently stopped working, and is also orphaned. apt-cacher-ng[1] would appear to be a more elegant solution, and seems to work out of the box with the default configuration. All I did on the server side (where apt-cacher-ng is actually installed) is open port 3142 in the firewall, and add this line to /etc/apt/apt.conf:

Acquire::http { Proxy "http://127.0.0.1:3142"; };

which forces all apt network traffic to go to apt-cacher-ng on the designated port. On the client side, I added this line to /etc/apt/apt.conf:

Acquire::http { Proxy "http://lenovo:3142"; };

where "lenovo" is the name of my apt-cacher-ng server in /etc/hosts. With these two settings, all packages downloaded to either server or client or saved in the cache for future use.

Note that on the server if you point your browser at

http://localhost:3142/acng-report.html

you will find some cache statistics, and a function for cleaning stale files out of the cache.

[1] http://www.unix-ag.uni-kl.de/~bloch/acng/

posted at: 07:31 | path: /Admin | permanent link to this entry

Wed, 20 Jul 2011

/Admin/iptables: Build A Router With iptables

This[1] is a deeper reference, but it did not quite get the job done for me. (Nor did a lot of other recipes I looked at either, for that matter....) The "Example Scenario: SOHO" here[2] got me a working router.

First make sure forwarding is enabled in your router OS. The standard way to do this on Debian is to edit /etc/sysctl.conf to turn on net.ipv4.ip_forward. My machine is not a full-time router, so I added a

up echo 1 > /proc/sys/net/ipv4/ip_forward

line to the /etc/network/interfaces clause that brings up my internal LAN interface, ie.

iface static inet static
  address 10.1.1.1
  netmask 255.255.255.0
  network 10.1.1.0
  broadcast 10.1.1.255
  up echo 1 > /proc/sys/net/ipv4/ip_forward

Then I added these lines to my "basic firewall":

-A POSTROUTING -o eth0 -j MASQUERADE

-A INPUT -s 10.1.1.0/24 -i eth4 -m state --state NEW,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.1.1.0/24 -i eth4 -m state --state NEW,ESTABLISHED -j ACCEPT
-A FORWARD -d 10.1.1.0/24 -i eth0 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -d 10.1.1.0/24 -o eth4 -m state --state NEW,ESTABLISHED -j ACCEPT

where eth0 is the outward/WAN interface, eth4 is the inward/LAN interface, and 10.1.1.0/24 is the IP address block used on the LAN. Note that only ESTABLISHED, not NEW, connections are allowed to come in on eth0/WAN.

To configure DHCP[3] add this line to rules.v4:

-A INPUT -i eth4 -p udp -m udp --sport 67:68 --dport 67:68 -j ACCEPT

and

apt-get install dnsmasq

Just configure the dhcp-range in /etc/dnsmasq.conf, ie.

dhcp-range=10.1.1.50,10.1.1.150,12h

and it should be all ready to go.

[1] http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch14_:_Linux_Firewalls_Using_iptables#Masquerading_.28Many_to_One_NAT.29
[2] http://fedorasolved.org/Members/kanarip/iptables-howto
[3] http://www.faqs.org/docs/iptables/lettingdhcprequests.html

posted at: 22:52 | path: /Admin/iptables | permanent link to this entry

Tue, 21 Jun 2011

/Admin/iptables: Build A Basic Firewall with iptables

This seems like a good starting point[1], blocking everything except SSH, established, loopback, and outgoing connections:

iptables -P INPUT ACCEPT
iptables -F
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -L -v
/etc/init.d/iptables-persistent save

On my Debian system, the last "save" line puts a reloadable copy of the current running iptables rules in /etc/iptables/rules.v4 & /etc/iptables/rules.v6. Thereafter, it is also possible (advisable?) to edit these files directly to add/modify rules. For instance to open up the http port, add the following line to /etc/iptables/rules.v4:

-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

Then load the new ruleset using:

iptables-restore < rules.v4

[1] http://wiki.centos.org/HowTos/Network/IPTables

posted at: 04:56 | path: /Admin/iptables | permanent link to this entry

Home

Contact

Resumé / C.V.

Links

Categories:

Archives:

Subscribe XML RSS Feed