Clayton's Tech Bits

Home

Contact

Resumé / C.V.

Links

Search this site:
Custom Search

Categories:

/ (223)
  Admin/ (85)
    Apache/ (7)
      HTTPS-SSL/ (4)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (3)
    Monitoring/ (2)
      munin/ (2)
    OpenVPN/ (1)
    SSH-Proxy/ (3)
    SSH-SSL/ (6)
    backups/ (15)
      SpiderOak/ (1)
      backuppc/ (5)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (1)
    commandLine/ (11)
    crontab/ (1)
    databases/ (8)
      MSSQL/ (2)
      MySQL/ (5)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (9)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (6)
    iptables/ (2)
    virtualization/ (8)
      VMware/ (1)
      virtualBox/ (7)
  Coding/ (11)
    bash/ (1)
    gdb/ (1)
    git/ (2)
    php/ (4)
    python/ (3)
      Django/ (1)
  Education/ (1)
  Hosting/ (23)
    Amazon/ (14)
      EBS/ (3)
      EC2/ (11)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (20)
    Awesome/ (3)
    CPUfreq/ (1)
    Chinese/ (1)
    Debian/ (5)
      WPA/ (1)
    audio/ (1)
    encryption/ (2)
    fonts/ (1)
    misc/ (4)
    router-bridge/ (2)
  SW/ (39)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (25)
      Drupal/ (8)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (6)
      WebERP/ (2)
      eGroupware/ (1)
    email/ (1)
    fileSharing/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (12)
    IMchat/ (1)
    circumvention/ (2)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (1)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (13)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)

Archives:

  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Fri, 27 Jan 2012


    /Admin/SSH-SSL: Passwordless Authentication with SSH

    How to use SSH keys to login to your server without giving a password -- perhaps contra-inuitively, this kind of passwordless login is usually more, not less, secure then a password login. (Not to mention convenient and time-efficient....)

    Say we want to login to server.com from our desktop without a password. The rdiff-backup wiki provides a somewhat obtuse and hard-to-read article[1] on the subject. For the basics, I prefer to start with this article[2].

    On your desktop, run:

    ssh-keygen -b 4096 -t rsa -f /home/username/.ssh/id_rsa
    Do not enter a pass-phrase!! Leave it blank

    (Note: we are creating a 4096 bit key here as recommended by nearlyfreespeech.net[3]. It is possible that some situations will require a 1024 bit key, and this key will not be useable in that situation. It is possible to have multiple keys, which may be invoked by the "ssh -i" option, for instance.)

    Now copy the public key "id_rsa.pub" to root@server.com:

    scp /home/username/.ssh/id_rsa.pub root@server.com:

    Go to server.com and append the new key to the authorized_keys:

    ssh server.com
    cd /root/.ssh/
    cat ../id_rsa.pub >> authorized_keys

    Restrict access to these keys on both your desktop and your root@server:

    chmod -R go-rwx ~/.ssh

    Test to verify you are not prompted for a password. In a terminal on your desktop, try a verbose ssh to server.com:

    ssh -v root@server.com

    If there are problems and you are prompted for a password (you should not be) the -v output should give you some clues.

    In some situations, one can make SSH key logins even more secure. For instance, on server.com, add some security directives to a particular key in /root/.ssh/authorized_keys by pre-pending the following:

    command="rdiff-backup --server --restrict-read-only/",no-port-forwarding,no-X11-forwarding,no-pty
    ie. in /root/.ssh/authorized_keys the key in qestion now contains the following, ALL ON ONE LINE, and note the single space before "ssh-rsa":
    command="rdiff-backup --server --restrict-read-only /",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AA ... uqdswe= user@desktop

    "no-pty" explicitly forbids terminal priveleges. "command" here restricts the session to running one and only one command: "rdiff-backup --server --restrict-read-only".

    Now if you try to ssh to root@server.com from a terminal, your terminal will just lock-up and stop responding. If you really do want to allow this (a terminal ssh to root@server.com without a password) just remove the "command" and "no-pty" directives from the server.com /root/.ssh/authorized_keys file.

    Note that this will only work from the account "username" on your desktop machine where you have generated a /home/username/.ssh/id_rsa file and then passed the public key to server.com. Trying to ssh from any other desktop account to root@server.com will result in a password prompt.

    [1] http://wiki.rdiff-backup.org/wiki/index.php/UnattendedRdiff
    [2] http://linuxgazette.net/104/odonovan.html
    [3] https://members.nearlyfreespeech.net/ckoen/support/faq?q=SSHKeys#SSHKeys

    posted at: 06:51 | path: /Admin/SSH-SSL | permanent link to this entry

    Fri, 20 Jan 2012


    /Admin/OpenVPN: Basic OpenVPN as an Internet Gateway

    Aka. How to bore through the Great Firewall if you do not want to use an SSH tunnel.

    This[1] is the OpenVPN documentation, but it is not altogether straight-forward to read, and it is missing some necessary detail.

    This[2] will get OpenVPN basically working and connected for you. First create your keys:

    cd /usr/share/doc/openvpn/examples/easy-rsa/2.0 . ./vars ./clean-all ./build-ca ./build-key server ./build-key <client> ./build-dh

    Here I would add a warning from [1] that when creating your certificates above you must enter something at the "Common Name" prompt. My first time I just accepted all the defaults and got a connection error when I tried to start OpenVPN on my client. The second time, with a "Common Name" on the server certificates, everything just worked.

    Then distribute the keys to server and client(s) per the references.

    This will get the VPN client to the VPN server. However, we want to route *all* network traffic on the client through VPN and out to the internet. There are two components to this: some additional OpenVPN configuration, and some routing configuration on the server. On my VPN server, this is my /etc/openvpn/server.conf:

    port 1348
    proto udp
    dev tap
    ca ca.crt
    cert server.crt
    key server.key # This file should be kept secret
    dh dh1024.pem
    server 10.10.10.0 255.255.255.0 # vpn subnet
    ifconfig-pool-persist ipp.txt
    keepalive 10 120
    comp-lzo
    user nobody
    ; group nobody
    persist-key
    persist-tun
    verb 10
    mute 20
    client-to-client
    client-config-dir ccd "route 134.33.0.0 255.255.0.0"
    
    ; push "route 192.168.1.0 255.255.255.0" # home subnet
    push "redirect-gateway def1"
    push "dhcp-option DNS 10.10.10.1"
    

    This should be the same as in [2] except that "group nobody" line did not work on my Ubuntu Lucid server for some reason, and the last two "push" lines are what is needed on the server end to tell connecting clients to redirect all network traffic to the VPN. (Though I am not convinced that last dhcp line is having any effect on my Debian box at the moment....) Also, per [1], note that if your network interface is DHCP, it may die periodically because it is unable to communicate with your DHCP server.

    On my VPN client, this is my /etc/openvpn/client.conf:

    client
    dev tap
    proto udp
    resolv-retry infinite # this is necessary for DynDNS
    nobind
    user nobody
    ; group nobody
    persist-key
    persist-tun
    ca ca.crt
    cert x60s.crt
    key x60s.key
    comp-lzo
    verb 4
    mute 20
    

    which is the same as in [2], except for commenting out "group nobody", which did not work on my Debian Testing client machine.

    Now for routing on the server. What a PITA. I tried at some length to get raw iptables to do the job, but in the end turned to my old faithful, firehol. Here is my /etc/firehol/firehol.conf on my server, which enables the routing of traffic between tap0 (VPN) and eth0 (internet):

    version 5
    
    # interface eth0 internet
    interface eth0 internet
       protection strong 10/sec 10
       server "https http icmp ssh"  accept
       server openvpn accept
       server ident reject with tcp-reset
       client all   accept
    
    interface tap0 vpn
       server all   accept
       client all   accept
    
    router internet2vpn inface eth0 outface tap0
       masquerade reverse
       client all      accept
       server ident    reject with tcp-reset
    

    (After the fact, I ran into this[3] very interesting post....)

    [1] http://www.openvpn.net/index.php/open-source/documentation/howto.html
    [2] https://www.debian-administration.org/article/Connecting_to_office_network_using_OpenVPN_tunnel
    [3] http://www.hermann-uwe.de/blog/howto-using-openvpn-on-debian-gnu-linux

    posted at: 09:58 | path: /Admin/OpenVPN | permanent link to this entry


    /Linux/misc: Running and Installing Debian from a USB Stick

    So unbelievably easy[1]:

    cat debian.iso > /dev/sdX
    sync

    I grabbed the latest net install[2] ISO, did the above, popped it into my new laptop, hit F12 during boot to get the boot menu, and picked the USB option. Never had such a swift and painless install....

    Update:

    And then there was Ubuntu Lucid. Not so easy. The ISO will not boot from USB per the above. After a bit of flailing around, it seems the easiest way (and so far the only way) that has worked for me is to use usb-creator, which is packaged with Lucid. And only works with X, there is no console version.

    Yes, that means you need a running Lucid desktop to do this. Lame. The package that needs to be installed is usb-creator-gtk. Note that you can invoke it from the following menu:

    System --> Administration --> Startup Disk Creator

    but that did not work for me either, there were errors. What I did was installed the sux package, logged into root in a terminal by invoking sux, and then invoked usb-creator-gtk as root. Then it worked, and the USB stick booted.

    [1] http://www.debian.org/releases/stable/i386/ch04s03.html.en
    [2] http://www.debian.org/CD/netinst/

    posted at: 07:51 | path: /Linux/misc | permanent link to this entry

    Sun, 15 Jan 2012


    /Admin/virtualization/virtualBox: Debian Host and Guest

    At least on Debian, there is one thing that is constantly breaking in my VirtualBox (VB) installation: missing kernel headers. The symptom is, that after a new kernel comes in, VB suddenly will not work any longer. It seems particulary insidious on the guest Debian OS, because "all" that stops working are certain conveniences, like (what I was just sorely missing) the ability to copy and paste between host and guest OS. I found it very easy to assume that something was just broken, when in fact all that was missing was the kernel headers package.

    So once and for all, here is the drill for getting VB working properly on both host and guest....

    On the host, this is the list of modules I have installed:

    virtualbox
    virtualbox-dkms
    virtualbox-guest-additions
    virtualbox-guest-additions-iso
    virtualbox-ose
    virtualbox-ose-dkms
    virtualbox-ose-qt
    virtualbox-ose-source
    virtualbox-qt
    virtualbox-source
    

    On the host, if VB is not working and

    modprobe vboxdrv

    does not find your VB kernel driver, ie.

    FATAL: Module vboxdrv not found.

    you are almost certainly missing headers, ie. there should be a headers package installed that exactly matches your current kernel:

    linux-image-3.1.0-1-686-pae
    linux-headers-3.1.0-1-686-pae

    Install it and all should be well. On the Debian guest OS, things are only slightly different. Here are my current VB modules on the guest:

    virtualbox-guest-dkms
    virtualbox-guest-source
    virtualbox-guest-utils
    virtualbox-guest-x11
    virtualbox-ose-guest-source
    virtualbox-ose-guest-utils
    virtualbox-ose-guest-x11
    

    if you cannot copy and paste between host and guest (my recent symptom) and the vboxguest kernel module is missing, same drill: install your headers. Then:

    /etc/init.d/virtualbox-guest-utils start

    and now

    # ps -ef | grep -i vbox
    clayton     1641     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --clipboard
    clayton     1650     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --display
    clayton     1655     1  0 18:47 ?        00:00:00 /usr/bin/VBoxClient --seamless
    

    you can see all the things that were not working perfectly before. ("Clipboard" being the shared clipboard whose absence meant my copying and pasting was not working.)

    One final tweak to get sound working on the guest OS:

    apt-get install pulseaudio pavucontrol

    Then start pavucontrol, and in the output section, turn off the mute button. (I did try installing a couple of simpler mixers to see what was going on, but they would not even start for some reason. And in any case, in my experience, pulseaudio works well and I do not mind having it around.)

    posted at: 07:10 | path: /Admin/virtualization/virtualBox | permanent link to this entry

    Fri, 30 Dec 2011


    /Admin/Monitoring/munin: Turn on the Apache Munin Plugins

    You need to enable extended status on Apache. Assuming the status module is enabled, create another file /etc/apache2/conf.d/extendedStatus containing the following:

    ExtendedStatus On <Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from 127.0.0.1 localhost </Location>

    (Note that /etc/apache2/mods-enabled/status.conf contains some of the above, but may not be enough.)

    Restart Apache, and now on the server itself, if you use a terminal-based web browser like w3m, for instance, this

    w3m localhost/server-status

    should show you Apache's status page. And now these plugins should work:

    apache_accesses -> /usr/share/munin/plugins/apache_accesses
    apache_processes -> /usr/share/munin/plugins/apache_processes
    apache_volume -> /usr/share/munin/plugins/apache_volume

    And finally, if due to some insoluble weirdness in your Apache configuration you cannot get it working, I have found (thanks to [1]) that putting the status stuff on another port can work, ie. /etc/apache2/mods-enabled/status.conf:

    <IfModule mod_status.c> # # Allow server status reports generated by mod_status, # with the URL of http://servername/server-status # Uncomment and change the ".example.com" to allow # access from other hosts. # Listen 8001 ExtendedStatus On <VirtualHost *:8001> <Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from localhost ip6-localhost # Allow from .example.com </Location> </VirtualHost> </IfModule>

    Then in the tree /usr/share/munin/plugins/apache_* files, replace

    my @PORTS = exists $ENV{'ports'} ? split(' ', $ENV{'ports'}) : (80);

    with

    my @PORTS = (8001);

    And hopefully you are good to go.

    [1] http://serverfault.com/questions/200320/apache-server-status-403-at-non-standard-port

    posted at: 09:04 | path: /Admin/Monitoring/munin | permanent link to this entry

    Fri, 16 Dec 2011


    /SW/graphics: Fixing Simple Photo Issues at the Command Line

    (Without having to startup a huge program like gimp or photoshop.)

    These utilities are also drawn from the ImageMagick[1] graphics suite. Imagemagick is available for all of Linux, Windows, and the Mac environment[2]. Some examples:

    I have not yet found an Imagemagick manual that has the gift of making difficult tasks seem easy. Here[3][4][5] are some suggestions for manuals.

    [1] http://www.imagemagick.org/script/index.php
    [2] http://www.imagemagick.org/script/binary-releases.php
    [3] http://www.imagemagick.org/script/command-line-tools.php
    [4] http://linux.about.com/library/cmd/blcmdl1_ImageMagick.htm
    [5] http://www.imagemagick.org/Usage/

    posted at: 09:22 | path: /SW/graphics | permanent link to this entry

    Thu, 15 Dec 2011


    /Linux/CPUfreq: Manual CPU Frequency Control in Linux

    In comes Gnome 3, out goes my beloved (out of necessity) CPU frequency applet. It seems Gnome 3 does not do applets, at least not yet. And I have a couple of miscreant laptops that love to overheat under load. One Lenovo is actually almost unusable unless I throttle back the CPU before extended heavy loads like a massive apt-get upgrade.

    [1] clued me in to a command-line option to the applet in the cpufreqd package. After installing cpufreqd one must first uncomment the following lines in /etc/cpufreqd.conf:

    enable_remote=1
    remote_group=root

    which basically enables the command-line tools. Now

    cpufreqd-get

    will list your options, and

    cpufreqd-set <n>

    is how you choose your CPU speed, and

    cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies

    displays the available frequencies, and

    cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq

    shows the current frequency in effect. And now I can install Gnome 3 on these handicapped machines....

    [1] http://www.go2linux.org/how-to-configure-cpufreqd

    posted at: 14:04 | path: /Linux/CPUfreq | permanent link to this entry

    Sun, 06 Nov 2011


    /Linux/Chinese: Chinese Input with IBUS

    Until recently I was a long time user of SCIM, and then, at around the same time I noticed that the Ubuntu Desktop default was IBUS[3], and that SCIM was on it's way out.

    Getting SCIM to work has always been somewhat dodgy, and I am happy to say that IBUS is less so.

    I recently discovered that there is a native Linux QQ chat client[1], but published by a Chinese company and apparently(?) not Open Source. For a long time I have run Skype on my machine despite the fact that it is not Open Source, and it's bad reputation for security[2]. But an application published by a Government-controlled Chinese company is more then I can tolerate, so I am running my QQ in a Debian VM, where I also eventually intend to exile Skype and others of its ilk.

    This is an absolutely minimal VM running only fluxbox. To get Simplified Chinese input going I:

    apt-get install ibus ibus-pinyin ttf-arphic-ukai
    Then I ran im-config as root to select IBUS as the system master input method. And then ibus-setup as my normal user to turn on the "Chinese Pinyin" input method. I believe I restarted X somewhere along the way, and after that Ctrl-Space had me typing Chinese.

    [1] http://im.qq.com/qq/linux/
    [2] http://blog.langex.net/index.cgi/Security/skype
    [3] http://code.google.com/p/ibus/

    posted at: 16:14 | path: /Linux/Chinese | permanent link to this entry


    /Hosting/Amazon/EC2: Increase EBS Root Volume Size of an EC2 Server (Complicated)

    Complicated because this is a RightScale image with multiple partitions, so resize2fs did not work the first time per[1].

    Pick the image you want to resize (this one has an 8G root volume):

    $ ec2-describe-images ami-e00df089
    IMAGE ami-e00df089 944964708905/rightimage_debian_6.0.1_amd64_20110406.1_ebs 944964708905 available public x86_64 machine aki-4e7d9527 ebs paravirtual xen
    BLOCKDEVICEMAPPING  /dev/sda   snap-b62f31da   8
    

    Start a server up with a 25G volume instead:

    $ ec2-run-instances -t t1.micro --key clayton --block-device-mapping /dev/sda=:25 ami-e00df089

    Log in and see (in part):

    # df -h
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/xvda2            5.0G  1.2G  3.6G  24% /
    

    Here is where things get funky, as "resize2fs /dev/xvda2" will not work per [1] because there are two other partitions: xvda1 is /boot and xvda3 is swap. [2] to the rescue. Hoping that it is the swap partition that is in the way, I got rid of it and rebuilt the partition table as follows: deleting partitions 2 and 3, creating a new partition 2 (accepting the defaults) and then writing the new partition to disk(w):

    # fdisk /dev/xvda
    d 2
    d 3
    n p 2
    w
    

    Do not forget to remove the swap line from /etc/fstab, and reboot. Now:

    # resize2fs /dev/xvda2

    works! And all is well. Now, as we have done before, create a new image to save our work:

    ec2-create-snapshot vol-e41af089
    SNAPSHOT snap-697ad00b vol-e41af089 pending

    Once the snapshot is finished:

    ec2-register -a x86_64 -b '/dev/sda=snap-697ad00b:25:false' -n 'Squeeze_64' -d '64 bit Squeeze' --kernel="aki-4e7d9527"

    [1] http://alestic.com/2009/12/ec2-ebs-boot-resize
    [2] http://bioteam.net/2010/07/how-to-resize-an-amazon-ec2-ami-when-boot-disk-is-on-ebs/

    posted at: 05:10 | path: /Hosting/Amazon/EC2 | permanent link to this entry

    Thu, 03 Nov 2011


    /Admin/databases/MSSQL: A MS-SQL Server GUI for Linux

    As it turns out there seem to be a number of candidates out there, none of them apparently packaged for Debian, most or all of them Java-based. I picked one that seemed to have a bit of history and advertising more then one developer: SQuirrel SQL[1]

    Squirrel SQL is kind enough to at least provide an RPM for download, which is convertable to a .deb using the alien package, and thence installable with "dpkg -i", ie. (as root):

    alien squirrel-sql-3.2.1-1.noarch.rpm
    dpkg -i squirrel-sql_3.2.1-2_all.deb
    /opt/SQuirreLSQLClient/squirrel-sql.sh

    will install Squirrel, and finally start it up. Unfortunately this RPM seems to be missing all of its database drivers. I found one[2] on the MicroSoft site, where I downloaded sqljdbc_3.0.1301.101_enu.tar.gz and unpacked it in /opt/SQuirreLSQLClient/plugins/.

    After this you will find two .jar files in /opt/SQuirreLSQLClient/plugins/sqljdbc_3.0/enu. In the Squirrel SQL GUI, go to Windows --> View Drivers, then right-click on "Microsoft MSSQL Server JDBC Driver" and select "Modify Driver". Then click on the "Extra Class Path" tab, and "Add" those two sqljdbc .jar files.

    Click on OK, and "Microsoft MSSQL Server JDBC Driver" should now have a check mark beside it. Click on Windows --> View Aliases then the + button to add your server connection information, which should be quite straight-forward with the possible exception of the URL field: there you need to put real values in fields demarked by <...>.

    [1] http://www.squirrelsql.org/
    [2] http://www.microsoft.com/downloads/details.aspx?FamilyID=A737000D-68D0-4531-B65D-DA0F2A735707&displaylang=pt-br&displaylang=en

    posted at: 07:23 | path: /Admin/databases/MSSQL | permanent link to this entry


    /Admin/databases/MSSQL: Connecting to MS SQL Server from Linux

    This reference was MOST helpful:[1]

    apt-get install sqsh freetds-bin freetds-common

    Edit /etc/freetds/freetds.conf and add something like this to the end:

    [MServer]
            host = msuckserver
            port = 1433
            tds version = 8.0
    

    Edit ~/.sqshrc to contain this:

    \set username=msqlloginname
    \set password=msqlloginpassword
    \set database=msqldbname
    \set style=vert
    

    For the case of only one database, it seems to be sufficient to delete the above database=msqldbname line from ~/.sqshrc. And then connect as follows:

    sqsh -SMServer
    Figuring out what to do next is a bit tricky. This will get the list of tables:
    sp_tables;
    go
    And this will get the table contents:
    select * from tablename;
    go

    [1] http://www.foscode.com/connect-microsoftsql-server-from-linux/

    posted at: 02:56 | path: /Admin/databases/MSSQL | permanent link to this entry

    Wed, 02 Nov 2011


    /Hosting/Amazon/EC2: Sources of Official Amazon Images

    I seem to always be looking this stuff up. Time to right it down.

    Debian:

    http://wiki.debian.org/Cloud/AmazonEC2Image
    http://support.rightscale.com/21-Community/RightScale_OSS

    Ubuntu:

    http://uec-images.ubuntu.com/
    http://support.rightscale.com/21-Community/RightScale_OSS

    CentOS:

    http://support.rightscale.com/21-Community/RightScale_OSS

    posted at: 13:12 | path: /Hosting/Amazon/EC2 | permanent link to this entry


    /Hosting/Amazon/EC2: Accidental Server Termination Protection

    Especially if you work at the command line a lot, it seems frighteningly easy to accidentally terminate the wrong server. Not any more:

    For termination prevention[1]:

    ec2-modify-instance-attribute i-57e64936 --disable-api-termination true

    And to re-enable termination:

    ec2-modify-instance-attribute i-57e64936 --disable-api-termination false

    Note that a termination-protected server can still be stopped and started, it is just the totally destructive "termination" that is locked out.

    [1] http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?Using_ChangingDisableAPITermination.html

    posted at: 11:49 | path: /Hosting/Amazon/EC2 | permanent link to this entry

    Wed, 26 Oct 2011


    /Admin/commandLine: Find & process files & directories

    Find files ending in ".db", in and below the current directory:

    find . -name "*.db" -print | xargs /bin/ls -al
    find . -name "*.db" -print | xargs /bin/rm -f

    Files last modified more then thirty days ago, in a specified directory:

    find /home/userid/trash_* -mtime +30 -type f -exec rm -rf {} \;

    Remove empty directories:

    find /path/to/base/directory -type d -empty -delete

    Find files with a particular name:

    find . -type f -name "*unison.tmp-bad" -exec ls -alht {} \; | less

    Calculate the size of files found by "find":

    find . -type f -name "*unison.tmp-bad" -exec ls -l {} \; | awk '{ s+=$5 } END { print s }'

    posted at: 12:22 | path: /Admin/commandLine | permanent link to this entry

    Tue, 25 Oct 2011


    /SW/business/Drupal: Drupal Migrate Module Basics

    The Drupal migrate module[1] is an impressive tool for migrating databases from other web applications (or anywhere, really) into the Drupal database. It even has fairly impressive documentation[2], but not unlike a lot of other big chunks of documentation, it can be hard to extract from the documentation where to start in creating a simple functional example of the migrate module at work. Hopefully this example will help to fill that gap a little.

    First install and enable the migrate family of modules in a sandbox Drupal site:

    drush dl migrate migrate_example migrate_extras migrate_ui
    drush en migrate migrate_example migrate_extras migrate_ui

    If you then point your browser at:

    drupal/?q=admin/content/migrate

    you will see the migrate module's UI, which shows a couple of example migrations ("beer" and "wine") already coded and setup for you by the migrate_example sub-module. You can play with importing and rolling back data with the UI, or with drush:

    drush help --filter="migrate"

    To create your own migration, you need to create a custom module containing the migrate code. Using a MySQL database from another application, and based upon the beer.inc example from migrate_example, I was able to get a working migrate setup with three fairly small files in a new sub-module called "migrate_test" in sites/all/modules/migrate/migrate_test/ :

    sites/all/modules/migrate/migrate_test/migrate_test.module:

    <?php /* * You must implement hook_migrate_api(), setting the API level to 2, for * your migration classes to be recognized by the Migrate module. */ function migrate_test_migrate_api() { $api = array( 'api' => 2, ); return $api; }

    sites/all/modules/migrate/migrate_test/migrate_test.info:

    name = "Migrate Test" description = "My migration data." package = "Development" core = 6.x php = 5.2 dependencies[] = taxonomy dependencies[] = imagefield dependencies[] = comment dependencies[] = migrate dependencies[] = content dependencies[] = date dependencies[] = migrate_extras files[] = migrate_test.module files[] = langex.inc ; Information added by drupal.org packaging script on 2011-09-18 version = "6.x-2.2" core = "6.x" project = "migrate" datestamp = "1316388105"

    sites/all/modules/migrate/migrate_test/langex.inc:

    <?php // This is the connection information for the database containing the source data. Database::addConnectionInfo('for_migration', 'default', array( 'driver' => 'mysql', 'database' => 'langex', 'username' => 'langex', 'password' => 'anypass', 'host' => 'localhost', 'prefix' => '', )); /** * To define a migration process from a set of source data to a particular * kind of Drupal object (for example, a specific node type), you define * a class derived from Migration. You must define a constructor to initialize * your migration object. By default, your class name will be the "machine name" * of the migration, by which you refer to it. Note that the machine name is * case-sensitive. * * In any serious migration project, you will find there are some options * which are common to the individual migrations you're implementing. You can * define an abstract intermediate class derived from Migration, then derive your * individual migrations from that, to share settings, utility functions, etc. */ abstract class LangexMigration extends Migration { public function __construct() { // Always call the parent constructor first for basic setup parent::__construct(); // With migrate_ui enabled, migration pages will indicate people involved in // the particular migration, with their role and contact info. We default the // list in the shared class; it can be overridden for specific migrations. $this->team = array( new MigrateTeamMember('John Doe', 'john.doe@gmail.com', t('contractor')), new MigrateTeamMember('Larry Brewer', 'lbrewer@example.com', t('Implementor')), ); } } /** * There are four essential components to set up in your constructor: * $this->source - An instance of a class derived from MigrateSource, this * will feed data to the migration. * $this->destination - An instance of a class derived from MigrateDestination, * this will receive data that originated from the source and has been mapped * by the Migration class, and create Drupal objects. * $this->map - An instance of a class derived from MigrateMap, this will keep * track of which source items have been imported and what destination objects * they map to. * Mappings - Use $this->addFieldMapping to tell the Migration class what source * fields correspond to what destination fields, and additional information * associated with the mappings. */ class LangexUserMigration extends LangexMigration { public function __construct() { // The basic setup is similar to BeerTermMigraiton parent::__construct(); $this->description = t('Language Exchange Network users'); $this->map = new MigrateSQLMap($this->machineName, array('userid' => array( 'type' => 'int', 'not null' => TRUE, 'description' => 'Account ID.' ) ), MigrateDestinationUser::getKeySchema() ); $query = Database::getConnection('default', 'for_migration') ->select('users', 'u') ->fields('u', array('userid', 'name', 'username', 'password', 'email_address', 'sex', 'signup_date')); // $this->source = new MigrateSourceSQL($query); $this->source = new MigrateSourceSQL($query, array(), NULL, array('map_joinable' => FALSE)); $this->destination = new MigrateDestinationUser(); // One good way to organize your mappings is in three groups - mapped fields, // unmapped source fields, and unmapped destination fields // Mapped fields // The migrate module automatically converts date/time strings to UNIX timestamps. $this->addFieldMapping('created', 'signup_date'); $this->addFieldMapping('pass', 'password'); $this->addFieldMapping('mail', 'email_address'); $this->addFieldMapping('name', 'username'); // Unmapped source fields $this->addFieldMapping(NULL, 'name') ->issueGroup(t('DNM')); $this->addFieldMapping(NULL, 'sex') ->issueGroup(t('DNM')); // Unmapped destination fields // This is a shortcut you can use to mark several destination fields as DNM // at once $this->addUnmigratedDestinations(array('theme', 'signature', 'access', 'login', 'timezone', 'language', 'picture')); } }

    Note that this is a "cross-database migration"[3]. The source data and the destination data are located on the same MySQL server, but in different databases.

    [1] https://drupal.org/project/migrate
    [2] https://drupal.org/node/415260
    [3] https://drupal.org/node/1014558

    posted at: 09:13 | path: /SW/business/Drupal | permanent link to this entry