My personal notes on all things technical and factual....

Fri, 16 Dec 2011

/SW/graphics: Fixing Simple Photo Issues at the Command Line

(Without having to startup a huge program like gimp or photoshop.)

These utilities are also drawn from the ImageMagick[1] graphics suite. Imagemagick is available for all of Linux, Windows, and the Mac environment[2]. Some examples:

Rotation: Do you sometimes turn your camera 90 degrees to get that "tall" shot? Rotate the photo back clockwise with the convert command:
convert -rotate 90 image1.jpg image2.jpg
Resize: How about a whole directory full of photos that you want to reduce in size, without having to edit every photo to do it? Again use convert to do the job, this time in a little Unix shell script. First create the directory "convert-folder" in the same directory as the one containing your "big" photo directory. Then cd into the "big" photo directory and run this scrip:
for i in *.jpg ; do convert $i -resize 800x800 ../convert-folder/$i ; done

This may take a while if you have a lot of photos. In this example, photos will be resized such that they have a maximum width or heighth of 800 pixels, with no change in aspect ratio.
mogrify: resize while preserving aspect ratio. 600x500 are therefore maximum dimensions of the result. Use of the quality option with a value less then 100 reduces the size of the resulting file a little:
mogrify -resize 600x500 -quality 70 Gabe.jpg

Note that the key difference between mogrify and convert are that morgify over-writes the original file, and convert writes to a different file.
format: file format can also be changed:
convert input.jpg output.png
Annotate: an image with some text:
# convert -font helvetica -fill white -pointsize 36 \
-draw 'text 10,50 "This is the annotation text"' \
floriade.jpg comment.jpg
Thumbnailing: all the JPEGs in the current directory (note that this is a Unix shell script):
for img in *.jpg ; do convert -sample 25%x25% $img thumb-$img ; done
Sub-Directories: shrinking a bunch of jpg files spread out through a number of sub-directories:
nice ionice -c3 find . -type f -name "*.jpg" -exec mogrify -resize 200x200 -quality 70 {} \;
(Note that "nice" and "ionice" lower the priority of the operations' claim on CPU and disk resources, respectively. Especially useful for a production server.)

I have not yet found an Imagemagick manual that has the gift of making difficult tasks seem easy. Here[3][4][5] are some suggestions for manuals.

[1] http://www.imagemagick.org/script/index.php
[2] http://www.imagemagick.org/script/binary-releases.php
[3] http://www.imagemagick.org/script/command-line-tools.php
[4] http://linux.about.com/library/cmd/blcmdl1_ImageMagick.htm
[5] http://www.imagemagick.org/Usage/

posted at: 09:22 | path: /SW/graphics | permanent link to this entry

Tue, 25 Oct 2011

/SW/business/Drupal: Drupal Migrate Module Basics

The Drupal migrate module[1] is an impressive tool for migrating databases from other web applications (or anywhere, really) into the Drupal database. It even has fairly impressive documentation[2], but not unlike a lot of other big chunks of documentation, it can be hard to extract from the documentation where to start in creating a simple functional example of the migrate module at work. Hopefully this example will help to fill that gap a little.

First install and enable the migrate family of modules in a sandbox Drupal site:

drush dl migrate migrate_example migrate_extras migrate_ui
drush en migrate migrate_example migrate_extras migrate_ui

If you then point your browser at:

drupal/?q=admin/content/migrate

you will see the migrate module's UI, which shows a couple of example migrations ("beer" and "wine") already coded and setup for you by the migrate_example sub-module. You can play with importing and rolling back data with the UI, or with drush:

drush help --filter="migrate"

To create your own migration, you need to create a custom module containing the migrate code. Using a MySQL database from another application, and based upon the beer.inc example from migrate_example, I was able to get a working migrate setup with three fairly small files in a new sub-module called "migrate_test" in sites/all/modules/migrate/migrate_test/ :

sites/all/modules/migrate/migrate_test/migrate_test.module:

sites/all/modules/migrate/migrate_test/migrate_test.info:

sites/all/modules/migrate/migrate_test/langex.inc:

Note that this is a "cross-database migration"[3]. The source data and the destination data are located on the same MySQL server, but in different databases.

[1] https://drupal.org/project/migrate
[2] https://drupal.org/node/415260
[3] https://drupal.org/node/1014558

posted at: 09:13 | path: /SW/business/Drupal | permanent link to this entry

Sun, 18 Jul 2010

/SW/business/Drupal: Youku & Tudou Videos in Drupal

For websites targeting users in mainland China, one of the issues is that many common services like Youtube and Google Video are blocked / censored from within China. For this specific case of video, there are a number of similar local Chinese video services that can be used instead.

Most ex-China video services seem to make it easy to embed their videos in other sites by prominently displaying a button which will generate the javascript code necessary for embedding a video in another site. For whatever reason, the Chinese sites do not seem to be so helpful.

First of all, in Drupal, turn on the PHP module. Then select "Create Content" and some content type to get to the content creation page. Select "PHP code" in the "Input Format" sub-section. And finally, for a youku[1] video, paste the following code into the content area:

This is the video generated by the above:

Youku video links are of the form http://v.youku.com/v_show/id_XMTkwNDgzMjE2.html

Note the part between the "id_" and the ".html", ie. XMTkwNDgzMjE2. This is the part of the link that must be swapped into the above code for XNDg5MTMwMDQ in order to make it work for a different video.

Similarly, for tudou[2] videos, use this code:

which produces this video:

Use a Flash Player to play Video from TUDOU - http://sites.google.com/site/annuairevin/flash-reader

Tudou video links are of the form http://www.tudou.com/programs/view/eD1xQ0WTlI8/

Swap "eD1xQ0WTlI8" in the latter link for "rsOg3YPXjDg" in the code above to make the code work for a different video.

[1] http://www.youku.com/
[2] http://www.tudou.com/

posted at: 07:02 | path: /SW/business/Drupal | permanent link to this entry

Sun, 30 May 2010

/SW/business/Redmine: Installing Redmine on Debian Linux

I have found Redmine to be extremely sensitive to the Ruby environment, often requiring that librairies be of exactly a certain version (not older, not newer). Configuration documentation is not sufficiently detailed. And error and diagnostic information is extremely difficult to obtain in a typical hosting account.

So if you are trying to install Redmine in a hosting account that supports Ruby on Rails, you would be well-advised to get the application running first in a controlled environment where you have root. In my case, that was my Debian Linux desktop. After flailing around for hours in the hosting account, everything simply fell into place when I went back to basics and installed the Redmine application locally.

These[1][2] were the main references for getting this working on Debian.

apt-get install ruby rake rubygems libmysql-ruby
gem install rails -v=2.3.5 (takes quite a while!)
gem install rake -v=0.8.3
gem install hoe -v=1.3.0

Checkout a fresh copy of the latest Redmine stable, currently:

svn co svn://rubyforge.org/var/svn/redmine/branches/0.9-stable redmine-0.9

Start the native Ruby web server and see what happens:

cd [...]/redmine-0.9
ruby script/server production

At this point I got a complaint about a missing database.yml. In my case, I was actually migrating Redmine from one hosting account where it suddenly broke, to another hosting account. So I installed the MySQL database from the old account to a database called mentage_redmine, then created a config/database.yml as follows:

production:
  adapter: mysql
  database: mentage_redmine
  host: localhost
  username: mentage_redmine
  password: ***********
  encoding: utf8
 
development:
  adapter: mysql
  database: mentage_redmine
  host: localhost
  username: mentage_redmine
  password: ***********
  encoding: utf8

Restart the server: this time an "Internal Server Error" that gave very specific instructions about adding a line pertaining to cookies to the config/environment.rb file. After adding this line, it basically worked.

In my particular case, again because I was migrating an existing application, in the new site I replaced the public/images, javascripts, stylesheets, and themes directories with those from the old site's public directory. After that, everything looked right as well.

[1] http://www.redmine.org/wiki/1/RedmineInstall
[2] http://www.redmine.org/wiki/1/HowTo_Install_Redmine_in_a_home_directory_on_Debian

posted at: 11:30 | path: /SW/business/Redmine | permanent link to this entry

Sat, 29 May 2010

/SW/business/Redmine: Installing Redmine in a Hostgator Hosting Account

To their credit, Hostgator[1] actually provides some documentation[2] about getting Ruby applications working in their hosting account. I, however, found them to be inadequate, and debugging information availability was little to none, so for me the magic key was getting Redmine working first on my desktop, then migrating to the Hostgator hosting account.

One of the first things one notices when looking at the rather sparse Redmine installation manual[3] is the need for specific versions of a number of Ruby librairies. This is what I found:

My Debian Desktop Hostgator Account

rails (2.3.5) rails (2.3.8)

rack (1.0.1) rack (1.1.0)

rake (0.8.3) rake (0.8.7)

hoe (1.3.0) hoe (2.6.0)

In the Hostgator account, I got the above librairy versions using "gem list --local". I then installed in the account the correct versions as follows:

gem install rails -v=2.3.5
gem install hoe -v=1.3.0

Now transfer Redmine from desktop to Hostgator, placing it in /home/mentage/redmine-0.9.

In /home/mentage/public_html, create the following symlink: redmine -> /home/mentage/redmine-0.9/public/

Hostgator is using Apache to server up Ruby applications. Create a public/.htaccess[2] file with the following contents:

AddHandler fcgid-script .fcgi 
     AddHandler cgi-script .cgi 
     Options +FollowSymLinks +ExecCGI 
  
     RewriteEngine On 
     RewriteCond %{REQUEST_URI} ^/notrails.* 
     RewriteRule .* - [L] 
  
     RewriteRule ^$ index.html [QSA] 
     RewriteRule ^([^.]+)$ $1.html [QSA]     
     RewriteCond %{REQUEST_FILENAME} !-f 
     RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

After that, it worked for me by pointing a browser at the redmine directory.

[1] http://www.hostgator.com/
[2] http://forums.hostgator.com/ruby-rails-support-t13038p8.html
[3] http://www.redmine.org/wiki/1/RedmineInstall

posted at: 13:08 | path: /SW/business/Redmine | permanent link to this entry

Mon, 11 Jan 2010

/SW/business/Drupal: My Favorite Drupal Modules

(As of Drupal version 6.x....)

Drupal is a very mature piece of software, with a vast number of modules to choose from, often with multiple modules vying to provide the same piece of functionality. After some trial and error, here are my candidates for "best of", modules someone new to Drupal should look into early:

First some easily overlooked core modules:

PHP filter: useful even if you do not know what PHP is (the language that Drupal happens to be coded in). There are a number of modules out there promising to "make it easy" to incorporate things like Youtube videos and Google Adsense ads into your site. You do not need those modules, because all you need to do to make this happen is to copy a bit of Javascript from the source site into the editing page for the node or block where you want the ad or video to appear. Generally speaking, Drupal will ignore user-entered Javascript, unless you turn on "PHP code" in the "Input format" section.
Taxonomy: If you need to organize content into categories, this is where you go.
Trigger: On the admin page there is an item called "actions" where you can define things you want your site to do automatically. And then under "trigger", you can specify the criteria for when that action will be taken. For example, this is the mechanism by which my sites send me an e-mail when a new comment has been posted and I need to take care of the moderation queue.

Third-Party modules:

Content Construction Kit (CCK)[1]: one of the modules that truly sets Drupal apart from the competition. The basic function of CCK is to allow the user, via the Drupal GUI, to create custom databases within Drupal. Which is to say, you can add custom fields to any content, and control read/write access on a per-field basis.
Views[2]: Views and CCK usually travel in a pair, since Views allows the user (via Drupal GUI) to construct and display custom queries of almost any data in the Drupal database. As a very simple example, Drupal does not come equipped with a comment RSS feed, and one can easily create such a feed with Views.
Image[3] / Image FUpload[4]: the image module seems to be the standard way to add photo upload and management to a Drupal site. The image module supports bulk uploads via FTP. If you wish to do bulk upload multiple photos using the Drupal site, add the "Image FUpload" module.
Lightbox2[5]: drop this module in for elegant AJAX "lightbox" displays of photos and galleries. ("Lightbox" refers to a standard effect where, if you click on a photo in a gallery, the whole page fades to a dark gray background while a white box pops up in the middle to display the photo. Delete the box and the background page brightens and again becomes the foreground.)
Boost[7]: As a web application Drupal can sometimes be a bit of a resource hog, particularly if there are a lot of modules installed. If the host server is a bit light on memory, the result can be sluggishness. Boost is very simple in concept: pages referenced by anonymous users are stored in an Apache cache, such that when cached pages are served up Drupal is taken completely out of the picture: no PHP or MySQL overhead, your website performs like a static site. Until the cache expires, of course, and pages have to be regenerated once per caching period.

Here is someone else's list of favorites[6].

[1] http://drupal.org/project/cck
[2] http://drupal.org/project/views
[3] http://drupal.org/project/image
[4] http://drupal.org/project/image_fupload
[5] http://drupal.org/project/lightbox2
[6] http://www.nicklewis.org/40-essential-drupal-6-modules
[7] http://drupal.org/project/boost

posted at: 03:40 | path: /SW/business/Drupal | permanent link to this entry

Thu, 17 Dec 2009

/SW/business/Drupal: Comparisons of Drupal and Joomla

Back some time ago when I was trying to figure out which Content Management System to use for a couple of website development clients, I do not remember seeing any definitive reviews. In the end, I think I made the correct choice (Drupal) for the situation despite the limited amount of information, but I see now that there might be some situations (event calendaring? desire for a simpler admin interface?) where Joomla might be the right choice.

Here are some references I would like to preserve for posterity:

http://www.alledia.com/blog/general-cms-issues/joomla-and-drupal-version-2/
http://www.topnotchthemes.com/blog/090224/drupal-vs-joomla-frank-comparison-ibm-consultant
http://www.communicopia.com/blog/joomla-versus-drupal
http://www.goodwebpractices.com/other/wordpress-vs-joomla-vs-drupal.html

posted at: 13:16 | path: /SW/business/Drupal | permanent link to this entry

Mon, 07 Dec 2009

/SW/business/Drupal: Drupal User Manual: How to Edit Content

This post aims to provide the basic knowledge required to begin editing Drupal content.

First and most basically, you require an account on the Drupal website you wish to edit. After login, if you click on any menu item to navigate to that page, if you have the privileges to edit that page, you will see an edit (编辑) button or link (this depends on the theme being used) at the top of the page. Click on edit / 编辑, then edit and save the page.

After you have logged in, if you have been given some admin privileges on the site, you will see an Admin (管理) link in your personal menu under "My account" (我的帐户). Another way to see and edit the pages on the site is to click on "My account" / 我的帐户, and then click on "Content management/Content" (内容管理/内容). You will then be presented with a list of all the content items on the site, which you may then choose to edit individually, or perform various bulk operations on (deleted large quantities of spam posts, for instance).

To create new content, click on "Create content" (创建内容) and select a content type. For ordinary website page content, you probably want to choose "Page" content. After you click on "Page" and write your content, do not forget to expand the "Menu settings" (菜单设置) and enter a "Menu link title" (菜单链接名称) and select the menu position from the "Parent item" (上级菜单项) dropdown list. (You probably want to select "Primary Links" or one of the items under "Primary Links".)

Finally, you will probably want to re-arrange the order of the "Primary Links" in the navigation menu at the top of the site. To do this, click on "Admin" --> "Site building/Menus" --> "Primary links" (管理 --> 站点构建/菜单 --> "Primary links"). To change the order or position of menu items, just grab the "+" with your mouse pointer and drag'n'drop. For dropdown menus it is also good practice to click the "Expanded" (展开) checkbox to enable submenus. And finally, click "Save Configuration".

posted at: 04:17 | path: /SW/business/Drupal | permanent link to this entry

Wed, 14 Oct 2009

/SW/business/SugarCRM: SugarCRM Built-in Data Tranfer Tool is Highly Developed

SugarCRM is fundamentally a big, complicated address book, and any migration to SugarCRM can often reasonably include a wish to import a bunch of contact data from another application. In my case, I have a MySQL database.

To get this database into SugarCRM is basically a two-step process:

First compare Sugar's existing Account/Contact fields and the fields in the database to see what matchs and what does not. Then in the Admin section of the user interface, use the "Studio" application to create new custom fields where necessary, by clicking on "Fields" under "Contacts" and/or "Accounts". And then do not forget to add those new fields to a form in the corresponding "Layouts" subsections. Click "Save & Deploy" and you are done.
Then export the MySQL database table into a .csv file (phpmyadmin works well for this) click on the "Accounts" tab in Sugar, then "Import" to process the .csv file. In the resulting display, just match up Sugar fields with the fields in the .csv file, then click "Import Now". Several iterations of this may be required to repair bad fields in the database, mainly messed-up e-mail addresses. Once the accounts are imported, just repeat the same process by clicking on Sugar's "Contacts" tab and "Import" to strip out the contact information in the same database. Note that selecting the same "Account ID" when importing an account and its matching contacts allows the contacts to continue to be associated within Sugar, under the same Account.

This all works quite well as of SugarCRM version 5.5. 5.2 does not work so well: e-mail addresses do not import as they *all* trigger a syntax error, Account/Contact association is lost, etc. If you are doing a substantial data import I recommend you go right for the not-yet-stable version 5.5. It is a release candidate so it will be the new stable very soon.

posted at: 05:31 | path: /SW/business/SugarCRM | permanent link to this entry

/SW/business/SugarCRM: SugarCRM Upgrades

With "normal" PHP web applications, upgrading is a matter of downloading a tarball containing the new version of the application, unpacking it in the web root, and then running an upgrade.php script inside the new version to update the table structure of the back-end database to the new layout.

Not so with SugarCRM, and it is slightly non-obvious what the upgrade procedure might be. The first thing you notice when you download a tarball is that there is no upgrade (or update) script. Some poking around in the admin portion of the user interface leads to the discovery of an "Upgrade Wizard", so one's first instinct is to feed the afore-mentioned tarball (actually, in this case, a zip archive) into the Wizard. This does not work, and in fact complains about a "missing manifest.php script" in the archive.

In fact, what this wizard is looking for is a specific kind of file: an "upgrade" or a "patch" zip from sugarforge. And this somewhat buried page is where you need to go looking for these things:

http://www.sugarforge.org/frs/?group_id=6

posted at: 02:18 | path: /SW/business/SugarCRM | permanent link to this entry

Mon, 28 Sep 2009

/SW/research: Do-It-Yourself RSS

Ever cursed a site you want to check periodically for not having an RSS feed? cron / msmtp / web2mail.com offers a slightly crude solution -- crude because you get the whole page every time, not just new content.

web2mail.com provides a means of receiving website content via e-mail. I use msmtp to send the request to web2mail.com via one of my e-mail accounts, and cron to specify the frequency and issue the e-mail.

(I first tried to send the e-mail directly from my laptop mail server to web2mail.com, and web2mail.com did not like my e-mail and bounced it. Thus the msmtp strategy.)

I setup one of my e-mail accounts in ~/.msmtprc as follows:

# 163.com
account 163
host smtp.163.com
auth plain
from user@163.com
user user
password password

Then in a terminal issue "crontab -e" and add the following entry:

25 14 * * 1,4 echo -e "Subject: http://www.website-you-want-sent.com/ \n\n This is message body" | msmtp -a 163 -v www@web2mail.com

which grabs the web page on the first and fourth days of the week at 14:25.

posted at: 02:44 | path: /SW/research | permanent link to this entry

Wed, 09 Sep 2009

/SW/browser/Chrome: Chrome vs. Firefox

I will probably not part ways soon with Firefox because of its vast library of very useful plugins. But Google's Chrome is quickly becoming my essential second browser. There is of course the much publicized feature that every tab in Chrome runs in its own thread, therefore the contents of one tab locking-up do not effect the other tabs.

But Chrome also generally seems to be faster, and I have found at least one huge (for those of us living in bandwidth-starved regions of Asia) Chrome advantage in the https domain: when bandwidth is truly awful and Firefox sits quietly going nowhere for half an hour, Chrome succeeds in bringing down the page, sometimes in just a couple of minutes. The difference between a couple of minutes and a half hour to never is truly vast. No contest. Note that I have not yet tested this for http.

[1] http://code.google.com/intl/en/chromium/

posted at: 13:10 | path: /SW/browser/Chrome | permanent link to this entry

/SW/business/KnowledgeTree: Installing a New KnowledgeTree instance

If this is a first install on a new server, this guide[1] is a must read. Some of the highlights:

apt-get install apache2 php5 php5-curl php5-mysql mysql-server mysql-client xpdf zip unzip catdoc pstotext

apt-get install python-reportlab python-imaging python-uno antiword sun-java5-jre sun-java5-bin lynx openoffice.org-java

And do not forget to bump memory_limit up to at least 64 in /etc/php5/apache2/php.ini.

Get the latest source tarball from here[2][3].

In your desired web root, make a copy of knowledgetree for this instance, then sim link a simpler name, ie.

tar -xvf ktdms-src-oss-3.5.4a.tgz 
mv kt-dms-oss/ kt-dms-oss-3.5.4a
ln -s kt-dms-oss-3.5.4a/ knowledgetree

Give ownership to the web server process:

chown -R www-data:www-data kt-dms-oss-3.5.4a

Create a new MySQL database for the new instance:

#  mysql -u root -p
Enter password: mr4gvc7s
mysql> create database apps_kt;
mysql> grant all on apps_kt.* to 'apps_kt'@'%' identified by 'password';
flush privileges;

Edit config/config.ini to agree:

dbName           = apps_kt
dbUser           = apps_kt
dbPass           = password
dbAdminUser      = apps_kt
dbAdminPass      = password

Now import the database from sql/mysql/install:

mysql -u root -p apps_kt < structure.sql
mysql -u root -p apps_kt < data.sql

Separate out the configuration and the data:

mv knowledgetree/var/ kt-var
mv knowledgetree/config/config.ini kt-config.ini
cd knowledgetree
ln -s ../kt-var/ var
cd config/
ln -s ../../kt-config.ini config.ini

Now login to the new KnowledgeTree instance with the default:

user: admin
password: admin

and change the admin password.

Anticipated path for upgrading to a new version of Knowledgetree:

create a copy of the code of the new version
check for changes between the old config.ini and the new one
symlink (ln -s) var and config/config.ini in the new code to the old location in the top-level directory
run the upgrade script: knowledgetree/setup/upgrade.php
move the symlink for the short name (ie. knowledgetree) to the new copy of the code
test the new code
if the new code is broken, move the short name symlink back to the old code, and hopefully it should still work and down-time should have been just minutes.

[1] http://wiki.knowledgetree.com/Debian_4_Installation_Instructions#Grab_a_few_more_packages
[2] http://www.knowledgetree.com/try-now/knowledgetree_open_source_download
[3] http://www.knowledgetree.com/products/opensource/downloadopensource

posted at: 12:46 | path: /SW/business/KnowledgeTree | permanent link to this entry

Tue, 01 Sep 2009

/SW/fileSharing/mldonkey: mldonkey P2P File Sharing

mldonkey[1] is kind of a special beast among P2P file sharing software in that it is basically a headless server, with GUIs being more or less an after-thought that in my experience do not really work. I started using mldonkey because I was specifically looking for headless software to run on the Donkey[2] and Kademlia[3] networks, and have found it quite simple to operate from a telnet command line.

Defaults are all quite sane except that Kad is turned off by default, ie. this setting must be changed in downloads.ini:

enable_kademlia = true

Connect to and control mldonkey thusly:

ssh user@server.com
mlnet&
telnet localhost 4000
auth admin ""

where "mlnet" starts the mldonkey server, and the "auth" line logs in to the server. At the admin prompt, here are a subset of commands[4] that I personally find useful:

[1] http://mldonkey.sourceforge.net/
[2] http://mldonkey.sourceforge.net/EDonkey2000
[3] http://en.wikipedia.org/wiki/Kad_Network
{4] http://mldonkey.sourceforge.net/MLdonkeyCommandsExplained

posted at: 06:26 | path: /SW/fileSharing/mldonkey | permanent link to this entry

Tue, 04 Aug 2009

/SW/business/SugarCRM: A Simple SugarCRM API Class

This should give anyone needing to use the SugarCRM API a quickstart:

And the following is a simple example of how to use the above class to create a simple contact record with a name and an e-mail address, and store it in Sugar's Contacts database.

To find out all the possible fields in the Contacts database, use this script:

Many thanks to this post[1] for getting me on my way.

[1] http://systemsconsciousness.com/2009/04/10/sugarcrm-soap-examples/

posted at: 08:14 | path: /SW/business/SugarCRM | permanent link to this entry

Home

Contact

Resumé / C.V.

Links

Categories:

Archives:

Subscribe XML RSS Feed