Deploying the application remotely using Capistrano

I encountered the following problems in deploying the application using Capistrano

1. The "set :deploy_to" setting in Capistrano's deployment script config/deploy.rb

The syntax of the setting is "set :deploy_to {dir_name}".  After remotely logging into the deployment server as a user, the Capistrano command "cap deploy:setup" creates a directory of {dir_name} under user's home directory and enables the group write access to the directory.

Initially, I wanted to set the deployment directory to be ~/{app_dir}, where "~/" is the user's home directory, and the {app_dir} directly contains the Ruby-on-Rails application files, therefore, I used "set :deploy_to ."

The "." indicates the current directory, which should the user's home directory after Capistrano remote logs in.

The result of the "cap deploy:setup" command was

* executing `deploy:setup'

* executing "mkdir -p . ./releases ./shared ./shared/system ./shared/log ./shared/pids"

servers: ["ip_addr"]

[ip_addr] executing command

command finished in 1055ms

* executing "chmod g+w . ./releases ./shared ./shared/system ./shared/log ./shared/pids"

servers: ["ip_addr"]

[ip_addr] executing command

command finished in 53ms

It created the directories "releases" and "shared" directly under ~/, which is a little messy. What is worse is, the . or ~/ has been enabled for group write "chmod g+w .". This affects the ssh authentication. If the user's home directory is group/world-accessible, sshd does not start (see sshd manual). So any subsequent ssh login by the user will fail:

02:33 PM ~ $ ssh mfmproject@{host}

Permission denied (publickey).

Solution:

The solution is easy and creates cleaner deployment directory structure. Instead of deploying directly under ~/, deploy under a subdirectory such as "~/webapp". Change the setting in deploy.rb to

set :deploy_to, "webapp"

* executing `deploy:setup'

* executing "mkdir -p webapp webapp/releases webapp/shared webapp/shared/system webapp/shared/log webapp/shared/pids"

servers: ["ip_addr"]

[ip_addr] executing command

command finished in 1055ms

* executing "chmod g+w webapp webapp/releases webapp/shared webapp/shared/system webapp/shared/log webapp/shared/pids"

servers: ["ip_addr"]

[ip_addr] executing command

command finished in 53ms

2. During the execution of "cap deploy:migrations" command, the Git repository requires ssh authentication of user accessing the repository from the deployment machine.

The "cap deploy:migrations" command executes

"git clone -q mfmproject@[ip_addr]:git/karen_app_0515.git webapp/releases/20110919185255 && cd webapp/releases/20110919185255 && git checkout -q -b deploy 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 && (echo 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 > webapp/releases/20110919185255/REVISION)"

Even though both Git repository and the web server are on the same machine, the Git repository still treats the clone request as a if it is coming from a user on a different machine, thus requires ssh authentication of the user.

If the command was issued in the terminal of the deployment machine, the passphrase to unlock the user's ssh RSA private key would be prompted, in order for the user to access the Git repo via ssh.

Since Capistrano script runs without user interactivity, a passphrase is never given when it is needed by the "cap deploy:migrations" command, therefore the error like this happens:

* executing "git clone -q mfmproject@[ip_addr]:git/karen_app_0515.git webapp/releases/20110919185255 && cd webapp/releases/20110919185255 && git checkout -q -b deploy 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 && (echo 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 > webapp/releases/20110919185255/REVISION)"

servers: ["ip_addr"]

[ip_addr] executing command

** [ip_addr :: err] Host key verification failed.

** [ip_addr :: err] fatal: The remote end hung up unexpectedly

command finished in 392ms

failed: "sh -c 'git clone -q mfmproject@[ip_addr]:git/karen_app_0515.git webapp/releases/20110919185255 && cd webapp/releases/20110919185255 && git checkout -q -b deploy 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 && (echo 94c0d6bb8db04921d1f6ae8659ba9fac8c4f3f00 > webapp/releases/20110919185255/REVISION)'" on [ip_addr]

Solution

Set up ssh agent forward option in Capistrano deploy.rb

ssh_options[:forward_agent] = true 




This uses local keys on the development machine (where Capistrano is running), instead of the keys on the deployment machine to access the Git repository.

3. In executing the "cap deploy:migrations" command, Capistrano uses the system-wide Ruby installation instead of the user-specific RVM ruby installation when running the bundle_install task. The system-wide Ruby installation requires sudo access which is disabled in deploy.rb.

The command fails, because sudo access to do bundle_install fails.

Solution

Configure Capistrano to use RVM ruby installation in deploy.rb:

# configure Capistrano to use user-installed rvm rubies instead of the system-wide rubies

# Note: this rvm setting is obtained from the output of "rvm info" command, or# "cat ~/.rvm/environments/default".set :default_environment, {

'PATH' => "/home/mfmproject/.rvm/gems/ruby-1.9.2-p290/bin:/home/mfmproject/.rvm/gems/ruby-1.9.2-p290@glob

al/bin:/home/mfmproject/.rvm/rubies/ruby-1.9.2-p290/bin:/home/mfmproject/.rvm/bin:/usr/local/sbin:/usr/local/bin

:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games", 'RUBY_VERSION' => 'ruby-1.9.2-p290',

'GEM_HOME' => '/home/mfmproject/.rvm/gems/ruby-1.9.2-p290',

'GEM_PATH' => '/home/mfmproject/.rvm/gems/ruby-1.9.2-p290:/home/mfmproject/.rvm/gems/ruby-1.9.2-p290@glob

al'

}

4. In executing "cap deploy:migrations", after bundle install, Capistrano reports "rake aborted"

* executing "cd webapp/releases/20110919223344 && rake RAILS_ENV=production db:migrate"

servers: ["ip_addr"]

[ip_addr] executing command

*** [err :: ip_addr] rake aborted!

*** [err :: ip_addr] You have already activated rake 0.9.2, but your Gemfile requires rake 0.8.7. Consider using bundle exec.

*** [err :: ip_addr]

*** [err :: ip_addr] (See full trace by running task with --trace)

*** [err :: ip_addr]

StackOverflow has a thread on this problem.

Solution

Add set :rake, 'bundle exec rake' to deploy.rb.

5. In executing "cap deploy:migrations", unable to access log file due to the symbolic link links to invalid files

* executing `deploy:migrate'

* executing "cd webapp/releases/20110920001304 && bundle exec rake RAILS_ENV=production db:migrate"

servers: ["ip_addr"]

[ip_addr] executing command

*** [err :: ip_addr] Rails Error: Unable to access log file. Please ensure that /home/mfmproject/webapp/releases/20110920001304/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.

** [out :: ip_addr] (in /home/mfmproject/webapp/releases/20110920001304)

The symbolic link is broken for two files:

mfmproject@OB-MFM2:~$ ls -l webapp/releases/20110920001304/log

lrwxrwxrwx 1 mfmproject mfmproject 17 2011-09-19 17:12 webapp/releases/20110920001304/log -> webapp/shared/log

mfmproject@OB-MFM2:~$ ls -l webapp/current

lrwxrwxrwx 1 mfmproject mfmproject 30 2011-09-19 17:12 webapp/current -> webapp/releases/20110920001304

mfmproject@OB-MFM2:~$

The link destination isn't valid because "webapp/" is a relative path. 

Solution

This should be fixed in deploy.rb, in the setting"set :deploy_to, webapp". Instead of relative path, provide the full path to webapp.

Note: I replaced the actual IP address of the deployment server by "ip_addr" in the logs.

Git operation between remote and local repositories

• Add the alias (stanford) to the remote repository

12:03 PM ~/Development/rails_projects/app_0515 $ git remote add stanford ssh://[username]@[ip_addr]/~/git/app_0515.git

12:04 PM ~/Development/rails_projects/app_0515 $ git remote

stanford

12:04 PM ~/Development/rails_projects/app_0515 $ git remote -v

stanford ssh://[username]@[ip_addr]/~/git/app_0515.git (fetch)

stanford ssh://[username]@[ip_addr]/~/git/app_0515.git (push)

• Fetch remote repository into a local copy

02:28 PM ~/Development/rails_projects/app_0515 $ git fetch stanford

remote: Counting objects: 55, done.

remote: Compressing objects: 100% (49/49), done.

remote: Total 51 (delta 2), reused 51 (delta 2)

Unpacking objects: 100% (51/51), done.

From ssh://[ip_addr]/~/git/app_0515

* [new branch] master -> stanford/master

• Diff between the local, and remote repository which has been fetched to a local copy

03:22 PM ~/Development/rails_projects/app_0515 $ git diff master stanford/master --stat

Gemfile | 5 ++---

1 files changed, 2 insertions(+), 3 deletions(-)

• Got an error pushing to remote repository. The error indicates git thinks the remote repository has recent pushes that are not synced into the local repository.

03:21 PM ~/Development/rails_projects/app_0515 $ git push stanford master

To ssh://[username]@[ip_addr]/~/git/app_0515.git

! [rejected] master -> master (non-fast-forward)

error: failed to push some refs to 'ssh://[username]@[ip_addr]/~/git/app_0515.git'

To prevent you from losing history, non-fast-forward updates were rejected

Merge the remote changes (e.g. 'git pull') before pushing again. See the

'Note about fast-forwards' section of 'git push --help' for details.

• Merge the pushes/changes in the remote repository into the local repository

03:22 PM ~/Development/rails_projects/app_0515 $ git merge stanford/master

Merge made by recursive.

• Push to the remote again, this time it succeeds.

03:22 PM ~/Development/rails_projects/app_0515 $ git diff master stanford/master --stat

Gemfile | 5 ++---

1 files changed, 2 insertions(+), 3 deletions(-)

03:22 PM ~/Development/rails_projects/app_0515 $ git push stanford

Counting objects: 10, done.

Delta compression using up to 2 threads.

Compressing objects: 100% (6/6), done.

Writing objects: 100% (6/6), 978 bytes, done.

Total 6 (delta 3), reused 0 (delta 0)

To ssh://[username]@[ip_addr]/~/git/app_0515.git

a0226b1..2523dd3 master -> master

03:22 PM ~/Development/rails_projects/app_0515 $ git diff master stanford/master --stat

03:22 PM ~/Development/rails_projects/app_0515 $

Git reference.

TextMate tips and shortcut keys

• Autocompletion: type the first few letters, "ESC" and "Shift-ESC"
• Incremental forward/backward search: Control-S / Control-Shift-S

Getting a USB GSM modem working on Ubuntu

I have USB GSM modem working on a MAC, and have been using it to send SMS. Now I need to get the same modem working on a Ubuntu 10.04. Here are the steps I took to get it working on Ubuntu

When I first plug the modem into the USB port, the modem is detected as a CD-ROM device. A CD-ROM icon appears on the desktop. Ubuntu also detects the modem as a storage device, as I can tell, from issuing a few commands.

• the device appears as /dev/sr# (# is number which varies), a byte stream device
• the command "lsusb" output shows the device as "Bus 002 Device 005: ID 1c9e:f000." From what I read on the Internet, "f000" indicates it is a memory device.
• the command "ls /dev/disk" shows there is disk "by-label" of "Modem", and "by-id" of usb-USBModem_Disk_1234567890ABCDEF-0:0.

So all evidences show the modem isn't detected as a modem. Via Google search, I found out it is the effect of the ZeroCD feature that the modem manufacturers have been putting in the modem. Here is a wiki description on it. Essentially, the USB modem has a bit flash storage which stores the driver of the modem. When the modem is plugged into a Windows or MAC machine, it first runs as a storage device which auto-installs the driver, the driver then switches the modem out of the storage mode into the modem mode. On a Ubuntu, the driver doesn't run and install, therefore the modem remains as a storage device.

To switch the USB modem out of the storage mode and into the modem mode, I found the Sakis3G script. It is simple and works very well.

$ gunzip sakis3g.zip

$ sudo cp sakis3g /usr/bin

$ cd /usr/bin

$ ls -l sakis3g

Here are the model of the modem and the version of Ubuntu that I'm using:

GSM modem: Zoom 4595 3G tri-band USB modem

Ubuntu: 10.04.3 LTS

Donald Knuth answered my question!

In American writing, when quoting something in a sentence, the punctuation comma or period which ends the sentence always goes inside the quotation marks, such as the following:


Whitman famously said "I contain multitudes."

However, I always wondered what if I'm writing a computer science paper and quoting a variable name, putting the punctuation inside the quotation marks seem strange and can appear confusing to the reader -- is the comma or period part of the variable name?

Alas, Donald Knuth answered my question in his Mathematical Writing

The normal style rules for English say that commas and periods should be placed in-side quotation marks, but other punctuation (like colons, semicolons, question marks, exclamation marks) stay outside the quotation marks unless they are part of the quotation. It is generally best to go along with this illogical convention about commas and periods, because it is so well established, except when you are using quotation marks to describe some text as a specific string of symbols. For example,

Good: Always end your program with the word "end".

So my intuition is right, in computer science writing, leave the symbols alone!

Comma (,) generates a newline in Ruby "puts"

ruby-1.9.2-p290 :010 > first = "ruby"
=> "ruby"
ruby-1.9.2-p290 :011 > second = "rails"
=> "rails"
ruby-1.9.2-p290 :012 > puts first + second
rubyrails
=> nil
ruby-1.9.2-p290 :013 > puts first, second
ruby
rails
=> nil

Test code inside Rails Console

11:13 AM ~/{application dir}/ $ rails console
Loading development environment (Rails 3.0.5)
ruby-1.9.2-p290 :001 > Gsm::Modem.convert_string_from_ascii_to_char("6900500068006F006E00650d0a")
=> "iPhone"