buidling geocode2 from source

So I've officially given up on figuring out how to get the right libraries installed for GTG so that I can test out its geolocalization features. But, if you've seen my previous posts, you know it's been a painful process since I just can't seem to get the right libraries installed to get things working. But I'm going to go out on a limb and guess that this might not be entirely my fault because, from what I can tell, it seems like this particular feature is a rather old project? So maybe it's not surprising that the necessary libraries are a bit out of synch from the latest downloads that come bundled with Debian/Gnome/GTG. (Update: this bug report seems to confirm that this might be the case after all.)

So, with that sentiment, I'm moving on and am now going to try and install the libraries directly to see if I can get localization working separately from GTG. I've been advised that I need GeoClue2 to get location information, Champlain in GIR to show locations on a map, and GTK3 in GIR to show a map on a window. So I'm going to break it down and tackle the problem piece-wise.

First step: get GeoClue2 working.

• Building the source: After digging around, I came across this, which looked a lot more promising. I also found this experimental deb release here, which is useful for those who don't want to deal with sources.
  [Note: I also came across this repo as well. But this is specifically for geocoding and is something different than GeoClue2.] So I pulled the source. To avoid getting the same install errors I did, first installed the gtk-doc package:

  git clone git://anongit.freedesktop.org/geoclue
  sudo yum install glib-devel gtk-doc json-glib-devel 
  sudo yum install intltool itstool libxml2-devel libtool libsoup-devel
  sudo yum install gobject-introspection-devel
  

  Now you should have all the pre-reqs for running GeoClue2:

  ./autoget.sh --prefix=/usr
  ./configure --prefix=/usr
  make
  sudo make install
  

  A couple of things to note here. First, if you don't use the --prefix=/usr flags, then the dbus files get put in /usr/local/share instead of /usr/share. Second, you have to use root credentials to do the install because you're installing dubs services.
• Now that everything compiles nicely, you can try out the demo by going to the demo folder and running ./where-am-i. You may get an error message like this:

  CRITICAL **: Failed to connect to GeoClue2 service: 
  Error calling StartServiceByName for org.freedesktop.GeoClue2: 
  GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildExited: 
  Launch helper exited with unknown return code 253
  

  If you do, it's because you didn't use the --prefix=/usr flag I mentioned before. So go back, do a make clean and repeat the build process.
  If you're sure you did add the flags, then it's possible that the install still went incorrectly. On my Fedora machine, it added the /usr prefix in front of a /etc install, which led to things being installed under /etc instead of /usr/etc. You can change the script, or just hack it for now (like I did) and move the necessary files:

  sudo mv /usr/etc/system.d/org.freedesktop.GeoClue2.* /etc/dbus-1/system.d/

  Now you should be able to run the GeoClue2 demo:

  cd demo
  ./where-am-i
  

  And you'll see output that looks like this:

  Latitude: 40.440601
  Longitude: -79.995903
  Accuracy (in meters): 15000.000000
  Description: Pittsburgh, Pennsylvania, United States
  

• Note if you want to get things installed on Ubuntu, I had to first install these packages:

  sudo apt-get install intltool itstool libxml2-utils 
  sudo apt-get install libgeocode-glib-dev libjson-glib-dev libsoup2.4-dev
  sudo apt-get install libgirepository1.0-dev 
  

  The rest of the install process (including the --prefix=/usr flags I mentioned before) is the same.
  For 'fun', I also tried this process on a Debian machine and ended up having a much harder time getting things to just work. So I decided to ditch that and just stick with Fedora (or Ubuntu) for now.

geolocalization & gtg, take 2

OK, I'm trying to install GTG again. This time, I'm using Ubuntu 13.10 Desktop (64-bit). To get things started, I ran the same pre-steps as before: added myself to the sudo list, installed some basic tools, and set up bzr, and grabbed the code from launchpad. I tried running:

sudo apt-get install python-liblarch python-liblarch-gtk

But it turns out that this installs v1.0 and GTG is expecting v2.1. No worries. Just follow the instructions to do a git clone. The trick after that is to run ./gtg -l (instead of just ./gtg). To eliminate the other warning messages that GTG throws, just do:

sudo apt-get install python-dateutil python-suds python-evolution

Now to get the geolocation stuff to work, installing geoclue is easy:

sudo apt-get install python-geoclue

To get python-clutter, I had to update my sources.list to reference squeeze, since it seems it's not listed in the wheezy dist. So that definitely fixes it so that apt-get can find the package, but it still gave me problems, just like before:

The following packages have unmet dependencies:
python-clutter:
Depends: libclutter-1.0-0 (>= 1.0.8) but it is not going to be installed
Depends: python (< 2.7) but 2.7.3-4+deb7u1 is to be installed
E: Unable to correct problems, you have held broken packages

So this is saying that, to use python-clutter, I need to downgrade my python to something < 2.7. OK, so I found that one of the easiest way to install older versions of python is to add PPAs like this to your sources.list:

sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
sudo apt-get install python2.6 python2.6-dev
ls -lsa /usr/bin/python* # you should see python2.6 listed here now

Now that I have a version that might work with python-clutter, I can set up a virtual environment that only has python2.6 and see if it works:

sudo apt-get install python-virtualenv
mkdir ~/<gtg_virtualenv>
cd <gtg_virtualenv>
virtualenv -p /usr/bin/python2.6 --no-site-packages .
source bin/activate

The --no-site-packages is important, as it says that we won't look at any of the packages that are in our host environment (like python2.7). Of course, that also means that you'll have to reinstall any packages that you need. And that explains why, if you go run GTG right off the bat, you'll get error messages like:

File "./gtg", line 26, in <module>
from gtk.gdk import Screen
ImportError: No module named gtk.gdk

And that's where my patient with this experiment ends. I couldn't seem to figure out an easy way to install the right set of packages. I tried the usual suspects (pygobject, etc), but none of them worked or they were already installed, so it didn't help. Oh well, back to the drawing board!

geolocalization & gtg, take 1

Came across the GTG project and was interested in possibly doing some development on it. The first major hurdle I encountered was getting a VM setup that worked with the right libraries. Since I was interested in working on the geolocalization features in GTG, I had to install the python modules for geoclue, clutter, and champlain. Turns out I was having a hard time getting python-clutter to install because it seemed that the apt sources were from an older version of Debian (squeeze) than what I was running (wheezy). Modifying the /etc/apt/sources.list, as suggested here, worked so that at least the module was found when I tried to sudo apt-get install it. But it turns out that it's only compatible with python version < 2.7. And Debian 7 (wheezy) comes with python 2.7 installed by default. I tried downgrading python, but then that cascaded and a whole slew of other dependencies also needed to be downgraded. So I decided to start from scratch again and just use Debian 6 (squeeze), which comes with python v2.6.6 installed by default. From there, it was a breeze in getting things setup. Just a few administrative things to install up front:

1. Add myself to the sudo list. After running the following commands, I had to reboot the VM. It seems a simple logout/login didn't work for me. After restarting, I was able to use the sudo

   su
   adduser myusername sudo
   

2. Add git and bzr so that I can checkin/checkout code. Xsel allows me to use a handy shortcut. Note that, if you're using VMWare, you'll still need to have VMWare Tools to be installed so that you can pass stuff between your VM and your host computer.

   sudo apt-get install git bzr xsel
   cd ~
   nano .bashrc 
   # inside .bashrc add this line:
   alias pbcopy='xsel --clipboard --input'
   # save the file and exit out
   source .bashrc
   

3. Created a sshkey for myself, pasted that into bzr, and then setup bzr.

   ssh-keygen -t rsa -C "email@address.com"
   ssh-add id_rsa.pub
   pbcopy "
   bzr launchpad-login <launchpad-username>
   

From here, I followed the setup guide here . Here's how I solved (or tried to solve) the errors I encountered:



• After downloading the code from launchpad and then running ./gtg, I encountered this error message:

  E: Unable to locate package python2

  I did have python installed, but it was mapped to python, not python2. Solution:

  sudo ln -s /usr/bin/python /usr/bin/python2

• Running ./gtg shows me this message:

  GTG can't find liblarch and liblarch_gtk

  The error messages provides a possible solution:

  To install missing libraries, run the following command in the current folder:
  git clone https://github.com/liblarch/liblarch ../liblarch

  I decided to use apt-get to install the missing libraries. But it turns out that liblarch (and liblarch-gtk) don't have packages for squeeze, only for wheezy. So first I had to edit the sources.list file and then run apt-get:

  sudo nano /etc/apt/sources.list
  # add this line to the file, feel free to change the url 
  # to whichever/wherever is most appropriate for you
  deb http://carroll.aset.psu.edu/pub/linux/distributions/debian/ wheezy main
  # save file and exit
  # to ensure you get the new package list from the url you just added, run this: 
  sudo apt-get update 
  

  I tried to run sudo apt-get install python-liblarch python-liblarch-gtk, but I got a bunch of errors saying I had unmet dependencies, so I went with aptitude instead, searched for python-liblarch, and then went with its recommendation of how to resolve the dependencies. I repeated the same step with installing python-liblarch-gtk.



...And, what do you know - I hit another wall. I'm able to install liblarch and liblarch-gtk, but when I'm back at the prompt, I get greeted with this error message:

/usr/bin/sudo: No such file or directory

Looking at my PATH variable suggests that the right path is there (i.e., /usr/bin is included), but looking inside that directory, sudo isn't listed. And which sudo return nothing. Looking online, suggests something reinstalling the OS and/or booting to a Live disc, mounting it and then copying the necessary files over. So, I'm back to square 1 and trying to figure out how to set everything up - this is frustrating!

setting up hyperkitty dev environment

Stumbled across one of Fedora's app called HyperKitty and was interested in some of their UI. To get started mucking around with the code, I followed these directions to setup the dev environment. I used a Fedora 19 image running on VMWare. The only changes I needed to do were:

• Run all the software updates for Fedora before doing any of the dev setup steps.
• Set up a launchpad account: https://launchpad.net/. Add a SSH key to the account.
• Tell bzr my launchpad id: bzr launchpad-login
• Fedora comes with Firefox, but sometimes I like developing on Chrome. To install Chrome, I used these instructions: http://anl4u.com/blog/how-to-install-chromium-browser-in-fedora-19/

When running the various python setup.py install commands, I ran into the following error message:

unable to execute gcc: No such file or directory
error: Setup script exited with error: command 'gcc' failed with exit status 1

To solve this, I just installed gcc (sudo yum install gcc). (Note that, by following the Development Setup Guide and running the install command after trying to setup mailman, that means that gcc gets installed inside the virtualenv. It's easy enough to run the install command again outside the virtualenv.) Other helpful tips:

• To exit out of a virtualenv, just deactivate it. In my case, that meant going to a terminal window that was running the virtualenv (as indicated by the (mailman) prefix in front of the prompt, and typing deactivate. It's also possible to just delete the virtualenv directory. In my case, this would mean removing mailman/lib and mailman/bin.
• If you open a new terminal window and want to stay in the virtualenv, then just reactivate it. In my case, I just run the command: source mailman/bin/activate

troubleshooting rvm install

In an effort to get reacquainted with Ruby on Rails, I embarked on installing rvm on my machine. Unfortunately, right out of the box, I ran into a few problems. Since I already have macports and the latest Xcode installed (along with its command lines tools), I started the installation process off by running :

$ \curl -L https://get.rvm.io | bash -s stable --rails --autolibs=enabled

However, running that gave me this error message:

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: http://curl.haxx.se/docs/sslcerts.html

I tried a bunch of options (like re-running the \curl command with -k/--insecure, installing a certificate from http://curl.haxx.se), but nothing seemed to work for me. I ended up just enabling insecure https downloads through the config file for curl by running:

echo "insecure" >> ~/.curlrc

Not the cleanest solution, because you have to remember to edit the .curlrc file and comment out the insecure (until next time you run into this problem). This led to a second error message in the installation process (when running the \curl command):

unable to convert "\x89" from ASCII-8BIT to UTF-8 for guides/assets/images/belongs_to.png, skipping

Actually there were a bunch of similarly worded error messages, but the above gives you a flavor of what I encountered. Seems like the solution is that to install a version of the rdoc gem that supports the conversion by running:

gem install rdoc

To verify that the conversion works, you can regenerate rdoc by running:

gem rdoc --all --overwrite

I also ran into this error message when trying to install json (gem install json):

unable to convert "\xCF" from ASCII-8BIT to UTF-8 for lib/json/ext/generator.bundle

This conversion error is the same issue as the rdoc conversion issue. Installing the latest rdoc and regenerating rdoc also got rid of the json error message as well. By default, the latest version of ruby is installed (which was 2.0.0), but I also decided to install the previous version (just in case I ran into problems with old code or whatnot):

rvm install 1.9.3

In my installation, ruby defaults to the latest version, but in case it doesn't for you, you can run this to let dvm know which version you prefer

rvm use 2.0.0

or, in case you're seeing the version of ruby resetting whenever you reset your terminal, you can run this:

rvm --default use 2.0.0

You can verify that the ruby version sticks by running:

ruby -v

bitbucket setup

While I do have a GitHub account, I've decided to give BitBucket a try. The biggest selling point for me is that they have unlimited private repos. I still think GitHub is better for the general open source community, since more developers are on that site so it's one less account they have to create when they are contributing to a particular project. But for tinkering on small, personal projects, it seems like BitBucket is more along the lines of what I'm looking for.

After creating a BitBucket account, I went ahead and setup SSH so that I could use a SSH connection (and avoid the username/password hassle of using HTTPS) to push to my BitBucket repo. BitBucket provides pretty thorough instructions on how to do that here: https://confluence.atlassian.com/display/BITBUCKET/Use+the+SSH+protocol+with+Bitbucket

Getting BitBucket up and running was relatively straightforward. I created an empty repo on the BitBucket site. Then, on the client side, I decided to go with EGit since I was using Eclipse to do my dev work. My version of Eclipse came with EGit installed, so there's no need to install additional software. Looking at this tutorial, I skipped to Section 5 to start configuring Git in Eclipse. Note that, instead of using '/home/username/' I used '/Users/username/' since I was doing this on a Mac. What I like about the EGit interface is the Git Staging Perspective that they have. It makes it super easy to see what hasn't been staged, what's been staged and waiting to be committed, and what's already been committed.

To push the changes to a cloud Git server, just run these commands:

cd /path/to/my/repo
git remote add origin ssh://git@bitbucket.org/username/helloworld.git
git push -u origin --all # pushes up the repo and its refs for the first time
git push -u origin --tags # pushes up any tags

setting up git

Continuing my installation streak, I wanted to setup Git. I find this particularly useful when mucking around with a new coding language. As I'm making my way through what works and what doesn't, it's a tremendous help to know that in case I accidentally get too creative in my coding experiments, I always have something to easily fall back to. Now there are already lots of good discussions (e.g., see here and here) about Git vs other versioning control systems (VCS) (I personally used to use Subversion before switching to Git). As a Mac user, Git was already installed, though it was an older version (v. 1.7.12). Since I use Macports to do nearly all of my installs, I use these commands to update everything:

sudo port selfupdate         # updates macports
sudo port outdated           # lists which ports have a newer version
sudo port upgrade outdated   # upgrades the ports to the newer versions

Macports is usually smart about building/installing any dependencies first, but sometimes you run into errors. Also, there are a few ports that take a long time to build, so be patient! During my install, I encountered an error building arpack:

 --->  Computing dependencies for arpack 
--->  Configuring arpack 
Error: org.macports.configure for port arpack returned: configure failure: 
command execution failed 
Please see the log file for port arpack for details: 
/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_arpack/arpack/main.log

To solve this, I first ran this:

 sudo port clean arpack 
 sudo port install arpack +openmpi -gcc45 -gcc47 

...and then continued running sudo port upgrade outdated

Note that others (http://mac-os-forge.2317878.n4.nabble.com/MacPorts-39476-arpack-s-openmpi-variant-conflicts-with-gcc45-td220687.html) have cited that they still encountered errors when building arpack. If that's the case, then try: sudo port install arpack -accelerate +atlas instead.
To check that the version was updated, run:

 git --version

installing apache, tomcat, and mysql on ec2

I'm a bit late jumping on the ec2 boat, but finally got around to checking out what ec2 is like and decided I'd try my hand at JSP while I'm at it. So with that in mind, I decided to go with a linux distro for my ec2 instance. For the most part, I followed these instructions to set up tomcat and mysql: http://coenraets.org/blog/2011/11/set-up-an-amazon-ec2-instance-with-tomcat-and-mysql-5-minutes-tutorial/ With that, I was up and running, I used MySQL Workbench (connecting over SSH) to setup a toy database that I could muck around with. Then I started hacking away on Eclipse EE, trying to do some simple JSP/Servlet stuff.
Inevitably, there were some additional tweaks I ended up needing to do after the 5-min tutorial setup, so I'll outline what I did here:

1. Open up MySQL port. To ensure that my code could access the database (and not just MySQL Workbench), I had to open up the mysql port (3306). To do this, go to the EC2 Dashboard; under Network and Security, click on Security Groups. Select the appropriate security group for your instance and, under 'Inbound', create a new rule for the mysql port. After click 'Add Rule', it should appear in the list of TCP Ports.
2. Automatically startup services. To automatically have tomcat and mysqld start up whenever the ec2 instance restarts, run these commands:

   sudo chkconfig --level 35 tomcat6 on
   sudo chkconfig --level 35 mysqld on

   Note: the different levels correspond to runlevels. To check that we have these services running with the right flags, we can run: chkconfig --list
3. Secure MySQL. To make MySql slightly more secure, add a password for root (by default, there's no password), remove the test database, and remove anonymous access to the database.

   # create a new password for root
   sudo mysqladmin -u root password 'new-password' 
   # login with the new password
   sudo mysql -u root -p                           
   # remove test database
   mysql> DROP DATABASE test;
   # remove anonymous access to database
   mysql> DELETE FROM mysql.user WHERE user='' ;    
   mysql> FLUSH PRIVILEGES;
   

4. Get phpMyAdmin working. Sometimes it's just easier to do DB stuff through a browser instead of a dedicated app like MySQL Workbench. To get phpMyAdmin up and running, I need to install Apache and php first. Having both Apache and Tomcat installed isn't problematic, since I have Apache handle the 80 port and Tomcat handle the 8080 port (which is the default for Tomcat anyways).
   
   To install Apache:

   sudo yum install php
   sudo yum install httpd
   

   To have the Apache service automatically run on startup:

   sudo chkconfig --level 35 httpd on

   Then I just went to https://gist.github.com/aronwoost/1105007 and followed the instructions in these sections: 'Setup phpMyAdmin', 'Make mod_write work', and 'Start apache'.
   
   First, to allow access from external IP's:

   sudo chmod 600 /etc/httpd/conf.d/phpmyadmin.conf
   sudo nano /etc/httpd/conf.d/phpmyadmin.conf
   #  Web application to manage MySQL
   #  #
   #  Order Deny,Allow
   #  Deny from all
     Allow from 127.0.0.1
   #
   Alias /phpmyadmin /usr/share/phpmyadmin
   Alias /phpMyAdmin /usr/share/phpmyadmin
   Alias /mysqladmin /usr/share/phpmyadmin
   

   Then, we add the blowfish_secret so that cookie authentication will work

   sudo chmod 600 /usr/share/phpmyadmin/config.inc.php
   sudo nano /usr/share/phpmyadmin/config.inc.php
   ...
   $cfg['blowfish_secret'] = 'put-a-magic-string-here'; 
   /* YOU MUST FILL IN THIS FOR COOKIE AUTH! */
   ...
   sudo chmod 644 /usr/share/phpmyadmin/config.inc.php
   

   Now we modify the httpd configuration to allow mod_rewrite to work:

   cd /etc/httpd/conf
   sudo nano httpd.conf
   

   A few lines below that, look for AllowOverride None and replace with AllowOverride All. And now start up Apache:

   sudo service httpd restart

   Now phpmyadmin is up and running at: 

   http://Your_EC2_IP_Address/phpmyadmin

   or

   http://ec2-*.*.compute.amazonaws.com/phpmyadmin