Postgres and Postgis are a PAIN

The following is from a blog post from 2009. Installing PostgreSQL and PostGIS is still (still!!) incredibly painful. I'm re-blogging this here (I hope the author forgives me!) I'm terrified of losing these instructions. I know I'll need them again:

_______________________________________________________________

Install PostgreSQL 8.4 and PostGIS 1.4.0 in Ubuntu 9.0.4

HTTP://BIODIVERTIDO.BLOGSPOT.COM/2009/10/INSTALL-POSTGRESQL-84-AND-POSTGIS-140.HTML

WEDNESDAY, OCTOBER 21, 2009

Install PostgreSQL 8.4 and PostGIS 1.4.0 in Ubuntu 9.0.4

I am a big fan of the new PostGIS 1.4.0 (and also of Paul Ramsey) . I always have troubles installing PostGIS in ubuntu so I thought that this time I was gonna document it and blog it here. So this is just a log of the steps required to install it on an EC2 instance with Ubuntu 9.04. I hope it can be useful for someone else.

Just for the record. The EC2 instance I used was ami-ccf615 from http://alestic.com .


Once login (totally fresh).

apt-get update
apt-get install vim

#The sources are still not available on the regular package servers... edit the sources 
vim /etc/apt/sources.list
  add deb http://ppa.launchpad.net/pitti/postgresql/ubuntu jaunty main
          deb-src http://ppa.launchpad.net/pitti/postgresql/ubuntu jaunty main

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8683D8A2
sudo apt-get update
sudo apt-get install postgresql-8.4

#This changes the port from 5433 to 5432
sudo sed -i.bak -e 's/port = 5433/port = 5432/' /etc/postgresql/8.4/main/postgresql.conf

sudo /etc/init.d/postgresql-8.4 stop
sudo /etc/init.d/postgresql-8.4 start
apt-get install postgresql-server-dev-8.4 libpq-dev
apt-get install libgeos-dev
wget http://postgis.refractions.net/download/postgis-1.4.0.tar.gz
apt-get install proj
tar xvfz postgis-1.4.0.tar.gz
cd postgis-1.4.0
./configure
make
make install
sudo su postgres

#change the postgres password to "atlas" so that you can later login
psql -c"ALTER user postgres WITH PASSWORD 'atlas'"

createdb geodb    (with password atlas)
createlang -dgeodb plpgsql
psql -dgeodb -f /usr/share/postgresql/8.4/contrib/postgis.sql
psql -dgeodb -f /usr/share/postgresql/8.4/contrib/spatial_ref_sys.sql
psql -dgeodb -c"select postgis_lib_version();"

#This should return 1.4.0

exit

Good Luck!

What's the Magic Vagrant Word?

The magic Vagrant command is:

sudo /etc/init.d/vboxdrv setup

Get comfy with that command. You keep needing it.

Leaping into the World of APIs and Data

I am taking a flying leap and creating an entry for GitHub's 3rd Annual Data Challenge. My topic: The Six Degrees of Baconjs. Much like the movie-buff parlor game, the premise is simple: in six degrees or fewer, you can connect every user on GitHub (users who have at least one contributions/contributor outside of their own personal repos) with any other.

I have never attempted anything like this before. From the 10,000 feet view, given an input of a user, I will need to query for a list of all contributors to the Baconjs project and work backwards through each of their other repos to find the next degree's list of users. It's going to be recursive and strange but lots of fun!  I'm starting my studies of the first step with the GitHub API docs and the terrific tutorials on Codecademy. They even have one specifically on the GitHub API!

Wish me luck! I'm going to need it!

Time Flies Like an Arrow and Fruit Flies Like a Banana

That's my new little motivational mantra. Someday, I will help create a computer that understands that sentence.

Back in the present, though, here's a run-down of what I've been up to in the Open Historical Map project:

Week 11: July 28 - August 3[edit | edit source]

1. We're jumping into the rendering now. Most of the work will be in ohm_mod_tile, especially renderd and the master daemon program.
2. I posted the IRC log here (tried to strip out the filler /leave, /join, etc)
3. RENDERING
   1. This file is proving vital but elusive: home/tim/ohm-carto/ohm-carto/mapnik 2008.xml (UPDATE: I have the file now, but...)
4. The bulk of the renderer is written in C and C++ (with a lot of Bash and some inline SQL for good measure.)
   1. I am v e r y s l o w l y and carefully modifying the renderer to accept the {t} parameter. It's been many years since I've even looked at C code (and that was in an introductory C class.)
   2. I'm relying heavily upon the 'git grep' command (tracking here).
   3. I'm logging my progress so I can keep track of what I've changed and where: git diff log.
5. I don't want to lose this reference (is this Momento?):
   1. on line 245 in mod_tile.c, ~~bzero~~ recv() (bzero is used to zero out the bits at &resp to prepare it for the map tiles) and sends &resp to get the tiles from the database over a websocket
6. I am really deep in the weeds here in C. I am going to take a little time to read up more in the man pages and go through some more of the tutorial over on Learn Code the Hard Way

Week 12: Aug 4 - Aug 10[edit | edit source]

1. Made some changes to my changes (meta!) to the renderd.py byte packing/unpacking directive.
   1. IRC log on discussion here
   2. updated diff log here
   3. link on languages and urls from Chippy
   4. link on parameterization of mapnik tiles from Chippy
2. The OHM Team had our monthly Google Hangout notes
3. Re-attempting to install dependencies to run test server, instructions here for Mapnik and here for mod_tile
4. After much searching (and an upgrade to gcc, I finally got it all running well enough to attempt to compile.
   1. Sadly, MediaWiki's spam filter is preventing me from posting a link to the very helpful blog that walked me through the task of upgrading my C/C++/Java compiler. If you visit charette.no-ip.com:81/programming/2011-12-24_GCCv47, you'll get to the instructions.
5. I am now working on fixing the massive list of warnings and errors that running make generated for me!

Updating GCC for C++ 11

I will write up a more detailed blog post on this topic soon, but I wanted to share this:

In a blog post from 2011, Stephane Charette detailed how to determine which version of gcc you are using and how to easily update and switch between versions. (The standard version of gcc included in Ubuntu 12.04 does not support C++ 11). After spending two days thrashing following various official wikis, his blog post was a miraculous!

Linux Scholarship, How to End a Freelance Gig and How to Thrive as a Remote Worker

All of the following can be especially helpful for my fellow OPW interns. The first deals with the Linux Foundation offering scholarships for their (rather expensive) training classes. The second is a summary of Kate Hamill's helpful post on how to wrap up a freelance gig. Finally, Brazen Life wrote a post on how to manage remote workers. I'm going to translate that into how to be an awesome remote worker.

Today, the Linux Foundation announced their Linux Training Scholarship Program. Five scholarships will be awarded, one to a winner in each of the following categories: Whiz Kids (high school or college grads 18 years or older, Women in Linux, SysAdmin Super Stars, Developer Do-Gooder ("developers who are using Linux for good") and Linux Kernel Guru (a Linux contributor "who has promise of becoming a Linux kernel developer or maintainer.")

If you're a Linux Lover, check it out! Applications are due by midnight (PDT) on Monday, September, 2nd.

---

Over on the Freelancers' Union website, Kate Hamill has a post on the three things that freelance pros do when they finish a gig. Step one is to make sure you really are finished with the project. Now is the time to check your work and make sure there aren't any loose ends. She also advises freelancers to send out an email to get confirmation (in writing!) from the client to be sure that they believe the project is finished. Step two is to send a thank you email to your client. Why email? Because then your client can easily search their email for you. Very, very few people systematically file away thank you cards. Finally, Hamill recommends connecting with your (now former) client through social media.

---

Finally, Brazen Life describes seven ways for managers to do a better job of managing their remote (or "virtual") workers. It's a great list of tips, especially if you flip it and turn it into seven ways to thrive as a remote worker:

1. Be clear on your role and responsibilities. Being remote means missing out on those micro-moments where you can check in to make sure you and your boss agree you're on the right track. Starting your day with a list of that day's essential tasks (and then sharing that with your manager) can be a useful way to accomplish this.

2. Use some of the great tools out there like Basecamp or GitHub to create shared goals and track them.

3.  Keep communicating! If your manager hasn't already, try to set up a regular scheduled time to catch up and keep your manager informed of your work.

4.  Seek out novel ways to collaborate and create those "water cooler" moments. Scott Hanselman describes one way to do this in his post on "Virtual Camaraderie."

5.  Seek out co-working spaces, local offices or create one. This can give remote workers in your region a place to come together in company nodes and share information, ideas and, yes, collaborate.

6.  If there isn't one already, help create a central place to keep the knowledge of the project. GitHub can be great for this. (That's what we use for the Open Historical Maps project.)

7.  Find ways to tailor your work to you. One danger in being a remote worker is the risk of being seen as a widget that can easily be replaced by the next Gun.io applicant. By tailoring your work to you, you can become an indispensable part of the team, no matter how far away you are.

That's it for now.

Some Bits on Binary Packing and Holding Broken Packages

The first part of my day was spent on binary packing in Python. When using this feature to share objects between languages, you pack the structure/object into binary and deliver it with instructions for how to parse or unpack it. The format of the instructions will look something resembling this:

self.fields = "5i41sxxx"

where the structure is comprised of 5 integers followed by a 41 bit string followed by 3 null bits added for padding to make the structure add up to a total of 64 bits. My current task on the OHM was to add the {t} variable to this binary structure. Misunderstanding, I added the length of the 5 bit string to the existing 41 bit string. However, I learned that, unlike for the 5 integers, 46s would be treated as a single 46-bit string (logical but not obvious to me at the time). Instead, we opted to insert a second 5-bit string before the padding (which was increased to a total of 61 to increase the size of the binary structure to a nice, even 128 bits.)

Later that day, I spent a lot of time searching for solutions to an installation problem I was having. My updater was howling about need to update the firmware. My initial attempts were met with a strange error about having "help broken packages" (which drew up a most disturbing mental image). After lots of searching, I finally found a helpful post on AskUbuntu which described in detail the steps needed to clear this error. After typing:

sudo apt-get install xserver-xorg-lts-precise
hwe-support-status --verbose 
sudo apt-get install linux-generic-lts-trusty \
xserver-xorg-lts-trusty libgl1-mesa-glx-lts-trusty \ 
linux-image-generic-lts-trusty

TBH, though, I kind of wish that I hadn't updated because now Spotify won't run in my browser and the delete key fails on a long press. Google's suggested search when searching straight out of the address bar is malfunctioning as well. Something good will come of it, though. I'll have some fresh learning to write about tomorrow night!

10 Ways to be Smarter

A recent article by Jessica Stillman in Inc. magazine summarizes a recent Quora posting on what can you can do to get smarter. In short:

1.  Be smarter about how you use your time on the Internet. Think more TED talks, Khan Academy and (presumably, Inc. magazine) and much less YouTube, Farmville and Cracked.com.

2.  Write 400 words a day about something you learned that day.

3.  Make an "I did it" list to remind yourself of your accomplishments.

4.  Play games: chess, Connect 4, bridge, (MtG, anyone?).

5.  Surround yourself with friends who are smarter than you.

6.  Read a lot. A lot.

7.  Explain what you've learned. You don't really know it until you can teach it to someone else so they will understand.

8.  Do random new things. You never know how what you learn will help you down the line.

9.  Learn a new language. (I hope programming languages count!)

10.  Take some time to digest what you've learned. Sit and think (or exercise and think.) Spend time thinking.

I'm going to start with #2 tonight.

How to Change Tab Size in Vim and Other Nifty *Nix Tricks

I learned two nifty *nix tricks this week that I'd like to save here both so I can find them again if I manage to lose my notes (or my muscle memory) and in the hopes that I can save another future dev a little bit of time.

To change your tab size in Vim, first at the command line, type:

vim $HOME/.vimrc

This will open the user preferences file for Vim, creating it if the files doesn't already exist. You will then be popped into your Vim editor, where you can enter your preferences. Remember to type the letter a to enter insert mode. Otherwise, anything you type will be interpreted as a Vim command. Ready?

Here's where you can paste all those nifty "yak-shaving" "time-saving" snippets you've been finding online! I used

set tabstop=4
set shiftwidth=4
set expandtab

Press the esc key to re-enter command mode then :wq to save and exit. Re-open Vim and voila! You've just set your tabs to 4 spaces!


My second trick is much shorter, but it's a great little command just the same. When working in Git, before you add any files using git add type

git diff

to see the changes you've made across all of the files in your working repository. If you've already started tracking the files using git add, type

git diff --cached

It gives you lovely output such as:

That's all for now. Happy coding, everyone!