Well, a few reasons:

1. Google is firmly convinced that “George London” is a long-dead opera singer.
2. “glondon”, “georgel”, and “londong” all sound dirty.
3. Rogueleader with one “r” was taken.
4. Because I’m a galactic fighter pilot. Pew. Pew. Pew.

.post img{
width: 100%;
}

[Today I read a mailing list question from an entrepreneur asking how to find developers who would work for no pay. That struck a chord since I wasted a lot of time trying to do that myself when I was starting out on LinerNotes. So I wrote this blurb that I wish I could send back in time to myself. Also NB that I’m still working solo, so take this with a grain of salt!]

The first thing you need to do is to reverse the way you’re approaching the problem.

Unless you’re extremely lucky, you’re not going to have a lot of success with “how can I get people to do what I want?” No one else cares what you want. They care what they want. So instead you need to think in terms of “how can I give other people what they want (and satisfy my own needs in the process)?”

No one is going to work for you for free (would you personally spend months helping a stranger out of the kindness of your heart?) They may work for no cash, but only if you can compensate them in some other way that’s as valuable as cash. So think about what the type of people you need (i.e. programmers) want and think about which of those things you can offer them. The best thing you can offer is traction (i.e. a hugely reduced risk of failure) because everyone wants a seat on a rocket ship. You might offer equity, but if you haven’t raised money yet then your equity is basically worthless.[1]  Or it might be some connection or hard expertise you can share. If you don’t have anything at all to offer, then you need to go focus on yourself for a while and figure out how to either develop yourself to be a more useful partner or how to acquire the resources you need to attract people.

Applying this framework will give you a “candidate archetype” of people with whom you can make a mutually agreeable transaction. Then figure out where that type of person hangs out (e.g. in the UT-Austin computer labs) and go talk directly to them. Computer people are friendly and most of them will talk to strangers as long as you’re humble in your initial contact.

If you’re talking to a lot of people and not finding anyone, the problem is probably with you. Either you’re talking to the wrong people (because you didn’t target well) or you don’t have enough to offer at this point as a partner.

[1] Rule of thumb – think about the amount of cash you’d sell your company for today, and assume the whole equity pie is worth less than 1/10 of that.

Dear reader, this is a unusual day. For the first time (possibly) ever, I’m actually writing my next future the economy post while NOT airborne.

When I left you, we were talking about productivity. I claimed that productivity is the root of all wealth and nearly all improvements in the human condition. And I told you about the two ways that productivity improves:

1. Doing the same thing faster
2. Doing different things

It’s pretty obvious how computers let you do #1. Once upon a time, rocket scientists had to compute thousands of ballistic equations using slide rules. “Computer” used to be the job title of a person who simply did computations all day.

Now scientists can do the same thing, but faster. And Moore’s law tells they’ll be able to do that about 50% faster ever year. That’s pretty good, unless of course I want to do something that currently takes a million years (e.g. perform a 1 second/byte calculation on 10 terabytes of data).

For that, I’m going to have to get creative. My only option is to choose an algorithm that does substantially fewer computations or does each computation substantially faster.

Computer scientists have a fancy way of talking about how long a certain type of algorithm will take. When you see expressions like “O(n^2)” (i.e. “big O notation”) or “polynomial time”, people are talking about how much slower an algorithm will run as the amount of input data increases. For some problems, like sorting a list of n integers, a good algorithm might take (n)*(log n) nanoseconds while a bad one takes n^2 nanoseconds. For a billion item list, that’s a difference of about 30 years.

By using the better algorithm, I’ve increased my productivity by about 4,000,000,000%.

These examples are most dramatic in computer science but they apply just as much to normal life. Say I need to attend a meeting in Houston. That’s a ~20 hour travel ordeal OR a 1 hour Skype conversation. If they’re both equivalently good[1], I can 20x my productivity by choosing wisely.

Next time, we’ll get a little meta and talk about how to choose better algorithms.

[1] In real life, alternative solutions are rarely perfect substitutes. But they’re often close enough along the attributes that matter.

When building websites, you’ll often want users to be able to search for something by name. On LinerNotes, users can search for bands, albums, genres etc from a search bar that appears on the homepage and in the omnipresent nav bar. And we need a way to match those queries to entities in our Postgres database.

At first, this might seem like a simple problem with a simple solution, especially if you’re using the ORM; just jam the user input into an ORM filter and retrieve every matching string. But there’s a problem: if you do

Bands.objects.filter(name="beatles")

You’ll probably get nothing back, because the name column in your “bands” table probably says “The Beatles” and as far as Postgres is concerned if it’s not exactly the same string, it’s not a match.

Users are naturally terrible at spelling, and even if they weren’t they’d be bad at guessing exactly how the name is formatted in your database. Of course you can use the LIKE keyword in SQL (or the equivalent ’__contains’ suffix in the ORM) to give yourself a little flexibility and make sure that “Beatles” returns “The Beatles”. But 1) the LIKE keyword requires you to evaluate a regex against every row in your table, or hope that you’ve configured your indices to support LIKE (a quick Google doesn’t tell me whether Django does that by default in the ORM) and 2) what if the user types “Beetles”?

Well, then you’ve got a bit of a problem. No matter how obvious it is to human you that “beatles” is close to “beetles”[1], to the computer they’re just two non-identical byte sequences. If you want the computer to understand them as similar you’re going to have to give it a metric for similarity and a method to make the comparison.

There are a few ways to do that. You can do what I did initially and whip out the power tools, i.e. a dedicated search system like Solr or ElasticSearch. These guys have notions of fuzziness built right in (Solr more automatically than ES). But they’re designed for full-text indexing of documents (e.g. full web pages) and they’re rather complex to set up and administer. ES has been enough of a hassle to keep running smoothly that I took the time to see if I could push the search workload to Postgres, and hence this article.

Unless you need to do something real fancy, it’s probably overkill to use them for just matching names.

Instead, we’re going to follow Starr Horne’s advice and use a Postgres EXTENSION that lets us build fuzziness into our query in a fast and fairly simple way. Specifically, we’re going to use an extension called pg_trgm (i.e. “Postgres Trigram”) which gives Postgres a “similarity” function that can evaluate how many three-character subsequences (i.e. “trigrams”) two strings share. This is actually a pretty good metric for fuzzy matching short strings like names.

To use pg_trgm, you’ll need to install the “Postgres Contrib” package. On ubuntu:

sudo apt-get install postgres-contrib

**WARNING: THIS WILL TRY TO RESTART YOUR DATABASE**

then pop open psql and install pg_trgm (NB: this only works on Postgres 9.1+; Google for the instructions if you’re on a lower version.)

psql

CREATE EXTENSION pg_trgm;

dx # to check it's installed

Now you can do

SELECT *

FROM types_and_labels_view

WHERE label % 'Mountain Goats'

ORDER BY similarity(label, 'Mountain Goats')

DESC LIMIT 100;

And out will pop the 100 most similar names. This will still take a long time if your table is large, but we can improve that with a special type of index provided by pg_trgm:

CREATE INDEX labels_trigram_index ON types_and_labels_table USING gist (label gist_trgm_ops);

or

CREATE INDEX labels_trigram_index ON types_and_labels_table USING gin (label gin_trgm_ops);

(GIN is slower than GIST to build, but answers queries faster.

That’ll take a while to build (possibly quite a while), but once it does you should be able to fuzzy search with ease and speed. If you’re using Django, you will have to drop into writing SQL to use this (until someone, maybe you, writes a Django extension to do this in the ORM.)

And as a frustrating finishing note, my attempt to implement this on LinerNotes was not ultimately succesful. It seems that that index query performance is at least O(n) and with 50 million entities in my database queries take at least 10 seconds. I’ve read that performance is great up to about 100k records then drops off sharply from there. There are some apparently additional options for improving query performance, but I’ll be sticking with ElasticSearch for now.

[1] Sorry, Googlebot! Not sorry, Bingbot.