Month: January 2013

[Note: this post is adapted from a long comment I made on one of Albert Wenger’s recent posts]

Albert, I think you’re absolutely right that the search game is just getting going. If you define search as “finding relevant web pages,” then Google has search locked up and solved. But if you define search as “providing the information you need to make the best decisions all the time” then there’s a vast and fertile green field ahead. At the end of the day, the goal of search isn’t to deliver links – it’s to deliver wisdom.

So how are we going to do that?

Let’s start with a thought experiment: imagine that you yourself are hired to be a search engine. Now imagine that your wife is a search user and so naturally any time she needs something you want to give her the absolute best and most useful answers you can.

Let’s assume that you have unlimited time, resources, and patience to search. But let’s add a constraint: the only way your wife can communicate is by barking 2-10 words at you and waiting silently.

How you would go about answering your wife’s queries?

For example, let’s say she asks “pizza broadway.” You can probably guess what she wants, but at least for me Googling that phrase returns junk. You, on the other hand, would take what you know about your wife’s tastes, then you’d compile a list of restaurants on Broadway, then you’d figure out which ones serve pizza, then you’d look at Yelp to see how well reviewed each pizza place is, and then perhaps you’d call your friends who eaten at those restaurants to get their opinion (which you would discount based on how much you trusted their taste in pizza.) Ultimately you’d hand your wife a simple summary telling her that the best pizza on Broadway was at “Pizza X,” located at 3333 Broadway (click here for directions or here to have it delivered.) You might even provide a summary of why you chose that answer if it was a close call.

Next consider a slightly more complicated (but much more important) query: “best college for my child.” Even with your human judgment and everything you know about your wife and children, that’s a mighty hard question to answer. But it’s not impossible, and it probably has an objectively correct answer (or at least there’s a small set of equivalently good options.)

So how would you get to the answer? Personally, I would start by fighting back against the initial constraint of this thought experiment, i.e. that you can’t communicate with the searcher. I’m sure you’d be able to do a much better job of answering the college query if you could have a conversation with your wife that lets her volunteer additional relevant information (e.g. whether your kid prefers small or big schools.) That additional information would let you figure out which variables to optimize and thus come up with a much better answer.

I’m using this thought experiment to try to make three points:

1) Search is a much larger and much more economically and socially valuable problem than “find the best link.”

2) A great answer to a hard question requires sophisticated aggregation of information from a variety of sources.

3) Instead of starting with an interface (e.g. “text in a box”) and figuring out how to
stretch that approach, one should start by figuring out what information one
needs from the user and then design the interface to most effectively gather
that information.

Facebook’s graph search is extremely exciting because it’s the first approach I’ve seen by a large player that recognizes each of those three principles:

1) Graph search lets you solve an entirely new class of problems. As a real life example – last week I spent about 20 minutes compiling a list of people to invite to my
birthday party. I could have done that in an instant by querying “close friends
who live in New York.”

2) The whole system is based on Facebook’s massive structured dataset. Cost-of-data-acquisition was what killed off the 1980’s “question answering” AI systems, but Facebook has basically solved that problem by conscripting an army of human volunteers.

3) It’s still a textbox, but the addition of the “concept chaining” system (i.e. “in New York”, “from 2003”) lets users massively but easily increase the expressive power of
their queries.

Obviously it’s going to be a long, long time before a computer can tell your children where to go to college. But it’s going to happen.

The search systems of the future aren’t going to be location-based OR mobile OR social. They’re going to be all of the above. They’re going to combine cutting-edge machine learning and NLP with crowd-sourcing and old-fashioned manual data entry to create comprehensive, varied knowledge bases. They’re going to use embarrassingly parallel systems to process data that’s organized with ontologies from linked data and stored in not-only-SQL data-stores. And they’re going to use best-practices from Human Computer Information Retrieval (HCIR) to expose interfaces that let searchers effortlessly deploy statistical reasoning to get the exact answers they need to live better lives.

Imagine a world where your mobile phone can instantly find every band similar to your favorite bands you’ve never heard, or can provide you all the core evidence for and against investing in an entrepreneur, or can save you hundreds of millions of dollars by correctly estimating how many factories you need to build to meet demand for your new widget.

Those answers are coming. And they’re coming soon.

[Edit: if you like this post, help more people see it! Please share, or upvote this post in the Hacker News “new” queue  ] 

I was not a normal child. At age 2, my favorite pastime was disassembling every child safety device my parents bought. By age 4, I had become an avid user of the original Macintosh (despite being unable to read.) When I hit school, I was… ”impatient” with the pace of the curriculum. Luckily, I had great parents. They were interested enough to provide me with an endless supply of books and extracurricular excursions. They were savvy enough to maneuver me into an excellent charter middle school. And they were affluent enough to send me to a private high school where the teachers stuffed me with knowledge instead of letting the students stuff me into lockers.

Not all children are so lucky. I’m no social scientist, nor am I particularly well informed about the particulars of the American public educational system. What I am is strongly convinced that educational opportunities make an exceptional difference in the life of a gifted child. And that every gifted child in America – irrespective of background – deserves at least the same opportunities I had.

Last week, I read an article in the New York Times about the continued under-availability of gifted and talented (G&T) education, especially to minority students. In our era of unbelievably insane and misguided budget madness, education is something of an easy target. Children, of course, don’t vote. And G&T education is often seen as a luxury which can be cut or as a competitor for a fixed number of education dollars (especially against programs like special education.)

So, naturally, while falling asleep last night I decided to estimate the actual cost vs. benefit of gifted education. The results were somewhat surprising, so I’d like to share them with you.

In short, the conclusion is that it literally costs the government $0 to spend up to $6,500 extra per pupil per year on gifted education. That’s approximately 50% more than the U.S.A. currently spends on the average pupil.

That sounds like a lot, right? Here’s how I got there:

First, we need a few simplifying assumptions:[1]

·      Students can be ranked into deciles by academic aptitude [2]

·      Annual income is related to academic aptitude by a power law distribution [3]

·      A well-targeted education program can raise the average annual income of a decile by 5% [4]

From there, the logic is simple. The annual median income for men in the USA is ~$45k [5]. We assume that the least a worker in the USA can make in a year is $10k [6]. We then roughly fit a power law distribution using income data reported by the U.S. Census Bureau (which has a pretty clear power law shape.)

(source)

This lets us predict annual income by academic decile:

From there, we can predict lifetime income by simply multiplying annual income by the average number of working years (assumed to be 43)[7]. We then multiply lifetime income by our education “bonus” multiplier (5%) to get the additional income each student is expected to generate over her lifetime. We then assume that the government will take back 30% of that extra income through taxes. We divide those collections by the number of years a student spends in school (13), and we have our annualized “free money” threshold, i.e. an amount of spending per pupil that will generate a positive return on investment for the government.[8]

This chart shows the conclusion:

If you’d like to see the calculations step-by-step or you’d like to adjust my assumptions, take a look at this Google doc.

What does this chart say, exactly? It says that if the government spends an additional $5,000 per year on a gifted student, it can expect to collect more back in income taxes over her life than it spent on her gifted education. In other words, up to a point (i.e. $6,500) spending on gifted education is literally free for the government.

That means that any debate about prioritizing gifted vs. special needs education is deeply misguided. It’s not an either/or question. The only reasonable course of action is to just spend the money on gifted education and make all other education spending decisions independently. The government need not even raise taxes to fund the programs; they could issue a 30-year municipal bond and expect to be able to pay back the interest with the extra taxes they collect.

Now this analysis is obviously an oversimplification. Everyone knows that being smart per se does not guarantee a high income.[9] It’s crude at best to assume a static and binary 5% boost in income from simply having a gifted program. And this analysis completely ignores the many social benefits of having better prepared doctors and scientists[10]. I discuss a number of other specific flaws in the footnotes.

Nevertheless, I think the conclusion is sound. Opportunities for accelerated education made a huge difference in my life, and it’s a tragedy that talented students continue to be denied those opportunities just because of their parents’ income. Whether the USA recognizes that fact, China and India certainly will.

Notes:

[1] Relevant data was hard to find on a cursory Google search. If you have better data or feel like making more sophisticated models, please post in the comments!

[2] The idea of innate academic aptitude is controversial enough even without specifying a yardstick. So choose your own! The method of measurement doesn’t affect the logic as long as a measurement can be made.

[3] It’s pretty clear that income itself follows a power law distribution. I’ve never seen a good study about how well academic aptitude predicts income, but to whatever extent America actually is a meritocracy: it will. For every maxim about “A students work for C students”, I’ve met very few unintelligent people who’ve earned large amounts of money.

[4] I pulled this number out of a hat. Intuitively and based on my own experience, I suspect it’s significantly higher. But I wanted to be conservative for the analysis.

[5] The median income for women is substantially lower, but I’m assuming that’s because homemakers are counted. Assuming men and women have equal earning potential if they actually do work, using male income shouldn’t affect the conclusion.

[6] A bit less than full-time minimum wage income, since unemployment rates are much higher for the less educated.  http://poverty.ucdavis.edu/faq/what-are-annual-earnings-full-time-minimum-wage-worker

[7] Working from 22-65

[8] I’m ignoring any time-value-of-money issues, both because that would be complicated and because I figure that wages should grow approximately at the interest rate, so discounting vs. wage inflation should cancel out.

[9] In an era where a good data scientist can earn $300,000…we’re getting close. Many top decile people will chose lower income careers, but they will have had at least the option to earn more if they chose.

[10] Medical, technological, and scientific discovery probably also follow a power law distribution. So we should expect that giving our brightest scientists more education will significantly increase the rate of discovery.