2) You are talking about the cold start problem. Basically, the problem is how do you deliver good recommendations to new users as quickly as possible. I am actually currently talking to a big company about this problem.

Google has talked about how they do this for Google News, especially because they have high item churn (news is not news in a day). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.... Essentially, they do clustering of users and news items.

Another approach is to give the user a hunch-like quiz when they join. Basically, you use a decision tree to cluster the users, getting as much information from them with as few questions as possible.

Another approach that requires almost NO user interaction is to use some ambient information about the user. Maybe you have a cookie for the user and other sites will share the user's behavior with you. Maybe you ask them to log in through facebook or twitter, and then can look at their profile and activity history to build a user profile. So getting creative about possible sources of ambient information is another approach.

Could you provide easy examples where you would use a pair-wise approach, and a list-wise approach? Like, let's say I wanted to build a recommendation system for college football pick-ems (Ie, will Iowa beat Michigan State? Etc.). Which method would be preferred, and why?

Another question:

Let's say (again) that I want to build a recommendation engine for college football picks. How do I know how much (more) data will help for training purposes? IE, will only Win/loss records suffice [probably not], or should I add in the opponent they beat (or were beaten by) as well? What about whether the game was away/at home? What about factoring in the individual players on each team, and their stats? What about the temperature that day? How you do end up knowing when you can stop looking at smaller and smaller variables with your training data, and have something that gives the best results?

If your model is linear, you will probably see diminishing returns as you implement more, different features. It also helps to look at model errors to see which features are pulling things the wrong way, and then you can add other features to compensate.

(feature engineering + linear models feels a lot like writing an old-school AI heuristic program, except you have a computer program give a numeric weight the rules you're writing, so you're free to write a lot (millions) of rules and still get a manageable model)

I had written a longer reply detailing what are listwise methods and how they differ from pairwise methods, but apparently it was eaten by the web hyenas.

Turian's explanation doesn't actually cover what listwise methods are. I've seen them presented in the learning to rank context (learning to rank is when you want to build a machine learning system that given a set of documents, say search results, ranks them from more to less relevant to a query), so I'll follow this context here.

There are two very obvious approach to design a learning system that outputs sorted data: the first is to learn to assign a real number to each "document" (and then you sort according to these numbers) and the second is to learn a classifier that predicts given a pair of "documents" if a <= b (and then you can use this classifier as a comparison function in a sorting algorithm). Both these approaches have a common flaw, however, which is that they are very easily myopic, and will make decisions only looking at a very small window. This is clearly suboptimal in the learning to rank context because, for example, mistakes in the top elements of the ranked list are a lot more important than mistakes further down the list, and sometimes you want to maximize diversity or something like that, and it's hard to do that in an elementwise or pairwise approach.

So they invented listwise approaches, and there are actually two sorts of these: you can either learn a classifier that scores entire sorted sequences of documents (with features like document trigrams, features connecting similar documents, etc) or you can learn one of the above models with a loss function that depends on the entire sorted list of documents.

So essentially a listwise approach is better if you can do it, as listwise approaches come closer to minimizing error measures you actually care about (like precision among the top 10 documents) instead of bogus measures (like number of document pairs misclassified). On the other hand, precisely because listwise approaches allow you to be more specific, they are less generic, and it might be cumbersome to adapt one of them to your recommendation system.

Also, it fundamentally depends on how the results of your recommendation system are used. If you present something like top k recommendations then listwise approaches can be better, but in some other scenarios you will actually care about all the individual decisions, so a pairwise approach can do just as well.

Much as I love them, I've been getting so frustrated with the declining quality of their news page that I'm not sure if this is a good example. I know I'm not the only one.