The main downside I see is that it will be harder to evaluate teams if they don't have an idea to talk about in the app and interview -- even the idea is totally throwaway.
It doesn't seem like a bad idea, but I think the RFS already fills this need (as long as people know they can apply with an RFS but then do something different)
I agree. If you don't have an idea, on what basis are they judging you? what is their filtering process? If you are in your early twenties, what exactly have you accomplished that would separate you from the herd? Nothing, except which schools you have attended. I'm guessing the main filtering process YC will do is to filter based on which schools you attended, so in that way, YC is basically adopting the recruiting process that investment banks do.
I wonder what % of those YC funds attended the Ivy League or other private u's vs. those who went to state schools?
Assuming the YC guys get about 3000 applications (based on published acceptance rates and class sizes), they can probably either do some a/b tests in the reviewing process, or just substitute extra manual labor to evaluate based on other metrics if something goes wrong. It's not all all-or-nothing single test.
Metrics only get you so far. They don't really tell you why you fail. They only tell you how far you are along your plan to succeed, and whether it is matching expectations.
But if things are failing, they can't tell you why, particularly in an experiment like this. Maybe it is genuinely a bad idea and nothing will make it succeed. Maybe it is a good idea but needs to be gone about differently.
In other words, you can show that you are doing what you think you need to do to be successful, and that you are not being successful, but you can;t show what changes you need to make in order to be successful.
The solutions do require manual labor, but they require creative labor. This area is more art than science.
If I was doing this I would assemble a list of reasons why this experiment will fail up front. I would revisit the lists in a year or so. I would ask which ones played into the problem. I would then ask what could be done about it. But that's me. Others have to find their own ways.
I meant for the application-reviewing process, which has much faster feedback, vs. the "are the applicants qualified" process, which takes waiting at least through the session, and possibly a few more years.
They should be able to tune the app-review and interview process pretty easily, at least to get the same amount of information as from teams with ideas. Figuring out if those teams, if accepted, are markedly different from those who came with existing ideas, that's a much harder problem.
It doesn't seem like a bad idea, but I think the RFS already fills this need (as long as people know they can apply with an RFS but then do something different)