1) Kagan said it was apt in saying that judges, like umpires, should not have a "team in the game," should not come onto the field rooting for one team over another.
OK answer, although she lost points for not using the Twins in her hypo in response to a question from a Senator from Minnesota.
As I have argued before, to the extent this is what the metaphor means, it does no work. No one believes a judge should be "rooting" for one party over another and we don't need an analogy to baseball to drive the point home. Besides, no one seriously believes an umpire "roots" for one team or that a judge "roots" for one party.
But there's more something going on here, tied to the complexity of law and the fact that different judges can reach different conclusions (which Kagan talks about later). An umpire may not be rooting for a particular team, but an umpire who interprets and enforces certain rules a certain way may benefit one team over another. An ump with a wider strike zone will make calls more to the benefit of a team with control pitchers who work the outside corner; an ump with a narrower zone benefits teams with patient hitters who work counts. Similarly, I would not say that Justice Scalia "roots" for the government in a challenge to abortion laws. But given that his reading of the relevant (far-less-determinate) rule is that Due Process does not provide a liberty to obtain an abortion, he is likely to find for the government in any challenge to a restriction on abortion. That does not mean Scalia is biased towards the government any more than it means Umpire X is biased towards the team with patient hitters. And it does not make their approach to law illegitimate. But the nature of the legal rules is such that one party always will benefit from that person's legal views. This is why you cannot evaluate anything solely based on outcomes.
2) Kagan also said the metaphor is right (and she believes this is what Roberts meant) in saying that judges must understand that they, like umpires, are not the most important people in the game. Policymakers (Congress and the Executive) are the more important players in the game, with judges playing the limited role of policing the constitutional boundaries of congressional and executive action and conduct.
I did not like this part of the answer, in part because it could give the metaphor new, different life. Neither courts nor umpires play a limited role. Umpires necessarily are involved in every single play--not one pitch is thrown in a baseball game that does not result in at least one pro forma call on the play. Moreover, I never read Roberts as using the analogy to say that judges should not vigorously exercise the power to police the constitutional boundaries; certainly his behavior on the Court does not indicate a belief in according great deference to the elected branches. So I hope the metaphor does not become a catchphrase for more constrained judicial review.
Plus, it's just wrong. Umpires don't "decide" the game--who wins and loses. That is done by the players who throw the ball, catch the ball, and hit the ball. So, sure, umpires are secondary to the players on the field. But judges (at least in non-jury cases) do decide the case; it is their job to determine who wins and who loses in litigation that has been brought before the court. So judges, of necessity, are a major player in litigation. As I argued in my LSA panel on the metaphor: We could have a baseball game without umpires and we would understand it to be baseball, but we could not have litigation or adjudication without judges.
3) The metaphor fails at the task for which many Senators and others (not clear if Roberts falls in this group) have put it: Making judging appear simplistic and law clearly determinate such that any judgment is unnecessary and, in fact, bad. Quoting in full:
[T]he metaphor might suggest to some people that law is a kind of robotic enterprise. That there's a kind of automatic quality to it. That it's easy. That we just sort of stand there, and we go "ball" and "strike" and everything is clear cut, and . . . there's no judgment in the process. And I do think that that's not right, and it's especially not right at the Supreme Court level, where the hardest cases go, and the cases that have been the subject of the most dispute go. . . . Judges do in many of these cases have to exercise judgment. They're not easy calls. . . . But we do know that not every case is decided 9-0 and that's not because anyone is acting in bad faith. It's because those legal judgments are ones in which reasonable people can reasonably disagree sometimes. . . . [L]aw does require a kind of judgment, a kind of wisdom.
It would have been nice if she also acknowledged that even umpires do not just go "'ball' and 'strike'" and that they exercise interpretation and judgment as well. Thus the broader point is that no decisionmaking is robotic or automatic and we should stop acting as if it ever is. Part of what always has bothered me about the metaphor is that it is based in the first instance on a misunderstanding of what umpires do.
Otherwise, this answer was spot-on in explaining why the metaphor does not work--rules always must be interpreted and judgment exercised. Stop pretending the rules are easy, mindless, or clear, obvious, and determinate. And stop acting as if a decision with which you disagree was illegitimate or based on person preferences (that accusation was a recurring theme towards any justice or decision with which a questioning Senator disagreed).
Is the metaphor gone forever? I doubt it. I expect to hear it repeated in the committee and floor debates on Kagan, as well as in the hearings on Obama's next nominee. But Kagan started to lay out a pretty good map of how to attack the analogy--certainly as much as is possible in the current (unfortunate) political context.