Objectively measuring "toughnesss"

AlanAllen

Inspired by the Toughest Vintage Set thread, I thought it would be interesting to attempt to rank similar sets by some objective measures using only the pop report. What metrics do you think are most telling in terms of building a set in 8+? I was going to use the following, but I thought I'd get some opinions first:

Average number of 8+ ((pop 8 + pop 9 + pop 10)/cards in set)
Percentage of cards graded 8+ ((pop 8 + pop 9 + pop 10)/pop)
Absolute number of cards with pop in 8+ of 5 or less (speaking for football here, the number for baseball would be higher)
Absolute number of cards with pop in 8+ of zero (one might say this trumps all).

Of course, this would ignore several important factors such as availability of quality raw examples, "submission bias," and popularity, but there's no accurate way to quantify those.

The method I'm planning on using would rank each set in my sample from 1-X in each metric, and simply add those numbers from each metric. The set with the lowest number would be the toughest.

Joe

mikeschmidt

I think that for sets before 1960 (trying to find a decent cutoff here), any sort of analysis may be skewed by star cards and their respective submission numbers. As such, I would suggest that for sets before 1960, the analysis focus on common cards and minor star cards. From there, I would say that the percentage of cards graded 8+ is probably one of the best measures of "toughest set".

An example of what I am proposing above: With 1952 Topps, every PSA-graded Mantle is a valuable card. With 1959 Topps, there are many dealers, etc. who may not be so inclined to get their Mantle card graded unless it is Excellent or better. Thus, I think that the %'s would tend to be skewed in such an analysis whereby their is an incentive in one set to get lower grade cards professionally graded.

As most pre-1960 sets have a minimum of 15 cards of most every common graded, I think % of 8s and higher becomes a useful tool. Just my $.02.

AlanAllen

I agree Marc, that's why submission bias is so hard to filter out. I was considering something like exclusing the top 5 submitted cards and doing something with the remaining data, but I'm not sure how to proceed with that. I've started collecting some data on football sets, and I'm seeing more submission bias in the 60's sets, wherein some have a ton of low pops, but the pops of the stars are many multiples of the pops of the commons, suggesting there is a good amount of raw commons floating around. Anyway, I'll publish the data on football sets later today, for what it's worth.

Joe

mikeschmidt

Joe -- it definitely becomes difficult to have any objective measure between sets. Whereas 1952 Topps may be put together in PSA 6 shape and be noteworthy and collected by many, other sets, such as 1954 Bowman, probably would receive little interest in grades PSA 6 and below but for star cards.

wolfbear

Joe - thanks for the post.
As a numbers freak, I think about this kind of stuff all the time.

How about this :
Muliply the number of cards in a set by 5 %.
Then remove all the data from those high profile cards from your stats.
Next, devide the number of remaining 8's and better into the adjusted total number of cards submitted.

The lower the number, the tougher the set.
If there's a flaw in my logic ? here, someone will point it out, I'm sure ...

AlanAllen

Good idea Paul, I'll go back and run the numbers excluding the top 5% pop from each set, and see if the data makes any better sense.

Joe

sagard

I don't think you will gain a lot by throwing out any cards from a given sets pop report. The '52 Mantle has been submitted alot, but it is not the most submitted '52 card. Throwing out cards like the '85 Topps McGwire that is two thirds of the submitted '85 Topps cards makes sense, but with the '52-'69 sets I'd just compare them as is.

I think the most important factors in toughness would be the percentage of cards graded meeting a grade threshold as well as taking the quantity of "low pop" cards. This will give you a quality of card stock comparison between years and the knowledge of what sets have very scarce graded cards.

Basically I agree with the first post.

I just wish I could try to collect PSA 8+ sets instead of raw ex+.

OhioTaz

One factor that I have not seen mentioned is 'demand'. A set with two PSA 8's for every card is easy if only one person is collecting the set.

I am not certain how to figure demand objectively. The only tool that we have is the registry. That obviously has significant flaws though. Any thoughts?

Sandy

qmayer

Why not find the % of cards graded in each grade? Wouldn't that eliminate the popularity bias and let the PSA grading standards take over?

Pop PSA 6s / Total submissions
Pop PSA 7s / Total submissions
Pop PSA 8s / Total submissions

I suppose you would have to consider that no one would really want to send in a 1972 common if they weren't sure it was at least an 8. That could skew the results a little bit, but we could subjectively factor in the age of the set as well.

Justin