Avoiding TPG report entanglements

RogerB

It seems that a lot of collectors (and sellers of coins) get hung up on TPG authentication reports. They use various numbers to show that a coin is rare, or not rare, or medium rare, or maybe just half-baked. Truthfully, using TPG report data for drawing conclusions or predicting future behavior is largely invalid.

Certainly, what follows will irritate some, confuse others, and bring a smile to most; but here goes.

The difficulty arises from misunderstanding the content of TPG reports.

First – Coins are sent to TPGs on owner initiative. This is a random event for each submission, and reasons for the submission are not established according to specific criteria. It might be a personal whim, desire to have a personal “grading set,” interest in authentication, or many other reasons.

Second – TPGs have their own corporate criteria for acceptance of a coin. These include damage, cleaning, authenticity, and others. None of these are publicly defined or explained, so from an outside perspective no one knows how many individual coins of any date/mint/variety/condition/authenticity have been submitted.

Third – Published “grades” are not uniformly applied by different companies, and also vary with time and place (and some claim, ownership). Grades are, unfortunately, unstable.

The bottom line is:

The quantity of pieces reported for any date/mint/grade is a highly biased population. TPGs never tell us how many were submitted and the quantity rejected; and we don't know how many of this item exist. Also, unstable “grading” results in no assigned “grade” being comparable to one assigned at some earlier (or later) time, or by a different TPG.

The result is that trying to use TPG report data as representative, objective or inclusive will fail all statistical tests for validity.

Aotearoa

We also don't know how many times the same coin may be listed under various grades.

Coinosaurus

The result is that trying to use TPG report data as representative, objective or inclusive will fail all statistical tests for validity.

I haven't done the math, but I suspect the correlation for TPG pops vs. Mint-reported mintage figures, for 18th and 19th century American coins, would be greater than 70% (statistically "significant"). TPG data, as a whole, is useful. Individual data points can have many problems as you point out.

ricko

I have worked with statistics, studies and surveys for many years....Even the best are merely a snapshot in time, representing that particular segment as studied under particular circumstances by specific humans. Such studies are granted relevance by interested parties and criticized by those whose views are not supported. My experience indicates that such a large segment as represented by TPG data, especially with all the known variables, is nothing more than interesting information and, for the most part, has the potential to be vastly in error. @RogerB has enumerated some of the issues, certainly the major points. Basically, at any time, that data is at best, anecdotal and - giving it the highest credit - better than nothing. Cheers, RickO

RogerB

@Coinosaurus said:

The result is that trying to use TPG report data as representative, objective or inclusive will fail all statistical tests for validity.

I haven't done the math, but I suspect the correlation for TPG pops vs. Mint-reported mintage figures, for 18th and 19th century American coins, would be greater than 70% (statistically "significant"). TPG data, as a whole, is useful. Individual data points can have many problems as you point out.

Well, not quite --- TPG data are "descriptive statistics" and are, as mentioned above, possibly 0.7 or greater in significance. (This would be stronger for older coins where behavior and profit drive a greater proportion of items to be submitted. Thus, there is a greater likelihood that TPG authentication rates will parallel initial production rates.)

However, what many try to do is called "inferential statistics." Many attempt to makes estimates or inferences or present conclusions. For this purpose, TPG data is a highly biased population, an unreliable data set, contaminated by unstable “grading” results.

Unfortunately, TPG collection bias prevents establishing a confidence interval. Meaning, the data are practically useless for anything more than filling pages. (A small sample suggested a confidence interval of close to zero.)

[Suggested reading: Liven Vaughn, Statistical Methods for the Information Professional: A Practical, Painless Approach to Understanding, Using and Interpreting Statistics. American Society for Information Science and Technology, Medford, NJ. 2003. This is a direct, simply written exposition on basic statistics for IT professionals, especially managers who what to understand how to interpret data from others, and how to ask the right questions of staff members.]

RogerB

Sometimes, anecdotal material is really worse than nothing.

Law enforcement professionals have to deal with witness failure nearly every day - from reporting a fender bender, to a daylight robbery with a dozen eyewitnesses. Psychologists have found that memories are reconstructed rather than played back each time we recall them. We all do the same when we present anecdotes and when we listen to anecdotes.

Think about all the anecdotes of "I saw a 1964-D Peace dollar." None has presented a specimen, yet a large proportion of coin collectors believe the anecdotes.

In coin collecting, much of our older material is anecdotal and subject to the same distortions as any other human recollection.

BAJJERFAN

@RogerB said:
It seems that a lot of collectors (and sellers of coins) get hung up on TPG authentication reports. They use various numbers to show that a coin is rare, or not rare, or medium rare, or maybe just half-baked. Truthfully, using TPG report data for drawing conclusions or predicting future behavior is largely invalid.

Certainly, what follows will irritate some, confuse others, and bring a smile to most; but here goes.

The difficulty arises from misunderstanding the content of TPG reports.

First – Coins are sent to TPGs on owner initiative. This is a random event for each submission, and reasons for the submission are not established according to specific criteria. It might be a personal whim, desire to have a personal “grading set,” interest in authentication, or many other reasons.

Second – TPGs have their own corporate criteria for acceptance of a coin. These include damage, cleaning, authenticity, and others. None of these are publicly defined or explained, so from an outside perspective no one knows how many individual coins of any date/mint/variety/condition/authenticity have been submitted.

Third – Published “grades” are not uniformly applied by different companies, and also vary with time and place (and some claim, ownership). Grades are, unfortunately, unstable.

The bottom line is:

The quantity of pieces reported for any date/mint/grade is a highly biased population. TPGs never tell us how many were submitted and the quantity rejected; and we don't know how many of this item exist. Also, unstable “grading” results in no assigned “grade” being comparable to one assigned at some earlier (or later) time, or by a different TPG.

The result is that trying to use TPG report data as representative, objective or inclusive will fail all statistical tests for validity.

Click on the "COUNTER" at the top of the PCGS home page. It gives numbers of how many coins were graded AND rejected for various means. There are also monthly stats somewhere on this site.

I expect that similar [if not more detailed] info is available in their 10-K and like reports.