Time at last for computer grading?
dpoole
Posts: 5,940 ✭✭✭✭✭
Discussions on some recent threads have stimulated this question again for me.
Immediate disclaimer: I'm a liberal arts guy. But even in the use of speech-recognition hardware and in the lay person's reading I've done, it seems to me that visual scanning of individual coins now can easily teach computers to assess features necessary for even fine graduations of technical grading, given definitions and enough examples fed into the system.
It seems to me that this would improve over time the opportunity to screen out counterfeits, to spot market unsanctioned doctoring and, of course to objectify grading and stabilize pop reports.
There would remain intangible market-relevant features such as eye appear and toning, which will always remain the purview of personal taste. But we would then leave that up to the market force of the seller/buyer and even to de-refined grading actvity, working from the baseline of certain authenticity and technical quality.
Those among you with technical knowledge should certainly comment about whether we are there yet, but I'm also curious:
Do you think this is a desirable direction for the hobby and for present and potential grading services?
Comments
Kind of like weather predicting I think. We'd have the European model, the NOAH model and the Weather Channel model, and none would agree. Think of it. PCGS, NGC and ANACS would all have different software and thus different results?? lol
There will still be a subjective component in that coins will vary in eye appeal that some will like more than others... I doubt computer grading could adequately deal with the entire scope of what is reviewed in the process.
Experience the World through Numismatics...it's more than you can imagine.
Never gonna happen. It would hurt the bottom line of the tpgs and imo would take a lot of fun out of the hobby
Not only that, but when there were bug fixes or updates to the models used to train software, the results within a single grading service would change.
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars • Variety Attribution
It is certainly possible, however, the software project alone would be daunting. Computers can be used for technical grading, eye appeal etc. (IMO) should never be considered in a coin's grade. Once the software is developed, we would be able to have standards.. wouldn't that be great? People who collect tarnish would still be able to do so, premiums and all.... Cheers, RickO
A big fly in the punch is especially in a series were coins can be weakly struck Take for instance the eagle breast feathers on a Morgan Dollar.
I do like the idea of being able to collect Tarnish, While most like all the colors, I am most attracted to the old original tarnish look, or what I call original toning.
I would like to see the results of a test group first. lol
Computers could very well do the grading. Humans will still do the pricing, though
Liberty: Parent of Science & Industry
Grading has too many nuances for computer grading. Sort of like art appreciation.
I have always had a problem with eye-appeal affecting the grade. If the coin technically grades as a 63, then it's a 63 and should be valued as such. Leave the eye-appeal to the seller and buyer. Grading is subjective enough without considering eye-appeal.
Just my eversohumble opinion.
Cheers
Bob
Who or what trains and validates the computer grading, then?
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars • Variety Attribution
I really don't think it will ever be possible.
It's been tried. Check out this paper, "Machine assisted visual grading of rare collectibles over the Internet ":
csis.pace.edu/~ctappert/srd2003/paper19.pdf
Text from pages 5-6:
Machine grading issues
Hypothetically, machines can be configured and trained to perform an evaluation of all of the
detailed technical grading1
aspects that humans are supposed to consider in the grading process
but rarely do. Such a should be able to consistently produce technical grades at a higher
accuracy and consistency level than humans.
There have been several previously documented attempts at automated coin grading and two
commercials attempts will be addressed. Two commercial companies, Professional Coin Grading
Service (PCGS) [6] and CompuGrade [7] systems in the market in the 1990’s but quickly
withdrew them due to their lack of commercial success.
In 1990 PCGS announced a computerized system for grading coins. The system, which was
known as the PCGS Expert, utilized robotics, image enhancement, image processing and an
online image database for its integrated computer system. [6]
CompuGrade, a company run by Jim Diefenthal, had a system, which could consistently grade
Morgan dollars to a standard of a tenth of a point. One aspect of the CompuGrade system was
that the system could be used as a grading teaching tool as a method for learning grading. [4]
No formalized research could be located which detailed the approach of the CompuGrade
system. However coin expert John Baumgart attended a presentation of CompuGrade at the 1991
Chicago ANA show and reported his observations of the CompuGrade system[2]. According to
Baumgart, the CompuGrade system reportedly graded coins based on digital images taken in a
multistep proprietary technology process within a controlled environment. First a defect map
was generated by subtracting an ideal coin from the coin that was being graded and thus
producing the defect map. This map contained all defects discovered such as bag marks and
scratches, which appeared on the coin. Then an algorithm, which rated the marks contained in
the defect map based on location and severity, came up with part of the grade. The next step
required that several more images of the coin were taken to ascertain the eye-appeal by
evaluating the coins luster. Lastly a human evaluator made sure there weren't any catastrophic
errors.
Some of the technical problems reported [2] with the CompuGrade system included:
· Toning could not be measured accurately by the system, as measuring toning in with an
automated system is a very complex task. Toning is a natural discoloration of a coin's
surface by the atmosphere over a long period of time. Many collectors often consider
toning to be very attractive and desirable and they tend to prefer coins with natural
toning. Toning is a subjective quality that is an important factor in determining the value
of a coin. Toning colors of major concern are white, copper, nickel, and gold depending
upon the metal composition of the coin. However these major colors can include: red,
red-brown, brown, white, full white, original color, dark color, light tone, pleasing tone,
rainbow tone, unusual tone, dark fields and light fields.
· Measuring abnormal die strikes created significant challenges. Die strikes are created
during the minting process of impressing the design from a die into a planchet to make a
coin. They are important because they indicate the completeness of detail (as in weak
strike, full strike, etc.).
· One algorithm for grading didn’t work for all coins in the Morgan Series due to frequent
design changes in the series.
· The system didn’t assure authenticity or detect counterfeits.
· The eye-appeal algorithm didn’t provide the entire picture of eye-appeal, which is
arguably subjective from person to person.
Avid collector and dealer Byron Reed pointed out that the CompuGrade system was “great from
the standpoint of determining detail, but was really poor when determining pretty verses ugly.”
[14] Still an important benefit afforded by the system was the maintenance of a database, which
included the previously graded coins. This database was potentially useful for identifying
specific individual coins by contact marks, wear and for the purpose of insurance, theft
protection, rarity evaluation, and provenance. [2, 14]
Both PCGS and CompuGrade attempted to build systems that they anticipated would become
commercially viable and profitable. They soon discovered that the development of software
could be a long and expensive process. In the wake of rising development costs, missed
deadlines, ever increasingly complex rule sets, the hope of all profitability diminished and both
companies quietly withdrew their systems from the market.
Computer grading - I'm a science guy and I wouldn't spend 2¢ submitting any coin for such "grading". I worked in artificial intelligence for a number of years and the obstacles to the kind of recognition necessary for coin grading are daunting. I'd give more trust to a trained parrot than a computer to grade my coins. But don't trust me, spend a day or two doing some hard research on the limitations of AI and see what the real experts think about the kind of "intelligence" computers possess.
Regards,
Old Pushkin
Cool! I'm an expert!
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars • Variety Attribution
If you collect coins minted prior to the civil war it will never work. Dipped coins, differences in strikes and the level of technology during this period make it impossible to do it correctly on a consistent basis.
You're famous
I could see it turning out to be a re-grade fiasco.
I picture it this way.
Some TPG will come out with the 'promise of a fool proof computer program grading system'
Great!!...all is well with the collecting world for about 5 years, then......
a new and improved computer program that will 'promise' to correct any glitches the first system had will be introduced and the 'buzz talk' in the coin forums will be to get your coins regraded that were graded by the first computer system.
On and on it'll go.
Then.....there'll be a computer generated sticker program that will 'absolutely, without a doubt (this time) guarantee the grade's accuracy"
Oh boy.....we'll all be wishing that we kept our coins in the blue Whitmans coin book.
"“Those who sacrifice liberty for security/safety deserve neither.“(Benjamin Franklin)
"I only golf on days that end in 'Y'" (DE59)
Why not computer assisted grading??? Let the human graders look at the coins and assign the grades... but then scan each coin as well to make sure that coin hasn't already been in at the service before. If the computer scanner says the coin has been there before, the human graders could then see what they did last time (or the many times the coin has been in) and be much better informed if the prior graders saw an issue or graded the coin differently than they think they see it today.
Oh wait, PCGS came out with that technology about 7 years ago and it's called Secure Plus!!! Too bad that service was not fully implemented and made mandatory from the start as things may be a little different today if it had been.
Michael Kittle Rare Coins --- 1908-S Indian Head Cent Grading Set --- No. 1 1909 Mint Set --- Kittlecoins on Facebook --- Long Beach Table 448
This is what I wanted to know about from someone familiar with the current status of thing. I am aware of earlier tries to computerize grading, but I wondered whether algorithms had reached the levels of sophistication that that shortcomings of the 1990s efforts could have been resolved. From a layman's point of view, Siri, self-driving cars and 99% accurate voice recognition suggested that we may have arrived. Alas, perhaps not yet.
Conceptually, though, it still seems a worthy goal, that a coin could be consistently technically graded according to some fixed standard.
I very much agree with you, Illini420, that Secure Plus at least reliably identifies coins, and so potentially gives a basis to stabilize grade inflation pressure.
Here's a warning parable for coin collectors...
It won't happen because of subjectivity of grading (human element), possible loss of revenue to TPgs (the coin would be correctly graded the first time), and, oh yeah, the most important reason--greed (no more possible "just-barely-made-it-top-pop-coin") because graders had a lucky night the night before.
TPGs don't want the random lottery effect taken away.
Inconsistency = more submissions.
As soon as a paper on AI is published saying it can't be done, it's obsolete. Technology moves on.
This thread is worthless w/o a picture!
Or better yet, a truly graded Compugrade coin.
Yah...OK... looks 63.4, but that spot bothers me.
One of the big problems is establishment of the ground truth which is necessary to both train and validate a computer grading system. If the past 30 years of grading has shown us anything, is that ground truth, which is determined by the market, is at best elusive, and at worst variable. You can't make a permanently invariant solution for a market that has a variable ground truth.
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars • Variety Attribution
I see LakeSammaman found an MS 63.4 example.
Here's an MS 63.8, apparently:
and an MS 63.5 PL which has the fractional part on the top title:
There will always be those who say it can't be done....and those who will go out and do it!! Cheers, RickO
This will absolutely come. It won't be PCGS or NGC who brings it.
Watson, for coins.
There is also a difference between science, dogma, and sometimes wishful thinking. More importantly, a difference between what is theoretically possible and what resources are available to bring theory into practicality. Nuclear fusion (not fission) as a viable energy source is theoretically possible, but years of research and hundreds of millions of dollars in investment have produced only a few fleeting positive results and no viable energy source (okay, I cherrypicked this example). Yes, in twenty, thirty, perhaps fifty years with a lot more investment there will likely be significant progress in fusion, but that doesn't mean it will happen in our lifetimes - even it we wish really, really hard that it would happen sooner. Grading coins using AI - if you want to invest in it fine. Some people may wish to invest nuclear fusion, some in black hole containment, some in AI coin grading, these are their choices, not mine.
Computers can crunch numbers with speed and accuracy far beyond the human mind's ability to do so, they can even solve theoretical math problems that were intractable just a few years ago, and look at computer algebra, commercial programs like Mathematica are miraculous. Computers can replace many tasks previously done by humans on assembly lines and in other areas of manufacturing, and so on and so on. Performing the tasks necessary to grade coins, recognition issues and their many subtleties, is another matter. Possibly, someday, maybe, but that doesn't mean the financial resources to invest in AI for some tasks are available or desirable. It is still impossible for AI to perform many relatively simple human tasks like grading coins (and the "Docs" will always be one step ahead : ).
Can be done, yes, will be done, perhaps in a different lifetime or a different universe.
As always, my advice is do your own research and be objective when you do it.
Just my opinion, and opinion is cheap (AI is expensive).
Hmmmm guidance perhaps but I'm thinking that, for example, bust coinage what with the strike variations etc. the particular characteristics of each and every one would have to be programmed. Daunting.
Possibly for 'technical' grades, but I can not see optical scanners getting luster or eye appeal very well.
But what if the computer made "a recommendation" based upon technical grade and a human finalizer made the last call based upon luster and eye appeal? Nothing would be missed.
Bump
"Computers could very well do the grading. Humans will still do the pricing, though
"
That boat sailed on day one, when Dr. Sheldon set up his grading scale in the late 1940's--it was based on prices for 1794 large cents at that time, and his 'science of cent values' remained in use, and applied to other types of coins, primarily because there was nothing better that both collectors and dealers would accept. Over the intervening years, it has become a patchwork of hand-waving and rationales that won't stand up to rigorous examination. In short, grading and pricing are ensnared in a jumble of circular reasoning. There are still dealers who 'grade' their coins for sale using dollars, rather than the 70-point scale--that is more honest IMO. In some cases involving eye appeal associated with wild color, the marketplace value for the color is what will drive the extended price, not the technical grade.
I don't see computer grading solving anything. Embrace the craziness, and figure out a way to use it to your own profit.
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
Maybe one day, but we've yet to develop a computer remotely as complex and specialized as the human eye. Until then, computer grading will never happen.
Andrew Blinkiewicz-Heritage
The human brain is the problem, not the eye. The more central problem, not totally solved yet, is how to use an assigned grade to facilitate a coin transaction. Remember, the grade is a proxy for a marketplace value (and vice-versa).
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
I'd like to see it and I think it can be done now. It's just going to take someone with the money and motivation to make it happen. Facial recognition software has been around for years, so has finger print identification software. Heck even my iphone activates by using my finger print for authentication. So I think the basic technology is probably there for someone to build on. Plus, there are plenty of PCGS/NGC/CAC samples to "teach/calibrate" the computer for the variations/subtleties. Of course, I doubt any existing tpg service is going to "help" develop it. Can't be much worse than the wide variations we have today between TPG services (or even within a TPG service.)
Worked in plastic acrylics used in Areospace all the way to optical lenses I have seen Tens of millions flushed down the preverbal inspection line toilet. Great on paper not so much applicable in real life. Particulars in air, static electricity and constant tweeking of the parameters just drive up price, increase rejection of acceptable products and add cost to the process. So would you pay more for a modern ms coin? Would you not want a really valuable coin looked at and crossed checked? Just not seeing the return on this. But I might be wrong.
Best place to buy !
Bronze Associate member
How much money and time (during which no revenue comes in) will be required to develop the algorithm? There are vested interests that will not help with this. How long will it take just to assemble the sample set? I don't think that this is cost effective.
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
I would NOT want my coins graded by computer -- with a few exceptions.
Most new U.S. Proof coins grade 69 or 70. Let a computer decide. But only if it's cheaper & quicker.
New bullion coins too.
Successful BST transactions with forum members thebigeng, SPalladino, Zoidmeister, coin22lover, coinsarefun, jwitten, CommemKing.
I don't doubt the day will come; and certainly hand held scanners comparing images of coins in hand with online databases shouldn't be insuperable. Advantages would be not getting stung by problems as many raw coins have problems. Remember how weak chess computers were back in the 1970s, and how that continued to be the reality until the 1990s; and then IBM, etc. started to put up a lot of money to essentially solve the game. We're around 20 years past those advances, now anyone can buy a program or download one stronger than the top human.
Computer grading won’t work for a multitude of reasons. One major one is who decides what a 63 (or any other grade looks like)? It’s still subjective. The second is the large training set required to teach a computer to grade (think machine learning and artificial intelligence). You’ve now trained it on a subjective data set.
Long story short, grading is too subjective to be taught to a computer.
Choice Numismatics www.ChoiceCoin.com
CN eBay
All of my collection is in a safe deposit box!
Chess has a ground truth. Coin grading doesn’t.
Choice Numismatics www.ChoiceCoin.com
CN eBay
All of my collection is in a safe deposit box!
No. It is time for the market to realize that the greater the number of grading intervals and the more subtle the differences, the more subjective it becomes. The appearance of increased precision has come at a cost of great accuracy. Let's go back to the good old days of AU50, 55, 58, and MS60, 63, 65, 67, and 70 and let the market distinguish the rest. It's also time to dump market grading.
As for the use of computers, how do you write standards for humans much less computers to quantify and account for eye appeal? I think it would be virtually impossible to write a decent computer program that accounts for ALL of the contingencies that the machine would encounter in normal grading.
I edited for you
When I think of this, it might help to take a huge step back and look at the big picture.
Coins don't have grades. A grade cannot be precisely determined as there is NO STANDARD. The mass, diameter, reflectivity, smoothness, and composition of a coin can be determined to many decimal places. Grades are just a representation of what we find important, and the various bits of what determines that wander all over the place over time. I don't see a way around this.
Color right now is "in". At times it has been commonplace to dip a coin virtually every time it traded hands. Strike is enormously important to people like Q.D. Bowers, but others barely notice it. Luster seems to be strongly favored in the grading room today. Stick around for 10 years and the emphasis on even the most measurable attributes will change.
Why bother grading coins?
So we can assemble the "best" set?
As a proxy for determining value?
For research purposes?
I don't think we grade them simply to rank them. Assembling the "best" set (for Registry glory) drives a good part of the market, but in practical terms we grade coins most often so that we can get a shorthand appraisal of the coin's value. Since the market value of a coin is tied enormously to eye appeal (maybe to a greater degree than any other attribute), a purely technical grade isn't all that useful to the market. This (and the TPG's constant need to keep grading coins) is the reason we've gradually drifted from a mostly technical grade to a mostly market grade over the last few decades. The trend is continuing.
The real issue is that we collect coins because we like them. What we like, on average, wanders all over the place over time. Bell bottoms anyone? When it comes to coins, this is true for even those parts of the equation that we can precisely measure.
The best we can do is to kinda sorta figure out which ones we like (and are willing to pay for) more than others. It's a big fat average and exceptions abound. That's just the way it is with ALL subjective evaluations.
@BryceM said: "I don't think we grade them simply TO RANK THEM. Assembling the "best" set (for Registry glory) drives a good part of the market, but in practical terms we grade coins most often so that we can get a shorthand appraisal of the coin's value. Since the market value of a coin is tied enormously to eye appeal (maybe to a greater degree than any other attribute), a purely technical grade isn't all that useful to the market. This (and the TPG's constant need to keep grading coins) is the reason we've gradually drifted from a mostly technical grade to a mostly market grade over the last few decades. The trend is continuing.
I have a different opinion. Decades ago, on a flight to teach my first coin grading seminar, I realized that in order to teach students to grade I should be able to define it. This is the definition I came up with and it is still what I start my classes with today:
Grading is a SUBJECTIVE observation made to access the condition of preservation and RELATIVE RANKING among similar objects. In actuality, it has NOTHING to do with value (we all know that all MS-65's are not equal) or even coins themselves. We grade everything both consciously and subconsciously. When I go to the store, I pick out the firm yellow bananas with a slight hint of green at one end and pass over the soft, black, oozing ones with flies buzzing all over their skin.
Think about this and perhaps you may modify your opinion.
"Grading is an objective observation made to access the condition of preservation and RELATIVE RANKING among similar objects. In actuality, it has NOTHING to do with value (we all know that all MS-65's are not equal) or even coins themselves."
Well, I thought about it. Condition census coins have been ranked on a relative scale (whether everybody believes the ranking is another matter). But a VF35 that magically becomes an EF40 five or ten years later is just a pawn in a game (or are we--the collectors--the pawns?). Grading is NOT objective---it is associated with businesses whose primary goal is to maintain a decent revenue stream. The lack of objectivity flows to decisions concerning 'liner' coins where liability is a factor in making a decision, and also to the re-grade game in which a hoped-for grade increase is possible, but a decrease can be stopped by the submitter.
My training is in the physical sciences and engineering---there is little about grading (as practiced during the last 50 years that I have been a coin collector) that is objective. In fact, a measurement system like today's coin grading would not be tolerated in the sciences or engineering. Would you accept a thermometer whose temperature measurement scale tends to creep up with time?
I do like your bananas analogy, which sums up my feeling about the state of grading. However, when you are selecting your bananas, you are doing so subjectively.
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
No