Thoughts on Automated Coin Grading – MS66, or MS67, or AU?
RogerB
Posts: 8,852 ✭✭✭✭✭
Concepts for discussion.
The subject of computer-assisted or automated coin grading comes up with surprising frequency in casual conversation among collectors. Usually it is quickly dismissed as unworkable with failures of several 1990s era prototypes given as examples.
Yet, worldwide industries, and even our own U.S. Mint, have extremely accurate, reliable, automated parts recognition and quality control system in daily use. These modern systems are capable of orienting parts (or coins) to a uniform position, scanning the surfaces, and outputting consistent and precise reports on every part (or coin) it processes. The U.S. Mint’s automated proof coin examination system processed over 10 million coins in 2017; all without human bias.
Industrial systems handle ten- or 100-times the Mint’s volume.
The quality control system at the US Mint orients and examines every proof coin and rejects those that do not meet pre-determined standards. This is almost exactly like what a “grading” company does, and is especially applicable to bulk product grading performed for telemarketers and other retail outlets. The only conceptual difference between an industrial quality control system and coin grading is that a “grading system” must provide alternative products paths for items of differing numeric “grades.” This is largely a software tweak with appropriate physical paths to packaging added.
While the about concept is most directly applicable to TPG bulk products, it can be used for any coin design, although the additional definition and orientation programming will require investment. Results will be consistent and standard to within system specifications and good workflow design will allow for diversion of ambiguous results to human evaluation, as needed.
Comments
good topic Roger. I think to discuss further it might need to be broken up into many groups. You start with the process to create the coin, develop a standard and go on from that point. Old coins also will need to be handled with TLC.
Question I would ask is technology and the cost of it cheap enough to invest the time & energy. what will be the upkeep cost. Will it add value and to whom? Is the market willing to pay that for it. I find education a big part of the fun in coin collecting. Trade offs.
Best place to buy !
Bronze Associate member
I don't think there is any issue with automated coin grading as long as everyone is willing to accept that coins would then be graded purely by their "technical grades."
Gobrecht's Engraved Mature Head Large Cent Model
https://www.instagram.com/rexrarities/?hl=en
All true. For Moderns. Very doable.
But the problem for coins that were produced using one uniform production technique has no meaningful equivalence with less well-manufactured coins with tolerances that currently-used quality-control techniques would throw out at the ineffective MS/PR68 level. That would be a instant rejection for an aerospace rivet. That 69/70 line is science.
How many images of 1884-O $1's in whatever grade are needed to program your AI database? Well or soft struck, die-rust pimples or not. And this is before any PMD is considered.
Now let's grade an 1808 50c in VF-XF. That is an art.
Doable? Certainly. But tens of thousands of man-hours at what cost? Coins will likely forever be a cottage industry.
I expect that we will see automated grading for MS69/70 Eagles in the next few years. I think people would accept computer results in that specific area and it wouldn’t be nearly as technically challenging as any other type of grading.
I have a question. Not picking on you but as you brought it up.... I should like to knw what exactly is a "technical grade?""
I am hoping with enough emails from concerned citizens that a blue ribbon panel of Presidential experts from the IT and numismatic community could come together to resolve this perplexing problem of grading disparities once and for all.
Agree to much data to be sorted, but see it as usable for the moderns that the market for is crashing as we type this.
Kkathy - Volume would be a major problem. The TPGs' volume, even for bulk orders is tiny compared to any manufacturing (or minting) operation.
I'd be concerned !!!
Someone is going to need to teach the computer to differentiate between tiny struck thrus and tiny impact damage. Some TPGS allow coins with tiny mint-made defects still grade MS/PR-70. Would /do collectors care?
I'll just be (near) polite and just say that I totally totally doubt the machines will ever be good enough to grade coins. Too many variables to be considered, too many highly fluid variables to be synthesized into a final "number", too many conflicting variables which affect human preferences one way versus another, and then there will probably be too many intelligent but foxy folks who will be trying to "game" and "beat" whatever AI system that happens to be devised.
They'll have AI to totally replace academic researchers long before they have AI to properly grade coins.
IMHO, Grading coins is more of an "art" than a "science".
Exactly right! How is a computer going to decide that a coin that is bright white yet softly struck is better than a beautifully toned coin with a small rim nick? I am the buyer, not the computer. It is time for a new hobby if the computer starts telling me what to buy or not buy.
In terms of colonials, how the heck is a computer going to know diagnostics of certain varieties? Is it an early die or terminal die state? I have seen coins that others would grade VG that are in actuality two or three grades higher. In addition I have seen slabbed and raw colonials that are a couple grades too high. It is an inexact science. If you are into cobs, that is a whole other ballgame. I’m lucky to find a small handful a year that I really like. Makes the chase a lot more challenging yet fun.
There have been several threads on this and other forums asking if it's possible to photograph "rub". I've tried a couple times to show it clearly, but it is not an easy thing to do. If anyone has successfully shown rub photographically, then I'd like to see it. Until rub can be viewed optically (the basis of being photographed) and objectively distinguished from other types of pre-circulation damage, computer grading can't happen.
http://macrocoins.com
Developing an automated, computer grading system is mostly a software problem. However, algorithm development would be a lengthy, expensive process. I don't think that it would pay for itself, unless a limited use were envisioned (e.g., grading MS70/PR70 modern coinsthat are untoned).
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
I'm not sure why the idea of computer grading and human grading are seemingly orthogonal in people's minds. I could easily see computer grading augmenting the existing process in order to speed it up and eliminate mistakes. I believe there are several folks looking at a coin now, and some sort of consensus is reached to arrive at the final grade. Why couldn't a computer be one of those "folks"? The computer would give an opinion based on the technical merits of the coin such as marks and hits, surface condition and wear, technical luster, and authenticity. The computer can look at a lot more details a lot faster than a human can, and make a basic technical judgement about the coin. One or more human graders can look at the coin more for aesthetic aspects, such as eye appeal, that would be more difficult to program accurately.
http://macrocoins.com
Would the computer be trained to ignore published grading standards too?
I'd expect a grading standards calibration file to be referenced for the objective grade. How many hits of what size, how much area of rub, how many spots and of what size, etc etc etc. These can be adjusted according to current market acceptability.
http://macrocoins.com
I think there is a significant difference between "Yes" and "No", (the binary decision making the mint system apparently uses), and a system that could consistently give a 60, 61, 62, 63, 64.....etc.
I'd also like to know the mistake rate of their Yes/No decision, i.e., how often a "good" coin gets rejected, or a "bad" coin gets accepted.
This isn't even considering the seemingly simple, but actually complex, decision about circulated vs. mint state....and the added complexity of dark colored vs. a light colored coin would be. (The Mint objects of interest are all bright white, freshly minted).
I'm an engineer, and have a great appreciation for what technology can do. But I think automatic coin grading has the same future as autonomous cars. As soon as you release "the version" that you are SURE works, some combination of sun angle, rain, and road conditions is still going to get someone killed.....
(Ok...nobody ever died grading coins. To my knowledge.)
It can be done for moderns, especially post 1971. Anything before that has variables based on date, mint, tarnish, damage, etc.
One other quick thought, (and this applies when you hear that YOUR job will someday taken by a machine):
Once you've spent the thousands of man hours required to write the software, tweak the hardware, and build the machine that will do this automatic grading.....You've probably spent enough money to hire a dozen graders for 5 years.
It just won't be cost effective.
The mirrors on modern proofs should be quantifiable. It's a grading service dream, actually. All of a sudden an MS70 with 7" mirrors is "better" than one with 6".
You blast some light at the coin, and measure how much comes back.
"You blast some light at the coin, and measure how much comes back."
I am waiting with baited breath. This has to beat my current system---I blast some money at a coin, and measure how much less comes back.
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
I do not think that computer grading will ever be "market acceptable". Computer grading would be absolute whereas TPG grading is subjective. While a computer could assign an accurate technical grade, subjective qualities of the coin would require a preferrence being programmed into Artiful Intelligence computer routines. Probably not happening in our life times.
OINK
I've written quite a few image processing algorithms and programs over the years (that was my regular "day job" from about 1990 to 2005 or so).
With large quantities of very similar modern coins, finding an anomaly (a defect) via automated inspection is fairly easy. The US Mint can do it.
The more subjective the grading becomes, the more difficult it is to do it via automated (computer) means.
I don't think there will be any successful computerized grading until such time that AI is advanced enough that we will have "Mr. Data" and the like walking around amongst us. I do think that will eventually happen, though.
I would have said the same thing about self driving cars just a few years ago. When it comes to coins I would not be surprised to see the 3-D printer copying technology perfected to a degree that is beyond our present imagination as well.
The eventual arrival of computer grading is inevitable - though when this will happen is still far in the future. AI will eventually allow the technical parameters of coin evaluation to be programmed....Factors such as colorful tarnish vs. ugly stains are another matter - in other words, eye appeal (which tends to evolve in human perception). It would be nice to eliminate the subjectivity from grading...and this would also eliminate gradeflation.....Cheers, RickO
A computer can do a lot of things faster and better than humans.......grading coins is not one of them....and never will be.
Eye appeal keeps being brought up. To me eye appeal should have nothing to do with grade. Eye appeal comes into play when you have two coins with the same grade.......one is OK and the other is beautiful. One cost $100 and the other may cost $200 or more. Leave eye appeal out of grading. Grade the coin based on what state it is in from which it started. JMHO.
Interesting how many appear to romanticize coins. They are nothing but (usually) metallic objects confined to specific dimensions, relief, reflectivity and color. An automated quality control system does not decide how the object is used, or even if it is suitable for anything - only that it deviates from specification in certain specifics.
Such a system, already in common manufacturing use, when scaled down and modified for coin grading, could bang out precisely and consistency graded ASEs and other bulk products as fast as fresh coins could be input and graded ones packaged. In the business world of electronics the kinds of detail some posters mention is trivial when placed next to parts and assembly specifications and definitions.
An automated system does not make an emotional judgement. It does not "like" or "dislike" - it only reports results against defined standard. People - coin collectors - impose their individual opinions as derived from their experiences, and that is, I feel, their best and proper role.
In effect: let the dumb machine do the grunt work; let smart people decide what they like -- and how much it is worth.
All for it if it brings grading fees down. There always be Coins high end / low end based on various individual preferences - this wb worked out between buyers and sellers.
the Colonel raises a good point, but if a collector/dealer of his ilk can learn those things over a period of time I would expect that enough information could be programmed. I would expect that there would need to be a baseline for grading coins and then the "open, free market" would price everything.
imagine the dilemma that would be placed before collectors and dealers. we would all actually need to LEARN things about the coins we collect and the nuances associated with their manufacture. right now, we trust the TPG's to know all the stuff we should know, we expect them to grade accordingly and then the coins are sold at pre-determined prices that the TPG's know about and participate in controlling.
we bitch about "market grading" but do we really want it to go away?? we pretend to know more than we actually do when in some regards all we are capable of is grading to our own level and reading a price guide. if the TPG's only technical graded we could decide what eye appeal was worth, whether a strike designation was really visible and what it was worth, we could add any premium to whatever asset a coin has that we feel certain about.
A thought - Based on several commercial systems I've seen within the past decade, everything, every nuance and subtly collectors mention about factual descriptions of a coin, can be defined within an automated identification and sorting system. (This can also include tarnish patterns and colors.)
While errors still occur in such systems, they are nearly always attributable to causes outside the system - usually from humans doing something inadvertent or intentionally incorrect. (This would occur if proof ASEs were being graded and someone added a Morgan dollar to the line.)
As the Colonel and others mention, cost effectiveness of this would be a deciding factor. Hucksters can push only a certain quantity of slabbed ASEs per year. Value might be achieved if any coin could be examined and "graded" but the software development cost for circulated (and easily human graded) coins might prevent reaching a profit point.
Probably a bit of hyperbole, but I bet I could find a millenial or Z'er who could make it work fairly well with an iphone app in a weekend.
http://macrocoins.com
I've written about computer grading on this forum a few times in the past, so I guess I'll do it again, to see if my response is as least as consistent as "the market's" taste for grading.
Short answer: Computer grading is possible.
Start of longer answer: But...
Rest of longer answer: Not all coins grade the same way. Circulated Barber coinage would probably be the easiest to do. The wear patterns are very consistent, and there aren't major design voids that show up in some dates and mints that don't in others. No need to look at luster on a moderately to heavily circulated coin, either, as you're only looking at the amount of wear. ASEs that are either 69 or 70 are graded simply by localization and quantification of surface defects, which is also not too difficult. Contrast that with uncirculated coins, even something relatively simple like a Morgan dollar, where in order to get good and consistent results, a human grader needs to tip and twirl the coin under a light they are used to in order to get a read on the surfaces. Even though it can only take a few seconds, there is a huge amount of data being acquired here by the grader. Things like fine scratches or wheel marks could be missed with the wrong light (i.e., too little data or corrupted data). Something might catch your eye that suddenly warrants pulling out a magnifier for closer inspection (i.e., need more data). In order for a computer to grade these coins, they need to be shown the coins with the same high quality imagery that a human grader can get, which is a significant undertaking, especially if you want to automate it.
Let's assume you can acquire this data consistently. The next problem you run into is that the computer has to be taught how to grade from that very large amount of imagery. This is a task for a deep neural network. The thing is, you have to train the network with a lot of data, which means you have to re-image all the types of coins you plan on grading in a manner consistent with the imagery you plan on using to do the grading. This new, huge amount of data represents your ground truth. Selecting what coins to image in this manner needs to consider distribution of grades across the population of coins. If you train on images of a bag of 1000 MS63 1884-O Morgan dollars, then every Morgan dollar will grade MS63. If you don't train on enough MS66 and MS67 Morgan dollars, you won't get a good distinction between the two grades when you actually grade them. In general, if you don't handle the biases in your input data correctly, you'll get results that need to be adjusted.
Having to adjust your results can simply mean re-training with more data, but then you have the same issue you have today, where grades shift over time. You'll then get people chasing coins that were graded by a certain version of the software because they'll upgrade with the current version. You could also get people suddenly stop grading coins of a certain type because they've heard that these are graded tighter all of the sudden due to algorithm re-training. Either way, you have the same inconsistency problem you have today.
Now let's assume, on top of all the other assumptions that you can successfully train such a network once and for all. Suddenly, the crack-out and re-submission game ends. If I'm a TPG, I don't want that.
So to summarize my long answer into a better short answer, computer grading is possible, but not a trivial undertaking, and not just because of technology.
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars • Variety Attribution
why not just put the slab on at the end of the Mint production line, call it a day.
Best place to buy !
Bronze Associate member
Yes then they can say 1st strike 2nd strike and so on and so on. Who will make $$$ on this every one but us. We will try but you know the out come.
Hoard the keys.
Photography is static, visible light. Automated grading would potentially use a number of technologies: video, dynamic lighting/motion, lasers, etc. It opens up a lot more possibilities for determining wear. It would not just be grading a picture.
My opinion as a software developer is similar to others here, automated grading is certainly possible for most coins. However, the economics likely make it unfeasible for perhaps all but the most trivial cases. Messydesk does a good job describing some of the difficulties in acquiring datasets.
Just a couple of clarifying comments. Anything that we can observe and identify can also be coded into modern software. While it's nice to think of our skills as superior to machine possibilities, in the examination and evaluation of objects and images, computer systems are much faster and accurate than people - even those who have lots of experience. A computer never has a bad day, or a hangover, or an emotional opinion, or get tired. Further, computer "imagery" is far more detailed, over a much wider range of parameters than any human - and this is not unusual; it's routine today. We examine a coin visually. A computer controlled system can do that at a thousand wavelengths beyond human vision and a million angles, and then combine those with relief measurements acoustical signatures and even add (although expensive) surface chemical analysis to catch "putty" and other nefarious actions. Alterations invisible to the very best authenticators can become obvious with appropriate data processing.
Everything I have mentioned is in corporate or non-classified government use today and available from systems providers.
Please maintain the discussion as long as members are interested.
Closing thoughts -
The purpose of this post was to bring to the attention of TPGs the opportunities to improve quality and consistency, at potentially lower cost for bulk products. Adding oft-seen collector coins, such as double eagles and silver dollars, might also be viable. It is up to the TPGs to determine if purchase of an automated grading system for these purposes is financially viable.
Video, dynamic lighting/motion, even lasers (combined with sensors) are all part of photography. Ultimately, the coin must be scanned in some way (photographed, video'd, laser profiled, SEM, whatever), and those scans analyzed (software). My point is that the output of the scans must contain all the relevant data, and the elusive "rub" is a particularly difficult thing to capture photographically.
http://macrocoins.com
Ordinary photography is "people scale." I refer to "research scale" imagery; much further outside the box.
@rmpsrpms said: "Video, dynamic lighting/motion, even lasers (combined with sensors) are all part of photography. Ultimately, the coin must be scanned in some way (photographed, video'd, laser profiled, SEM, whatever), and those scans analyzed (software). My point is that the output of the scans must contain all the relevant data, and the elusive "rub" is a particularly difficult thing to capture photographically.
Maybe a team of professional TPGS graders could examine it for "rub" after all the imaging is finished.
It's all just photography. Shining light and recording the reflections.
Actually, SEM is not.
http://macrocoins.com
We see only a little of what is to be seen. That is "people scale" photography. Research imagery is much broader in wavelengths and higher resolution.
Out of curiosity who is going to program the computer? Is it a human? If so, then how can this work? I have been watching these guess the grade threads on the forum and the grades posted are all over the place. Everyone has a different opinion and in their eyes they are right. Some people like white coins, others heavily toned, others with a bit of PL, others prefer a light bluish tone. Some people don’t mind scratches, others a small rim bump. That’s great! They are all opinions and the collector is happy with what he or she has bought. How will the computer be able to distinguish a poorly struck coin in XF from an XF with buckled dies? How would it know that a certain date comes struck softly on the eagles left wing? Good luck with pre-1828 coinage. On top of that, how to place a value on the coin. Is the computer going to do that too?
Let's be honest, there is no desire for the services to implement computer grading and think of the can of worms that it would open:
(1) The services rely on changing grades (cycles of loosening and tightening) for revenue. If that disappears so does a large part of their revenue stream. It is even more problematic if you use algorithms so someone can objectively prove with ease that the standards change (no more "grading is subjective").
(2) Let's say a rich collector has an expensive 6 or 7 figure coin and disagrees with the grading service on the grade. The coin is sent in under the guarantee. The service denies the claim. The party is angry and sues. That party can now use discovery (or if a third party through subpoena power) can force the service to reveal its algorithm. This would (1) create a way to to reveal information that would undermine market dominance and allow for easy replication and (2) give ammo to those with an ax to grind. This is true regardless of whether the plaintiff's claim has merit or not. If he can survive a motion to dismiss or analogous pleading, discovery would likely be permitted.
You're speaking like someone who knows about these techniques, but exactly what are you referring to? I'm familiar with most of the imaging techniques used for physical failure analysis (as a small part of my job), so which techniques are you thinking would be useful for grading? Remember, it must be fast, non-destructive, and cheap...
http://macrocoins.com
Yes, the technologies to do this have been available for a long time. My naysing about rub is just more hyperbole, since of course this can be overcome, and likely in a clever way. I have a feeling that it has already been prototyped, possibly multiple times, by one or more TPGs (just a hunch...).
I think ultimately the market is not able to accept a purely objective/technical grading standard for reasons above and more. I'd wager that many, many coins in BU holders would end up as high end AU's, and that is just not where folks want to go. Hmm, I came full-circle back to rub, didn't I?
http://macrocoins.com
Maybe, but a big part of the allure of coin collecting for me is the mystery, possibility, randomness, uniqueness, and opportunity that is all contained in one little metal disk. Reducing this to a simple grade is already borderline preposterous. Trying to refine this further with an algorithm is cold and takes something of the soul out of what is supposed to be a slightly mysterious hobby.
Yup. When I'm fleeced, I'd much prefer that it be done with an aura of mystery, even uniqueness. But not randomness, since I would hope that I'd be viewed as 'special'.
RMR: 'Wer, wenn ich schriee, hörte mich denn aus der Engel Ordnungen?'
CJ: 'No one!' [Ain't no angels in the coin biz]
And hence the "real" reason computer grading has not taken hold.
However, if there were a service that did technical grading by computer, don't you think it would be competitive? I really don't know.
http://macrocoins.com
Digital photography has come a very long way in a very short time, but ultimately the end product to be acceptable requires the human brain. A computer graded coin will at least require a human reviewer.