AI Grading- Appears Much Closer Than I Thought - Implications?
JCH22
Posts: 341 ✭✭✭✭
Came across a very intersting Master Thesis--- “Development of an Automated Coin Grading System: Integrating Image reprocessing, Feature Extraction, and ML Modeling, by Jianzhu Chen, submitted to Virginia Polytechnic Institute for his Masters in Electrical and Computer Engineering in 2024.
Apart from being a mouthful, is a long read—94 pages—
Summary line—he was able to develop/demonstrate a promising model nicely matching NGC & PCGS grading of Franklins, including toners.
His model is far from prime time, but given the exponential increases in AI, the future of non-human coin grading appears something that's not too far off. Might almost be there with something like generic bulk Silver Eagles.
I guess in the future, discussions will center around whose algorithm is tighter, which is better for which series/issues etc. Perhaps even a Grand Master Computer-- most strict Algo--- can review the others---give whatever the equivalent of an Algo sticker might be?
Will there be a separate market for human graded holders? Would these be thought less valuable, or more, given they were graded by us subjective & fallible humans? Registry?
Would very much appreciate any thoughts on what others might have good/bad, consequences/ramifications ... Seems just a matter of time....
Comments
if the resolution from the pcgs' coin fingerprinter is high enough, ai might like to use that more than a picture with potential lighting issues
where there are enough examples to do so, a sample "ms70" version of each coin could be made from a composite of other lesser graded coins... without the date and mm, which can vary
hmmmm. how would it handle die gouges and die cracks
and removing doubt about a details-scratched vs a lesser long hit
or graffiti
It could be trained using a human graded set, like some of the ANA grading sets.
It would also need a die variety set so it can learn die varieties.
Finally it would need @FlyingAl axial lighting, and probably a rotating image like Heritage is increasingly using for auctions.
It's completely doable, but standardized photography would be necessary to prevent deceiving the AI grader. Ironically, our hosts would be best positioned to standardize the imaging techniques.
implications
we'd still need graders to prevent mechanical errors since ai isn't always right
would continued human involvement mean that ai use might be slower because you have to light the coi then press a button, then check for mechanical errors or to label something as details or reverse a details opinion
There will be real blowback with ai systems ability to grade more consistently certain criteria nicks, scratches etc. vs humans. PR issues would develop with current standards on iconic coins- classics. I don't see it in the near term.
Moderns however should be doable. Here is one scenario:
A major TPG goes with AI on a 100 pt scale. Thus separating the human 70 pt and giving the choice to the collector without rendering either obsolete by force, letting the market decide desirability.
hmmm... see below
pr issues - think of all the embarrassing regrade results and the grade guarantees
and moderns... would 70s be possible for every new mint issue? then there are regrades to get from a 69 to a ....
...100?
they better use 100 to cloud the grade guarantee issue
for example a 69 regrade might make smoeone happy to receive a 99 since 69/70 < 99/100... versus continuing to use 70 might find a bunch of 69's as 70s or worse a bunch of 70s as 69s
There is definitely a business case for it. The training is the key and those who train it are putting themselves out of a job.
It doesn't take a grader to train it. You just need a grading set to train it.
The human also isn't necessarily out of a job. Their job just changes to finalizers and QC. They're might be fewer humans but not zero.
Now all they will need to do is train AI to become actual collectors! They won't need humans to grade them or buy them.
Just make sure the power outlet is carefully guarded. The human janitor may just decide to pull the plug.
(This was actually the theme of a circa 1960 comic book story. A computer had taken over the world and the janitor saved the world by doing just that.)
If it happens then Grading will become totally meaningless.
I see little hope or acceptance of an AI only grading program. I think the vast majority of collector will want a human finalizer. The current AI programs have one huge flaw. They are all dirivitive. They only build on something else. There is no original thinking. If even the smallest flaw, mistake or false fact gets included them it becomes part of the AI programs bedrock thinking.
I do assume that there will be future generations of AI and I won't pre-judge them, but they will have to teach AI that it can both make mistakes and how to correct them. James
Mr_Spud
I see it as a great way to grade moderns and really push the volume out. The grading on those issues is a lot less suggestive (or should be). No toning, circulation, etc to really worry about. Mostly splitting the 69s and 70s. Just need QC on something that drops lower that might be due to mint error (struck through, delam, etc).
You could have written this same thing about TPGs rather than AIs in 1990. That didn't age well.
The AIs (LLMs) are more creative than you give them credit. See Paul Schrader's comments on films.
When humans are being completely "creative", it is called insanity. AIs neural networks work very similarly to human. Human creativity is also trained and "derivative". (See literature in "perception filters".)
There's a reason that movie plots contain all the same themes as Greek plays.
what/who is being used now to prevent mechanical errors?
does anyone know the ratio of mechanical errors now? 1/1000 1/100000 ?
Well, the humans also make the "mechanical errors" so...
AI is used for some of the most complex medical analysis, business/financial decisions and space exploration. Think grading coins is easily doable for most coin series. As noted by @jmlanzaf, a comprehensive grading set needs to be feed into the program. This is a lot of work but for most series, there are plenty of examples. A human finalizer would still be needed and any AI 'errors' would require updates to the program so it will always evolve. The good point, but probably an issue with the crackout players, is the same grade will come back every time
.
Great point about shifting away from Sheldon. Trying to envision the fallout from such a move. Can see moderns as the guinea pigs, but classics no doubt would follow as capabilities increase. Otherwise, the markets would be bifurcated. Is true they are sort of now with EAC, but that’s its own discrete niche.
Will current top census coins be at jeopardy of “downgrading” on a 100pt scale? Would the move inject all kinds of uncertainty in the market/value for those, as well as more common coins?
Will there be a new registry scramble—or just parallel human and a machine registries.
What would be seem more favorably by the market--- a gold bean old holder 65, or a machine 97?
Will the investor be looking for coins with “machine appeal” as opposed to a collector’s eye appeal?
Market seems headed to some kind of fundamental changes. TPGS profoundly changed the market in their own way. AI will have its legacy—what that will be be is sure an open question….
Not for me...
Experience the World through Numismatics...it's more than you can imagine.
@jmlanzaf . I'm not sure if you are calling 1990's graders robots or implying they were perceived that way. All I know is you can be first in line to buy the coins with no human finalizer. And I would say Boccaccio's The Decameron is the single most plagarized work in the world. Finally, when a Greek play writer made a mistake all the other writers didn't immeadiately start making the same mistake. James
Neither. I'm saying that people said that collectors would not want 3rd party graded coins.
No one said there would be no human finalizer. It remains to be seen. If there isn't, it's because it has been determined that it isn't needed. In that case, sign me up.
How much money is spent on resubmitting coins now either attempting to get the human to give it a modicum of grace or, if you prefer, to get the human to fix their prior mistake. People are constable complaining about grades but the thought of an AI suddenly makes you feel like the humans are infallible.
LLMs don't infect the system unless they are in "learn mode" which they are usually not when they are being used. An AI built to grade coins would not be running in learn mode.
If you are worried about a 100 pt scale differential do not submit your sheldon for AI crossover.
100p pt scale makes sense for moderns the program will be precise, maybe too precise but that can be tweaked.
Would not a collector want an AI92 versus an MS69?
I'd like to see the formula a computer comes up with for grading coins. Could be very useful to get a forumula for coin grading. Take a lot of the human eye out of the equation.
The substantial truth doctrine is an important defense in defamation law that allows individuals to avoid liability if the gist of their statement was true.
AI grading is sort of like the registry sets. Once you define it, changing the algorithm would cause a massive revolt among collectors. Or think of it this way - while a computer can flawless repeat a given algorithm, that assumes there is a starting point where the market universally accepts the algorithm to begin with. What if collector A thinks grading should be 5% based on the strike and collector B thinks it should be 10%?
_Will there be a separate market for human graded holders? Would these be thought less valuable, or more, given they were graded by us subjective & fallible humans? Registry?
Would very much appreciate any thoughts on what others might have good/bad, consequences/ramifications ... Seems just a matter of time...._
I could see technology in this arena evolving to a point for a phone app or something along those lines, but I would
not be interested in submitting coins to a TPG to have AI grade it.
AI and new technology is often touted as the answer to quality of worker, labor costs and/or labor shortages, improved productivity, consistency, workmanship issues, etc... and for many applications it certainly can be a solution to some of those drivers. It can also create an entirely new set of issues to contend with at the same time it addresses the motivating issues. I don't see coin grading as an area I would be fan of it.
There are "separate markets" (different pricing) for PCGS, NGC, CACG, ANACS and ICG. So, why not?
It’s going to happen. Nothing in the world can stop it nor should it. It will be better and more consistent than what we have now.
The things AI can do now in its infancy is astonishing. In a few years time, the world and workplace will be unrecognizable. I wish I was younger.
Humans will be freed to do far more interesting things.
Or will the vast majority of humans be reduced to the level of serfs who live only to serve the whims of the super-rich who control AI (and everything else)?
There are "separate markets" (different pricing) for PCGS, NGC, CACG, ANACS and ICG. So, why not?
@jmlanzaf - sure I could see a new TPG popping up with this sort of grading process. Could I be a potential buyer of such a product? sure, possibly. I don't necessarily restrict my purchases by TPG brand.
Would I be inclined to be a submitter to such a TPG? No, not likely. At least not until well vetted. If I was one to pursue grading of Silver Eagles or other modern coinage, sure I might consider, but those are not my areas of interest.
we'll be the hamsters on the wheel
No
AI will almost certainly come into coin grading, and it will probably creep in un-noticed rather than arrive with a flare of trumpets. Here's how I see AI infiltrating the coin grading process.
The currently-existing TPGs will first begin experimenting with AI grading algorithms. I would be very surprised if they haven't already started tinkering with this. They are in the best position to attempt this, since they already have a constant stream of fresh coins to experiment with and have an array of grading experts readily available to assist with programming and training the AI. A brand-new AI-only startup grading company would not have those advantages.
The outputs of the AI will be continually monitored and compared with the current human-grader opinions of those same coins. The output presumably gets better and better over time.
At some point, I would assume the algorithms will be able to successfully match the human graders 99.99% of the time. At that point, they will let the AI take over the mundane grading process entirely and the human graders will revert to being the reviewers if/when a grade is disputed and/or the people who open the packages and line up the coins for the AI camera to examine them. They will also need to keep humans in the loop for grading scarce coins, not because of some need for human oversight of expensive coins but simply because the training database for those coins would be too small to provide reliable outputs.
Whether the TPGs actually tell us when the AI takes over completely, will be up to them. Coin collectors are an inherently conservative lot, and we like to know who the people are that are handling and grading our coins. But I strongly suspect that by that stage, we the collectors would not be able to tell the difference between an AI-graded coin and a human-graded one, so we wouldn't care. So I wouldn't expect to see any difference in price or market esteem.
A difference that makes no difference is no difference.
Roman emperor Marcus Aurelius, "Meditations"
Apparently I have been awarded the DPOTD twice.
I agree with this with a few important caveats.
Most coins can be graded accurately using just the Trueview. Play GTG with a random assortment of Trueviews and you'll be close on most!
The problem is that certain coins can slip through the cracks, including coins with rim problems, hairlines, artificial toning, repairs, concealed scratches, etc.
In order to address the problem coins, you need better photography. That means higher resolution, better lighting, and more angles. TPGs will have liability issues if they use AI to straight-grade problem coins that would have been detected by humans.
Other issues are easier to solve. AI will quickly learn to mimic human aesthetic values, what the nerds call "alignment." That means giving credit for nice original album toning, and recognizing that marks on Lady Liberty are more significant than marks on the dentils.
Photography needs to be improved to do live AI grading, but the Trueview database is already sufficient for a proof-of-concept.
Have anyone ever think about the possibility that someone might use ai to create fake coin that can be graded? That’s a scary thought.
Is it fake if it can't be differentiated from an original?
Like what? And once we're all unemployed, what money will we use to do those "interesting things"?
Futurists have always overlooked the realities and downsides of their envisioned utopias.
Being "freed" from work means being "freed" from independence.
There's no doubt the world is changing, and some of the changes might be for the better, but many will not be for the better.
How, exactly? You can create transfer dies now if you wish. Creating counterfeits is about tooling not "intelligence".
Robot tax. There are solutions that have already been discussed.
This has all happened before. Data switches eliminated phone operators, for example. Streetlights eliminated lamplighters. The entire course of human history has been about technology replacing jobs done by humans.
Farming technology replaced 80% of farmers which fueled the industrial revolution.
@jmlanzaf is correct. Technology has taken over human activity throughout time. We find different stuff to do.
In my occupation, AI has freed up about 2 hours a day for me. I have automated a lot of low level administrative nonsense I had to deal with. That doesn’t scare me at all. I am far more productive. If the day ever comes that it can replace me, I’ll do what every other person in history has done. I’ll transition and find something else to do.
It’s coming much faster than most people realize. If your only information is what is written about by journalists, you have no idea what is coming.
But with AI we are talking about replacing virtually all jobs. We aren't talking about a handful of buggy whip makers having to change jobs, we're talking about countless numbers of people becoming unemployed.
And what will a robot tax do - allow us all to sit at home and still collect a check? I don't need a crystal ball to know how that will turn out.
People have floated a guaranteed minimum wage without a robot tax. I am not convinced that there isn't a dystopian side to all those retired people. But there are ways to use GDP to fund human livelihood.
But you are thinking about AI incorrectly, I think. AI makes humans more efficient, much like computers before them. Take any job you want. I'm a chemist. If I'm more efficient, one of two things happens: they hire fewer chemists or they do more chemistry. It is not automatic that people get displaced. We might just get more work done.
Robots - the dumb kind - have already replaced hundreds of thousands of auto workers. They have also made cars that are still affordable. A handmade care would be astronomically expensive.
It is NOT a handful of buggy whip makers. In 1950 there were 342,000 telephone operators in the US. Today there are less than 2000.
https://conversableeconomist.blogspot.com/2020/02/telephone-switchboard-operators-rise.html
In 1900, 40% of THE ENTIRE POPULATION were farmers and it is down to 1.3% today. That is MILLIONS of people.
https://wellwisp.com/percentage-of-americans-that-are-farmers/
I'm not smart enough to know how AI and other technological development will ultimately shape the future workforce. But humans have ALWAYS found a way. No one in 1950 expected telephone operators to disappear or computer techs to appear, but they did. How many hundreds of thousands of secretaries were replaced by word processors?
Yours is a well thought out comment. The only issue here is that grading is a moving target. I remember when an Old Copper that was 5% RD was slabbed as RB. That ship sailed over 25 years ago. You need 20-25% RD on an Old Copper to get a RB designation now.
PCGS, NGC and CAC all have slightly different standards for Uncirculated coins. Which of these will AI adopt, or will it be a mixture of them, and if so, how will this be determined?
Lastly, different graders have slightly different standards, and graders come and go. You also need to figure that into the mix.
I see AI as a helpful tool. In everything. It does a good job of accumulating and organizing data. It doesn't think or analyze.
"Seu cabra da peste,
"Sou Mangueira......."
I am guessing e-bay will put an AI coin grading feature on their create listing page for coins.
I doubt it. But if they do it won't work very well because it will be dependent on the imaging.
Yeah, that is true.
A lot of ebay sellers photos are terrible.
Who really cares if a silver eagle grades 69 or 70? That is nothing but a money grab. It’s bullion, nothing more and nothing else. Let’s see how AI will be able to grade pre-1830 U.S. as well as colonial coins that may strike up poorly. Even later dated material such as mintmarked Buffalo nickels from the 20’s or say a high end 1896-O dollar.
A very large number of people care whether ASE’s grade 69 or 70. The fact that you apparently don’t doesn’t have any bearing on that.
Mark Feld* of Heritage Auctions*Unless otherwise noted, my posts here represent my personal opinions.
I'm going to go out on a limb here. Anything graded ms70 is no longer bullion.
The substantial truth doctrine is an important defense in defamation law that allows individuals to avoid liability if the gist of their statement was true.
Sorry, but unless they can download JA's brain into a computer, I will take a JA personally graded coin over an AI coin any day. I have no issues with AI for very simple things, but it is not nearly where it would need to be to effectively and accurately grade coins. Example case: using AI for legal research -- cannot be done, AI makes up cases with correct citations according to the published versions but they are still completely bogus and AI cannot recognize its error because it cannot learn specifically in that way. Just finished a three hour CLE course regarding the folly of AI legal research as a basic level tool.