Options
Has anyone attempted to develop computer grading?
TigersFan2
Posts: 1,442 ✭✭
It seems like the current grading system by PCGS, NGC, ANACS and others of using human eyes has an element of being subjective. Has anyone every attempted to develop a computer grading method where the coin is photographed at a high resolution or a laser scan and a computer algorithm then examines the detailed features and calculates a grade? Or would such a system have inherent inaccuracies and inconsistencies that would make it worse than human eyes?
I expect that a computer grading system would still need a human element to do the initial determination whether the coin is cleaned or exhibits other non-gradable damage.
I love the 3 P's: PB&J, PBR and PCGS.
0
Comments
I expect that a computer grading system would still need a human element to do the initial determination whether the coin is cleaned or exhibits other non-gradable damage.
This is a true statement. In my mind, the biggest bottleneck that currently exists is the lack of a dataset (i.e. human labeled images). It would require the cooperation of at least a handful of top notch graders to grade some tens of thousands of coins for the express purpose of putting themselves out of the job (either as a grader or an upgrader) which I don't think will happen anytime soon.
Once there is a dataset, I can't imagine any other technological hurdles that one would face that hasn't already been solved in a similar context.
Someone out there knows more about it than me....so I'll shut up.
(That's why I doubt we will be losing our jobs to a robot anytime soon....unless your job is to put nuts on screws.)
When stealer and I discussed this playfully a few months back, the thought we came to was to have the computer teach itself to grade by throwing thousands of grades coins at it and letting it pick up on the differences, rather than programming the computer with the grading criteria. Hence why stealer says the biggest hurdle is the initial sample of coins needed to teach it.
That is the general approach for machine learning. I imagine the TrueViews would aid in this process significantly: there are many manually graded coins with photographs taken in a controlled manner. It would be "merely" a matter of training the algorithm to know that "coins which look like X set of images = MS65"
Modern algorithms are pretty good at this, essentially operating as human assisted training. It's an area with a great deal of active research by most of the top tech companies. The sample sets might not be large enough yet but it's something which could be tuned at PCGS' leisure, as they have the images and know the grades.
I think the problem with a LOT of automation technologies is that the cost of developing, programming, testing, and getting a fully working system actually running would be many, many times the cost of simply hiring 3 or 4 people to actually do the job....for several years.
(That's why I doubt we will be losing our jobs to a robot anytime soon....unless your job is to put nuts on screws.)
The cost of developing a learning machine is actually quite low. All it takes is a computer (which can be had for ~$10 these days), a timeshare on a server (which is pretty cheap thanks to AWS and Azure) and a high school student that knows how to apply programming libraries.
I still stand by the concept that just about anything under the sun can be achieved by gathering a group of college students and telling them that their grades depends on the success of the project.
When stealer and I discussed this playfully a few months back, the thought we came to was to have the computer teach itself to grade by throwing thousands of grades coins at it and letting it pick up on the differences, rather than programming the computer with the grading criteria. Hence why stealer says the biggest hurdle is the initial sample of coins needed to teach it.
That is the general approach for machine learning. I imagine the TrueViews would aid in this process significantly: there are many manually graded coins with photographs taken in a controlled manner. It would be "merely" a matter of training the algorithm to know that "coins which look like X set of images = MS65"
Modern algorithms are pretty good at this, essentially operating as human assisted training. It's an area with a great deal of active research by most of the top tech companies. The sample sets might not be large enough yet but it's something which could be tuned at PCGS' leisure, as they have the images and know the grades.
This is true. PCGS and NGC both have amazing access to a nearly unlimited dataset as long as people keep sending them coins. The only problem is that TrueViews would not do the job - namely that the lighting is inconsistent depending on the type of coin and how the photographer wants the coin to be displayed.
The only way to get past this is to have an automated system that does all the photography, preferably taking multiple images of the coin at different, but fixed, angles relative to the lighting.
There are of course other issues such as data storage, size and quality of images etc. but that can all be tuned for over time.
I think the problem with a LOT of automation technologies is that the cost of developing, programming, testing, and getting a fully working system actually running would be many, many times the cost of simply hiring 3 or 4 people to actually do the job....for several years.
(That's why I doubt we will be losing our jobs to a robot anytime soon....unless your job is to put nuts on screws.)
The cost of developing a learning machine is actually quite low. All it takes is a computer (which can be had for ~$10 these days), a timeshare on a server (which is pretty cheap thanks to AWS and Azure) and a high school student that knows how to apply programming libraries.
I still stand by the concept that just about anything under the sun can be achieved by gathering a group of college students and telling them that their grades depends on the success of the project.
Sure...but now throw in these factors:
- Speed. Can you develop a system to feed the coins through the "grader" at a rate that will rival what graders at PCGS grade at now?
- Maintenance. Assuming "speed" requires moving parts.....Someone needs to keep those moving parts working. Then there's cleaning the system, adjusting it, etc. And the computer technician to replace parts, poke at the software, etc.
- Insert and removal. Coins aren't going to get into the system solo. You need to hire someone to put the coins on the "conveyer belt", and take them off at the other end.
- Error correction and quality control: Once the system gets out of synch, someone is going to have to make sure that seated dime doesn't end up in the Morgan dollar slab.
At some point, you've hired so many people to make the system work, you were probably better off (financially) to just train them to grade coins!
I'm an engineer, and I share the philosophy that "anything is possible", and it's getting more possible as time and technology progresses.
The problem I see is that the financial aspects of the whole thing just won't catch up to simple well trained humans. At the very best, you are replacing expensive and knowledgeable coin graders with expensive and knowledgeable computer and automation experts.
(This, of course, ignores the possible advantage of repeatability and consistency, which the computer system might have. But if the financials don't support it, nobody will buy it!)
I may be wrong....but I won't be the first to under-estimate the long term effects of computers, automation, and robotics.
In fact a board member here brought a COMPUGRADED coin to our coin club meeting last night.
MY COINS FOR SALE AT https://www.pcgs.com/setregistry/collectors-showcase/other/bajjerfans-coins-sale/3876
I think the problem with a LOT of automation technologies is that the cost of developing, programming, testing, and getting a fully working system actually running would be many, many times the cost of simply hiring 3 or 4 people to actually do the job....for several years.
(That's why I doubt we will be losing our jobs to a robot anytime soon....unless your job is to put nuts on screws.)
The cost of developing a learning machine is actually quite low. All it takes is a computer (which can be had for ~$10 these days), a timeshare on a server (which is pretty cheap thanks to AWS and Azure) and a high school student that knows how to apply programming libraries.
I still stand by the concept that just about anything under the sun can be achieved by gathering a group of college students and telling them that their grades depends on the success of the project.
Sure...but now throw in these factors:
- Speed. Can you develop a system to feed the coins through the "grader" at a rate that will rival what graders at PCGS grade at now?
- Maintenance. Assuming "speed" requires moving parts.....Someone needs to keep those moving parts working. Then there's cleaning the system, adjusting it, etc. And the computer technician to replace parts, poke at the software, etc.
- Insert and removal. Coins aren't going to get into the system solo. You need to hire someone to put the coins on the "conveyer belt", and take them off at the other end.
- Error correction and quality control: Once the system gets out of synch, someone is going to have to make sure that seated dime doesn't end up in the Morgan dollar slab.
At some point, you've hired so many people to make the system work, you were probably better off (financially) to just train them to grade coins!
I'm an engineer, and I share the philosophy that "anything is possible", and it's getting more possible as time and technology progresses.
The problem I see is that the financial aspects of the whole thing just won't catch up to simple well trained humans. At the very best, you are replacing expensive and knowledgeable coin graders with expensive and knowledgeable computer and automation experts.
(This, of course, ignores the possible advantage of repeatability and consistency, which the computer system might have. But if the financials don't support it, nobody will buy it!)
I may be wrong....but I won't be the first to under-estimate the long term effects of computers, automation, and robotics.
Very fair points. When I was playing around with the idea a couple of months ago, I realized the need for (at the very least) one human operator per machine who can put coins in the "eyepiece" of the computer. On the other hand, I was not nearly as ambitious in thinking that the system would also serve as a replacement for the slabbing process - if I were tasked with designing the system, I would at the very least have two disjoint computers: one for grading and one for slabbing.
Successful BST transactions with forum members thebigeng, SPalladino, Zoidmeister, coin22lover, coinsarefun, jwitten, CommemKing.
In February 1991 Merton announced that a company which he was associated with, Compugrade, had started grading coins using computer aided grading.
"Everything is on its way to somewhere. Everything." - George Malley, Phenomenon
http://www.americanlegacycoins.com
There was also COMPUGRADE
In fact a board member here brought a COMPUGRADED coin to our coin club meeting last night.
I've owned a few coins in their slabs- they only graded Morgans, if I remember correctly. The coolest thing was the grade- a coin could grade MS 64.2. Their slabs bring a nice premium as well.
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars
Leo
The more qualities observed in a coin, the more desirable that coin becomes!
My Jefferson Nickel Collection
Modern algorithms are pretty good at this, essentially operating as human assisted training. It's an area with a great deal of active research by most of the top tech companies. The sample sets might not be large enough yet but it's something which could be tuned at PCGS' leisure, as they have the images and know the grades.
Yes, I think this will help immensely, having the images, taken under fairly controlled conditions, with numbers assigned to them is the perfect food for a machine learner.
Also have seen a peace dollar
I just received a coin in a MS64 grade. High in details (tough for the date), nice luster and darker gold toning but the condition, the number of marks and their locations (all toned over, btw), for this coin, played a major role in its 64 grade. But I haven't seen this level of detail in higher grade holders so a computer grade should be a 4 or 5 digit number to basically tell a story about each component of the coin. I don't need the number since I already know what to look for but it would help seeing a coin when the photography is not enough.
Leo
64.v.w.x.y.z
MY COINS FOR SALE AT https://www.pcgs.com/setregistry/collectors-showcase/other/bajjerfans-coins-sale/3876
at some point in the nuances of grading, eye appeal always factors in , and how does one program this peculiarity into the algorithms ?
Won't work.
bob
The problem is training the program.
Which is why you will see it for moderns much sooner than for all coins.
The way these neural networks work is that you feed in 100s or 1000s of inputs and correct the network to produce the same results as the expert. When that's done, you've trained it to grade one series or one year of that series (or even just the weakly struck group of that year). The volume exists for say ASEs.
You need an upfront network to separate PF PFCAM MS and MSPL. Then separate network(s) to grade within. And you always have the option of reject-to-human for special cases (say a pocket piece).
But by the time you've trained it to grade say the Barber dime sold last month, you've graded all 9 known examples.
But anything with enough volume is possible to do. You could be scanning the coins today to feed into so e future program along with the human assigned grades. When you have a network that agrees 95 or 99% of the time, you are done. It's really how do you do the cutover? Do you have people resubmit? Tough sell when most will get the same grade. AND: how do you make money at it.
ANA 50 year/Life Member (now "Emeritus")
It's actually quite possible, given the improvements in scanners and computer power.
The problem is training the program.
Which is why you will see it for moderns much sooner than for all coins.
Mostly because one modern is pretty much exactly like another. OTOH one can have 100 ms63 1886 Morgans all of which likely will be different.
MY COINS FOR SALE AT https://www.pcgs.com/setregistry/collectors-showcase/other/bajjerfans-coins-sale/3876
A few guys could grade it sight unseen if Pistareen described it. [ let me expound ]
at some point in the nuances of grading, eye appeal always factors in , and how does one program this peculiarity into the algorithms ?
It would be up to the humanoid finalizer to decide on eye appeal. For moderns you might have computer only grading and for classics maybe computer assisted grading.
MY COINS FOR SALE AT https://www.pcgs.com/setregistry/collectors-showcase/other/bajjerfans-coins-sale/3876
It's actually quite possible, given the improvements in scanners and computer power.
The problem is training the program.
Which is why you will see it for moderns much sooner than for all coins.
Mostly because one modern is pretty much exactly like another...
And most are graded 69 or 70.
Successful BST transactions with forum members thebigeng, SPalladino, Zoidmeister, coin22lover, coinsarefun, jwitten, CommemKing.
NGC
Silver Eagles (1986-Date) 7,988,291
Morgan 3,073,190
Gold Eagles (1986-Date) 1,190,370
PCGS
Morgan 2,959,339 (plus another 120K DM and PL)
Silver Eagles, MS 1,880,091 (plus another 700K PR and SP)
ANA 50 year/Life Member (now "Emeritus")
As for eye appeal... IMO (and many disagree), I do not believe that should have any place
in coin grading. Eye appeal is up to the buyer.... and it varies from collector to collector - that very fact should eliminate it from grading. Some art, and artists, are found to be pleasing to
some people... and not others. The same applies to coins. Remove that factor and the
science becomes more realistic and achievable - given the issues noted by many in the
foregoing posts. Cheers, RickO
Some thoughts...
I wonder if computer grading could be used to supplement human grading. For example, distinguishing between MS60-70. The human already determines uncirculated. The computer then determines the MS60-70 grade by counting/measuring strength of strike and number of marks.
It seems like it would be difficult for a computer to grade circulated coins as coins do wear differently. But what I would like about a computer graded coin is that it could provide data of why it came to a specific grade. A computer would likely be able to break down the grade between obverse, reverse and various fields. With human grading, it would significantly increase the time/cost of grading if the humans would provide us with an explanation with each coin of why the grade was determined. I believe we've all looked at a PCGS or NGC coin and wondered how the grade was determined
PCGS owns the patent and I believe it was based on a design from board member here, but I don't recall who.
It is over 19 years. Shouldn't be the patent expired?
K
Images are 2D...for proper grading I would think you need a 3D scan. Just an thought on a Friday afternoon.
K
Not really. Human graders use a series of stereoscopic views of a coin. The same input could be given a computer, but now you have to produce something that can repeatably produce such input.
Keeper of the VAM Catalog • Professional Coin Imaging • Prime Number Set • World Coins in Early America • British Trade Dollars
"Seu cabra da peste,
"Sou Mangueira......."