Watch Out EDL & All You Other Gold CAC Bust Collectors...AI Told Me Your Coins Are For Sale!
TomB
Posts: 23,053 ✭✭✭✭✭
I understand that is a pretty whacked title, but I input a question about numismatics into an AI chatbot tonight and got really strange responses. Obviously, numismatics is a rather thin and niche market and area of research, but AI can make some astoundingly poor declarations.
I didn't ask anything amazing, but just started the conversation with something like a request to find gold coins with CAC stickers in older holders. These aren't coins I'm actively looking for, but wanted to see what AI would come back with since I don't often use AI platforms (or at least don't often use them knowingly). The chat bot came back with some strange interpretations and gave me some links. It said that finding gold coins in older holders is tough because dealers submit all such coins to CAC and then those that receive gold CAC stickers are immediately cracked out and sent in raw to PCGS and NGC in a quest for a higher grade on the holder.
I thought I had misinterpreted what it wrote so I asked for clarification and it responded that any coin in an OGH that might receive a gold CAC sticker would then be cracked out for a regrade attempt and that dealers and collectors use CAC as a guide for what coins to crack out. It even mentioned that the only coins that survived this were the "ultra-rare Doily holder" and it gave me links to coins on ebay or HA archives like 1938-D Buffalo nickels in MS65 and stated that these OGH coins with gold CAC stickers would be regraded in an attempt to go MS66. I just shook my head and corrected it by stating that Doily holders are not ultra-rare and that the financial aspect of the business transaction it suggested with gold CAC stickers was preposterous. It thanked me and then proceeded to send me a list of links for coins with gold CAC stickers. One coin with a gold CAC population of one in its grade, but that exists in very large numbers otherwise, was said to be the "sole survivor" of the issue until I corrected all of its misinterpretations.
I decided to roll with the conversation and asked it to find coins in OGH with gold CAC stickers that were available to purchase now and it proceeded to list coin after coin from HA archives and dealer website archives. None of the coins were currently available for purchase. It thanked me again for the correction and then sent me this link-
https://forums.collectors.com/discussion/1100205/gold-cac-busties
Apparently, the AI platform believed that all the coins in this link are for sale and that they regularly and universally are sold within and between collectors. Sort of like an interlibrary loan or something like that. I told it that made no sense and it responded with a few dealer names and the most fluffed up BS type of hyperbole you can imagine. It had read the dealer "About Us" pages and took them as truth and then embellished them to the point that it suggested one dealer had routine access to collections that had been stored for decades in bank vaults all in OGH and many with gold CAC stickers while another on the East coast would have access to collections that were hundreds of years old and passed down through families and that these came up regularly and had monster coins in OGH with gold CAC stickers and another dealer who is quite large and sells "meh" coins was listed as only handling the finest pieces in existence. Again, I disabused it of its inaccuracies.
Finally, it told me it did one last thorough scan of the web and found an OGH with gold CAC sticker from a small dealer website that appeared to not be listed anywhere else and it gave me the coin issue and grade and a complete description as it appeared on the website. I was intrigued about its success so I looked up the dealer and saw they had a simple placeholder website and no coins for sale. I then asked the AI platform about the coin it described to me and asked for a link and it responded with an apology that it had fabricated the entire existence of the dealer and the coin based upon what might sound plausible because it was trained to please people and all of its failures previously in our conversation conditioned it to make certain to end the discussion with success. It apologized profusely. I told it I thought it was hilarious and the AI platform told me it would bow out of the conversation since it was hallucinating.
So, EDL & Company...when are you going to sell me your coins?
Comments
AI is utter garbage!
Some of it is. Much of it isn’t. Your comment was way too broad to be accurate.
Mark Feld* of Heritage Auctions*Unless otherwise noted, my posts here represent my personal opinions.
And yet every major academic researcher i know uses it daily.
All comments reflect the opinion of the author, even when irrefutably accurate.
@TomB that is a wild journey you were taken on!
Out of curiosity, which AI model were you using?
Nothing is as expensive as free money.
Did it happen to tell you anything about the BU roll market?
If it was Copilot, he's going to hear from cladking.
All comments reflect the opinion of the author, even when irrefutably accurate.
A few things to note about current AI chat models:
If they don't natively know the answer, they search the web. They only search the top 10 or so findings and summarize them. Therefore you're often not getting a complete picture, especially if it's a niche topic. The story it gave you about CAC stickers might have been in just one forum post that happened to come up in its search and didn't have any competing information along with it.
They won't "admit" they don't know something. They are currently built to always find some answer, even if that sometimes means fabricating a possibility.
The very latest / frontier models are better in both cases, being more "honest". You have to pay to access them.
Rare-Change.com - Low listing fee
It also depends on how you construct the prompt. You can give it specific search parameters as well as specific verification parameters. That cuts WAY DOWN on hallucinations.
That's why cladking's copilot is such an echo chamber. He's got it using himself as the source material. The LLMs always try to please the user, so it always tells cladking that he's right.
All comments reflect the opinion of the author, even when irrefutably accurate.
I am a proponent of AI although I don’t use it very often for numismatics. I think what’s important with AI in general is when you’re looking for something very specific it’s a good idea to use more than one AI app. The information you might find on one AI app is garbage while on another one you’ll find exactly what you’re looking for.
I was doing some Google searches so I typed in the question on Google and it's AI model is the one I interacted with. I don't have an account and only use it as a guest in private mode so it has even more limitations than it might have otherwise. My questions were quite specific, but I know enough about actual reality to realize so much of what I received was downright wrong and I politely corrected it time after time while attempting to guide it down a more realistic path.
In honor of the memory of Cpl. Michael E. Thompson
This thought came to mind and I mean it really for myself. Kind of speaking out loud. For certain things it seems like a waste of time.
I laughed and thought "maybe the humans need to be programmed ".
I haven't really delved into AI to much so I only know so much. This thread speaks volumes.
Student of numismatics and collector of Morgan dollars
Successful BST transactions with: Namvet Justindan Mattniss RWW olah_in_MA
Dantheman984 Toyz4geo SurfinxHI greencopper RWW bigjpst bretsan MWallace logger7 JWP BruceS bigjpst
JWP PROMETHIUS88