Home U.S. Coin Forum

I created Numi: The World's Most Powerful Coin Grading and Identification AI, powered by ChatGPT4.

2

Comments

  • CoinscratchCoinscratch Posts: 8,658 ✭✭✭✭✭

    @Batman23 said:
    It is hard enough for a person to grade from photos. I don't think a computer is going to be even close.

    I could see the scenario of the coin being scanned in person in a machine. I cannot imagine a scenario where a digital image is accurately scanned since the image can be manipulated in so many ways.

  • SIowhandSIowhand Posts: 339 ✭✭✭✭✭

    Well, chatgpt is not really AI. Ok, got that out of the way.

    I know it’s fun to poke fun at some of these results but you have to remember that we are barely at the very beginning of this technology. There will come a time when AI can do trivial things like grading coins better than any person. It will happen.

    There are some spaces that this technology has been in use for several years.

    I guarantee that some, if not most of you, have had a conversation on the phone with AI and have not known it. You did not realize you weren’t speaking with a human. It gets better everyday.

    I’m not going to out myself, but customers of a certain insurance company that is popular with older folks, has used it for several years.

  • AUandAGAUandAG Posts: 24,761 ✭✭✭✭✭

    @SIowhand said:
    Well, chatgpt is not really AI. Ok, got that out of the way.

    I know it’s fun to poke fun at some of these results but you have to remember that we are barely at the very beginning of this technology. There will come a time when AI can do trivial things like grading coins better than any person. It will happen.

    There are some spaces that this technology has been in use for several years.

    I guarantee that some, if not most of you, have had a conversation on the phone with AI and have not known it. You did not realize you weren’t speaking with a human. It gets better everyday.

    I’m not going to out myself, but customers of a certain insurance company that is popular with older folks, has used it for several years.

    Sorry, but you are so mistaken on so many levels. It's like thinking that we humans can stop global warming/cooling. Won't happen to any extent that is significant. AI will likely be used for more evil than good and some involved (like Elon) are worried that this will be so.
    bob :)
    vegas baby!

    Registry: CC lowballs (boblindstrom), bobinvegas1989@yahoo.com
  • Batman23Batman23 Posts: 4,999 ✭✭✭✭✭

    @gumby1234 said:
    @Batman23 toning shouldn't subdue the luster. Heres an extremely toned dime. Look at the blazing luster.

    Nice dime with booming luster throughout. And your photo shows it. Will AI give it MS? This is not the type of coin photo I was talking about. Look at the first image of Johnny010's 1880 half eagle he posted in this thread. The field area in front of Liberty's face does not show the luster and can be interpreted as wear. The second photo of his coin in the slab shows the full luster better.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    I ran an experiment where I ran the same coin through ten times with different photos for each test. I wanted to see how the model did with consistency.

    The grades ranged from XF40 - MS64 :( It's quite disappointing that the grades were so inconsistent. But that just means I'll have to go back to the model and adjust it to see if I can get things to be more consistent. I'll have to experiment with more angles. I need to stop by my local coin shop soon so I can get my hands on some slabbed coins with various grades. That will better help me fine-tune the model.

    Illinois State Quarter - 2003 D

    Test #1
    AI Grade: AU 55
    Photos Used
    Full AI Response

    Test #2
    AI Grade: AU 50 - AU 53
    Photos Used
    Full AI Response

    Test #3
    AI Grade: XF 40
    Photos Used
    Full AI Response

    Test #5
    AI Grade: AU 55
    Photos Used
    Full AI Response

    Test #6
    AI Grade: MS 64
    Photos Used
    Full AI Response

    Test #7
    AI Grade: AU 55
    Photos Used
    Full AI Response

    Test #8
    AI Grade: AU 58
    Photos Used
    Full AI Response

    Test #9
    AI Grade: MS 60 - MS 61
    Photos Used
    Full AI Response

    Test #10
    AI Grade: AU 55
    Photos Used
    Full AI Response

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    An interesting observation comparing v1.0 to v1.25. The results it got from @johnny010 coins are much more accurate

    v1.25 had a lot of new logic added for normalizing and processing all images holistically

    $5 Gold Liberty Head - 1880
    v1.0 AI Grade: AU 55 - AU 58
    v1.25 AI Grade: MS 64
    Actual Grade: MS 65
    Photos Used
    Full AI Response

    $5 Gold Indian Head Half Eagle - 1914
    v1.0 AI Grade: XF 40
    v1.25 AI Grade: MS 64
    Actual Grade: MS 65
    Photos Used
    Full AI Response

    $1 Gold Indian Princess Head - 1862
    v1.0 AI Grade: AU 55
    v1.25 AI Grade: MS 62
    Actual Grade: MS 65
    Photos Used
    Full AI Response

  • jmlanzafjmlanzaf Posts: 34,174 ✭✭✭✭✭

    @dcarr said:

    @AlbumNerd said:
    @johnny010

    Here you go!

    $5 Gold Liberty Head - 1880

    • AI Grade Given: AU 55 - AU 58

    $5 Gold Indian Head Half Eagle - 1914
    AI Grade Given: XF 40

    $1 Gold Indian Princess Head - 1862
    AI Grade Given: AU 55

    >

    Where can I buy "AI"-graded coins ?

    @AUandAG said:

    @SIowhand said:
    Well, chatgpt is not really AI. Ok, got that out of the way.

    I know it’s fun to poke fun at some of these results but you have to remember that we are barely at the very beginning of this technology. There will come a time when AI can do trivial things like grading coins better than any person. It will happen.

    There are some spaces that this technology has been in use for several years.

    I guarantee that some, if not most of you, have had a conversation on the phone with AI and have not known it. You did not realize you weren’t speaking with a human. It gets better everyday.

    I’m not going to out myself, but customers of a certain insurance company that is popular with older folks, has used it for several years.

    Sorry, but you are so mistaken on so many levels. It's like thinking that we humans can stop global warming/cooling. Won't happen to any extent that is significant. AI will likely be used for more evil than good and some involved (like Elon) are worried that this will be so.
    bob :)
    vegas baby!

    He could be right about everything and AI still gets used for evil... kind of like the internet.

  • gumby1234gumby1234 Posts: 5,584 ✭✭✭✭✭

    @alaura22 said:
    How about I use "A EYE" to choose my coins :)

    Or even 2 eyes.

    Successful BST with ad4400, Kccoin, lablover, pointfivezero, koynekwest, jwitten, coin22lover, HalfDimeDude, erwindoc, jyzskowsi, COINS MAKE CENTS, AlanSki, BryceM

  • scotty1419scotty1419 Posts: 928 ✭✭✭

    Interesting work!

    I think it'd be very interesting to test this with a lot of problem coins - bad surfaces etc.

    A lot of AU/MS coins, I wouldn't complain if it gets off on a 58 vs 63, but it'd be interesting if it could ALSO interpret different kinds of cleaning...

  • ElmerFusterpuckElmerFusterpuck Posts: 4,722 ✭✭✭✭✭

    A cool topic. Like humans, that AI will need to look at a huge amount of coins to hone "its" grading skills, along with some grading books. Would be interesting to see what grade the AI would be assign to a very weakly struck MS 1940-S walking liberty half dollar.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    .> @alaura22 said:

    How about I use "A EYE" to choose my coins :)

    Time for you to get back home dad

  • CoinscratchCoinscratch Posts: 8,658 ✭✭✭✭✭

    Sounds like our boys at PCGS and others will have job security for foreseeable future 😁

  • jmlanzafjmlanzaf Posts: 34,174 ✭✭✭✭✭

    @Coinscratch said:
    Sounds like our boys at PCGS and others will have job security for foreseeable future 😁

    I don't know. Do we know that the OP is exceptional at this?

    I used to think that grading couldn't be done via AI consistently. Now, I'm not so sure. The evolution in a year has been astounding. And it's mostly language, the image AI is just getting rolling.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @jmlanzaf said:

    @Coinscratch said:
    Sounds like our boys at PCGS and others will have job security for foreseeable future 😁

    I don't know. Do we know that the OP is exceptional at this?

  • jmlanzafjmlanzaf Posts: 34,174 ✭✭✭✭✭

    @AlbumNerd said:

    @jmlanzaf said:

    @Coinscratch said:
    Sounds like our boys at PCGS and others will have job security for foreseeable future 😁

    I don't know. Do we know that the OP is exceptional at this?

    Lol. No offense intended. I just meant that if a larger entity wanted to hire a full team to do it, they might get better results.

  • CoinscratchCoinscratch Posts: 8,658 ✭✭✭✭✭

    The Oracle would have to break down the images into little particles that fly through the air. And then somehow put them all back together again in order to come up with a reasonable explanation.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    I did a complete rework on Numi's internal logic and the results are looking significantly better. The model is now on v1.26 Alpha.

    My current goal is to fine-tune the model so that grades are at most two grades off consistently. Eventually, this will move down to one grade off, then right on the money.

    The model is exceptional at identifying high grade Proof Coins. Here are the results for a Proof 69 Lincoln Memorial Cent - 2005 S

    The model is getting within two grades in the mint state range. Here are the results for a MS-65 Lincoln Wheat Cent - 1938 D

    The model is somewhat struggling in the AU range. But it's a lot closer than before. Here are the results for a AU-55 Liberty Head Eagle - 1868 S

    The sweet spot for accuracy appears to be around 6-8 pictures of a coin.

    I would love more help from the community with photos. Especially photos of coins that are from Poor (P) - Extremely Fine (XF)

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭
    edited November 15, 2023 11:46PM

    Numi v1.27 alpha is now live!

    Video Demo

    Changes include

    • Fine-tuned grading criteria to align more closely with third-party grading standards
    • Changed response format to be more clear and concise
    • Added Numista integration back in after fixing prior deployment issues

    Users can now follow up on their conversation and ask Numi for additional facts about their coins such as:

    • Mintage figures
    • Coin values
    • Coin details such as composition, diameter, weight, etc
    • Info is pulled from Numista's database

    Feel free to share any photos you'd like tested! I will post the results here

  • MetroDMetroD Posts: 2,196 ✭✭✭✭✭

    @AlbumNerd said:
    [...]
    I would love more help from the community with photos. Especially photos of coins that are from Poor (P) - Extremely Fine (XF)

    Don't know the rules regarding usage, or if you are already aware of it, but:
    PCGS Photograde

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @MetroD said:
    Don't know the rules regarding usage, or if you are already aware of it, but:
    PCGS Photograde

    TrueView and PhotoVision have great pictures but the problem is that they only include one photo of each side. This has been a problem with AI grading in the past because the AI is basing its grade on only one angle. And using only one angle and lighting misses a lot of key details for an accurate grade. My AI is built to handle analyzing photos from multiple angles.

    @Vasanti said:
    I would happily pay for something that helps to identify ancients, even if it just gave you a starting point and a list of possibilities backed up by a database.

    Feel free to post a few photos and I'll post the results! I haven't tested any ancient coins yet, so I'm curious to see what the response would be.

  • VasantiVasanti Posts: 455 ✭✭✭✭



  • MetroDMetroD Posts: 2,196 ✭✭✭✭✭

    @AlbumNerd said:

    TrueView and PhotoVision have great pictures but the problem is that they only include one photo of each side. This has been a problem with AI grading in the past because the AI is basing its grade on only one angle. And using only one angle and lighting misses a lot of key details for an accurate grade. My AI is built to handle analyzing photos from multiple angles.

    Understood.

    How about this? Not in your preferred grade range, but multiple angles are shown.

  • yosclimberyosclimber Posts: 4,788 ✭✭✭✭✭
    edited November 16, 2023 1:15PM

    @Vasanti said:
    I would happily pay for something that helps to identify ancients, even if it just gave you a starting point and a list of possibilities backed up by a database.

    wildwinds.com is great - you can just guess the date range / ruler to get started.
    The photos you posted look easy.
    For ones you can't figure out, there are people here on the World and Ancient Coins forum that can help.
    Doing the IDs yourself is one of the fun parts of ancients, in my view.

    Or did you mean having a smartphone app that you could use at a coin show to ID ancients?

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @Vasanti said:

    Thanks for the photos! Here are the results

    AI Guess: Follis of Byzantine Emperor John II Komnenos
    Full AI Response

    AI Guess: Unable to determine. The best guess is a coin consistent with Byzantine follis coins
    Full AI Response

    AI Guess: 40 Nummi - Emperor Phocas (Nicomedia)
    Full AI Response

    Numi hasn't been tuned for ancient coins yet so it took some prodding. But these are the results.

    If you want, you can provide another ancient coin with photos taken at various angles. That would help the AI see all the details in multiple lighting, which would help it better identify the coin.

    I did however come across this Flickr album containing ancient coins shown on last week's E-Sylum newsletter. The results are much better as the photos were more detailed.



    AI Guess: Judea Capta coin depicting Emperor Vespasian
    Full AI Response



    AI Guess: Roman Sestertius depicting Emperor Hadrian
    Full AI Response

    @MetroD
    Thanks for the link!

  • knovak1976knovak1976 Posts: 402 ✭✭✭✭




    Before I send them in….😉

  • gumby1234gumby1234 Posts: 5,584 ✭✭✭✭✭

    Why does it say Lincoln wheat cent for a 2005?

    Successful BST with ad4400, Kccoin, lablover, pointfivezero, koynekwest, jwitten, coin22lover, HalfDimeDude, erwindoc, jyzskowsi, COINS MAKE CENTS, AlanSki, BryceM

  • FlyingAlFlyingAl Posts: 3,217 ✭✭✭✭✭

    Try this one:

    Coin Photographer.

  • yosclimberyosclimber Posts: 4,788 ✭✭✭✭✭
    edited November 16, 2023 3:10PM

    @AlbumNerd said:
    ...

    AI Guess: 40 Nummi - Emperor Phocas (Nicomedia)
    Full AI Response

    Indeed a guess.
    It's easy for someone who has identified a few Roman coins to simply read the ruler name on this one -
    DN IVSTINIANVS = Justinian.
    Not FOCAS.
    https://www.wildwinds.com/coins/byz/justinian_I/t.html

  • BigAlBigAl Posts: 1,187 ✭✭✭

    I’m impressed, the tech is cool, but needs some obvious tweaking. I think the AI is mistaking a weak strike for wear. So likely it will do better grading modern coins, rather than $5 Indians or O mint Morgans. If the tech could “learn” you could feed it pics of already graded coins so that it would learn the idiosyncrasies of specific issues and dates. Then sell the tech to a TPG.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @knovak1976 Here are the results. I'd love to know the real grade when you know!

    1950 Franklin Half Dollar [Proof?]


    AI Grade: PR-68
    Full AI Response

    1951 Franklin Half Dollar


    AI Grade: MS-65
    Full AI Response

    @FlyingAl Would love to know the real grade!
    1942 Walking Liberty Half Dollar


    AI Grade: MS-65
    Full AI Response

    @gumby1234
    Just a spelling mistake c;

  • alaura22alaura22 Posts: 3,182 ✭✭✭✭✭
    edited November 16, 2023 5:40PM

    Well, you missed the 1942 by 3 grades...... and it's a proof coin not an MS coin
    It's an PR68!

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @alaura22 said:
    Well, you missed the 1942 by 3 grades...... and it's a proof coin not an MS coin
    It's an PR68!

    Thanks for the photos and info! This type of testing helps a ton :smile:

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    I have just pushed a massive update to Numi's logic. [v1.28 alpha]

    I have now added custom logic for Ancient Coin identification and grading. Thank you @Vasanti @yosclimber for the extremely helpful feedback! I was able to fine-tune the model to handle Ancient Coins much better. The key was having Numi reference Numista's database and always provide a link to a Numista page. This forced Numi to be more careful and thorough with its analysis.

    I'm not sure how accurate the grades are since I don't know how to grade ancients. Would love feedback on that.

    I think you may be pleased as these results

    Denarius - Sextus Pompeius
    AI Grade: Choice Very Fine (Ch VF)



    Numista Link

    Denarius - Julis Caesar (CAESAR; Elephant
    AI Grade: Choice Very Fine (Ch VF)



    Numista Link

    It's still not perfect though. It's still struggling with @Vasanti ancient photos

    Overall a massive upgrade, especially within one day, and considering Numi couldn't even handle any ancient coins yesterday. I will continue to fine-tune. Any more photos of ancient coins, especially photos taken from multiple angles, will greatly assist.

  • knovak1976knovak1976 Posts: 402 ✭✭✭✭

    Thanks! Both of my Franklin halves are from original boxed proof sets that I removed from the cellophane wrapper last week. I will be sending them in for grading in a couple of weeks and will let you know their grades. 😉

  • JBKJBK Posts: 15,567 ✭✭✭✭✭

    @Vasanti said:
    I would happily pay for something that helps to identify ancients, even if it just gave you a starting point and a list of possibilities backed up by a database.

    I'll say up front that I am not a huge fan of front-end AI. I just read the most ridiculous description of a coin on ebay, obviously AI generated.

    I hope the OP doesn't miss other potential uses for this technology, such as the identification of ancients.

    Also, how about using it to identify varieties of all kinds. The AI should be able to easily pick up the markers once trained to do it.

  • VasantiVasanti Posts: 455 ✭✭✭✭

    @AlbumNerd said:
    I have just pushed a massive update to Numi's logic. [v1.28 alpha]

    I have now added custom logic for Ancient Coin identification and grading. Thank you @Vasanti @yosclimber for the extremely helpful feedback! I was able to fine-tune the model to handle Ancient Coins much better. The key was having Numi reference Numista's database and always provide a link to a Numista page. This forced Numi to be more careful and thorough with its analysis.

    I'm not sure how accurate the grades are since I don't know how to grade ancients. Would love feedback on that.

    I think you may be pleased as these results

    Denarius - Sextus Pompeius
    AI Grade: Choice Very Fine (Ch VF)



    Numista Link

    Denarius - Julis Caesar (CAESAR; Elephant
    AI Grade: Choice Very Fine (Ch VF)



    Numista Link

    It's still not perfect though. It's still struggling with @Vasanti ancient photos

    Overall a massive upgrade, especially within one day, and considering Numi couldn't even handle any ancient coins yesterday. I will continue to fine-tune. Any more photos of ancient coins, especially photos taken from multiple angles, will greatly assist.

    If you can do it, you should have your AI scrape sites like Wildwinds and resources like NGC. You should be able to have a huge number of pictures that are tied to accurate descriptions.

  • seatedlib3991seatedlib3991 Posts: 710 ✭✭✭✭✭

    What a game changer it would be to get detailed grading notes. gook luck. james

  • TurtleCatTurtleCat Posts: 4,600 ✭✭✭✭✭

    Just saw you’re starting to do ancients. That is very cool. I don’t have the patience or eyesight or resources to track down all the random stuff I have gotten. Would love to get some idea of what some of the dug ancients I have are.

  • TurtleCatTurtleCat Posts: 4,600 ✭✭✭✭✭

    As an aside, do I have to pay for ChatGPT to sign up to use this?

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @JBK
    Also, how about using it to identify varieties of all kinds. The AI should be able to easily pick up the markers once trained to do it.

    There is some logic already in Numi to try to identify varieties. I haven't tested it out yet so I'm unsure of its current effectiveness. But it is a planned feature.

    @Vasanti
    If you can do it, you should have your AI scrape sites like Wildwinds and resources like NGC. You should be able to have a huge number of pictures that are tied to accurate descriptions.

    I'm not sure there would be value in doing that. I am unable to train the actual dataset used by GPT4 as that is controlled by OpenAI. If those sites are already public on the internet, OpenAI has most likely already scraped them. Numista integration has proven pretty helpful in identifying coins and pulling up descriptions.

    @seatedlib3991 said:
    What a game changer it would be to get detailed grading notes. gook luck. james

    Banknote identification and grading is something I hope to add quite soon :)

    @TurtleCat said:
    As an aside, do I have to pay for ChatGPT to sign up to use this?

    Currently yet, the only way to access Numi is if you already have a paid ChatGPT Plus subscription. It's currently $20 a month and is charged monthly. ChatGPT's latest features have been so popular since their launch last Monday, that OpenAI has paused new subscribers lol

    If I can validate that Numi is indeed powerful enough to meaningfully help the community, I will find a way to use set up a custom UI so folks can access it for free. [Hopefully]. It'll depend how much the API will cost to use GPT4.

  • jmlanzafjmlanzaf Posts: 34,174 ✭✭✭✭✭

    I am continually mildly surprised that people still think that AI will not change absolutely everything.

  • CoinscratchCoinscratch Posts: 8,658 ✭✭✭✭✭

    @jmlanzaf said:
    I am continually mildly surprised that people still think that AI will not change absolutely everything.

    Mildly? Said Musk, only Mildly?

  • alaura22alaura22 Posts: 3,182 ✭✭✭✭✭

    AI WILL change everything..............if you allow it to :/

  • TwoSides2aCoinTwoSides2aCoin Posts: 44,288 ✭✭✭✭✭

    Wildly not mildly, said I.

  • jmlanzafjmlanzaf Posts: 34,174 ✭✭✭✭✭

    @Coinscratch said:

    @jmlanzaf said:
    I am continually mildly surprised that people still think that AI will not change absolutely everything.

    Mildly? Said Musk, only Mildly?

    Yes, only mildly. People can be very skeptical of change. And the free public release of ChatGPT is imperfect. But it is developing quickly with all the big tech companies involved. And what I've seen from my colleagues on campus is already amazing and disruptive. I'm actually buying GPT4.0 this weekend to start playing with the plug ins.

    This is the most exciting, most transformative technology of our lifetimes.

  • ElmerFusterpuckElmerFusterpuck Posts: 4,722 ✭✭✭✭✭

    I'm detecting some people in this thread have really tightened their vise-like grip on their buggy whips. I'll never take a car over a horse! Nevah I say! NEVAH!!!!!

  • Mr_ColomboMr_Colombo Posts: 47 ✭✭✭

    @AlbumNerd
    Why the rush? As you know, OpenAi DevDay was on Nov 6th which rolled out OpenAI's custom model GPT creator. Your post was dated 9 Nov, It will take much more than the 2-3 days of input to Chatgtp4 to claim "The World's Most Powerful Coin Grading and Identification AI"

    IMO, i believe one day we will get close to accurately grading MS 65 or higher coins with AI. It will always be hard with lower grade coins due to the innumerous variables.

  • AlbumNerdAlbumNerd Posts: 193 ✭✭✭

    @Mr_Colombo said:
    @AlbumNerd
    Why the rush? As you know, OpenAi DevDay was on Nov 6th which rolled out OpenAI's custom model GPT creator. Your post was dated 9 Nov, It will take much more than the 2-3 days of input to Chatgtp4 to claim "The World's Most Powerful Coin Grading and Identification AI"

    IMO, i believe one day we will get close to accurately grading MS 65 or higher coins with AI. It will always be hard with lower grade coins due to the innumerous variables.

    @Mr_Colombo said:
    @AlbumNerd
    Why the rush? As you know, OpenAi DevDay was on Nov 6th which rolled out OpenAI's custom model GPT creator. Your post was dated 9 Nov, It will take much more than the 2-3 days of input to Chatgtp4 to claim "The World's Most Powerful Coin Grading and Identification AI"

    IMO, i believe one day we will get close to accurately grading MS 65 or higher coins with AI. It will always be hard with lower grade coins due to the innumerous variables.

    I've always been a big tech nerd, so it's fun to push these state of the art models to its absolute limits. I'm using this project as an way to stay on top of AI development, and in the process, find cool ways to learn about area of numismatics I wouldn't have considered before. I never thought I would dive into ancient coins, but here we are lol

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file