Building an Auction Catalog Database
Boosibri
Posts: 12,714 ✭✭✭✭✭
As some of you know, one of my favorite things to do with the coins in my collection is to trace the provenance of the coin back to other notable collections. The process of that is usually searching through my library or through various online sources. The online sources, including the NNP are very tedious and frankly frustrating. The search functionality of the Newman Portal is only effective when you open each catalog individually. With the recent upgrade, the catalog listings are now inaccurate and difficult to use. For example, the Hans M.F. Schulman listings now have over 8900 listings including English sales from the 1700's.
I found in my searches that I relied upon the same 100 sales or so each time. I decided then to create my own searchable database by ingesting the catalogs into a Claude AI agent to then build and populate a SQL database. Each catalog hOCR is uploaded from Archive.org or a PDF is uploaded for the agent to OCR and include in the database. Many times the OCR is poor especially for older catalogs or annotated sales where the writing obscures the text. This is generally an issue for lot codes, but the AI can usually infer the lot code based on sequencing.
I'm up to about 50 catalogs in the database now. I'll upload all of the Henry Christensen, Schulman, Almanzar, Glendenings, Spink and other similar catalogs. These are all intended to supplement CoinArchives or ACSearch which have newer listings. The next effort will be to try and add the Virgil Brand inventory and invoices from the NNP into the dataset. The OCR on the NNP is a challenge so I would need to see if Claude and read a PDF output and assign the dataset correctly.
Key gaps due to a lack of a digital copy will be: Norweb/Christies, UBS/Swiss Bank Corp, Ponterio.
Here is a view of the output and a list of the catalogs ingested so far.
Comments
Cool project Boosibri. I went into the NNP yesterday for the first time in a few months and was disappointed to see the upgrades have made it even harder to use, and it wasn't easy before.
Re: missing catalogs - there used to be a bunch of Ponterio catalogs in there before the update... I'll send an email to see if they can restore access.
They have some still but there are many missing
Latin American Collection
When you have finished the current holding, send me a list and I will see what I have spare. If I listed everything spare, I would lose the will to live.
Very impressive @Boosibri
Here's an organized list of many generally publicly available auction catalogs, usually hosted on BnF's Gallica or archive.org, which should provide a good seed set:
https://www.rnumis.com/auctions_top.php?elookback=0
Really helpful. Some additional sources and a very helpful structure. The Gallica user experience is what the NNP hopefully will be one day.
Latin American Collection
I like to have auction catalogs stored locally onto my computer. Most are from archive org or from the auctioneers’ web sites. Having these stored locally allows me to do text searches to find certain coins. However even though the scans are often good, the compressed pdf files at archive are typically of very poor quality, making the pictures useless. So this requires downloading the scanned image files from archive. Then I need to search through the image files to find the high res scan of the coin I’m seeking. There certainly needs to be a better way. I suppose I could just use the higher resolution scanned images to make my own PDF or other format files, eschewing compression for quality. I’m not that well versed in this sort of computer workflow so I probably am doing it the most inefficiently way possible.
End of the work day after having a Claude running in the background most of the day has got me to 108 catalogs and about 25% of the way through Ponterio and going.
Latin American Collection
Now this is AI I can get behind. Cool project!