States of Research Data
RogerB
Posts: 8,852 ✭✭✭✭✭
A correspondent asked about what happens to all the unused letters and other data researchers accumulate. Are unneeded letters thrown away or what….?
First, nothing gets discarded including duplicates taken from different sources. There are no “orphan” documents.
Second, every page gets an identifier and a file name.
Here’s what happens, ordered by the project result:
1. Published book or article. Keep everything including images, drafts, etc. in the database and in an off-site archive. (No cloud storage.)
2. Book or article in process. Most research is complete and is being analyzed, correlated and checked across all available sources.
3. Article in research. Subject is well defined, but much work to do in finding connections between documents and locating multiple sources.
4. Ancillary material. Materials not directly related to any of the first three, above, but shows promise of an interesting subject or possibly a significant unknown event.
5. Collateral material. Documents, lists, diagrams, etc. copied because they had an interesting subject, or might have relevance to something see elsewhere.
6. Stuff. Not of any present interest or singular subjects. No planned use. However, this can change without warning. Ex. Giles Anderson’s coin collection; letter to mint director suggesting recessing date on SL quarter; Mints and FRB’s holding back rare and interesting gold coins, and so forth.
How it works:
The identifier is printed on each page and includes source location of the original. The document file name has the date and subject coded. These are key fields in the database. If I’m working on a step 3 article, I can easily search every document file or location for relevant items. This includes all of step 5 or 6. A search of 100,000 pages takes about 5 seconds sometimes a little longer if a lot of PDF files are involved.
Nothing makes correlation, cross checking, validation and analysis much easier – but being able to sort through large amounts of data really helps…. Until it doesn’t ! 
Comments
This approach also results in a lot of incomplete and complete but unpublished articles. I currently have maybe 25 complete articles that are too long or complicated for hobby publications. (The Journal of Numismatic Research - JNR - approach I tried failed...Expensively!)
As Dick Johnson said, a writer's most valuable asset is their own research files.
RogerB is more organized that most. The ability to digitally search one's own research files is unusual. Most of us rely on memory, and people like Bowers or Breen or Newman had extraordinary abilities to recall documents not seen in many years. That's always a great tool to have, but the computer changes everything and can't be ignored.
Research is tedious, and Roger does it well....The small research projects I have undertaken (not in numismatics) have always developed like home repairs... much more than initially expected. I commend the research people, patience and dedication are imperative qualities....Cheers, RickO
I do not trust my memory for either completeness or accuracy. Hence, the habitual - compulsive ? - identification of documents and sources.
Early on, I did not mark every page, but quickly learned that was a bad idea. The first drafts of my first book were soundly teased for having "too many footnotes" (...a parody on Emperor Joseph II's comment "And there are simply too many notes, that's all. Cut a few and it will be perfect," from the movie Amadeus).
To give readers an idea of the kind of mixture found in nearly every box and volume, here’s a sample from E-229 Box 092. There are over 500 pages in this box; most are routine reports and acknowledgements. These were of interest when I copied the documents in June 2016.
18990518 NO Asst M-R Ingalls missing.pdf
18990518 NO Asst M-R position vacant.pdf
18990519 P Poor ring to gold-blistered strips.pdf
18990519 SF Die hardener dead.pdf
18990520 NO M-R weighed correct.pdf
18990523 HQ DE of 1849.pdf
18990523 Lafayette portrait design.pdf
18990525 HQ Confusion Seattle payments.pdf
18990526 P Poor ring to gold-blisters.pdf
18990527 HQ Mint for Juneau Alaska.pdf
18990531 HQ Date on Lafayette dol.pdf
Super info Roger.
I'd be interested in this one.
18990523 Lafayette portrait design.pdf
PDF attached. Let me know when you upload it so I can delete the file from the message boards and not hog space.
Got it, thanks!
OK. PDF removed.