Imagine how much unanticipated historical perspectives might become uncovered if everyone uploaded paraphenelia of long deceased ancestors like this; after indexing, and searched as one hyper-amalgamated crowdsourced knowledge graph, can show who was where doing what in say the 1920s, 1930s, 1940s in a way that mainstream history might fall short of capturing.
dogline
Also, just to clarify, I scanned all 7488 pages in personally (Fujitsu ScanSnap ix500). With Claude's help, I found some undocumented SANE features to auto crop and fix the scans, then had a Python script in Linux auto scan them and put them into a Postgres database as I went. Other scripts would add transcription, summaries, and auto index everything.
"mistral-ocr-latest" did really good handwriting transcription, considering how tight and small some of the handwriting is. Then back to Claude API calls to summarize by month and collect people and places from all of the entires.
Claude then created static html pages from what started as a Flask app. Published on Dreamhost.
show comments
jlpk
Nice work! For others with journals in the U.S., but not feeling up to all the scanning and transcription work, I volunteer with the American Diary Project (https://americandiaryproject.com/) based in Cleveland Ohio. You can donate journals to be archived and shared. It's only been established in the past few years, and all scanning/transcription is done by volunteers, but are currently evaluating more automated pipelines like OPs. So great to see it in practice!
reaperducer
Fun fact: "Government mule" isn't just an expression, it's a real thing. And the U.S. government, including the Forest Service, still employs teams of mules to carry things to places that can't be reached any other way.
show comments
toomuchtodo
Well done! Have you uploaded these scans to the Internet Archive? If not, please consider doing so.
Imagine how much unanticipated historical perspectives might become uncovered if everyone uploaded paraphenelia of long deceased ancestors like this; after indexing, and searched as one hyper-amalgamated crowdsourced knowledge graph, can show who was where doing what in say the 1920s, 1930s, 1940s in a way that mainstream history might fall short of capturing.
Also, just to clarify, I scanned all 7488 pages in personally (Fujitsu ScanSnap ix500). With Claude's help, I found some undocumented SANE features to auto crop and fix the scans, then had a Python script in Linux auto scan them and put them into a Postgres database as I went. Other scripts would add transcription, summaries, and auto index everything.
"mistral-ocr-latest" did really good handwriting transcription, considering how tight and small some of the handwriting is. Then back to Claude API calls to summarize by month and collect people and places from all of the entires.
Claude then created static html pages from what started as a Flask app. Published on Dreamhost.
Nice work! For others with journals in the U.S., but not feeling up to all the scanning and transcription work, I volunteer with the American Diary Project (https://americandiaryproject.com/) based in Cleveland Ohio. You can donate journals to be archived and shared. It's only been established in the past few years, and all scanning/transcription is done by volunteers, but are currently evaluating more automated pipelines like OPs. So great to see it in practice!
Fun fact: "Government mule" isn't just an expression, it's a real thing. And the U.S. government, including the Forest Service, still employs teams of mules to carry things to places that can't be reached any other way.
Well done! Have you uploaded these scans to the Internet Archive? If not, please consider doing so.
https://help.archive.org/help/uploading-a-basic-guide/
https://help.archive.org/help/managing-and-editing-your-item...
Trail Crew Stories and Mountain Gazette might also be interested in this.
https://www.trailcrewstories.com/
https://mountaingazette.com/