Monday, January 26, 2026

Improving searchability using Natural Language Processing and Wikidata

The Python Natural Language Processing Toolkit and FastAPI form the foundation for new, major improvements in the searchability of various numismatic projects, beginning with American Numismatic Society type corpora such as OCRE and Hellenistic Royal Coinages. These improvements currently apply to all online type corporate published by the ANS and are slowly reindexing in the ANS collections database, MANTIS, to improve large swaths of the ANS's collection of Greek and Roman coins that have been linked to these coin type URIs. This work began in the fall and was finally put into production over the course of December 2025.

The Problem: Words that lack meaning

There are 273 results in Online Coins of the Roman Empire when searching the term "serpent" and 589 when searching for "snake". Searching in MANTIS and other Numishare-based platforms is based quite literally on the explicit entry of words within the obverse and reverse type descriptions of coins and types. Such description is subject to idiosyncrasies of terminology related to regional linguistics. There is nothing inherent within the search index software upon which Numishare is built, Apache Solr, to suggest that a serpent and a snake are two words for the same type of animal, and the use of these two words is not only inconsistent across Roman Imperial Coinage volumes, but also within the same section of the same volume (e.g., for Galba in RIC I). Similarly, this plays out in MANTIS, as there are five objects featuring the word "puma," one "cougar," and two with the extremely rarely-used "catamount." Some residents of the United States refer to these as "mountain lions," although there are no results for this exact phrase in MANTIS. If a researcher is interested in all of the depictions of reptiles, birds, cats, architecture, etc. on coins and medals, there is simply no way to capture all possible results without the introduction of a semantic layer on the descriptive text.

 

A search for cougar in MANTIS, with one result instead of eight

The Solution: NLP + Wikidata

The first step in equating "serpent" with "snake" for merging the results for either term is the use of the Python Natural Language Toolkit to parse textual descriptions of coins into their constituent parts of speech. 

As an example, we'll use this type description: "Asclepius, nude, standing front, head left, 
leaning on small staff with serpent coils." After tokenizing the description into `words`, we'll analyze their parts of speech:

tagged_words = nltk.pos_tag(words)

[('Asclepius', 'NNP'), 
('nude', 'NN'), 
('standing', 'VBG'), 
('head', 'NN'),
('leaning', 'VBG'), 
('on', 'IN'), 
('small', 'JJ'), 
('staff', 'NN'),
('with', 'IN'), 
('serpent', 'NN'), 
('coils', 'NNS')]

Although there is potential in interpreting types of movement and action on coins (verbs), our focus in this phase is on the nouns, whether proper or not. The nouns are essentially concepts, like 'Asclepius,' 'staff,' and 'serpent.'

There is nothing inherent within the Python NLTK platform to suggest the semantic meaning of these words, so we need to introduce the intermediary step of reconciling them to their related concepts in Wikidata.org. This was done relatively easily in OpenRefine. This also has the added benefit--which is essential for a scholarly dataset--of introducing human vetting (by me) in order to control the quality of the output. These semantic assertions should not be left to an LLM without human verification, which is one of several reasons why we cannot use an LLM to improve searchability of the data, because the results are not consistent or dependable.

Reconciling terms in OpenRefine
 

Following the reconciliation of the few thousand concepts that appear throughout Hellenistic Royal Coinages, OCRE, and CRRO, the lookup table between search term, Wikidata URI, and preferred label is loaded into SQLite, and I used FastAPI in Python to create an easy lookup mechanism that will convert pre-processed type descriptions into related concepts in JSON, which are then integrated into the NUDS records in Numishare. Once the concept keywords are in the NUDS record, then Numishare makes use of them to index preferred labels, alternative labels, and labels related to hierarchical parent concepts into Solr for search. 

FastAPI response that uses NLP to look up terms in SQLite table after reconcilation

In order to facilitate the most efficient indexing process possible, the alternative and hierarchical labels are also extracted via Wikidata SPARQL in an intermediary process and stored in SQLite and served via FastAPI. Wikidata has request limits based on time, so one could never index a collection as large as OCRE without pausing frequently to refrain from violating Wikidata's API policy.

Wikidata SPARQL query extracting labels and hierarchical labels for concepts

Technical Workflow Summary

The technical workflow is thus summarized as follows:

  1. Parse descriptions, extract iconographic concepts (Python: NLTK)
  2. Reconcile concepts to a controlled vocabulary system, Wikidata.org (OpenRefine)
  3. Extract alternative labels and hierarchical labels from Wikidata (Python/SPARQL)
    1. "snake" = "serpent"
    2. "snake" is a type of "reptile" which is a type of "animal" etc.
  4. Create a fast lookup mechanism for integration into the web (Python: FastAPI)
  5. Index terms for improved search (Numishare) 

This workflow was repeated for each ANS type corpus following the developing of the prototype on the relatively small and well-curated Coinage of the Roman Republic Online.

Results

Following the implementation of NLP into the indexing workflow, there are now more than 700 results for "serpent" or "snake" in OCRE. Either term yields the same results, as expected. There are more than 4,500 results for "architecture" since altars, gates, arches, and temples are all classified as architecture in Wikidata. This is a dramatic improvement in searchability, but it is certainly not perfect yet (although perfection is not really achievable). There are some limitations, illustrated below:

  • Applies to typology databases (currently), not MANTIS
  • Depends upon detailed textual descriptions, not images
  • Wikidata labels and hierarchical structure may be inconsistent or incomplete
  • LLM may be more intuitive, but the cost overhead is prohibitive, particularly in real-time. 

Although the results of this work do not apply to all of MANTIS yet, the connection of tens of thousands of coins in the ANS collection to coin type URIs makes it possible to extract the iconographic concepts from the NUDS data in OCRE and other projects for indexing in MANTIS under the same process. After updating the indexing code in Numishare and reindexing all of the ANS' coins from CRRO, the number of coins grew from 123 to 350 when searching for "architecture," since more than 200 Roman Republican coins in the ANS collection link to the 29 types featuring some kind of architecture. The original 123 results in MANTIS included the word "architecture" explicitly in the type description or elsewhere in the record.

Search of HRC for "animals", including eagles, elephants, horses, etc.
 

After the conclusion of the indexing of ANS coins connected to type URIs, we will implement NLP on descriptions department by department, although the results of this work may take some time to vet, for lack of staffing in other departments.

Nevertheless, this is an enormous enhancement in the searchability of our numismatic collections, and the code is reusable by other projects, whether they are numismatic in nature or not. The instructions for deploying the services and processing data have not yet been fully documented in the Github repository

On January 23, I presented an ANS Longtable about this work. The slides have been uploaded to Zenodo, and the recording should be available on YouTube in the near future. 

Thursday, November 6, 2025

Updates to RRDP specimens

At long last, we have migrated the objects accessible in the SITNAM public database into a CollectiveAccess back-end. SITNAM was initially launched as a database that mirrors the functionality of the ANS collection, MANTIS, but for objects from public or private collections that don't have permanent, stables URIs. The test case was for the Roman Republican Die Project, for which there are tens of thousands of Republican coin images--largely from auction catalogs--pasted into cards and binders. The specimen data were stored in spreadsheets, but migrating these data into CollectiveAccess has opened the door to publishing improved bibliographic data in SITNAM that was not present in the first iteration of the database.

A coin from the Cisterna di Latina hoard

  • Several hundred coins contained references to Coin Hoards of the Roman Republic, and so those URIs have been added into the hoard field for these records, and points for the hoard show up in maps of related specimens, e.g., this coin cited in Hersh and Walker's article on the Mesagne hoardThe hoards also propagate into the map-based search interface in SITNAM. Hoard is now a search facet in SITNAM. Likewise, coins from SITNAM (and other contributors to CRRO) will appear in the example specimens on hoard pages (see figure, below).
  • There is an improvement in publishing bibliographic references (which link to the ANS library catalog, if applicable) associated with public collections that have Nomisma URIs. Previously, only the collection name was present in the record, but not the associated reference work, for example, R. Wegeli and P. Hofer, Die Munzen der romischen Republik bis zum Jahre 27 (Bern, 1923) associated with coins in the collection of Bern, Switzerland.
  • Auction records are better connected with the organization responsible with the sale, often linking to the VIAF URI for the entity. The seller is now a facet in the search interface, enabling a user to see all coins sold by the Classical Numismatic Group, for example
  • Fixed typos or missing provenance records for about 1,000 coins in RRDP (about 2% of the collection). The provenance records are now stored in relational tables, which has made it possible to link a single specimen from Levantine Coins Online (a coin of Yehud) into the auction catalog record for a handful of Roman Republican coins from the RRDP dataset. This is the groundwork for greatly improving provenance research.
  • External links to collections databases (which are not Nomisma contributors) or auction databases, like CoinArchives are now visible in SITNAM and should also be visible on the type pages for CRRO. 
  • Auction dates are stored in the database, but not yet searchable in SITNAM, and so that could be one area to improve in the future: to search for all coins sold within a particular date range.

 

Several specimens from La Grajuela Hoard (GRJ). 
 

Several hundred coins of Yehud for the Levantine Coins Online project have subsequently been migrated into the SITNAM CollectiveAccess back-end, which greatly simplifies the long-term sustainability of disparate projects that need to publish coins from auctions or excavations that do not have stable URIs maintained by the holding individual or institution.


Friday, October 31, 2025

Department of Classics at Cologne joins Nomisma

More than 100 Seleucid and Ptolemaic coins from the Department of Classics at the University of Cologne have been integrated into the Hellenistic Royal Coinages platform, the first tranche of a collection consisting of over 12,000 specimens. This is, in fact, the second collection housed at the University of Cologne to join Nomisma, the other associated with the Department of History and part of the NUMID consortium.

A Department of Classics coin among CPE I.2 B126

 

Tuesday, October 21, 2025

Princeton University joins Nomisma LOD cloud

The Princeton University numismatic collection has bee integrated into the Nomisma.org Linked Open Data cloud, with almost 3,600 coins linked to coin types in Online Coins of the Roman Empire, Coinage of the Roman Republic Online, and the various portals that fall under Hellenistic Royal Coinages. The majority of the coins have been photographed and are accessible as zoomable IIIF images. Although many coins are from excavations, e.g., Antioch, the contextual information is not yet expressed is LOD to populate geographic visualizations associated with types and Nomisma concepts as this phase. At some point, we should expect to see a large submission of Byzantine coinage from Princeton when the new joint typology project from Princeton-Dumbarton Oaks is ready for publication.

 
 

A Princeton coin among among others of Seleucid Coins (part 2) 2363b

Thursday, July 17, 2025

Art Institute of Chicago joins the numismatic Linked Open Data cloud

The Art Institute of Chicago is the latest collection to join the growing international Nomisma.org Linked Open Data ecosystem, providing more than 200 Roman Imperial coins for Online Coins of the Roman Empire. This is the first portion of their numismatic collection to be integrated into the LOD cloud, and other ancient coins will be integrated in due course.

A screen shot of the coin type represented by http://numismatics.org/ocre/id/ric.4.ss.97_denarius, showing a description of the type, example coins in various museum collections, and a map of circulation.
AIC coin 1920.1008 on RIC Septimius Severus 97 (denarius)
 

The images provided by the AIC are IIIF compatible, providing zooming functionality in the OCRE user interface, and they are also granted public domain licenses regarding their reuse in publications. 

Wednesday, July 2, 2025

Samarian coinage added to Levantine Coins Online

It had gone unannounced on this blog at the time, but in late March, 376 coin types from the upcoming A Corpus of Samarian Coinage, by Oren Tal, Haim Gitler, and Mati Johananoff, were published to Levantine Coins Online. The ANS has recently cataloged our relevant coins with these new Samarian URIs, which have subsequently been exported from Mantis into the Nomisma.org SPARQL endpoint. The ANS now contributes 255 of the 377 total coins linking to LCO, providing photographic examples of 162 of the 376 published Samarian types.

A screenshot of http://nomisma.org/id/samarian_coinage, with map and type examples

At least one of the ANS's objects is from IGCH 1504, providing a hoard coordinate point the related coin type and Nomisma.org numismatic concepts.

The type data were prepared for publication by Jessica Schillig. 

Tuesday, May 27, 2025

FileMaker Pro finally retired at the ANS

After about two decades, the American Numismatic Society has finally turned the lights off in its FileMaker Pro database. The last two departments, Medieval and Modern European were migrated into CollectiveAccess about one month ago, although not in an entirely completed form--the places, people, organizations, dynasties, and denominations still require further normalization by curator, David Yoon, over the next year before these entities are merged into the relational database system. Nevertheless, FileMaker Pro cataloging has ceased, marking a major step in a once-in-a-generation task of normalizing and migrating systems.

The ANS database system was a bespoke DOS-based system pioneered in the 1980s called PRIME. The ANS was among the first major numismatic collections to implement a database for cataloging, although it was never designed as a relational database (a technology which did exist at the time). The database was ostensibly a flat "spreadsheet" data entry system with more than 100 fields, most of which were uncontrolled free-text fields. Before the World Wide Web, curators never envisioned their catalog would become public, and quality and completeness were inconsistent across departments as curators developed their own idiosyncratic data entry processes.

At some point during the 2000s, this PRIME system was picked up and moved into FileMaker Pro, with little to no intermediary normalization. Indeed, many MSDOS special characters (non-ASCII and non-UTF-8) for tabs or other such breaks were retained within FileMaker. Since its inception in 2011, many thousands of lines of PHP code were necessary to transform the ANS' cataloging data into something generally usable on the web with Mantis. The code became more and more complex as we sought to reconcile type or hoard reference patterns to URIs in OCRE, Hellenistic Royal Coinages, or Coinhoards.org. The FileMaker->Mantis publication process was a house of cards.

In 2015, we decided that migrating from FileMaker Pro was a necessity and decided upon the open source collection management software, CollectiveAccess. CollectiveAccess was built on PHP and MySQL, with a fully customizable metadata entry system that could accommodate the significant complexity our curators require. Over several years, we customized and tested a numismatic data entry system for CollectiveAccess, but it wasn't until January 2023 that we officially began the data cleanup and migration process. Significant time and effort was invested over two years in reconciling entities to URIs in Linked Open Data controlled vocabulary systems--not only Nomisma.org, but also Wikidata and Geonames. Most ancient places in CollectiveAccess align to Nomisma and most modern ones align to Geonames. I am not certain of the exact percentage, but there is significant overlap between the rulers, kingdoms, artists, dynasties, issuers, etc. in our numismatic collection and Wikidata. This reconciliation will enable better quality and more complete query of our collection.

I myself worked on Medals and Decorations, as well as Greek, Roman, and Byzantine. Sami Norling, now of the Smithsonian Institution, helped tremendously with North and Latin American, Islamic, and South Asian. David Yoon completed East Asian several months ago and continues to iron out loose ends in Medieval and Modern.

Once these data have been migrated into a proper relational database system, the difficult and time-consuming work of normalization and reconciliation has already been completed. Going from PRIME to FileMaker Pro was kicking the can down the road and building a sustainable curatorial database. If we migrate from CollectiveAccess to another system in another 10 or 20 years, it will be much easier to migrate rigorously curated relational data than the free text fields that made up the ANS database for the last 40 years. This is a significant milestone in the ANS's technological history.