Linked Data More Than a Millennium Old

Organizing and comparing more than 12,000 pages of text written by dynasties separated by one hundred and fifty years is no easy task. But this is exactly what Professor Anna Shields in Princeton’s East Asian Studies (EAS) department and her team are working toward in their Tang History Database, with support from the Center for Digital Humanities’ dataset curation grants.

The project focuses on two massive texts: the Old and New versions of the history of the Tang Dynasty (618-907 CE). Shields and her colleague Professor Wen Xin are creating a digital, open-access database of these two texts that allows for side by side comparisons. For even more analysis, the project will tag names of people in the Histories to connect them to the Chinese Biographical Database (CBDB) and tag locations to data in the Chinese Historical Geographic Information System (CHGIS) database.

Unlike existing, larger databases of Chinese texts, this project focuses exclusively on two works to make it easier to see patterns and differences between the two. And the differences are significant. The Old Tang History was “compiled relatively quickly over the course of four years [around 945 C.E.] from existing sources after the fall of the Tang dynasty,” said Shields, so although it offers a wealth of primary sources, later editors found it insufficient and inaccurate. As a result, during the Song dynasty (960-1279), nearly 150 years after the end of the Tang dynasty, editors produced a revised New Tang History (1060 C.E.). These editors also sought to make the lessons of history clearer. “Heroes became more heroic and villains more villainous, is one way I’ve put it,” said Shields, whose current book research focuses on the biographies in the texts. “The biographies in particular were rewritten to make the life stories more clearly exemplary, whether those were positive or negative examples.” Finally, the new version introduces cultural norms more in line with those of the Song dynasty.

To create the database, the project team searched for the “cleanest” version of the Tang histories. They chose an open-source digital version to use as a base. Shields then recruited students with reading abilities in traditional and simplified Chinese characters to check that text against the print versions. In working on the project, these students will learn more about data visualization, machine learning, and linked open data.

Also essential is team member Jeff Heller, the data and project coordinator of the EAS department. Heller has a background in data analysis and development, as well as a BFA in graphic design. Before Princeton he worked at Apple, as a “personal trainer for computers” as he puts it, offering training sessions for customers and colleagues. In order to teach people, he says, “I implemented the use of analogies, metaphors, and different modes of language. I focused on paring down systems to their bare elements.” This skill has helped Heller share his knowledge with his project colleagues, even though he does not have as much experience in Chinese history and language.

For Shields, whose research explores “how definitions of literature and the literary sphere were reimagined,” the ability to compare these works allows for new insights. In the past, scholars have focused on comparing individual biographies. “No one has attempted to read across the biographies for this kind of large-scale ideological and cultural revision before,” Shields said, adding that this comparative database allows her to attempt to quantify changes and move away from more “impressionistic” scholarship.

As Shields and her team continue their work, they see substantial potential for additional research. By connecting names to the Chinese Biographical Database, for example, scholars can see how individuals appear or disappear within the accounts and analyze their social networks, Shields says. They might use geographical data to explore the meanings of specific places or migration patterns. Shields hopes that the database encourages researchers to see the Histories differently. “This project explicitly approaches the Histories as representations rather than ‘data’ about the Tang,” she says. Placing the versions next to each other underscores that each is a constructed narrative, rather than a source of information to be mined uncritically.