Managing Complex Structured Data In a Fast Evolving Environment

Abstract: Criminal data comes in a variety of formats, mandated by state, federal, and international standards. Specifying the data in a unified fashion is necessary for any system that intends to integrate with state, federal, and international law enforcement agencies. However, the contents, format, and structure of the data is highly inconsistent across jurisdictions, and each datum requires different ways of being printed, transmitted, and displayed. The goal was to design a system that is unified in its approach to specify data, and is amenable to future "unknown unknowns". We have developed a domain-specific language in Common Lisp which allows the specification of complex data with evolving formats and structure, and is inter-operable with the Common Lisp language. The resultant system has enabled the easy handling of complex evolving information in the general criminal data environment and has made it possible to manage and extend the system in a high-paced market. The language has allowed the principal product of Secure Outcomes Inc. to enjoy success with over 50 users throughout the United States.