Language specification

Data Definition Language

Write structured data using six composable value types.

Overview

The Arcadia Data Definition Language (DDL) describes structured data for exchange between humans and machines. A document is a sequence of Unicode code points describing one value: a Boolean, Number, String, Void, List or Map. Lists and maps can contain other values, including nested collections.

Start with the complete example and type reference below. The three translation sections then define the formal rules. Data Definition Language Schema (DDLS) describes constraints on DDL data; it is separate from the DDL grammar. This page does not claim that proposed DDLS versions are implemented.

DDL selects specific rules from the shared lexical specification. The conformance notes summarize these selections, verified reader boundaries and migration guidance.

Your first DDL document

This fictional Stone Lantern record is a complete DDL document, verified by the FirstDocument reader fixture. It is an example of DDL structure, not an ADL asset or a DDLS schema.

{
  name: "Stone Lantern",
  enabled: true,
  size: 1.5,
  layers: ["stone", "light",],
  details: {note: void,},
}

The outer map has five entries in source order. Each name is followed by a colon and a value. layers contains a list of two strings; details contains another map whose note has the value void. The commas before closing brackets are permitted. Map names are not quoted strings, and the keywords true, false and void cannot be used as names.

Six-type quick reference

Examples below are covered by DocumentationContract and the existing semantic fixtures. Each invalid example is a complete input, not just a value of a different type.

Translation pipeline

1. Lexical translation: Unicode code points become words (tokens).
2. Syntactical translation: whitespace, newlines and comments are discarded; the remaining words form one sentence.
3. Semantical translation: the sentence denotes a typed value; duplicate names in a map are rejected.

Lexical translation

Lexical translation turns the input sequence of Unicode code points into words (tokens).

The lexical translation of the Data Definition Language is based on Common Lexical Translations.

DDL selects only the double-quoted alternative of the shared string category. Within this specification, \(\text{Lexical.String}\) is restricted by

Single quotes do not delimit DDL strings or character literals. An apostrophe inside a double-quoted string is ordinary text, as in "It's ready"; a double quote inside it must be escaped as \". The broader shared Lexical.String category remains available to other languages.

DDL also defines its own name profile. Within this specification, Lexical.Name matches [A-Za-z_][A-Za-z0-9_]*: the first character is an ASCII letter or underscore; subsequent characters are ASCII letters, digits or underscores. Names are nonempty and case-sensitive. Examples include name, _, __, _1 and layer_2.

The exact lowercase words true, false and void are reserved keywords, not names. Keyword prefixes such as true_ are names. Map keys must be names, not quoted strings or digit-leading forms such as 1name. This DDL profile replaces the shared Name production's requirement for a letter after leading underscores; it does not redefine other languages' name rules.

The DDL label Lexical.Newline denotes a CR, LF or CRLF line ending. CRLF is recognized as one newline word. It plays the end-of-line role described by the shared lexical categories; the label is retained for existing DDL grammar references.

The lexical grammar consists of

• a set of non-terminals \(\textit{DDL.Lexical.NonTerminals}\) and the set of terminals \(\textit{DDL.Lexical.Terminals}\) which are disjoint
• a set of production rules \(\textit{DDL.Lexical.ProductionRules}\), specified in this section, and
• a starting symbol \(\text{DDL.Lexical.Words}\) which is element of \(\textit{DDL.Lexical.NonTerminals}\).

The two rules, defined in terms of the Common Lexical translations, are defined as follows:

The lexical translation translates a sequence of Unicode code points into words. This resulting sequence of words is then consumed by the syntactical translation.

Lexical.Colon denotes :, as defined in Common Lexical Translations. The current reader recognizes spaces and tabs, CR, LF and CRLF, // line comments and /* ... */ block comments. Comments are not nested. Names and keywords are case-sensitive. See conformance for DDL-specific selections and reader edge cases.

Syntactical translation

The syntactical translation translates a sequence of words provided by the lexical translation into a sentence.

The syntactical grammar consists of

• a set of non-terminals \(\textit{DDL.Syntactical.NonTerminals}\) and the set of terminals \(\textit{DDL.Syntactical.Terminals}\) which are disjoint
• a set of production rules \(\textit{DDL.Syntactical.ProductionRules}\), specified in this section, and
• a starting symbol \(\textit{DDL.Syntactical.Sentence}\) which is an element of \(\textit{DDL.Syntactical.NonTerminals}\).

Important: The following words are removed from the sequence of words before its translation into a sentence:

• \(\text{Lexical.Whitespace}\) (also called \(\text{DDL.Lexical.Whitespace}\)),
• \(\text{Lexical.Newline}\) (\(\text{DDL.Lexical.Newline}\)), and
• \(\text{Lexical.Comment}\) (\(\text{DDL.Lexical.Comment}\)).

The sentence consumes the entire filtered input: exactly one value, followed by end-of-input. A comma after the top-level value is not allowed. Whitespace and comments may follow that value before end-of-input. In the rules below, \(\epsilon\) is the empty sequence.

DDL.Syntactical.Value

The sentence \(\text{DDL.Syntactical.Value}\) is defined by

DDL.Syntactical.String

The sentence \(\text{DDL.Syntactical.String}\) is defined by

DDL.Syntactical.Number

The sentence \(\text{DDL.Syntactical.Number}\) is

DDL.Syntactical.Boolean

The sentence \(\text{DDL.Syntactical.Boolean}\) is

DDL.Syntactical.Void

The sentence \(\text{DDL.Syntactical.Void}\) is

DDL.Syntactical.Map

A map may be empty. Each nonempty entry has a name, a colon and a value; commas separate entries, and one trailing comma is allowed. Quoted strings and reserved keywords are not map names.

The sentence \(\text{DDL.Syntactical.Map}\) is

DDL.Syntactical.List

A list may be empty. Otherwise commas separate values, with one optional trailing comma. A comma alone does not form an empty collection.

The sentence \(\text{DDL.Syntactical.List}\) is

The resulting sentence is consumed by the semantical translation.

Semantical translation

Semantical translation maps a sentence to a typed value. The Data Definition Language has six basic types: \(\textit{List}\) and \(\textit{Map}\), which are the so called aggregate types, and \(\textit{Boolean}\), \(\textit{Number}\), \(\textit{String}\), and \(\textit{Void}\), which are the so called scalar types.

The type \(\textit{Value}\) is defined as the union of all the types above. \[\begin{aligned} \textit{Value} =&\;\textit{List}\\ \cup&\;\textit{Map}\\ \cup&\;\textit{Boolean}\\ \cup&\;\textit{Number}\\ \cup&\;\textit{String}\\ \cup&\;\textit{Void} \end{aligned}\]

The translation of a sentence into values is described by syntax-directed translations (see Aho, Sethi and Ullman for background).

At the end of translation, a value form \(x\) has an attribute \(x.\text{value}\) of type \(\textit{Value}\). In each table, numbered occurrences associate a grammar production with its semantic action. The operator \(:=\) assigns an attribute; \(\circ\) concatenates lists, \([a]\) is a singleton list and \([]\) is an empty list. The attribute \(\textit{codePoints}\) means the word's associated text: the original numeric spelling for Number, the name spelling for Name, but the decoded contents (without quotes) for String. It does not always mean the raw source lexeme.

The current reader builds DDL nodes. Boolean and Void nodes store keyword spellings, Number nodes store numeric text, and String nodes store decoded UTF-8 text. The rules below describe the denoted values, not a promise that every node is converted to a native numeric or Boolean runtime type. Semantic analysis is an explicit operation; DefaultReader_run parses but does not itself invoke duplicate-name checking.

Scalar types

The scalar rules translate a single word into its denoted value.

Boolean type

The type \(\textit{Boolean}\) has two values, \(\textit{true}\) and \(\textit{false}\), expressed by true and false (\(\text{Lexical.True}\) and \(\text{Lexical.False}\)). The older names DDL.Lexical.true and DDL.Lexical.false refer to these same keywords.

The semantical translation is defined as follows:

Number type

The type \(\textit{Number}\) retains a sequence of code points satisfying \(\text{DDL.Syntactical.Number}\). Its representation is textual: for example, 01 and +1 remain distinct spellings. Decimal notation allows an optional sign, an optional decimal point (with digits on at least one side), and an optional exponent introduced by e or E with an optional sign and at least one digit.

The semantical translation is defined as follows:

Representation and precision: Reading a Number does not perform arithmetic or convert it to int or float. The grammar does not impose a fixed machine-number range, but storage is bounded by implementation resources. Conversions performed by consumers have their own range, precision and error behavior; this specification does not promise arbitrary-precision numeric operations.

String type

The \(\textit{String}\) type represents decoded text, not its quoted source spelling. For example, "\u{41}" denotes the same text as "A". DDL strings are delimited only by double quotes; single-quoted strings and character literals are not supported. The reader decodes \", \\, \b, \f, \n, \r, \t, \v, \0 and braced \u{...} escapes. Each Unicode escape requires one to six digits from 0-9, a-f or A-F inside braces. The decoded value must be at most U+10FFFF and outside U+D800 through U+DFFF (the surrogate range). Malformed escapes and non-scalar values cause a lexical error; \uXXXX is no longer accepted. Supplementary scalar values are supported directly as UTF-8 or through braced escapes, with identical decoded text. Unescaped code points U+0000 through U+001F are rejected. Embedded NUL is part of the text, not a terminator. No Unicode normalization is performed. Surrogate-pair escapes are not supported: use the scalar value itself, for example \u{1F3EE}.

The semantical translation is defined as follows:

Void type

The type \(\textit{Void}\) has a single value, \(\textit{void}\), expressed by the keyword void.

The semantical translation is defined as follows:

List type

The type \(\textit{List}\) represents an ordered list of values of type \(\textit{Value}\). A list is expressed by \(\text{DDL.Syntactical.List}\). The rules prepend each element to the translated remainder; empty bodies and trailing commas contribute an empty list.

The semantical translation is defined as follows:

Example

// A list with three numbers 1, 2, and 3.
[ 1, 2, 3 ]

Map type

The type \(\textit{Map}\) represents an ordered list of entries. Each entry is a pair: a name spelling (\(\textit{Name}\), not a seventh value type) and a value of type \(\textit{Value}\). Name comparison is case-sensitive. Names must be unique within each map; duplicate names cause a semantic error, including in nested maps. The same name may occur in different maps. A value of type \(\textit{Map}\) is expressed in the language by the sentence \(\text{DDL.Syntactical.Map}\) (as defined in the syntactical grammar).

The semantical translation is defined as follows:

Value type

The type \(\textit{Value}\) is the union of the types \(\textit{Boolean}\), \(\textit{List}\), \(\textit{Map}\), \(\textit{Number}\), \(\textit{String}\), and \(\textit{Void}\).

The semantical translation is defined as follows:

Errors

Failure categories describe where translation stops; they do not prescribe diagnostic wording.

Lexical error

The input cannot be read as words: for example +, -, +e2, 1e, "\q", "\u{G}" (a non-hexadecimal escape digit), "\u0041" (missing braces), an unterminated string or an unterminated block comment. The tested reader status is LexicalError.

Syntactical error

The words cannot form a complete sentence: for example {x 1} (missing colon), [1 2] (missing comma), an empty document, true false (a second value), or true, (a top-level trailing comma). The tested reader status is SyntacticalError.

Semantical error

A parsed sentence violates a value constraint: {x:void, x:1} repeats a map name. Explicit semantic analysis reports SemanticalError; parsing alone does not reject this sentence.

Conformance and reader edges

The regression fixtures exercise the scanner/parser and explicit semantic analysis separately. These observations apply to the current implementation, not to every possible DDL implementation. The approved DDL-specific rules and reader boundaries are summarized below.

DDL strings use double quotes only. "text" and "It's ready" are valid; 'text', 'a' and '' fail with LexicalError. There is no single-quoted character-literal form. This is an intentional DDL restriction of the shared string grammar, not a reader/specification mismatch. The writer also emits double-quoted strings.

Map-entry names follow [A-Za-z_][A-Za-z0-9_]*, including _, __ and _1. They are case-sensitive ASCII names; true, false and void are reserved. Digit-leading and quoted keys are invalid. This is the intended DDL name profile, not an unresolved difference from the shared grammar.

A document contains one top-level value, followed only by whitespace or comments and then end-of-input. Top-level trailing commas such as true, and [1,], fail with SyntacticalError. Trailing commas inside collections remain valid: [1,] and {x:1,}.

Migration: remove a comma after the complete document value. Do not remove collection trailing commas unless desired for style.

Unicode escapes use only \u{...}, with one to six hexadecimal digits inside braces. For example, "\u{41}" decodes to "A"; "\u{1F3EE}" has the same decoded UTF-8 text as a literal U+1F3EE. Empty braces, more than six digits, non-hexadecimal digits, values above U+10FFFF and surrogate values U+D800 through U+DFFF fail with LexicalError.

Migration: the former unbraced \uXXXX syntax is rejected, not retained as an alias. Replace \u0041 with \u{0041} (or \u{41}). Do not wrap surrogate halves separately; use the complete scalar value instead. Other escape forms such as \n remain unchanged.

A numeric word requires at least one mantissa digit, and an exponent, when present, requires at least one digit. The scanner rejects +, -, +e2 and incomplete exponents such as 1e+ with LexicalError. Valid forms such as +1, .1, 1. and 1e2 retain their exact spelling. This error-classification rule does not change the accepted numeric forms or the validation contract of direct Number-node API calls.

The short escapes follow the shared string translation rules: \b denotes U+0008 (backspace), \f U+000C, \t U+0009, \v U+000B and \0 U+0000 (embedded NUL). Braced escapes decode to Unicode scalar values; UTF-16 surrogate-pair escapes and normalization are not supported.

The repository's contract matrix and verification handoff are in Documentation/Specifications/Data Definition Language/Verification.md. The maintained cases are discoverable through the test executable's --list or CTest.

References

• Common Lexical Translations: shared word productions (see DDL-specific conformance notes).
• Mathematical Definitions, Notations and Proofs: grammar and list notation.
• DDLS specification: schema constraints.
• Aho, Sethi and Ullman, Compilers: Principles, Techniques, and Tools, first edition, p. 305: syntax-directed translation.