databricks/sjsonnet

databricks/sjsonnet

Language: Scala

License: Apache-2.0

Stars: 311

Forks: 67

Open issues: 10

Created: 2018-06-15T03:57:54Z

Pushed: 2026-06-09T16:34:06Z

Default branch: master

Fork: no

Archived: no

README:

Sjsonnet

A Scala implementation of the Jsonnet configuration language, running on JVM, GraalVM, Scala Native and JavaScript.

Usage as a CLI

We release standalone executables JARs, Scala Native and GraalVM in the github release page:

$ chmod +x sjsonnet.jar

$ ./sjsonnet.jar
Missing argument: file
Expected Signature: Sjsonnet
usage: sjsonnet [sjsonnet-options] script-file
-A --tla-str [=] Provide top-level arguments as string. 'If is
omitted, get from environment var
-J --jpath Specify an additional library search dir (left-most wins unless
reverse-jpaths-priority is set)
-S --string Expect a string, manifest as plain text
-V --ext-str [=] Provide 'external' variable as string. 'If is
omitted, get from environment var
-V --ext-code [=] Provide 'external' variable as Jsonnet code. If
is omitted, get from environment var
-V --tla-code [=] Provide top-level arguments as Jsonnet code. 'If
is omitted, get from environment var
-c --create-output-dirs Automatically creates all parent directories for files
--debug-importer Print some additional debugging information about the importer
-e --exec Evaluate the given string as Jsonnet rather than treating it as a
file name
--ext-code-file = Provide 'external' variable as Jsonnet code from the
file
--ext-str-file = Provide 'external' variable as string from the file
--fatal-warnings Fail if any warnings were emitted
file The jsonnet file you wish to evaluate
-m --multi Write multiple files to the directory, list files on stdout
-n --indent How much to indent your output JSON
-o --output-file Write to the output file rather than stdout
-p --preserve-order Preserves order of keys in the resulting JSON
--reverse-jpaths-priority If set, reverses the import order of specified jpaths (so that the
rightmost wins)
--strict Enforce some additional syntax limitations
--throw-error-for-invalid-sets Throw an error if a set operation is used on a non-set
--tla-code-file = Provide top-level arguments variable as Jsonnet code
from the file
--tla-str-file = Provide top-level arguments variable as string from
the file
-y --yaml-stream Write output as a YAML stream of JSON documents
--no-trailing-newline Do not add a trailing newline to the output
--yaml-debug Generate source line comments in the output YAML doc to make it
easier to figure out where values come from.
--yaml-out Write output as a YAML document

$ ./sjsonnet.jar foo.jsonnet

Usage as a library

Sjsonnet can be used from Java and Scala:

com.databricks
sjsonnet_3
${sjsonnet.version}

// Java
sjsonnet.SjsonnetMain.main0(
new String[]{"foo.jsonnet"},
new DefaultParseCache,
System.in,
System.out,
System.err,
os.package$.MODULE$.pwd(),
scala.None$.empty()
);

// Scala
sjsonnet.SjsonnetMain.main0(
Array("foo.jsonnet"),
new DefaultParseCache,
System.in,
System.out,
System.err,
os.pwd, // working directory
None
);

Or from Javascript:

Since 0.5.3, the output is a CommonJS module.

const { SjsonnetMain } = require("./sjsonnet.js");
// or
import { SjsonnetMain } from "./sjsonnet";

console.log(
SjsonnetMain.interpret("local f = function(x) x * x; f(11)", {}, {}, "", (wd, imported) => null)
);
// => 121

console.log(
SjsonnetMain.interpret(
"local f = import 'foo'; f + 'bar'", // code
{}, // extVars
{}, // tlaVars
"", // initial working directory

// resolver callback: receives a base directory and the imported path string,
// returns the resolved path
(wd, imported) => wd + "/" + imported,
// loader callback: receives the full path and returns the file contents
(path, binary) => "local bar = 123; bar + bar"
)
);
// => '246bar'

Note that since Javascript does not necessarily have access to the filesystem, you have to provide an explicit import callback that you can use to resolve imports yourself (whether through Node's fs module, or by emulating a filesystem in-memory)

Async imports

SjsonnetMain.interpret is synchronous, which is awkward in browsers where files come from fetch or FileReader. Use SjsonnetMain.interpretAsync instead: the loader returns a Promise and the call returns a Promise of the result. Imports are statically discovered from each parsed file's AST, loaded concurrently, then evaluated synchronously against the populated cache.

const result = await SjsonnetMain.interpretAsync(
"local lib = import 'lib.libsonnet'; lib.greet('world')",
{}, // extVars
{}, // tlaVars
"", // initial working directory
(wd, imported) => imported, // resolver, same shape as `interpret`
// loader: returns a Promise of the file contents (string for `import` /
// `importstr`, or bytes for `importbin`)
async (path, binary) => {
const response = await fetch("/files/" + path);
return binary ? new Uint8Array(await response.arrayBuffer()) : await response.text();
}
);

Running deeply recursive Jsonnet programs

The depth of recursion is limited by running environment stack size. You can run Sjsonnet with increased stack size as follows:

# JVM
java -Xss100m -cp sjsonnet.jar sjsonnet.SjsonnetMain foo.jsonnet

# Scala Native
SCALANATIVE_THREAD_STACK_SIZE=100m ./sjsonnet foo.jsonnet

# ScalaJS (Node)
node --stack-size=100m

Architecture

Sjsonnet is implemented as an optimizing interpreter. There are roughly 4 phases:

• sjsonnet.Parser: parses an input String into a sjsonnet.Expr, which is a

Syntax Tree representing the Jsonnet document syntax, using the Fastparse parsing library

• sjsonnet.StaticOptimizer is a single AST transform that performs static

checking, essential rewriting (e.g. assigning indices in the symbol table for variables) and optimizations. The result is another sjsonnet.Expr per input file that can be stored in the parse cache and reused.

• sjsonnet.Evaluator: recurses over the sjsonnet.Expr produced by the

optimizer and converts it into a sjsonnet.Val, a data structure representing the Jsonnet runtime values (basically lazy JSON which can contain function values).

• sjsonnet.Materializer: recurses over the sjsonnet.Val and converts it into

an output ujson.Expr: a non-lazy…