CLI
Open core · self-host + all Splyntra Cloud plans
The splyntra command-line tool ships with both SDKs — it is installed alongside
pip install splyntra and npm install @splyntra/sdk. Its main job is evaluation: push
datasets and gate CI on regressions without writing code, so the same
evaluation flow available in the SDK works from a shell or a
CI pipeline.
Configuration
The CLI reads two environment variables:
| Variable | Purpose |
|---|---|
SPLYNTRA_API_KEY | Your Splyntra API key, sent as a Bearer token |
SPLYNTRA_EVAL_ENDPOINT | The evaluation service endpoint |
Set both before running any command:
export SPLYNTRA_API_KEY="..."
export SPLYNTRA_EVAL_ENDPOINT="https://ingest.splyntra.com" # or your self-hosted endpoint
splyntra eval push
Upload a dataset of evaluation items from a JSONL file. Each line is an item with
input, expected_output, and optionally context (which powers groundedness
scorers).
splyntra eval push --name support-qa --file dataset.jsonl
| Flag | Description |
|---|---|
--name | Dataset name |
--file | Path to the dataset JSONL file |
splyntra eval run --gate
Run scorers against your caller-produced results and, with --gate, exit non-zero on a
regression versus the dataset baseline — turning the command into a CI gate.
splyntra eval run --dataset <id> --file results.jsonl --scorers exact_match,groundedness --gate
| Flag | Description |
|---|---|
--dataset | The dataset ID to score against |
--file | Path to the results JSONL file (input, actual per line) |
--scorers | Comma-separated scorer names, e.g. exact_match,groundedness |
--gate | Exit non-zero on a regression versus the baseline |
Because --gate sets the process exit code, dropping the command into a CI step fails
the build automatically when quality regresses. See
CI regression gates for a full walkthrough and
Scorers for the available scorer names.
Next steps
- Evaluation — datasets, runs, and the leaderboard.
- CI regression gates — wiring the gate into CI.
- Gate CI on evals — an end-to-end guide.