Slovary Pack Format v1

An open, self-contained format for dictionary content that the Slovary app can install. Anyone can build a compatible pack: this document is the complete specification, and the reference converter is published alongside it.

The app also imports the widely used StarDict format (.ifo/.idx/.dict or .dict.dz); StarDict files are converted to this format on import.

We do not host, distribute or curate third-party dictionary content. This specification exists so that users can bring their own.

1. Overview

A dictionary is delivered as one or two files:

The vector pack is optional. Without it the dictionary is fully searchable lexically (exact, prefix, substring, fuzzy, full-text); with it the dictionary also participates in meaning-based search.

Several dictionaries can travel together in a bundle — one file, one link, installed in a single step. See §6a.

2. Dictionary pack (.slovarypack)

A plain SQLite 3 database. No extensions, no triggers, no attached files.

2.1 pack_meta

CREATE TABLE pack_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);

2.2 entries

CREATE TABLE entries (
    local_id   INTEGER PRIMARY KEY,   -- 1..entries_count, dense, pack-local
    term_upper TEXT NOT NULL,         -- headword, upper-cased, no markup
    definition TEXT NOT NULL,         -- article body, restricted HTML (§4)
    lang       TEXT NOT NULL          -- same value as pack_meta.lang
);
CREATE INDEX idx_entries_term ON entries (term_upper);

local_id is local to the pack. It is never a global identifier. The app assigns its own table index when installing and addresses entries as (pack_id, local_id).

Why this matters: the app's bundled corpus historically used a composite global id (table_index << 32 | source_rowid). With user-supplied packs that scheme collides as soon as two independently built packs pick the same table index, and the failure is silent — wrong articles for right queries. Packs therefore never carry global ids.

term_upper must be upper-cased with toUpperCase() semantics for its language and must contain no HTML. Searching and sorting both use this column.

2.3 entries_fts

CREATE VIRTUAL TABLE entries_fts USING fts5(
    definition_plain,
    content='',
    contentless_delete=1,
    tokenize='unicode61 remove_diacritics 2',
    detail=none
);

Contentless FTS5 index over the markup-stripped definition text. Row i of the index must correspond to entries.local_id = i.

If the index is missing the app rebuilds it on import — that is slower but not an error.

FTS5 caveat inherited from the main corpus: the trigram tokenizer cannot match strings shorter than three characters. This format uses unicode61 instead, so short words work, but substring search is handled by the app's own cascade rather than by FTS.

3. Vector pack (.vectors.zip)

A ZIP archive with exactly three members. Vectors are INT8-quantised per-dimension.

3.1 vectors_meta.json

{
  "format_version": 1,
  "pack_id": "org.gnu.gcide",
  "entries_count": 189421,
  "count": 189421,
  "d": 384,
  "embedding_model": "intfloat/multilingual-e5-small",
  "model_revision": "2026-05-01",
  "quantization": "int8-per-dim-minmax",
  "query_prefix": "query: ",
  "passage_prefix": "passage: ",
  "vmin": [ /* d floats */ ],
  "vdiff": [ /* d floats */ ]
}

Dequantisation of code c (0…255) in dimension j:

value[j] = vmin[j] + (c + 0.5) * vdiff[j] / 255.0

3.2 vectors.sq8

Dense little-endian uint8 array of count × d bytes, row-major, ordered by ascending local_id (row i ⇒ local_id = i + 1). No header, no padding.

3.3 ids.bin

Little-endian int64 array of count entries: the local_id for each row, in the same order. Present so that a pack may ship vectors for a subset of entries in a later format version; in v1 it is the identity sequence and is verified as such.

3.4 What the app does with it

The query is encoded on-device with the declared model and scored against every vector by inner product (exhaustive scan, no ANN index). Top matches are resolved back to entries rows. Expect roughly 400 bytes per entry on disk.

4. Allowed markup in definition

The article body is rendered inside a WebView. Only this tag set survives import; everything else is stripped, including all attributes not listed.

Structural blocks: <b-q>, <tb-q>, <g-m>, <t-p>, <v-t>, <o-u>, <s-o>, <f-r>, <d-t>, <e-x>, <x-p>, <n-m>, <o-t>, <r-f>, <i-n>

Inline: <b>, <i>, <u>, <sup>, <sub>, <br>, <b-r>, <n-r>, <b-b>, <ab-r>, <c-r>, <s-m>, <k-f>, <t-r>, <ya-z>, <r-l>, <s-n>, <m-k>, <c-b>, <c-g>, <e-t>

Cross-reference: <r> — its text content is looked up as a term when tapped.

Image: <img src="..."> where src is a relative path inside the pack's optional image directory. Remote URLs are removed.

Removed unconditionally: <script>, <style>, <iframe>, <object>, <embed>, <link>, <meta>, event-handler attributes (onclick, onerror, …), javascript: and data: URLs, and anything else not on the list above.

This is a hard security boundary, not a style preference. The article WebView exposes a JavaScript bridge to the app; a script tag surviving import would reach it. Sanitisation happens on import, before the row is written, so the stored data is already safe and no renderer can be tricked later.

5. Validation on import

The app refuses a pack and reports the reason when:

1. format_version has an unknown major version.
2. pack_meta is missing a required key.
3. entries_count disagrees with COUNT(*) from entries.
4. local_id values are not a dense 1..entries_count sequence.
5. A vector pack's pack_id, entries_count, d, or embedding_model does not match its dictionary pack / the app's on-device encoder.
6. vectors.sq8 is not exactly count × d bytes.

Warnings that do not block installation: missing FTS index (rebuilt), missing optional metadata, unknown extra pack_meta keys.

The imported file is never opened as the app's working database. Rows are read out of it read-only, sanitised, and written into a fresh app-owned database; the source file is then discarded.

6. Building a pack

The reference converter builds a pack from StarDict or from a simple JSONL file, and a companion script produces the vector pack. Both are published with this specification.

Minimal JSONL input — one object per line:

{"term": "abacus", "definition": "A counting frame...", "lang": "en"}

# dictionary pack
python make_pack.py --in dict.jsonl --out org.example.mydict.slovarypack \
    --pack-id org.example.mydict --name "My Dictionary" --lang en \
    --license public-domain

# from StarDict instead
python make_pack.py --in mydict.ifo --out org.example.mydict.slovarypack ...

# optional vector pack (needs the E5 model; GPU recommended)
python make_vectors.py --pack org.example.mydict.slovarypack \
    --out org.example.mydict.vectors.zip

Host the resulting files anywhere. In the app: Settings → Dictionaries → Import, paste the URL or pick the file.

6a. Bundle: many dictionaries in one file

A bundle is a plain ZIP that carries whole dictionaries instead of one. It exists so a collection can be handed over as a single link: importing twenty dictionaries and their vectors by pasting forty URLs is not a workable instruction, for a user or for a reviewer.

Rules:

• The name carries the link between the two. A vector member is applied to

the dictionary whose pack_id matches the part of the name before .vectors.zip; a vector member with no matching pack is ignored. The contents are still validated as in §3 — a mismatched pack_id or entry count inside vectors_meta.json is refused even when the file name fits.

• Subdirectories are ignored; only the base name of each member is used.

A member whose name would escape the extraction directory is skipped.

• Order does not matter. Packs are installed in name order, and each one's

vectors immediately after it.

• Nothing else is read. Other members (README, licences) are left alone.
• A member that fails does not abort the rest. With dozens of dictionaries

in one file, losing all of them to one bad member would be the wrong trade; the app installs what it can and reports what it could not.

• Extension: .zip works. .slovarybundle is the suggested name when you want

the file to be self-describing — the app identifies the format by looking inside, never by the extension.

An importer distinguishes a bundle from a StarDict archive by looking for a member ending in .slovarypack; only the ZIP central directory is read, so this costs nothing on a large file.

The listing, and why it is the first member

A corpus bundle runs to hundreds of megabytes and a reader rarely wants all of it. bundle.json describes the contents so an importer can ask before downloading anything:

{
  "format": "slovary-bundle",
  "format_version": 2,
  "entries_count": 357644,
  "dictionaries": [
    {"pack_id": "org.gnu.gcide", "name": "GCIDE", "lang": "en",
     "entries_count": 124187, "size": 58720256, "vectors_size": 41943040,
     "pack_version": "2026-08-13", "license": "GPL-3.0-or-later"}
  ]
}

size and vectors_size are the member sizes in bytes (vectors_size absent or 0 means the bundle carries no vectors for that dictionary). Everything here is advertising: the app still validates each member on install, so a listing that lies costs the bundle its credibility, not the app its integrity.

Two fields decide what the importer offers before anything is fetched, and leaving them out makes a technically valid bundle behave worse than it should:

• pack_version is compared against the version already installed. Equal

means "you have this" — the row arrives marked and unticked. Different means an update, and arrives ticked. Omit it and every dictionary looks new, so reopening the same link offers to download the whole collection again. Use the same value as the pack's own pack_meta.pack_version.

• lang groups the rows and is what the app un-hides after installing:

a dictionary in a language the user had switched off would otherwise install correctly and then be invisible in search. Use the pack's pack_meta.lang.

ZIP64 is not read. A bundle over 4 GB, or with more than 65535 members, parks 0xFFFFFFFF in the fields that hold member offsets, and the importer refuses to guess: it falls back to downloading the whole file rather than fetching a range that would point at another dictionary's bytes. Keep a bundle under 4 GB if selective installation matters.

It is written first so that it lands in the opening kilobytes of the archive, and STORED like every other member. That combination is what lets an importer read it with a single ranged request (Range: bytes=0-32767) instead of downloading the file. A second request for the tail yields the ZIP central directory, which gives the offset and length of every member — so the dictionaries the user actually ticked can then be fetched one ranged request each. Members must therefore be STORED, not deflated: their byte range then IS the file.

A bundle without a listing (or one built by format_version 1, where dictionaries was a count) is still valid — an importer simply falls back to taking the whole thing.

Reference builder: scripts/db/pack_tools/make_bundle.py.

7. Version history