Example 12 – continuously trained Czech chatbot

This example turns the one-shot language experiment from Example 10 into a versioned continual-learning process. A new community batch can fine-tune the existing model without changing its architecture or restarting from scratch.

See latest version in Github: https://github.com/havelm3/cognia-chatbot

Reused Example 10 components

• Czech syllable tokenization with a safe character fallback;
• the 96-cell word-memory projection and delayed/recurrent connections;
• intent control, persistent dialogue memory, and previous-topic references;
• grammar-guided decoding, response obligations, and verb valency;
• softmax readout, eligibility traces, normalized covariance, and pairwise credit;
• the Release trainer, GPU-native indexed one-hot input, and EligibilityEvery=4.

The lexicon now contains 764 entries instead of the original 182. It covers people, technology, software, data, knowledge, places, and common objects in nominative, accusative, locative, and genitive forms, plus 36 verb paradigms, adverbs, prepositions, and conjunctions. The valency column connects verbs to their required accusative object or about + locative frame. VerbValencyController no longer has a hand-written list of a few objects; it loads all suitable forms directly from the lexicon. The maintained source is grammar/lexicon.base.tsv, while grammar/generate-lexicon.php deterministically creates grammar/lexicon.tsv.

Grammar remains a decoder safety rail rather than a source of meaning. As the neural model improves, its influence can be reduced through a wider candidate set or softer penalties, but regression tests must prove that intelligibility does not degrade. Removing grammar completely is not an automatic objective.

Why the vocabulary stays fixed

A checkpoint has an exact number of sensory and output neurons. Changing the vocabulary changes those dimensions and makes the old checkpoint incompatible. New words are therefore composed from known syllables, with unknown syllables falling back losslessly to characters. Vocabulary changes require a separate, explicit major model migration.

Continual-learning cycle

community JSONL
  → license, consent, PII checks, and deduplication
  → syllable delta dataset
  → new content plus replay of prior capabilities
  → training from the latest checkpoint
  → candidate checkpoint
  → grammar + memory + FineWeb regression tests
  → releases/current.bin only after passing

Replay prevents catastrophic forgetting. The default mixture gives repeated new content a 25% share and includes up to 2,000 older sequences. Every checkpoint records its parent and a manifest, creating an auditable model lineage. Failed candidates remain under output/ and are never deployed.

Community contributions

community/SCHEMA.md defines the JSONL format. The validator accepts Czech data with explicit consent under CC0 1.0 or CC BY 4.0. It rejects empty or oversized dialogues, duplicates, and basic private-data or secret patterns. A public site still needs human moderation, contributor reputation, audit logs, poisoning defenses, and a data-withdrawal process.

Running the sample batch

Build the Release cognia_train target and retain the Example 10 replay dataset and checkpoint. Create a candidate without the long evaluation suite:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\continuous-train.ps1 `
  -Batch .\community\incoming\example-batch.jsonl `
  -ReplaySequences 20 -NewShare 0.25 -Epochs 1

Production training with regression gating and promotion:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\continuous-train.ps1 `
  -Batch .\community\incoming\batch-2026-08.jsonl -Evaluate -Promote

Without -Promote, the pipeline only creates a candidate. Promotion requires -Evaluate. The gate permits at most a 2% relative cross-entropy increase and a one-percentage-point drop in top-10 or reply accuracy on any protected suite.

The chat launcher uses releases/current.bin when available and otherwise falls back to the dialogue checkpoint from Example 10:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\run-planner-chat.ps1

What the community service still needs

This example implements the model and data pipeline, not a complete public portal. The web service must isolate uploads from accepted data and add malware scanning, rate limits, moderator roles, signed release manifests, and immutable audit records. Safety and factual evaluation suites should be added before a checkpoint is published; language loss alone does not guarantee truthful or safe answers.