Scorers¶
All scorers implement cbtone.scorers.Scorer and return ToneScore (see
Concepts). Build one with cbtone.scorers.get_scorer(name, **kw); heavy
dependencies are imported lazily.
--scorer |
Family | Needs | Languages | Notes |
|---|---|---|---|---|
dict |
Dictionary | core install | en | Phrase lexicon hawkdove_v1.csv shipped in the package, with a crude stemmer and negation handling. Written for cbtone following published construction principles; not a replication of any published list. Optional Loughran-McDonald features are downloaded at run time, never shipped. |
fomc-roberta |
Finance-tuned transformer | [transformers]; Hugging Face access to the gated model |
en | gtfintechlab/FOMC-RoBERTa, revision pinned; CC-BY-NC-4.0, so non-commercial. Training data came from FOMC text, which matters for transfer to other banks. The paper uses a replication (fomc-roberta-repl) because access to the gated model was never granted: rebuild it with make fomc-roberta, or download it from thanatchon36/cbtone-fomc-roberta-repl (CC-BY-NC-4.0), then load it with --scorer ft --model-dir <dir>. |
llm |
Local LLM | a running Ollama and the model pulled | en, id, ms, th, zh | Default qwen3:4b, temperature 0, five prompt variants P1 to P5, optional --masked input and --evidence quotes. Calls are cached by content hash and resumable; the model tag and digest are recorded. |
ft |
Fine-tuned small model | [finetune] (training) or [transformers] (inference); --model-dir |
the languages it was trained on | XLM-RoBERTa-base fine-tuned on LLM silver labels. It refuses training rows that come from non-commercial evaluation-only data. |
The [api] extra is reserved for an optional hosted-API scorer; no such scorer is shipped yet, and none is ever required to reproduce results.
Contamination¶
A language model may have seen central-bank texts and the outcomes that followed. For that reason every LLM result is reported with:
- a masked variant (
--masked): bank names, dates, numeric rate levels, currencies and country names replaced by placeholders; - a post-cutoff split: documents dated after the model's documented training cutoff (or its public release date when none is documented).
A claim of predictive signal is made only where it survives both.
Choosing a scorer¶
There is no default winner: the pre-registered questions compare them. Read RESEARCH_QUESTIONS.md for what is tested, and DECISIONS.md for the model and prompt choices.
To write your own, see CONTRIBUTING.md.