Harbinger 24B

Overview

Harbinger 24B is a large language model developed by Latitude Games, optimized for immersive, decision-driven text adventures and role-playing scenarios. Built upon the Mistral Small 3.1 Instruct architecture, Harbinger 24B is distinguished by its narrative coherence, strong adherence to instructions, and the ability to maintain engaging storytelling across extended outputs. Its development process emphasizes reducing narrative clichés and the artifacts commonly exhibited by generative models, providing a more authentic and consequence-rich experience for interactive fiction and AI-driven roleplay.

Model Architecture and Technical Characteristics

Harbinger 24B is finetuned on top of the Mistral Small 3.1 Instruct base, which itself derives from the Mistral Small 3.1 Base. The resulting Harbinger 24B model comprises 23.6 billion parameters and utilizes the BF16 tensor type for efficient computation and inference. ChatML formatting is employed throughout all training and interaction stages, ensuring robust contextual handling and system-user-assistant role separation. The architecture is designed to favor second-person, present-tense narration, supporting the model's specialized role in text-based adventure and role-playing contexts.

Training Methodology and Direct Preference Optimization

The training process for Harbinger 24B integrates extensive supervised fine-tuning (SFT) alongside advanced alignment strategies. Initially, the model is trained using multi-turn datasets curated from a variety of sources, with a focus on "Wayfarer-style" adventures and diverse roleplay scenarios. Datasets are specifically rewritten and balanced to remove persistent generative clichés and maximize narrative consistency.

The second major phase implements Direct Preference Optimization (DPO), an optimization protocol that aligns the model's generations with user and player preferences. Preference data is collected from implicit player feedback within interactive platforms and is used to fine-tune the reward model, which in turn further shapes Harbinger 24B toward outputs that minimize repetitive or formulaic content. Notably, DPO is applied using datasets such as gutenberg-dpo-v0.1, on which human-authored chapters are paired with less favorable AI outputs, fostering a reduction in clichés and improved narrative variance.

The DPO process is inspired by methodologies from RLHF (reinforcement learning from human feedback), and is similarly applied in related models like Muse-12B, further supporting the advancement of narrative quality in alignment with user expectations.

Narrative Focus and Applications

Harbinger 24B is particularly optimized for text-based gaming experiences in which player choices lead to significant and consequential outcomes. The model excels in maintaining narrative coherence over long sessions, crafting engaging, vivid scenes primarily in the second-person, present-tense voice—a style ideally suited for interactive fiction and role-playing game masters. By virtue of its training, Harbinger 24B produces stories with reduced reliance on formulaic storytelling and a higher degree of narrative polish, supporting games and applications where the illusion of free will and meaningful repercussions is critical. The model's primary applications include immersive text adventures and general role-playing scenarios where the "unforgiving essence" of decisions is a central feature.

Data Challenges and Sentiment Analysis

A known challenge in language model training involves the tendency of synthetic data generators to produce outputs with a pronounced positivity bias and an increased frequency of narrative clichés. Analysis of synthetic data generated by models such as GPT-4.1 and Claude 3.7 Sonnet reveals this skew in sentiment scores, which can ultimately affect downstream storytelling quality. To counteract this, Harbinger 24B incorporates both diverse data sourcing and targeted optimization strategies.

Techniques such as careful data balancing, system-level prompts, and DPO-based fine-tuning are used to reduce such biases, decrease the presence of clichés, and enhance the model's utility in delivering authentic, consequence-driven narratives. Further details on these methodologies are outlined in Latitude's blog post on reward models and synthetic data generation.

Usage and Limitations

Harbinger 24B is intended to be engaged primarily in settings that leverage its narrative strengths, such as interactive storytelling games and AI-powered roleplay. It is constructed to respond to prompts in a specific format, ideally within ChatML dialogue roles, and is especially effective when guided by initial system prompts that establish narrative style and voice. Notably, the model is best utilized for second-person, present-tense storytelling and may offer less optimal results when deviations from this style are attempted.

Some limitations are inherent to the focused scope of its training. General-purpose deployments or applications outside narrative domains may not fully capitalize on Harbinger 24B's optimizations. The use of predominantly synthetic training data and the persistent challenge of sentiment bias in such data streams also introduce nuances to its output consistency. License information for Harbinger 24B, as of the latest public releases, has not been explicitly stated.

Related Models and Ecosystem

Harbinger 24B is part of a broader ecosystem of narrative-optimized language models developed by Latitude Games, including the Wayfarer model line and Muse-12B, which similarly employ preference optimization and alignment techniques. These models provide a comparative framework for evaluating Harbinger 24B's performance and application in the context of text-based roleplay.

External Resources

• Latitude Games on Hugging Face – Model repository for Harbinger 24B, Muse-12B, and the Wayfarer line.
• Quantized GGUF weights for Harbinger-24B – Direct access to quantized model weights in GGUF format.
• Latitude.io Blog: Synthetic Data, Preference Optimization, and Reward Models – Technical description of Harbinger 24B's training methodology.
• Wayfarer-12B on Hugging Face – Related model for passage-style adventures.
• Gutenberg-dpo-v0.1 Dataset – Dataset utilized for optimizing narrative quality and reducing clichés.
• Phi-3 Technical Report (arXiv) – Reference on synthetic data in language model training.
• Training a Helpful and Harmless Assistant with RLHF (arXiv) – Details on methods for alignment and feedback.
• Direct Preference Optimization (NeurIPS 2023) – Foundational paper on DPO technique.
• Safetensors Documentation – Technical explanation of safetensors format.