Wayfarer Large 70B Llama 3.3

Overview

Wayfarer Large 70B Llama 3.3 is a generative AI language model created by Latitude Games and designed specifically for challenging adventure role-play scenarios. Unlike many contemporary models that display an inherent positivity bias, Wayfarer Large 70B deliberately emphasizes conflict, tension, and the possibility of failure within narrative outputs—key elements valued in compelling storytelling and games. The model builds upon the robust Llama 3.3 70B Instruct architecture, enhancing its capabilities through specialized training on datasets curated for interactive adventure narratives.

Model Architecture and Training

Wayfarer Large 70B Llama 3.3 is a large language model with 70.6 billion parameters, grounded in the Llama 3.3 70B Instruct base model. The model's training pipeline employed a blend of unique data sources chosen to foster its narrative complexity, a focus on high-stakes storytelling, and proficiency in role-play dialogue. Notably, the model was trained to excel in second-person, present-tense storytelling, aimed at immersing users in interactive, game-like narration.

The parameterization and underlying structure are inherited from the Llama 3.3 architecture, incorporating bfloat16 (BF16) tensor types and further optimized by quantization techniques to facilitate efficient local inference.

Training Data and Methodology

The curation of the training dataset for Wayfarer Large 70B was tailored to foster narrative stakes and role-play depth. Approximately one third of the training data consisted of simulated playthroughs of AI Dungeon adventure scenarios, in which distinct user archetypes engaged with complex settings mediated by artificial narrators and players. These simulations typically extended up to 8,000 tokens or until in-game character death. Improvements with each iteration of the data focused on reducing repetition and improving narrative consistency across stories.

A further third of the data targeted role-play interactions, supporting the model's facility at rendering dynamic character exchanges and unpredictable dialogue essential to interactive storytelling environments. The remainder of the dataset consisted of a SlimOrca Sonnet instruct subset, which served to clarify distinctions between instructive language and fictional role-play, and to accentuate the model's intended inclination toward negative, high-stakes outcomes. This data strategy was critical for counteracting the pervasive "plot armor" and excessive optimism often found in language model storytelling.

Features, Applications, and Usage

The principal use case for Wayfarer Large 70B Llama 3.3 lies in the domain of adventure role-play, particularly where users seek complex, unforgiving, and sometimes perilous experiences. Typical users interact with the model through systems that invoke second-person narration ("you are") to create immersion. Its outputs commonly include elements such as tension, setbacks, and failure, making it well-suited for narrative-driven games and interactive fiction where the stakes of choice are high.

The model follows the native Llama 3 prompt format, and recommended inference settings include a temperature of 0.8, a repetition penalty of 1.05, and a minimum probability (min_p) of 0.05. An example prompt illustrates the model's approach:

<|start_header_id|>system<|end_header_id|>
You're a masterful storyteller and gamemaster. Write in second person present tense (You are), crafting vivid, engaging narratives with authority and confidence.<|eot_id|>
<|start_header_id|>user<|end_header_id|>
> You peer into the darkness.<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>
You have been eaten by a grue.
GAME OVER<|eot_id|>

This prompt structure demonstrates the model's direct, high-stakes storytelling style.

Performance, Limitations, and Comparisons

Quantitative benchmark scores for Wayfarer Large 70B Llama 3.3 have not been formally published. User feedback within platforms such as AI Dungeon has indicated a favorable reception among those seeking more challenging narrative experiences, though this assessment remains anecdotal. One limitation is that the model was trained exclusively in second-person, present-tense narrative, which may impact its ability to generalize to other styles of output. Nevertheless, the depth and scale of the model's architecture, combined with the underlying instruct layer of Llama 3.3 70B, may partially mitigate this constraint in practice.

The Wayfarer model family also includes the Wayfarer-12B, an earlier open-source release with a reduced parameter count. This smaller version allows for broad experimentation and comparison, whereas the 70B variant benefits from greater narrative depth and contextual awareness due to its scale.

Development Timeline and Open-Source Context

The development of the Wayfarer models progressed from the release of Wayfarer-12B in January, which was made openly available to the public. Due to demand for increased narrative sophistication and challenge, the team at Latitude Games subsequently scaled their efforts to produce the Wayfarer Large 70B Llama 3.3, which was likewise open-sourced for the community. The open release encourages further research, experimentation, and the development of derivative models, as evidenced by the availability of quantized model weights and listings of finetuned and merged variants across community repositories.

Licensing and Availability

While the explicit license of Wayfarer Large 70B Llama 3.3 is not detailed in its primary repository, the model is distributed as open source, allowing unrestricted download and use for research, development, or local deployment. Additional documentation and model metadata are available through Hugging Face model cards and official project resources.

Further Resources

• Wayfarer Large 70B Llama 3.3 Model Card
• Wayfarer-12B
• Wayfarer Large 70B Quantized Weights
• Llama 3.3 70B Instruct Base Model
• SafeTensors Documentation
• Text Generation Task on Hugging Face
• Hugging Face Model Cards Documentation
• Models Derived from Wayfarer Large 70B
• Models Merged with Wayfarer Large 70B
• Models Quantized from Wayfarer Large 70B
• AI Dungeon Platform