Llama 3.1 8B Stheno v3.4

Overview

Llama 3.1 8B Stheno v3.4 is a generative AI language model developed through a multi-stage fine-tuning process on the Llama 3.1 architecture. Designed for improved multi-turn dialogue, creative writing, and roleplaying, this model incorporates custom dataset curation and implemented instruction tuning techniques. Its development was facilitated by computational resources provided by Backyard.ai, emphasizing iterative dataset refinement for performance and alignment. Details on the model and its public documentation can be found at the Hugging Face model repository.

Model Architecture and Technical Features

Llama 3.1 8B Stheno v3.4 is constructed upon the Llama 3.1 architecture and contains approximately 8.03 billion parameters, employing the Brain Floating Point 16 (BF16) tensor format for efficiency and performance. This model targets several key capabilities including maintaining coherency in multi-turn conversations, adherence to system prompts, enhanced reasoning, and improved spatial awareness. According to statements from the developer, direct tuning on the Llama 3.1 base model was observed not to perform as strongly as with other foundational models such as Nemo, particularly noting the Llama 3.1 instruct tuning as "overbaked" due to post-supervised preference optimization, which impacted downstream model behavior. These technical observations are documented in the model’s release notes.

Training Strategy and Dataset Composition

The fine-tuning of the Stheno v3.4 model utilized a two-stage process. The first stage involved instruction-based, multi-turn dialogue fine-tuning using carefully curated conversational datasets. This was followed by a second stage focusing on creative writing and roleplay, leveraging both original and synthetic data sources. The dataset strategy for Stheno v3.4 included substantial modifications compared to prior Stheno versions. Notably, single-turn instruction data was replaced with richer prompt-response pairs, including those generated by Claude 3.5 Sonnet and Claude 3 Opus, with an emphasis on diversity and adherence to high-quality standards.

To further enhance dialogue depth and contextual consistency, multi-turn conversational datasets were introduced, distinct from equivalents used in models like Magnum. Roleplay capability was strengthened through the integration of a 55% increase in roleplaying examples derived from the Sonnet3.5-Charcard-Roleplay set, which underwent additional filtering and cleaning procedures. Creative writing coverage was extended by 40% with new examples. Additional subsets were introduced to target system prompt adherence, broad reasoning skills, and spatial reasoning, with all entries subjected to comprehensive filtering for error reduction and dataset quality. The final training mixture included both human- and Claude-generated content, striking a balance between authenticity and scalability.

The dataset design and turn structure are informed by the ShareGPT format, where a single "turn" refers to a paired human-model interaction. In this system, "4 turns" would include one system-level prompt followed by four exchanges each from both the human and the model, as visualized in the provided dataset charts.

Prompting, Usage, and Performance Insights

For optimal prompting, the model is designed to work with the Llama 3 Instruct formatting, and the Euryale 2.1 Preset has been identified as effective for engaging diverse model behaviors. The developer recommends using a temperature of 1.4 and a minimum probability (min_p) of 0.2, but encourages users to adjust these settings according to specific task requirements, as the model exhibits distinct characteristics compared to previous Stheno iterations. Over its first month of availability, Llama 3.1 8B Stheno v3.4 was downloaded 214 times, as reflected in publicly available release statistics.

Technical visualizations accompanying the release include bar graphs of turn distributions and histograms of token lengths. These provide further context for understanding the conversational depth and complexity emphasized during the model's fine-tuning, as demonstrated above.

Limitations and Observations

The developer's documentation notes shortcomings encountered with the Llama 3.1 instruct base, describing its output as less consistent than expected, potentially attributable to post-supervised fine-tuning preference optimization (AHP) applied after the primary supervised fine-tuning (SFT) phase. This may affect the model’s responsiveness in instruction-following and conversational flows compared to alternative foundation models such as Nemo. The iterative process of dataset curation and model evaluation remains ongoing, with the possibility of future refinements, particularly around error filtering and dialogue balance.

Datasets and External Resources

A key dataset utilized in the fine-tuning process is the Gryphe/Sonnet3.5-Charcard-Roleplay collection, which contributed to the model’s roleplay training examples after being subjected to additional filtering and cleaning steps. Visual assets used in the project’s model card are sourced from Pixiv Artwork.

Further Reading and Helpful Links

• Llama 3.1 8B Stheno v3.4 model card and documentation
• Gryphe/Sonnet3.5-Charcard-Roleplay dataset
• Pixiv Artwork: Original Image Source