Pygmalion 2 13B

Overview

Pygmalion-2 13B is an instruction-tuned generative language model tailored for conversational and fictional text generation. Developed by PygmalionAI and previously referred to as Metharme, the model builds upon Meta AI's Llama-2 13B architecture and has been refined to optimize interactions involving storytelling, roleplay, and nuanced dialogue. Public documentation and release notes reveal that Pygmalion-2 13B is primarily intended for entertainment and creative writing applications, supported by a mix of proprietary and community-driven training datasets.

Model Architecture and Technical Foundations

At its core, Pygmalion-2 13B leverages the Llama-2 13B architecture by Meta AI, benefiting from the transformer-based design and 13 billion parameters configured in a BF16 tensor format. The model keeps the underlying architectural specifications consistent with Llama-2 but undergoes further supervised fine-tuning to adapt its behavior for structured instruction compliance, conversational ease, and fictional engagement. The development and training of Pygmalion-2 13B were sponsored by SpicyChat, with model weights and training artifacts distributed under the Llama-2 license terms, permitting both research and broader commercial or non-commercial usage.

Training Data and Methodology

The model’s conversational and creative strengths derive from its training corpus, which is a composite of conventional instruction data, roleplay interactions, fictional narratives, and dialogues containing synthetically generated instructions. Notably, Pygmalion-2 13B incorporates the proprietary PIPPA dataset curated by PygmalionAI, in addition to various public instruction datasets and roleplaying forum data. Supervised fine-tuning techniques were employed to instill the model with a strong aptitude for following prompts and imitating role-specific dialogue, enhancing its ability to produce contextually appropriate and imaginative responses in diverse creative writing scenarios.

Prompting Format and Usage

Effective interaction with Pygmalion-2 13B is achieved through a structured prompting scheme that uses explicit tokens to demarcate roles and conversation turns. According to official documentation on model interaction, prompts are constructed using <|system|>, <|user|>, and <|model|> tokens. The <|system|> token serves to inject narrative context, behavioral instructions, or constraints on response characteristics, such as specifying "Enter RP mode. Pretend to be {{char}} whose persona follows: {{persona}}. You shall reply to the user while staying in character, and generate long responses." The <|user|> and <|model|> tokens, respectively, designate the user's input and the model's subsequent response. This scheme supports multi-turn dialogues through token chaining, enabling persistent role identity and scenario consistency across conversation sessions.

Applications and Intended Use

Pygmalion-2 13B was conceived with a primary focus on entertainment-oriented creative writing, including character-driven roleplay, collaborative storytelling, and dialogue simulation. Its conversational tuning and narrative orientation make it well-suited for applications in text-based fiction, gamified chat interactions, and as an auxiliary tool for authors in developing character dialogue or imaginative scenarios. The model’s design intentionally emphasizes adaptability to natural language instructions, facilitating intuitive prompt guidance across various creative contexts.

Limitations and Ethical Considerations

The training and fine-tuning pipeline for Pygmalion-2 13B did not incorporate specialized safety alignment or adversarial filtering for harmful content. Both the original Llama-2 base model and the subsequent fine-tuned variant have been exposed to training data containing language and themes that may be inappropriate or offensive in general contexts. As a result, the model is prone to generating outputs that are factually unreliable, potentially misleading, or socially undesirable from a non-fiction prompt. The developers explicitly restrict its application scope to fictional entertainment and do not recommend its deployment in environments or tasks demanding factual precision or safety-sensitive dialogue handling. Ongoing use beyond creative entertainment may require additional risk and output management controls.

Model Family and Licensing

Pygmalion-2 13B is part of a broader family of models curated by PygmalionAI, which includes additional fine-tuned, merged, and quantized variants designed for specific performance or accessibility needs. Consistent with the open-model ethos, Pygmalion-2 13B is available under terms compatible with the Llama-2 license, allowing both research experimentation and wider deployment across sectors, provided license compliance.

External Resources

• Pygmalion-2 13B Model Card and Weights
• Meta Llama-2 13B Base Model
• PIPPA Fine-Tuning Dataset
• SpicyChat (Model Training Sponsor)
• Axolotl (Training Framework)