Nous Hermes Llama 2 70B

Overview

"Nous Hermes Llama 2 70B" is a large language model developed by Nous Research, incorporating contributions in fine-tuning and dataset curation from the developers Teknium and Emozilla, with compute sponsorship from PygmalionAI. This model is a fine-tuned variant of the Llama 2 70B architecture, building upon the methodology and dataset used in the earlier "Hermes on Llama-1" release to ensure continuity of style and performance. The emphasis in this iteration is on generating longer, more coherent responses, reducing the likelihood of hallucinations, and eliminating certain types of filtering present in contemporary language models.

Model Architecture and Fine-Tuning Approach

The foundation of "Nous Hermes Llama 2 70B" is the Llama 2 70B architecture, known for its scalability in large language model applications. The fine-tuning process for Hermes involved training on sequences of up to 4096 tokens, leveraging high-memory GPUs to manage computational demand.

During training, the team employed bitsandbytes quantization with settings tailored for efficient memory usage. Specifically, the configuration included 4-bit quantization using the nf4 algorithm, with double quantization enabled and computation in the bfloat16 format. The model was fine-tuned using PEFT version 0.5.0.dev0 to support Parameter-Efficient Fine-Tuning. This technical setup was chosen to balance swift inference and the retention of generative behavior.

Training Data and Curation Techniques

The dataset underpinning the fine-tuning of "Nous Hermes Llama 2 70B" is composed mainly of synthetic outputs generated by GPT-4, with an emphasis on curating for both quality and task diversity. The curation process intended to avoid low-quality, repetitive, or filtered samples, leading to a dataset capable of producing knowledge-rich and stylistically consistent outputs.

Key dataset sources integrated into the training regime included the GPTeacher set from Teknium, WizardLM, the Nous Research Instruct Dataset, GPT4-LLM, and Microsoft’s Unnatural Instructions, the Airoboros dataset, Camel-AI’s domain expert datasets, and CodeAlpaca. These sources contribute to the diversity of knowledge, styles, and instruction-following capability embedded in the final model.

Evaluation and Benchmark Performance

"Nous Hermes Llama 2 70B" has undergone evaluation across several prominent benchmark suites to assess its performance in knowledge recall, reasoning, task completion, and general instruction-following ability. Notable among these are the GPT4All benchmarks, the BigBench suite, and AGIEval.

On the GPT4All suite, the model demonstrated accuracy scores such as 0.57 on ARC Challenge, 0.84 on ARC Easy, 0.84 on BoolQ, 0.65 on HellaSwag, 0.38 on OpenBookQA, and 0.77 on Winogrande. The results reflect consistent performance across a diverse range of reasoning and comprehension tasks.

In the BigBench benchmark, Hermes exhibited strengths in reasoning about sports (0.74), date understanding (0.73), and causal judgment (0.66), while presenting results on complex logical deduction and geometric reasoning tasks. AGIEval evaluations show competence in SAT English (0.86) and SAT English without passage (0.53), further underlining the model’s proficiency in language understanding and test-style assessments.

Applications and Use Cases

The "Nous Hermes Llama 2 70B" model is designed for general-purpose natural language processing tasks, making it suitable for use in conversational AI, knowledge generation, instruction following, and creative text synthesis. Applications include chatbot deployments, virtual assistants, automated content generation, and specialized implementations like roleplaying bots, which benefit from the model’s response length capabilities and stylistic versatility.

For operation, the model expects prompts in the Alpaca formatting convention. For example, a standard interaction starts with "### Instruction:" followed by the query, optionally an "### Input:" section for additional context, and "### Response:" for the generated output. This prompt structure helps guide the model toward structured, coherent replies.

Model Limitations and Future Directions

While Hermes Llama 2 70B focuses on reducing hallucination rates and excludes some forms of synthetic data filtering, the model retains limitations typical of transformer-based large language models. These include potential knowledge gaps due to the predominance of synthetic data, variance in reasoning stability for multi-step or ambiguous queries, and difficulty with highly specialized domains absent from the training data.

Future development directions, as outlined by Nous Research, include the continued refinement of data sources and the application of advanced filtering techniques to further enhance dataset relevance and response reliability.

Release and Model Family

The update and release of this Hermes model as part of the Nous Research Flagship LLM Series was finalized in August 2024. By leveraging the same dataset used for the original "Hermes on Llama-1," continuity in behavior and application scope across versions is maintained.

Helpful Links

• Hugging Face: Nous Hermes Llama 2 70B Model Page
• Nous Research: Collection of Models
• GPTeacher Project
• Wizard LM on GitHub
• Nous Research Instruct Dataset
• GPT4-LLM Dataset
• Unnatural Instructions Dataset
• Airoboros Dataset
• Camel-AI Repository
• CodeAlpaca Dataset
• BigBench Benchmark
• AGIEval Benchmark
• Llama 2 70B Architecture