---
license: mit
language:
- en
- ja
base_model:
- nvidia/Llama-3.1-Nemotron-70B-Instruct
pipeline_tag: text-generation
extra_gated_fields:
  Affiliation: text
  Country: country
  I want to use this model for:
    type: select
    options:
    - Research
    - Commercial
    - label: Other
      value: other
  Job title:
    type: select
    options:
    - Student
    - Research graduate
    - AI researcher
    - AI developer/engineer
    - Cybersecurity researcher
    - Reporter
    - Other
  geo: ip_location
library_name: transformers
datasets:
- trendmicro-ailab/Primus-FineWeb
tags:
- cybersecurity
---

# Llama-Primus-Nemotron-70B-Base

<img src="https://i.imgur.com/yzitCm9.jpeg" alt="Llama-Primus-Nemorton" width="60%">

- [Introduction](#introduction)
- [Benchmark Result](#benchmark-result)
- [Training Datasets](#training-datasets)
- [Acknowledgments](#acknowledgments)

## Introduction

The **Llama-Primus-Nemotron** series builds upon `nvidia/Llama-3.1-Nemotron-70B-Instruct` through continued training. Following the same methodology as described in the [Primus paper](https://arxiv.org/abs/2502.11191), we first performed pre-training on large-scale cybersecurity corpora (over **10B** tokens) to obtain **Llama-Primus-Nemotron-Base**. We then conducted supervised-finetuning and applied [DELLA](https://arxiv.org/abs/2406.11617) to merge with the original Nemotron, resulting in **Llama-Primus-Nemotron-70B-Instruct**.

_Llama-Primus-Nemotron-Base_ achieves an **11.19%** improvement in aggregate scores across several public cybersecurity benchmarks.


## Benchmark Result

### Cybersecurity

| **Metric** (5-shot, **w/o** chat template)     | **Llama-3.1-Nemotron-70B-Instruct** | **Llama-Primus-Nemotron-70B-Base** |
|-------------------------------------------|-------------------------------------|----------------------------------------|
| **CTI-Bench (MCQ)**                       | 0.6900                              | 0.7148                                 |
| **CTI-Bench (CVE → CWE)**                 | 0.6590                              | 0.7410                                 |
| **CTI-Bench (CVSS, _lower is better_)**   | 1.1893                              | 1.0281                                 |
| **CTI-Bench (ATE)**                       | 0.3905                              | 0.4540                                 |
| **CyberMetric (500)**                     | 0.9380                              | 0.9280                                 |
| **SecEval**                               | 0.7177                              | 0.7208                                 |
| **CISSP (Exam Questions)**                | 0.8527                              | 0.8703                                 |
| **_Aggregate_**                           | 3.0586                              | 3.4008  **↑11.19%** 🔥                 |


CTI-Bench(CVSS) is scored using Mean Absolute Deviation (_lower is better_), CTI-ATE uses F1 score, and the others use accuracy. The aggregate score (_Agg._) is the sum of all benchmarks, with CTI-Bench(CVSS) negated.

References:

-  **CyberMetric**: [CyberMetric: A Benchmark Dataset based on Retrieval-Augmented...](https://arxiv.org/abs/2402.07688)
-  **CTI-Bench**: [CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence](https://arxiv.org/abs/2406.07599)
-  **SecEval**: [SecEval: A Comprehensive Benchmark for Evaluating Cybersecurity Knowledge of Foundation Models](https://xuanwuai.github.io/SecEval/)

## Training Datasets

#### Pre-training:

- **Primus-Seed-V2 (0.457B):** An enhanced version of [Primus-Seed](https://huggingface.co/datasets/trendmicro-ailab/Primus-Seed), enriched with blogs, news, books, websites, Wikipedia, MITRE and Trend Micro knowledge.  
- **Primus-FineWeb (2.57B):** Cybersecurity text filtered from FineWeb-edu-score-2. [Link](https://huggingface.co/datasets/trendmicro-ailab/Primus-FineWeb)  
- **Primus-Nemotron-CC (7.6B):** Cybersecurity text filtered from Nemotron-CC.

> **Note:** Datasets *Primus-Seed-V2* and *Primus-Nemotron-CC* are not yet open-sourced and are currently under discussion. Feel free to reach out if you're interested.

> **Disclaimer:** No Trend Micro customer information is included.

## About _Primus_

_Primus_ is Trend Micro's pioneering family of lightweight, state-of-the-art open cybersecurity language models and datasets. Developed through our cutting-edge research initiatives and advanced technology, these resources share the innovative foundation that powers our enterprise-class [Trend Cybertron](https://newsroom.trendmicro.com/2025-02-25-Trend-Micro-Puts-Industry-Ahead-of-Cyberattacks-with-Industrys-First-Proactive-Cybersecurity-AI) solution. As an industry leader in cybersecurity, Trend Micro is proud to contribute these powerful, efficiency-optimized models and datasets to the community, while maintaining the excellence and reliability that define our global security standards.

## Acknowledgments

We would like to thank **NVIDIA** for generously providing computing resources (**Taipei-1**), which enabled the training and development of this model.

## License

This model is based on the MIT license, but you must also comply with the Llama 3.1 Community License Agreement.