Merge pull request rllm-org#104 from kingroryg/main

SEvenLLM Benchmark Implementation | ASET - Arcadia Impact

Author: alexandraabbas

README.md

@@ -139,6 +139,16 @@ Inspect supports many model providers including OpenAI, Anthropic, Google, Mistr
    ```
    inspect eval inspect_evals/gdm_in_house_ctf
    ```
+  
+- ### [SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence](src/inspect_evals/sevenllm)
+  [SEvenLLM](https://arxiv.org/abs/2405.03446) is a benchmark to elicit, and improve cybersecurity incident analysis and response abilities in LLMs for Security Events.
+ <sub><sup>Contributed by: [@kingroryg](https://github.com/kingroryg)</sub></sup>
+   ```
+   inspect eval inspect_evals/sevenllm_mcq_zh
+   inspect eval inspect_evals/sevenllm_mcq_en
+   inspect eval inspect_evals/sevenllm_qa_zh
+   inspect eval inspect_evals/sevenllm_qa_en
+   ```
 
 - ### [SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Security](src/inspect_evals/sec_qa)
   LLMs should be capable of understanding computer security principles and applying best practices to their responses. SecQA has “v1” and “v2” datasets of multiple-choice questions that aim to provide two levels of cybersecurity evaluation criteria. The questions are generated by GPT-4 based on the “Computer Systems Security: Planning for Success” textbook and vetted by humans.

requirements.txt

@@ -1,6 +1,4 @@
 datasets>=2.21
 pillow
 requests
-tiktoken
-
-
+tiktoken

src/inspect_evals/_registry.py

@@ -57,6 +57,7 @@
 from .pubmedqa import pubmedqa
 from .race_h import race_h
 from .sec_qa import sec_qa_v1, sec_qa_v1_5_shot, sec_qa_v2, sec_qa_v2_5_shot
+from .sevenllm import sevenllm_mcq_en, sevenllm_mcq_zh, sevenllm_qa_en, sevenllm_qa_zh
 from .squad import squad
 from .swe_bench import swe_bench
 from .truthfulqa import truthfulqa

src/inspect_evals/sevenllm/README.md

@@ -0,0 +1,71 @@
+# SEvenLLM
+
+[SEvenLLM](https://arxiv.org/abs/2405.03446) is a benchmark to elicit, and improve cybersecurity incident analysis and response abilities in LLMs for Security Events.
+
+<!-- Contributors: Automatically Generated -->
+Contributed by [@kingroryg](https://github.com/kingroryg)
+<!-- /Contributors: Automatically Generated -->
+
+<!-- Usage: Automatically Generated -->
+## Usage
+
+First, install the `inspect_ai` and `inspect_evals` Python packages with:
+```bash
+pip install inspect_ai
+pip install git+https://github.com/UKGovernmentBEIS/inspect_evals
+pip install jieba sentence_transformers rouge # packages required for the sevenllm benchmark
+```
+
+Then, evaluate against one or more models with:
+```bash
+inspect eval inspect_evals/sevenllm_mcq_zh --model openai/gpt-4o
+inspect eval inspect_evals/sevenllm_mcq_en --model openai/gpt-4o
+inspect eval inspect_evals/sevenllm_qa_zh --model openai/gpt-4o
+inspect eval inspect_evals/sevenllm_qa_en --model openai/gpt-4o
+```
+
+After running evaluations, you can view their logs using the `inspect view` command:
+
+```bash
+inspect view
+```
+
+If you don't want to specify the `--model` each time you run an evaluation, create a `.env` configuration file in your working directory that defines the `INSPECT_EVAL_MODEL` environment variable along with your API key. For example:
+
+```bash
+INSPECT_EVAL_MODEL=anthropic/claude-3-5-sonnet-20240620
+ANTHROPIC_API_KEY=<anthropic-api-key>
+```
+<!-- /Usage: Automatically Generated -->
+
+
+## Dataset
+
+The dataset described in the paper is a bilingual (English and Chinese) instruction dataset specifically created to train LLMs for cybersecurity tasks. It consists of 28 well-defined tasks, categorized into understanding tasks and generation tasks related to cybersecurity incident analysis.
+
+1. Understanding Tasks (e.g., key entity recognition, malware feature extraction, vulnerability intelligence extraction)
+2. Generation Tasks (e.g., attack strategy analysis, trend prediction, summary generation)
+3. Size: benchmark contains 1,300 test samples with multiple-choice and query-answer questions.
+
+>A phishing campaign targeted employees at a major company using fake login portals. The attacker utilized domain phishing[.]com and sent emails requesting password updates. This attack exploited a vulnerability in the email server to bypass spam filters.
+> 
+> Task: Key Entity Recognition
+> 
+> Instruction: Identify key entities such as attacker, victim, attack tools, and indicators of compromise from the text.
+
+> Output:
+>
+>    Attacker Information: Unknown (phishing campaign organizer)
+> 
+>    Victim: Employees of a major company
+> 
+>    Domain Name: phishing[.]com
+> 
+>    Vulnerability Exploited: Email server spam filter bypass
+> 
+>    Attack Tool: Fake login portals
+
+
+## Scoring
+
+All evaluations involve key metrics such as Rouge-L, which measures overlap between model-generated and reference responses, and semantic similarity, which uses multilingual sentence transformers to assess the meaning alignment of outputs.

src/inspect_evals/sevenllm/__init__.py

@@ -0,0 +1,8 @@
+from .sevenllm import (
+    sevenllm_mcq_en,
+    sevenllm_mcq_zh,
+    sevenllm_qa_en,
+    sevenllm_qa_zh,
+)
+
+__all__ = ["sevenllm_mcq_en", "sevenllm_mcq_zh", "sevenllm_qa_en", "sevenllm_qa_zh"]

src/inspect_evals/sevenllm/scorers.py

@@ -0,0 +1,129 @@
+import re
+
+import jieba
+from inspect_ai.scorer import Score, Scorer, Target, mean, scorer, stderr
+from inspect_ai.solver import TaskState
+from rouge import Rouge
+from sentence_transformers import SentenceTransformer, util
+
+SENTENCE_TRANSFORMER_REPO = (
+    "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
+)
+ROUGE_L_THRESHOLD = 0.2
+
+
+def split_sentences(text: str, is_zh: bool) -> list[str]:
+    """Split text into sentences (for ROUGE-L scoring)."""
+    if is_zh:
+        sentences = re.findall(r"[^。！？]+[。！？]?", text)
+    else:
+        sentences = re.split(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?|!)\s", text)
+    sentences = [s.strip() for s in sentences if s.strip()]
+    return sentences
+
+
+def tokenize_sentence(sentence: str, is_zh: bool) -> str:
+    """Tokenize sentences (for ROUGE-L scoring)."""
+    return " ".join(jieba.cut(sentence)) if is_zh else sentence
+
+
+@scorer(metrics=[mean(), stderr()])
+def rouge_l_scorer(is_zh: bool) -> Scorer:
+    """ROUGE-L scorer for the SEVENLLM benchmark.
+
+    This scorer calculates the ROUGE-L score to evaluate the similarity between
+    the model-generated output and the reference text on a per-sentence basis.
+    The ROUGE-L metric focuses on the longest common subsequence (LCS) between
+    the sentences, which captures the structure and fluency of the text. The scorer
+    is designed to handle both English and Chinese texts, with sentence tokenization
+    and splitting adjusted based on the `is_zh` flag.
+
+    The routine:
+    - Splits the target reference and model-generated text into sentences.
+    - Tokenizes each sentence (with support for Chinese tokenization).
+    - Computes the ROUGE-L similarity score for each pair of sentences.
+    - Aggregates the scores, with sentences scoring above a defined threshold
+      (default: 0.2) contributing to the final score proportionally.
+
+    The total score reflects the percentage of sentences in the reference text
+    that have sufficiently similar counterparts in the generated output,
+    scaled to 100%.
+
+    Args:
+        is_zh (bool): A flag indicating whether the input text is in Chinese (`True`)
+        or English (`False`).
+
+    Returns:
+        Scorer: A scoring function that computes the aggregated ROUGE-L score
+        and provides the result alongside the model-generated output and a
+        detailed explanation.
+    """
+    rouge = Rouge()
+
+    async def score(state: TaskState, target: Target) -> Score:
+        test_sentences = split_sentences(target.text, is_zh)
+        generated_sentences = split_sentences(state.output.completion, is_zh)
+        num_sentences = len(test_sentences)
+        per_sentence_score = 100.0 / num_sentences  # Adjust scaling as needed
+        total_score = 0.0
+
+        for test_sentence in test_sentences:
+            max_similarity = 0.0
+            for gen_sentence in generated_sentences:
+                test_sentence_tok = tokenize_sentence(test_sentence, is_zh)
+                gen_sentence_tok = tokenize_sentence(gen_sentence, is_zh)
+
+                scores = rouge.get_scores(test_sentence_tok, gen_sentence_tok)
+                similarity = scores[0]["rouge-l"]["f"]
+                if similarity > max_similarity:
+                    max_similarity = similarity
+            if max_similarity >= ROUGE_L_THRESHOLD:
+                total_score += per_sentence_score
+
+        return Score(
+            value=total_score,
+            answer=state.output.completion,
+            explanation=f"Accumulated ROUGE-L score: {total_score:.4f}",
+        )
+
+    return score
+
+
+@scorer(metrics=[mean(), stderr()])
+def semantic_similarity_scorer() -> Scorer:
+    """Semantic similarity scorer for the SEVENLLM benchmark.
+
+    This scorer evaluates the semantic similarity between the model-generated output
+    and the reference target text using a pre-trained SentenceTransformer model.
+    It computes cosine similarity between the embeddings of the two texts, providing
+    a quantitative measure of their alignment in meaning.
+
+    The routine:
+    - Encodes both the model's output and the reference target text into embeddings
+      using a SentenceTransformer.
+    - Computes the cosine similarity between the two embeddings as the similarity score.
+    - Returns the score along with the model output and an explanation string.
+
+    The score is aggregated using mean and standard error (stderr) metrics to evaluate
+    model performance across the benchmark.
+
+    Returns:
+        Scorer: A scoring function that computes semantic similarity and provides
+        both the similarity score and a detailed explanation of the result.
+    """
+    model = SentenceTransformer(SENTENCE_TRANSFORMER_REPO)
+
+    async def score(state: TaskState, target: Target) -> Score:
+        model_output = state.output.completion
+        target_text = target.text
+        embeddings = model.encode([model_output, target_text], convert_to_tensor=True)
+        cosine_similarity = util.pytorch_cos_sim(embeddings[0], embeddings[1])
+        similarity_score = cosine_similarity.item()
+
+        return Score(
+            value=similarity_score,
+            answer=model_output,
+            explanation=f"Semantic similarity score: {similarity_score:.4f}",
+        )
+
+    return score

src/inspect_evals/sevenllm/sevenllm.py

@@ -0,0 +1,140 @@
+"""SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence
+
+Hangyuan Ji, Jian Yang, Linzheng Chai, Chaoren Wei, Liqun Yang,
+Yunlong Duan, Yunli Wang, Tianzhen Sun, Hongcheng Guo, Tongliang Li,
+Changyu Ren, Zhoujun Li
+
+https://arxiv.org/abs/2405.03446
+
+# Eval for MCQs (Understanding) in Simplified Chinese (Zh)
+inspect eval inspect_evals/sevenllm_mcq_zh
+
+# Eval for MCQs (Understanding) in English (En)
+inspect eval inspect_evals/sevenllm_mcq_en
+
+# Eval for QAs (Generation) in Simplified Chinese (Zh)
+inspect eval inspect_evals/sevenllm_qa_zh
+
+# Eval for QA (Generation) in English (En)
+inspect eval inspect_evals/sevenllm_qa_en
+"""
+
+import json
+import re
+from typing import Any
+
+from inspect_ai import Task, task
+from inspect_ai.dataset import Dataset, Sample, json_dataset
+from inspect_ai.scorer import choice
+from inspect_ai.solver import generate, multiple_choice, prompt_template
+
+from inspect_evals.sevenllm.scorers import rouge_l_scorer, semantic_similarity_scorer
+
+BENCHMARK_DATASET_URL = "https://huggingface.co/datasets/Multilingual-Multimodal-NLP/SEVENLLM-Dataset/raw/main/test.jsonl"
+# This prompt has been inspired from the inference work done by the authors.
+# See: https://github.com/CSJianYang/SEevenLLM/blob/main/infer.py
+# The prompt remains the same for both the languages.
+TEMPLATE = r"""
+Below is an instruction that describes a task, paired with an input that provides further context.
+Write a response that appropriately completes the request. If you are given options (A, B, C, D),
+return just the option letter and not the entire text. If you're required to return a JSON, only return a JSON and
+nothing else.
+
+{prompt}
+"""
+
+
+@task
+def sevenllm_mcq_zh() -> Task:
+    """Inspect task implementing the SEvenLLM benchmark for MCQs in Simplified Chinese."""
+    return Task(
+        dataset=get_sevenllm_dataset(language="zh", data_format="mcq"),
+        solver=[prompt_template(template=TEMPLATE), multiple_choice()],
+        scorer=choice(),
+    )
+
+
+@task
+def sevenllm_mcq_en() -> Task:
+    """Inspect task implementing the SEvenLLM benchmark for MCQs in English."""
+    return Task(
+        dataset=get_sevenllm_dataset(language="en", data_format="mcq"),
+        solver=[prompt_template(template=TEMPLATE), multiple_choice()],
+        scorer=choice(),
+    )
+
+
+@task
+def sevenllm_qa_zh() -> Task:
+    """Inspect task implementing the SEvenLLM benchmark for QA in Simplified Chinese."""
+    return Task(
+        dataset=get_sevenllm_dataset(language="zh", data_format="qa"),
+        solver=[prompt_template(template=TEMPLATE), generate()],
+        scorer=[rouge_l_scorer(is_zh=True), semantic_similarity_scorer()],
+    )
+
+
+@task
+def sevenllm_qa_en() -> Task:
+    """Inspect task implementing the SEvenLLM benchmark for QA in English."""
+    return Task(
+        dataset=get_sevenllm_dataset(language="en", data_format="qa"),
+        solver=[prompt_template(template=TEMPLATE), generate()],
+        scorer=[rouge_l_scorer(is_zh=False), semantic_similarity_scorer()],
+    )
+
+
+def contains_zsh(text: str) -> bool:
+    """Return True if the text contains a simplified-chinese character."""
+    # Regular expression to match Simplified Chinese characters
+    # CJK Unified Ideographs range: \u4e00-\u9fff
+    pattern = re.compile(r"[\u4e00-\u9fff]")
+
+    return bool(pattern.search(text))
+
+
+def record_to_sample(record: dict[str, Any]) -> Sample:
+    """Applies transformations to each record in the dataset for the Task."""
+    instruction = record["instruction"]
+    record_format = "qa" if isinstance(instruction, str) else "mcq"
+    text = instruction if isinstance(instruction, str) else instruction["question"]
+    record_language = "zh" if contains_zsh(text) else "en"
+
+    sample = {
+        "id": record["id"],
+        "input": f"{text}\n\n{record['input']}\n\n",
+        "metadata": {
+            "category": record["category"],
+            "cot": record["thought"],
+            "language": record_language,
+            "format": record_format,
+        },
+        "target": json.dumps(record["output"], ensure_ascii=False)
+        if record_format == "qa"
+        else str(record["output"]),
+    }
+
+    if record_format == "mcq":
+        sample["choices"] = [
+            str(instruction["choice"]["A"]),
+            str(instruction["choice"]["B"]),
+            str(instruction["choice"]["C"]),
+            str(instruction["choice"]["D"]),
+        ]
+
+    return Sample(**sample)
+
+
+def get_sevenllm_dataset(language: str, data_format: str) -> Dataset:
+    """Get a filtered dataset from the SEvenLLM benchmark based on language and format."""
+    # Cannot use `hf_dataset` here because the benchmark jsonl file contains json of
+    # multiple schemas, which violates PyArrow validation checks.
+    dataset = json_dataset(
+        json_file=BENCHMARK_DATASET_URL,
+        sample_fields=record_to_sample,
+    )
+
+    return dataset.filter(
+        lambda sample: sample.metadata["format"] == data_format
+        and sample.metadata["language"] == language
+    )