aidial-rag-eval

v0.6.0 safe
3.0
Low Risk

Library for RAG evaluation

🤖 AI Analysis

Final verdict: SAFE

The package exhibits low risk across multiple dimensions with no detected network calls, shell execution, obfuscation, or credential harvesting patterns. The slight increase in metadata risk due to the author's limited presence on PyPI does not significantly alter the overall safe assessment.

  • No network calls detected.
  • No shell execution patterns found.
  • Low risk of obfuscation or credential harvesting.
Per-check LLM notes
  • Network: No network calls detected, which is normal if the package does not require external services.
  • Shell: No shell execution patterns detected, indicating no immediate risk of command injection or execution.
  • Obfuscation: No obfuscation patterns detected, indicating low risk.
  • Credentials: No credential harvesting patterns detected, indicating low risk.
  • Metadata: The author has only one package on PyPI, suggesting it might be a new or less active account.

📦 Package Quality Overall: Medium (6.0/10)

◈ Medium Test Suite 6.0

Partial test coverage signals detected

  • Test runner config found: pyproject.toml
◈ Medium Documentation 7.0

Some documentation present

  • Documentation URL: "Documentation" -> https://github.com/epam/ai-dial-rag-eval
  • Detailed PyPI description (6444 chars)
○ Low Contributing Guide 2.0

No contributing guide or governance files found

  • No CONTRIBUTING, CODE_OF_CONDUCT, or governance files found
◈ Medium Type Annotations 5.0

Partial type annotation coverage

  • 72 type-annotated function signatures detected in source
✦ High Multiple Contributors 10.0

Active multi-contributor project

  • 6 unique contributor(s) across 49 commits in epam/ai-dial-rag-eval
  • Active community — 5 or more distinct contributors

🔬 Heuristic Checks

✓ Outbound Network Calls

No suspicious network call patterns found

✓ Code Obfuscation

No obfuscation patterns detected

✓ Shell / Subprocess Execution

No shell execution patterns detected

✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

Email domain looks legitimate: epam.com

✓ Suspicious Page Links

All external links appear legitimate

✓ Git Repository History

Repository epam/ai-dial-rag-eval appears legitimate

⚠ Maintainer History score 2.0

1 maintainer concern(s) found

  • Author "EPAM RAIL" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with aidial-rag-eval 
Your task is to develop a comprehensive mini-application named 'RAGScoreAnalyzer' that leverages the capabilities of the 'aidial-rag-eval' library. This tool aims to evaluate Retrieval-Augmented Generation (RAG) models by providing detailed performance metrics based on user queries. Here’s a detailed breakdown of the project requirements and steps to follow:

1. **Project Setup**: Begin by setting up your development environment. Ensure you have Python installed, and create a virtual environment. Install necessary packages including 'aidial-rag-eval', along with any other dependencies such as pandas for data manipulation and matplotlib for visualization.

2. **Define Core Features**:
   - **Query Input Interface**: Implement a simple command-line interface where users can input their queries to be evaluated against a predefined set of questions and answers.
   - **Evaluation Metrics**: Utilize 'aidial-rag-eval' to calculate key performance indicators such as precision, recall, F1 score, and others relevant to RAG model evaluation.
   - **Visualization**: Use matplotlib to graphically represent the evaluation results, making it easier for users to understand the performance metrics visually.

3. **Implementation Steps**:
   - **Step 1**: Import the necessary modules from 'aidial-rag-eval'. Initialize the evaluation framework with your dataset.
   - **Step 2**: Design the query input function, ensuring it accepts natural language inputs and processes them efficiently.
   - **Step 3**: Integrate 'aidial-rag-eval' functions to compute the evaluation metrics based on the provided queries and expected outputs.
   - **Step 4**: Develop the visualization module to display the computed metrics using charts and graphs.
   - **Step 5**: Test the application thoroughly with different sets of queries to ensure reliability and accuracy.

4. **Enhancements**:
   - Consider adding a feature to save the evaluation results into a CSV file for further analysis.
   - Explore incorporating real-time feedback mechanisms to adjust the evaluation parameters dynamically.

By following these guidelines, you will create a robust and user-friendly tool that not only evaluates RAG models but also provides insightful visual representations of the evaluation outcomes.