🤖 AI Analysis

Final verdict: SUSPICIOUS

The package exhibits some concerning behaviors, particularly regarding metadata and credential handling, though the overall risk does not reach a critical level.

Rapid repository activity and limited maintainer history.
Use of environment variable for GITHUB_TOKEN.

Per-check LLM notes

Network: The network call is likely to fetch resources from GitHub, which is normal for many CLI tools.
Shell: Executing shell commands like 'claude --version' may be intended for checking the version of an external tool, but it could also indicate more invasive actions depending on the context.
Obfuscation: No obfuscation patterns detected.
Credentials: The code retrieves a GITHUB_TOKEN from environment variables for authentication purposes, which is common practice but should be handled securely.
Metadata: The repository's recent and rapid activity, along with the maintainer's limited history, suggest potential suspicious behavior.

📦 Package Quality Overall: Medium (5.6/10)

✦ High Test Suite 9.0

Test suite present — 3 test file(s) found

3 test file(s) detected (e.g. test_classifier.py)

◈ Medium Documentation 5.0

Some documentation present

Detailed PyPI description (6790 chars)

◈ Medium Contributing Guide 7.0

Some contribution signals present

Governance file: security.py

◈ Medium Type Annotations 5.0

Partial type annotation coverage

72 type-annotated function signatures detected in source

○ Low Multiple Contributors 2.0

Single-author or unverifiable project

1 unique contributor(s) across 17 commits in calintzy/agentscore
Single author with few commits — possibly a personal or throwaway project

🔬 Heuristic Checks

⚠ Outbound Network Calls score 1.5

Found 1 network call pattern(s)

rse_github_url(url) with httpx.Client(timeout=TIMEOUT, headers=_make_headers()) as client:

✓ Code Obfuscation

No obfuscation patterns detected

⚠ Shell / Subprocess Execution score 2.0

Found 1 shell execution pattern(s)

tr: try: result = subprocess.run( ["claude", "--version"], capture_ou

⚠ Credential Harvesting score 2.5

Found 1 credential access pattern(s)

vnd.github+json"} token = os.environ.get("GITHUB_TOKEN") if token: headers["Authorization"] = f"Bearer

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

✓ Suspicious Page Links

All external links appear legitimate

⚠ Git Repository History score 5.0

Git history flags: Repository has zero stars and zero forks

Repository has zero stars and zero forks
All 17 commits happened within 24 hours

⚠ Maintainer History score 4.0

2 maintainer concern(s) found

Author name is missing or very short
Author "" appears to have only 1 package on PyPI (new or inactive account)

✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with agentscore-cli

Create a comprehensive tool named 'AgentEvaluator' using the Python package 'agentscore-cli'. This tool will serve as a quality assessment platform for AI agents across various development environments. Your task is to develop a user-friendly command-line interface (CLI) that allows developers and testers to evaluate multiple AI agents simultaneously based on predefined metrics. The tool should support a wide range of AI agent types, including but not limited to chatbots, recommendation systems, and decision-making models. Here’s a detailed breakdown of what your project should include:

1. **Setup**: Begin by installing the 'agentscore-cli' package via pip. Ensure that your application can initialize a new evaluation project, which includes setting up directories for storing agent configurations, test cases, and evaluation results.
2. **Agent Configuration**: Develop a feature where users can define and save configurations for different AI agents. These configurations should include details such as the agent's endpoint URL, API keys if necessary, and any specific parameters required for interaction.
3. **Test Case Management**: Implement a system for managing test cases. Users should be able to add, modify, and delete test cases. Each test case should contain input data and expected outcomes, which will be used to evaluate the agent's performance.
4. **Evaluation Metrics**: Utilize 'agentscore-cli' to define and apply evaluation metrics. These could include accuracy, response time, consistency, and user satisfaction scores. The tool should calculate these metrics automatically after running each test case against the agent.
5. **Reporting**: After evaluating an agent, generate detailed reports summarizing the performance across all test cases and metrics. These reports should be easy to read and understand, highlighting strengths and areas for improvement.
6. **Comparison Tool**: Include a feature that allows users to compare the performance of different agents based on the same set of test cases. This comparison should be visual and should highlight differences in performance metrics clearly.
7. **Integration with Development Environments**: Ensure that 'AgentEvaluator' can be integrated into popular development environments like VSCode or PyCharm. This integration should facilitate seamless setup and execution of evaluations within these environments.
8. **Documentation and User Guide**: Provide comprehensive documentation and a user guide that explains how to install, configure, and use 'AgentEvaluator'. Additionally, include examples and best practices for setting up effective evaluations.

By completing this project, you will have built a powerful and versatile tool that significantly enhances the process of developing and testing AI agents.