agent-eval-rpc

v0.83.0 suspicious
4.0
Medium Risk

Python RPC client for @tangle-network/agent-eval — judge content against rubrics over HTTP or stdio RPC. Eval logic runs in the Node runtime; this package is a thin wire client.

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package shows moderate risk due to potential shell execution capabilities, though it lacks signs of obfuscation, credential theft, and other malicious activities. The non-HTTPS link adds slight concern.

  • moderate shell risk due to subprocess calls
  • presence of a non-HTTPS link
Per-check LLM notes
  • Network: Network calls appear to be related to health checks and RPC communication, which could be legitimate for a service agent.
  • Shell: Shell executions involving subprocess calls might indicate the package is intended to run external commands, but this could also be used for malicious purposes like executing arbitrary code.
  • Obfuscation: No obfuscation patterns detected, indicating low risk of malicious intent related to code obfuscation.
  • Credentials: No credential harvesting patterns detected, suggesting that the package does not pose a risk for stealing secrets or credentials.
  • Metadata: The presence of a non-HTTPS link is concerning, but there are no other red flags like typosquatting or suspicious email domains.

🔬 Heuristic Checks

⚠ Outbound Network Calls score 9.0

Found 6 network call pattern(s)

  • try: with httpx.Client(timeout=probe_timeout) as c: r = c.get(f"{se
  • try: with httpx.Client(timeout=self.timeout_s, base_url=self.base_url) as c:
  • self._http = http_client or httpx.Client(timeout=timeout_s) def __enter__(self) -> HostedClient:
  • try: r = httpx.get(f"{url}/healthz", timeout=1.0) if r.status_code
  • t res.rejected == [] r = httpx.get( f"{receiver['url']}/v1/runs", headers={
  • rt res.accepted == 3 r = httpx.get( f"{receiver['url']}/v1/runs/py-traces/traces",
✓ Code Obfuscation

No obfuscation patterns detected

⚠ Shell / Subprocess Execution score 8.0

Found 4 shell execution pattern(s)

  • try: proc = subprocess.run( [self.cli_path, "rpc", method],
  • try: r = subprocess.run( ["pnpm", "exec", "tsx", "--version"],
  • (SERVER_TS)] ) proc = subprocess.Popen( runner, env=env, cwd=REPO_ROOT,
  • ubprocess proc = subprocess.run( ["node", str(CLI_DIST), "rpc", method],
✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

⚠ Suspicious Page Links score 2.0

Found 1 suspicious link(s) on the package page

  • Non-HTTPS external link: http://127.0.0.1:5005
✓ Git Repository History

Repository tangle-network/agent-eval appears legitimate

⚠ Maintainer History score 2.0

1 maintainer concern(s) found

  • Author "Tangle Network" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with agent-eval-rpc 
Create a Python-based educational assessment tool that leverages the 'agent-eval-rpc' package to evaluate student submissions against predefined rubrics. This tool will facilitate remote learning environments where automated feedback can be provided to students immediately after they submit their work. Here’s a detailed outline of the project:

1. **Setup Environment**: Begin by setting up your Python environment. Ensure you have Python installed, along with pip. Install the 'agent-eval-rpc' package via pip.

2. **Define Rubrics**: Create a set of rubrics that will be used to evaluate student submissions. These rubrics should include criteria such as correctness, completeness, and creativity. Store these rubrics in a structured format that the 'agent-eval-rpc' package can read and process.

3. **Integration with 'agent-eval-rpc'**: Use the 'agent-eval-rpc' package to connect to a Node.js server that runs the evaluation logic. Your Python app should be able to send student submissions to the server for evaluation, and receive back the results.

4. **User Interface**: Develop a simple user interface where teachers can input student submissions and select which rubric to use for evaluation. The UI should also display the evaluation results.

5. **Evaluation Results**: The application should provide detailed feedback based on the rubrics. For example, if evaluating a programming assignment, it could break down the score into sections like syntax correctness, functionality, and comments.

6. **Feedback Mechanism**: Implement a feature that allows for manual override of the automatic scores. Teachers should be able to adjust scores directly within the application and add personalized feedback comments.

7. **Reporting**: Include a reporting feature that generates summary reports for all submissions, highlighting common areas of strength and weakness among students.

8. **Security Considerations**: Ensure that any personal information stored about students is handled securely, adhering to data protection regulations.

9. **Testing**: Thoroughly test the application using various types of submissions and rubrics to ensure accuracy and reliability.

This project aims to streamline the assessment process in educational settings, making it more efficient and providing valuable insights into student performance.