agentchaos-sdk

v0.1.0 suspicious
6.0
Medium Risk

Evaluate agent system robustness through controlled, runtime, non-intrusive LLM API fault injection.

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package exhibits moderate risk due to its obfuscated code and lack of historical context, which could indicate potential malintent.

  • High obfuscation risk
  • Minimal package history
Per-check LLM notes
  • Network: No network calls detected, which is normal unless the package requires external services.
  • Shell: No shell executions detected, indicating no immediate signs of malicious activity.
  • Obfuscation: The code shows signs of obfuscation with partial comments and use of eval which can be risky.
  • Credentials: No clear patterns indicating credential harvesting were detected.
  • Metadata: The package is new with minimal activity and no history, raising some suspicion.

🔬 Heuristic Checks

✓ Outbound Network Calls

No suspicious network call patterns found

⚠ Code Obfuscation score 8.0

Found 4 obfuscation pattern(s)

  • # report = await agentchaos.eval(my_agent, "Write a prime checker") # print(report.summary(
  • rable to: {vuln}" async def eval( agent_fn: Callable, query: str = "Write a Python fu
  • try: return str(eval(expression)) except Exception as e: retu
  • result = str(eval(json.loads(tc.function.arguments).get("expression", "0")))
✓ Shell / Subprocess Execution

No shell execution patterns detected

✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

✓ Suspicious Page Links

All external links appear legitimate

⚠ Git Repository History score 2.5

Git history flags: Repository has zero stars and zero forks

  • Repository has zero stars and zero forks
⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Only one version has ever been released — brand new package
  • Author "AgentChaos Contributors" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with agentchaos-sdk 
Create a mini-application named 'LLMResilienceTester' using the Python package 'agentchaos-sdk'. This tool aims to evaluate the resilience of language model APIs by injecting faults into their runtime environment. Here’s a step-by-step guide on how to develop this application:

1. **Setup Project Environment**: Start by setting up a virtual environment for your project. Install the 'agentchaos-sdk' package along with other necessary dependencies like 'requests' for making HTTP requests.

2. **API Configuration Module**: Develop a configuration module where users can specify details about the LLM APIs they want to test, such as endpoint URLs, authentication tokens, and request parameters. Allow users to choose from a predefined list of popular LLM APIs or provide a way to input custom API configurations.

3. **Fault Injection Mechanism**: Utilize the 'agentchaos-sdk' package to implement a fault injection mechanism. This could include simulating network delays, random response errors, or even temporary unavailability of the API server. The goal is to mimic real-world scenarios where API availability might be compromised.

4. **Test Execution & Logging**: Design a user-friendly interface that allows users to select which types of faults to inject and at what frequency. Execute these tests against the specified LLM APIs and log the results, including response times, error rates, and any unexpected behaviors observed during the test runs.

5. **Report Generation**: After completing the test execution, generate a comprehensive report summarizing the findings. This report should highlight the strengths and weaknesses of each tested API under various fault conditions. Consider visual aids like graphs and charts to make the data more accessible.

6. **User Feedback Loop**: Implement a feedback loop where users can provide additional insights or suggestions based on the test outcomes. Use this feedback to continuously improve the fault injection patterns and test coverage in future iterations of 'LLMResilienceTester'.

By following these steps, you will create a powerful yet easy-to-use tool for evaluating the robustness of language model APIs, leveraging the capabilities of the 'agentchaos-sdk' package.