asiai

v1.13.0 suspicious
7.0
High Risk

Multi-engine LLM benchmark & monitoring CLI for Apple Silicon

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package exhibits multiple high-risk behaviors including potential network and shell execution risks, as well as obfuscation techniques that could be leveraged for malicious activities.

  • High network risk due to HTTP POST and HEAD methods
  • Potential persistent backdoor through shell execution with 'launchctl'
  • Encoded eval indicating high obfuscation risk
Per-check LLM notes
  • Network: The network calls include HTTP POST and HEAD methods, which could be used for legitimate purposes but also might indicate data exfiltration or command and control communication.
  • Shell: The shell execution patterns involve running commands like 'clear', 'open', and 'launchctl' which can be benign, but the use of 'launchctl' to load/unload profiles suggests potential for persistent backdoor behavior.
  • Obfuscation: The presence of encoded eval suggests potential for executing arbitrary code, indicating high risk.
  • Credentials: No clear patterns indicative of credential harvesting were found, suggesting low risk.
  • Metadata: Suspicious non-HTTPS links and lack of maintainer information suggest potential risk, but no concrete evidence of malice.

📦 Package Quality Overall: Medium (5.6/10)

◈ Medium Test Suite 6.0

Partial test coverage signals detected

  • Test runner config found: pyproject.toml
◈ Medium Documentation 7.0

Some documentation present

  • Documentation URL: "Documentation" -> https://asiai.dev
  • Detailed PyPI description (22210 chars)
○ Low Contributing Guide 4.0

No contributing guide or governance files found

  • Development Status classifier >= Beta
◈ Medium Type Annotations 5.0

Partial type annotation coverage

  • 361 type-annotated function signatures detected in source
◈ Medium Multiple Contributors 6.0

Limited contributor diversity

  • 2 unique contributor(s) across 100 commits in druide67/asiai
  • Two distinct contributors found

🔬 Heuristic Checks

⚠ Outbound Network Calls score 9.0

Found 6 network call pattern(s)

  • encode("utf-8") req = urllib.request.Request( url, data=data,
  • POST", ) with urllib.request.urlopen(req, timeout=timeout) as resp: return 20
  • urllib.request req = urllib.request.Request(webhook_url, method="HEAD") with urllib.requ
  • , method="HEAD") with urllib.request.urlopen(req, timeout=5) as resp: results.append(
  • s(payload).encode() req = urllib.request.Request( f"{base_url.rstrip('/')}/v1/chat/completion
  • try: with urllib.request.urlopen(req, timeout=timeout) as resp: for r
⚠ Code Obfuscation score 4.0

Found 2 obfuscation pattern(s)

  • uct`` — instruction-following eval (deterministic). The 4th quality pillar, distinct from ``--a
  • iai bench --code` dev-quality eval (no judge LLM). These grade tool-calling # reliability (the |
⚠ Shell / Subprocess Execution score 10.0

Found 6 shell execution pattern(s)

  • uiet: subprocess.run(["clear"], check=False) snap = collect_s
  • png else svg_path subprocess.Popen( ["open", card_to_open], std
  • p(plist, f) result = subprocess.run( ["launchctl", "load", profile.plist_path],
  • file.plist_path): subprocess.run( ["launchctl", "unload", profile.plist_path]
  • h) try: result = subprocess.run( ["launchctl", "list", profile.label],
  • }" try: result = subprocess.run( ["tail", "-n", str(lines), profile.log_path],
✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

Email domain looks legitimate: free.fr>

⚠ Suspicious Page Links score 4.0

Found 2 suspicious link(s) on the package page

  • Non-HTTPS external link: http://127.0.0.1:8899
  • Non-HTTPS external link: http://192.0.2.10:8899
✓ Git Repository History

Repository druide67/asiai appears legitimate

⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Author name is missing or very short
  • Author "" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with asiai 
Create a Python-based utility named 'LLMBench' that leverages the 'asiai' package to benchmark and monitor multiple Large Language Models (LLMs) specifically optimized for Apple Silicon processors. Your goal is to develop a tool that not only evaluates the performance of these models but also provides insightful visualizations to help users understand their capabilities better.

Step-by-Step Guide:
1. Setup: Begin by installing the necessary packages including 'asiai', 'matplotlib', and 'pandas'.
2. Integration: Integrate 'asiai' into your project to enable benchmarking and monitoring functionalities.
3. Benchmarking: Develop a feature within LLMBench that allows users to select from a predefined list of LLMs (e.g., GPT, BERT, etc.) to benchmark on Apple Silicon hardware. This feature should measure various aspects such as inference time, memory usage, and power consumption.
4. Monitoring: Implement a real-time monitoring dashboard that tracks the performance metrics of selected LLMs during runtime. This dashboard should update dynamically as the models run different tasks.
5. Visualization: Use 'matplotlib' to create graphs and charts that visually represent the benchmark results. These visualizations should be easy to interpret and highlight key differences between models.
6. Reporting: Generate comprehensive reports summarizing the benchmark data. These reports should include both quantitative data and qualitative insights based on the benchmarking process.
7. User Interface: Design a simple command-line interface (CLI) for users to interact with LLMBench. This interface should allow them to easily choose which models to benchmark and view the results.

Suggested Features:
- Support for adding custom LLMs to the benchmarking process.
- Export options for benchmark results in formats like CSV or JSON.
- Comparative analysis tools that allow users to compare the performance of two or more models side-by-side.
- A logging system that records all benchmarking activities and errors for future reference.

How 'asiai' is Utilized:
'asiai' will serve as the backbone of LLMBench's benchmarking and monitoring capabilities. It will provide the essential functions to run tests on LLMs, collect performance data, and facilitate real-time monitoring. By leveraging 'asiai', LLMBench aims to offer a robust, efficient, and user-friendly solution for evaluating large language models on Apple Silicon devices.