asqi-engineer

v0.5.4 suspicious
6.0
Medium Risk

ASQI quality checks for AI systems

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package exhibits concerning behaviors such as executing shell commands and using obfuscation techniques, which could indicate attempts to bypass security measures. While these actions may have legitimate purposes, they raise significant red flags.

  • High shell risk due to potential unsanitized command execution
  • Use of obfuscation suggesting attempts to evade simple code analysis
Per-check LLM notes
  • Network: The network calls may be part of legitimate functionality, but could also indicate external communication that might not be necessary.
  • Shell: The shell execution patterns are concerning as they suggest the package executes external commands which can pose a risk if not properly sanitized or controlled.
  • Obfuscation: The use of AST parsing indicates an attempt to bypass simple code analysis tools, which is suspicious but not definitive proof of malicious intent.
  • Credentials: Fetching API keys from environment variables can be legitimate, but the lack of proper handling and fallback strategies suggests potential misuse or insecure practices.
  • Metadata: The package shows signs of low maintainer activity and poor metadata quality, raising some suspicion but not definitive evidence of malice.

📦 Package Quality Overall: Low (4.8/10)

✦ High Test Suite 9.0

Test suite present — 14 test file(s) found

  • 14 test file(s) detected (e.g. test_entrypoint.py)
◈ Medium Documentation 7.0

Some documentation present

  • 1 documentation file(s) (e.g. conf.py)
  • Detailed PyPI description (8868 chars)
○ Low Contributing Guide 2.0

No contributing guide or governance files found

  • No CONTRIBUTING, CODE_OF_CONDUCT, or governance files found
◈ Medium Type Annotations 5.0

Partial type annotation coverage

  • 287 type-annotated function signatures detected in source
○ Low Multiple Contributors 1.0

Unable to verify contributor count: no GitHub repository found

  • No GitHub repository linked — contributor count unavailable

🔬 Heuristic Checks

⚠ Outbound Network Calls score 3.0

Found 2 network call pattern(s)

  • } async with httpx.AsyncClient() as client: response = await client.post(
  • s.stderr) async with httpx.AsyncClient() as client: response = await client.post(
⚠ Code Obfuscation score 10.0

Found 5 obfuscation pattern(s)

  • using AST parsing instead of eval(). """ import ast import logging from typing import Any, Di
  • "eval") result = eval(code, {"__builtins__": {}}, context) # nosec B307
  • ox_arg: log = eval(task, limit=limit, log_dir=temp_dir, sandbox=sandbox_arg)[0]
  • else: log = eval(task, limit=limit, log_dir=temp_dir)[0] print(
  • try: module = __import__(module_name, fromlist=[evaluation]) task_func = getattr(module, evaluation)
⚠ Shell / Subprocess Execution score 10.0

Found 5 shell execution pattern(s)

  • try: result = subprocess.run( [sys.executable, "scripts/generate_schemas.
  • capture process = subprocess.Popen( garak_cmd, stdout=subproces
  • or "/sbin/mount" subprocess.run( [mount_cmd, "--bind", str(output_mount), st
  • put and capture process = subprocess.Popen( harbor_cmd, stdout=subprocess.PIPE,
  • n" ) result = subprocess.run( [sys.executable, "-c", probe], env=
⚠ Credential Harvesting score 10.0

Found 5 credential access pattern(s)

  • ot api_key: api_key = os.environ.get("API_KEY") if not base_url and not api_key: base_url =
  • rameters""" api_key = os.environ.get("API_KEY") if not api_key: # Fallback to provide
  • stem_config.get("api_key") or os.environ.get("API_KEY", "") return CustomOpenAIModel( model=
  • sut.get("api_key") or os.environ.get("API_KEY") or os.environ.get("OPENAI_API_KEY") ) mod
  • ataset self.api_key = os.environ.get("API_KEY") or os.environ.get("OPENAI_API_KEY") config.openai
✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

✓ Suspicious Page Links

All external links appear legitimate

✓ Git Repository History

No GitHub repository linked

  • No GitHub repository link found
⚠ Maintainer History score 6.0

3 maintainer concern(s) found

  • Author name is missing or very short
  • Author "" appears to have only 1 package on PyPI (new or inactive account)
  • Package has no PyPI classifiers (low effort / metadata quality)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with asqi-engineer 
Your task is to develop a comprehensive mini-application called 'AI Quality Auditor' which leverages the 'asqi-engineer' Python package to perform quality checks on various AI models and their outputs. This application will serve as a tool for developers and data scientists to ensure that their AI systems meet specific quality standards before deployment. Here’s a detailed plan for building this application:

1. **Project Setup**: Start by setting up your Python environment and installing the 'asqi-engineer' package. Ensure you have the necessary dependencies installed as well.
2. **Core Functionality**:
   - **Model Input**: Allow users to input the details of the AI model they wish to audit. This could include the model architecture, training dataset information, and any relevant hyperparameters.
   - **Quality Checks**: Utilize the 'asqi-engineer' package to run a series of predefined quality checks on the inputted model. These checks should cover aspects such as accuracy, bias, fairness, robustness, and performance metrics.
3. **User Interface**: Develop a simple yet effective command-line interface (CLI) for interacting with the application. Users should be able to easily input model details and review the results of the quality checks.
4. **Reporting**: Implement a feature that generates a detailed report summarizing the results of the quality checks. This report should highlight any areas where the model falls short of the expected standards and provide recommendations for improvement.
5. **Customization**: Allow users to customize the quality checks based on their specific requirements. For example, they might want to focus more on fairness or robustness depending on the use case of the AI system.
6. **Integration**: Consider integrating the application with popular AI frameworks like TensorFlow or PyTorch, allowing seamless testing of models built within these frameworks.
7. **Documentation**: Write comprehensive documentation that explains how to use the 'AI Quality Auditor', including examples and best practices for ensuring high-quality AI systems.

By following these steps, you will create a valuable tool that helps maintain the integrity and reliability of AI systems across different industries.

💬 Discussion Feed

Leave a comment

No discussion yet. Be the first to share your thoughts!