agent-observability-sdk

v0.2.1 suspicious
4.0
Medium Risk

Evaluation judges for AI voice agents (hallucination, response accuracy, intent, tool correctness, etc.)

πŸ€– AI Analysis

Final verdict: SUSPICIOUS

The package shows low risks in network, shell, obfuscation, and credential areas, but its novelty and limited maintainer history raise some concerns about potential supply-chain risks.

  • New package with limited maintainer history
  • Potential supply-chain risk due to package novelty
Per-check LLM notes
  • Network: Network calls are likely for legitimate purposes like sending telemetry data or updating agent configurations.
  • Shell: No shell execution patterns detected.
  • Obfuscation: No obfuscation patterns detected, indicating low risk.
  • Credentials: No credential harvesting patterns detected, indicating low risk.
  • Metadata: The package is new and the maintainer has few packages, which may indicate potential risk.

πŸ”¬ Heuristic Checks

⚠ Outbound Network Calls score 3.0

Found 2 network call pattern(s)

  • try: with httpx.Client(timeout=config.timeout_s) as client: resp =
  • als/v0" try: with httpx.Client(timeout=config.timeout_s) as client: resp = clie
βœ“ Code Obfuscation

No obfuscation patterns detected

βœ“ Shell / Subprocess Execution

No shell execution patterns detected

βœ“ Credential Harvesting

No credential harvesting patterns detected

βœ“ Typosquatting

No typosquatting candidates detected

βœ“ Registered Email Domain

No author email provided

βœ“ Suspicious Page Links

All external links appear legitimate

βœ“ Git Repository History

No GitHub repository linked

  • No GitHub repository link found
⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Package is very new: uploaded 2 day(s) ago
  • Author "Plivo Labs" appears to have only 1 package on PyPI (new or inactive account)
βœ“ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

πŸ’‘ AI App Starter Prompt

Use this prompt to build a project with agent-observability-sdk
Create a comprehensive evaluation tool for AI voice agents using the 'agent-observability-sdk' Python package. This tool will serve as a critical component in assessing the performance of AI voice assistants across various metrics such as hallucination detection, response accuracy, intent recognition, and tool correctness. Your task is to develop a fully-functional mini-application that not only integrates these evaluation capabilities but also provides an intuitive user interface for input and output. Here’s a step-by-step guide on how to proceed:

1. **Setup**: Begin by installing the necessary packages including 'agent-observability-sdk'. Ensure you have Python and pip installed on your system.
2. **Project Structure**: Organize your project into modules for better management. For example, separate modules for data handling, evaluation logic, and UI.
3. **Evaluation Logic**: Implement functions to evaluate AI voice responses based on the provided criteria. Use the 'agent-observability-sdk' to perform evaluations like hallucination checks, accuracy assessments, and tool correctness verification.
4. **User Interface**: Develop a simple yet effective command-line interface (CLI) where users can input test queries and see the results of the evaluations.
5. **Testing**: Test your application thoroughly with different types of AI voice responses to ensure it works as expected across various scenarios.
6. **Documentation**: Write clear documentation explaining how to use your application, including setup instructions and examples of usage.

**Suggested Features**:
- Detailed reports for each evaluation metric.
- Support for multiple languages or dialects.
- Option to save evaluation results for future reference.
- Integration with external tools for more advanced analysis.

Utilize the 'agent-observability-sdk' package to its full potential by leveraging its built-in evaluators and any additional functionalities it offers. Remember, the goal is to create a versatile tool that can be easily adapted for different AI voice agents.