> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcalmo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Grafana

> Connect Calmo to Grafana for comprehensive monitoring, alerting, and visualization through AI assistance

# Grafana Integration

Connect Calmo to your Grafana instance to enable monitoring, alerting, and visualization through AI assistance. This integration provides access to **26 specialized tools** across **8 categories** for complete observability and incident management workflows.

## Overview

The Grafana integration transforms how your team handles monitoring and observability by providing:

* **Advanced Dashboard Intelligence** - AI-powered dashboard analysis and panel query optimization
* **Multi-Source Data Querying** - Direct access to Prometheus, Loki, and other configured datasources
* **Intelligent Alerting** - Comprehensive alert rule management and status monitoring
* **Incident Response** - Integrated incident management with Grafana Incident
* **OnCall Coordination** - Complete on-call schedule and user management with Grafana OnCall
* **Safe Operations** - Read-only tools enabled by default with controlled write access

## Key Capabilities

When connected, Calmo gains access to **26 Grafana tools** across **8 categories**:

| Category                 | Tools   | Capability                                               |
| ------------------------ | ------- | -------------------------------------------------------- |
| **Dashboard Management** | 3 tools | Search, view, and analyze Grafana dashboards             |
| **Data Sources**         | 3 tools | Manage and query Prometheus, Loki, and other datasources |
| **Prometheus Querying**  | 5 tools | Execute PromQL queries and retrieve metadata             |
| **Loki Querying**        | 4 tools | Query logs and metrics using LogQL                       |
| **Alerting**             | 2 tools | View alert rules and their statuses                      |
| **Incident Management**  | 4 tools | Manage incidents in Grafana Incident                     |
| **OnCall Management**    | 5 tools | Manage on-call schedules and users                       |
| **Admin & Teams**        | 1 tool  | View and manage teams                                    |

## Prerequisites

* Grafana instance with API access enabled
* Admin access to create service accounts and API keys
* Calmo account with team or personal workspace

## Setup Process

### Step 1: Access Your Grafana Instance

**Locate Your Grafana URL:**

1. Navigate to your Grafana instance
2. Note your instance URL (e.g., `https://your-grafana-instance.com`)
3. Ensure API access is enabled on your instance

### Step 2: Create Service Account and API Key

**Create Service Account:**

1. Log in to your Grafana instance as an administrator
2. Navigate to **Administration** → **Service accounts**
3. Click **Add service account**
4. Configure service account:
   * **Display name**: "Calmo Integration"
   * **Role**: Editor or Viewer (based on desired permissions)
5. Create the service account

**Generate API Key:**

1. Click on the created service account
2. Navigate to **Tokens** tab
3. Click **Add service account token**
4. Configure token settings:
   * **Display name**: "Calmo API Token"
   * **Expiration**: Set appropriate expiration date
5. Copy the generated token immediately

**Required Permissions by Tool Category:**

| Tool Category                | Required Permissions                               |
| ---------------------------- | -------------------------------------------------- |
| **Dashboard Management**     | `dashboards:read`                                  |
| **Data Sources**             | `datasources:read`                                 |
| **Prometheus/Loki Querying** | `datasources:query`                                |
| **Alerting**                 | `alert.rules:read`                                 |
| **Incident Management**      | `incidents:read`, `incidents:write` (for creation) |
| **OnCall Management**        | `oncall:read`                                      |
| **Admin & Teams**            | `teams:read`                                       |

### Step 3: Connect to Calmo

1. Navigate to **Integrations** in your Calmo dashboard
2. Click **Grafana** integration
3. Enter your **Grafana URL** (including protocol: https\://)
4. Enter your **API Key**
5. Configure tool permissions:
   * ✅ **Read-only operations** enabled by default
   * ❌ **Write operations** disabled for safety
6. Test the connection using the built-in connection test
7. Complete the integration setup

## Tool Categories & Configuration

### **Dashboard Management** (Safe)

**Default: Enabled** - Essential for dashboard analysis and visualization

* **search\_dashboards** - Search for dashboards by title or metadata
* **get\_dashboard\_by\_uid** - Retrieve full dashboard details using unique identifier
* **get\_dashboard\_panel\_queries** - Get panel queries and datasource info from dashboards

*Use Cases: Dashboard discovery, performance analysis, panel optimization, visualization insights*

### **Data Sources** (Safe)

**Default: Enabled** - Datasource management and configuration

* **list\_datasources** - View all configured datasources
* **get\_datasource\_by\_uid** - Get datasource details by UID
* **get\_datasource\_by\_name** - Get datasource details by name

*Use Cases: Datasource inventory, configuration validation, connection testing*

### **Prometheus Querying** (Safe)

**Default: Enabled** - Direct PromQL query execution and metadata retrieval

* **query\_prometheus** - Execute PromQL queries against Prometheus datasources
* **list\_prometheus\_metric\_metadata** - Retrieve metric metadata from Prometheus
* **list\_prometheus\_metric\_names** - List available metric names
* **list\_prometheus\_label\_names** - List label names matching a selector
* **list\_prometheus\_label\_values** - List values for a specific label

*Use Cases: Metrics analysis, performance monitoring, capacity planning, troubleshooting*

### **Loki Querying** (Safe)

**Default: Enabled** - LogQL query execution and log analysis

* **query\_loki\_logs** - Query logs and metrics using LogQL
* **list\_loki\_label\_names** - List all available label names in logs
* **list\_loki\_label\_values** - List values for a specific log label
* **query\_loki\_stats** - Get statistics about log streams

*Use Cases: Log analysis, error investigation, application debugging, security monitoring*

### **Alerting** (Safe)

**Default: Enabled** - Alert rule management and monitoring

* **list\_alert\_rules** - List alert rules and their statuses
* **get\_alert\_rule\_by\_uid** - Get alert rule details by UID

*Use Cases: Alert monitoring, rule validation, alerting optimization, incident correlation*

### **Incident Management** (Mixed Safety)

**Default: Read operations enabled** - Grafana Incident integration

**Read Operations (✅ Enabled by default):**

* **list\_incidents** - List incidents in Grafana Incident

**Write Operations (⚠️ Disabled by default):**

* **create\_incident** - Create an incident in Grafana Incident
* **add\_activity\_to\_incident** - Add activity to an incident
* **resolve\_incident** - Resolve an incident

*Use Cases: Incident response, escalation management, post-mortem analysis, team coordination*

### **OnCall Management** (Safe)

**Default: Enabled** - Grafana OnCall integration

* **list\_oncall\_schedules** - List schedules from Grafana OnCall
* **get\_oncall\_shift** - Get details for a specific OnCall shift
* **get\_current\_oncall\_users** - Get users currently on-call for a schedule
* **list\_oncall\_teams** - List teams from Grafana OnCall
* **list\_oncall\_users** - List users from Grafana OnCall

*Use Cases: On-call coordination, schedule management, escalation planning, team availability*

### **Admin & Teams** (Safe)

**Default: Enabled** - Team and user management

* **list\_teams** - View all configured teams in Grafana

*Use Cases: Team organization, user management, access control, collaboration*

## Team vs Personal Configuration

### Team/Organization Setup

* Shared Grafana instance access across team members
* Organization-level monitoring policies and alert configurations
* Centralized incident response and on-call management
* Team administrators control incident creation and resolution permissions

### Personal Setup

* Individual Grafana instance connections
* Personal dashboard preferences and custom queries
* Private metric analysis and debugging sessions
* Full control over enabled tool capabilities

## Security & Best Practices

### ⚠️ Safety Recommendations

1. **Service Account Approach** - Use dedicated service accounts instead of personal API keys
2. **Minimal Permissions** - Grant only necessary permissions for intended use cases
3. **Token Rotation** - Regularly rotate API tokens according to security policies
4. **Instance Security** - Ensure Grafana instance uses HTTPS and proper authentication
5. **Access Monitoring** - Monitor API usage and access patterns

### 🔒 Permission Levels

| Risk Level | Operations                                       | Recommendation         |
| ---------- | ------------------------------------------------ | ---------------------- |
| **Low**    | View dashboards, query metrics/logs, read alerts | ✅ Safe to enable       |
| **Medium** | List incidents, view on-call schedules           | ✅ Generally safe       |
| **High**   | Create incidents, modify alert rules             | ⚠️ Enable with caution |

## Configuration Management

### Updating Grafana Connection

1. Navigate to **Integrations** → **Grafana**
2. Click **Edit Configuration**
3. Update Grafana URL or API key as needed
4. Modify tool permissions based on team requirements
5. Test connection to verify changes
6. Save configuration updates

### Managing Multiple Environments

* Connect separate Grafana instances for different environments
* Use different service accounts for production vs development
* Configure environment-specific monitoring and alerting policies
* Maintain separate incident response workflows per environment

## Advanced Features

### Multi-Datasource Support

* **Unified Querying** - Query multiple datasources through single interface
* **Cross-Datasource Correlation** - Correlate metrics and logs across different sources
* **Datasource Health Monitoring** - Monitor datasource connectivity and performance
* **Query Optimization** - Intelligent query optimization across different backends

### Advanced Analytics

* **Custom PromQL Queries** - Complex metric analysis with advanced aggregations
* **LogQL Intelligence** - Sophisticated log analysis with pattern recognition
* **Alert Correlation** - Cross-reference alerts with system events and metrics
* **Performance Insights** - Automated performance bottleneck detection

### Incident Response Integration

* **Automated Incident Creation** - AI-powered incident creation based on alert patterns
* **Escalation Management** - Intelligent escalation based on severity and on-call schedules
* **Timeline Tracking** - Comprehensive incident timeline with activity logging
* **Post-Incident Analysis** - Automated post-mortem generation and analysis

## Monitoring Workflows

### Real-Time Monitoring

* **Live Dashboard Analysis** - Real-time dashboard data analysis and insights
* **Metric Streaming** - Continuous metric monitoring with anomaly detection
* **Log Stream Analysis** - Real-time log analysis with pattern recognition
* **Alert Processing** - Intelligent alert filtering and prioritization

### Incident Response

* **Alert Correlation** - Automatic correlation of related alerts and incidents
* **On-Call Coordination** - Intelligent routing based on schedules and expertise
* **Escalation Automation** - Automated escalation based on response times
* **Communication Integration** - Seamless integration with communication tools

### Performance Analysis

* **Capacity Planning** - Resource utilization analysis and capacity recommendations
* **Performance Trending** - Long-term performance trend analysis
* **Optimization Insights** - Query and dashboard optimization recommendations
* **Anomaly Detection** - AI-powered anomaly identification across metrics and logs

## Troubleshooting

### Common Issues

**Authentication Failed**

* Verify API key is correct and hasn't expired
* Check service account permissions in Grafana
* Ensure API access is enabled on Grafana instance
* Verify network connectivity to Grafana instance

**Datasource Access Denied**

* Confirm service account has datasource query permissions
* Check datasource configuration and accessibility
* Verify datasource health in Grafana interface
* Review Grafana logs for detailed error information

**Dashboard Not Found**

* Verify dashboard exists and is accessible with current permissions
* Check dashboard UID and ensure it's correct
* Confirm user has read access to target dashboard
* Review dashboard sharing and permission settings

**Query Execution Failed**

* Verify datasource is healthy and responsive
* Check query syntax for PromQL or LogQL compliance
* Ensure time range is appropriate for data availability
* Review datasource-specific troubleshooting guides

### Getting Help

1. **Test Connection** - Use the built-in connection test feature
2. **Update Credentials** - Regenerate API key if authentication issues persist
3. **Check Documentation** - Refer to Grafana official documentation for API setup
4. **Contact Support** - Reach out to [support@getcalmo.com](mailto:support@getcalmo.com) for integration assistance

## Data Types & Analysis

### Dashboard Data

* **Dashboard Configurations** - Panel layouts, query definitions, and visualization settings
* **Panel Analytics** - Query performance, data sources, and refresh patterns
* **Usage Metrics** - Dashboard access patterns and user interaction data
* **Performance Data** - Dashboard load times and query execution metrics

### Metrics Data

* **Time Series Metrics** - Prometheus metrics with full dimensional data
* **Aggregated Data** - Pre-computed aggregations and rollups
* **Custom Metrics** - Application-specific and business metrics
* **Infrastructure Metrics** - System and infrastructure performance data

### Log Data

* **Structured Logs** - JSON and structured log entries with full context
* **Unstructured Logs** - Free-form log entries with pattern extraction
* **Log Metadata** - Log source, timestamp, and classification information
* **Error Logs** - Exception and error logs with stack traces

### Alert Data

* **Alert Rules** - Alert definitions, thresholds, and evaluation criteria
* **Alert History** - Historical alert firing and resolution data
* **Alert States** - Current alert states and evaluation results
* **Notification Data** - Alert routing and notification delivery status

### Incident Data

* **Incident Details** - Incident metadata, severity, and status information
* **Activity Timeline** - Incident response activities and timeline events
* **Resolution Data** - Incident resolution steps and outcome documentation
* **Team Coordination** - On-call assignments and escalation history

The Grafana integration provides comprehensive monitoring and observability capabilities, enabling your team to analyze dashboards, query multiple datasources, and manage incidents efficiently through AI-powered assistance while maintaining strict operational controls.

***

*For additional help with Grafana integration, contact our support team at [support@getcalmo.com](mailto:support@getcalmo.com).*