GCP Cloud Logging — SKILL.md

---
name: "GCP Cloud Logging"
description: "Skill for GCP Cloud Logging — auto-generated from documentation"
version: "1.0.0"
author: "skynet"
category: "infrastructure"
agents: ["claude-code", "codex", "gemini"]
tags: ["gcp-logging", "infrastructure", "auto-generated"]
---

# GCP Cloud Logging

---
name: GCP Cloud Logging
description: Use this skill when you need to collect, store, search, analyze, monitor, and alert on log data and events from Google Cloud and other applications. Essential for debugging, monitoring application performance, security analysis, and compliance requirements.
metadata:
  author: skynet
  version: 1.0.0
category: infrastructure
---

# GCP Cloud Logging

## Overview
Google Cloud Logging is a fully managed service that performs at scale and can ingest application and platform log data. It provides real-time log management and analysis capabilities for troubleshooting, monitoring, and auditing.

## Prerequisites
```bash
# Install and authenticate gcloud CLI
gcloud auth login
gcloud config set project YOUR_PROJECT_ID

# Enable Cloud Logging API
gcloud services enable logging.googleapis.com
```

## Core Operations

### Viewing Logs
```bash
# View recent logs from all resources
gcloud logging read --limit=50

# View logs from specific resource
gcloud logging read "resource.type=gce_instance" --limit=20

# View logs with severity filter
gcloud logging read "severity>=ERROR" --limit=30

# View logs from specific time range
gcloud logging read --since="2024-01-01" --until="2024-01-02"

# View logs with custom filter
gcloud logging read 'resource.type="k8s_container" AND resource.labels.namespace_name="production"' --limit=25
```

### Writing Custom Logs
```bash
# Write a simple log entry
gcloud logging write my-log "Hello from custom application" --severity=INFO

# Write structured log entry
gcloud logging write my-log '{"message": "User login", "user_id": "123", "action": "login"}' \
  --severity=NOTICE --resource=type=global

# Write log with custom timestamp
gcloud logging write my-log "Scheduled task completed" \
  --severity=INFO \
  --timestamp="2024-01-15T10:30:00Z"
```

### Log-based Metrics
```bash
# Create log-based metric
gcloud logging metrics create error_count \
  --description="Count of application errors" \
  --log-filter='severity>=ERROR AND resource.type="gae_app"'

# List existing metrics
gcloud logging metrics list

# Update metric filter
gcloud logging metrics update error_count \
  --log-filter='severity>=ERROR AND jsonPayload.component="auth"'

# Delete metric
gcloud logging metrics delete error_count
```

### Log Sinks
```bash
# Create Cloud Storage sink
gcloud logging sinks create audit-logs-sink \
  storage.googleapis.com/my-audit-bucket \
  --log-filter='protoPayload.serviceName="cloudaudit.googleapis.com"'

# Create BigQuery sink
gcloud logging sinks create analytics-sink \
  bigquery.googleapis.com/projects/PROJECT_ID/datasets/logs_dataset \
  --log-filter='resource.type="gce_instance" AND severity>=WARNING'

# Create Pub/Sub sink
gcloud logging sinks create alert-sink \
  pubsub.googleapis.com/projects/PROJECT_ID/topics/log-alerts \
  --log-filter='severity>=ERROR'

# List sinks
gcloud logging sinks list

# Update sink filter
gcloud logging sinks update analytics-sink \
  --log-filter='resource.type="gce_instance" AND severity>=INFO'
```

## Advanced Filtering

### Common Filter Patterns
```bash
# Application logs with specific labels
gcloud logging read 'resource.type="k8s_container" 
  AND resource.labels.container_name="web-server" 
  AND jsonPayload.level="error"'

# Time-based filtering with severity
gcloud logging read 'timestamp>="2024-01-15T00:00:00Z" 
  AND severity>=WARNING 
  AND resource.type="gae_app"'

# HTTP request logs
gcloud logging read 'httpRequest.status>=400 
  AND resource.type="http_load_balancer"'

# Audit logs for specific operations
gcloud logging read 'protoPayload.methodName="v1.compute.instances.insert" 
  AND protoPayload.serviceName="compute.googleapis.com"'
```

### Regular Expression Filtering
```bash
# Filter logs with regex pattern
gcloud logging read 'jsonPayload.message=~"user.*login.*failed"'

# Multiple regex conditions
gcloud logging read 'resource.type="gce_instance" 
  AND jsonPayload.message=~"(error|exception|failure)"'
```

## Decision Tree: Choosing Log Storage Strategy

```
Need to store logs long-term?
├── Yes → Compliance/Audit requirements?
│   ├── Yes → Use Cloud Storage sink with lifecycle policies
│   └── No → Use BigQuery sink for analytics
└── No → Real-time processing needed?
    ├── Yes → Use Pub/Sub sink with Cloud Functions
    └── No → Use default Cloud Logging retention (30 days)
```

## Monitoring and Alerting

### Setting Up Log-based Alerts
```bash
# Create alerting policy (requires Cloud Monitoring)
gcloud alpha monitoring policies create --policy-from-file=policy.yaml
```

Example policy.yaml:
```yaml
displayName: "High Error Rate Alert"
conditions:
  - displayName: "Error rate condition"
    conditionThreshold:
      filter: 'resource.type="gae_app"'
      comparison: COMPARISON_GREATER_THAN
      thresholdValue: 10
      duration: 300s
notificationChannels:
  - projects/PROJECT_ID/notificationChannels/CHANNEL_ID
```

### Log Router Configuration
```bash
# View current log router configuration
gcloud logging describe-log-router

# Create custom log router
gcloud logging routers create custom-router \
  --description="Route critical logs to multiple destinations"
```

## Programmatic Access

### Using Client Libraries (Python Example)
```python
from google.cloud import logging

# Initialize client
client = logging.Client()

# Write log entry
logger = client.logger('my-application')
logger.log_text('Application started', severity='INFO')

# Read logs
for entry in client.list_entries(filter_='severity>=ERROR'):
    print(f'{entry.timestamp}: {entry.payload}')
```

### REST API Examples
```bash
# List log entries via REST API
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  "https://logging.googleapis.com/v2/entries:list" \
  -d '{"resourceNames":["projects/PROJECT_ID"],"filter":"severity>=ERROR"}'

# Write log entry via REST API
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://logging.googleapis.com/v2/entries:write" \
  -d '{
    "entries": [{
      "logName": "projects/PROJECT_ID/logs/my-app",
      "resource": {"type": "global"},
      "textPayload": "Custom log message",
      "severity": "INFO"
    }]
  }'
```

## Troubleshooting

### Common Issues and Solutions

**Error: "Permission denied" when reading logs**
```bash
# Grant required IAM permissions
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="user:EMAIL" \
  --role="roles/logging.viewer"
```

**Error: "Log sink creation failed - bucket not found"**
```bash
# Verify bucket exists and has correct permissions
gsutil ls gs://BUCKET_NAME
gsutil iam ch serviceAccount:cloud-logs@system.gserviceaccount.com:objectCreator gs://BUCKET_NAME
```

**Error: "Invalid log filter syntax"**
```bash
# Test filter syntax before using
gcloud logging read 'resource.type="gce_instance"' --limit=1 --dry-run
```

**Logs not appearing in BigQuery sink**
```bash
# Check sink configuration
gcloud logging sinks describe SINK_NAME

# Verify BigQuery dataset permissions
bq show --format=prettyjson DATASET_NAME
```

**High logging costs**
```bash
# Analyze log volume by resource type
gcloud logging read 'timestamp>="2024-01-01"' \
  --format="value(resource.type)" | sort | uniq -c | sort -nr

# Set up exclusion filters for noisy logs
gcloud logging sinks update _Default \
  --add-exclusion="name=health-checks,filter=jsonPayload.path=/health"
```

### Performance Optimization
```bash
# Use specific resource filters to reduce query time
gcloud logging read 'resource.type="gce_instance" AND resource.labels.instance_id="INSTANCE_ID"'

# Limit time range for better performance
gcloud logging read 'timestamp>="2024-01-15T10:00:00Z" AND timestamp<="2024-01-15T11:00:00Z"'

# Use appropriate page sizes
gcloud logging read --limit=100  # Instead of very large limits
```

### Debug Mode
```bash
# Enable verbose output for troubleshooting
gcloud logging read --verbosity=debug 'resource.type="gce_instance"'

# Check quota usage
gcloud logging operations list
```