# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

TSAI Chatbot is a multi-tenant AI chatbot platform that integrates Angular frontends with a FastAPI backend, serving content across multiple WordPress sites. A major focus is **AI-generated articles and comments** for WordPress sites, created either by characters through the Creator app or via backend scripts. The system consists of embeddable chat widgets (Banner, SAS), standalone chat applications (Creator, Player), and a unified API for AI chat, content generation, and CMS integration.

## Environments

The stack runs in three environments:

- **tsai server** - Production Amazon EC2 instance hosting `turmansolutions.ai` (and subdomains) and `johnturman.net`
  - Repo path: `/tsai/repo`
- **scike server** - Production Amazon EC2 instance hosting `scike.ai` (and subdomains)
  - Repo path: `/data/tsai`
- **Development** - Local development environment

When referring to production servers, use "tsai server" or "scike server" to distinguish between them.

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                        Angular Frontends                            │
│  ┌─────────┐  ┌────────┐  ┌─────┐  ┌────────┐  ┌───────────────────┐│
│  │ Creator │  │ Player │  │ SAS │  │ Banner │  │ Builder Frontend  ││
│  │ (i18n)  │  │        │  │     │  │(widget)│  │                   ││
│  └────┬────┘  └───┬────┘  └──┬──┘  └───┬────┘  └─────────┬─────────┘│
│       └───────────┴──────────┴─────────┴─────────────────┘          │
│                              │                                       │
└──────────────────────────────┼───────────────────────────────────────┘
                               ▼
┌──────────────────────────────────────────────────────────────────────┐
│                         FastAPI Backend (/api)                       │
│  Endpoints: /chat, /chat-lc, /article, /image, /cms, /site, /user   │
│  Services: OpenAI, Anthropic Claude, LangChain, Tavily Search       │
└────────────────────────────────┬─────────────────────────────────────┘
                                 │
            ┌────────────────────┼────────────────────┐
            ▼                    ▼                    ▼
    ┌──────────────┐    ┌──────────────┐    ┌──────────────┐
    │  WordPress   │    │   Drupal     │    │ External APIs│
    │ (articles,   │    │ (prompts,    │    │ OpenAI/Claude│
    │  comments,   │    │  widgets,    │    │   Tavily     │
    │  images)     │    │  characters) │    │              │
    └──────────────┘    └──────────────┘    └──────────────┘
```

### Key Directories

- `/api` - FastAPI backend (Python)
- `/chatbot` - Angular 22 multi-project workspace
- `/builder` - Site orchestration system (frontend + backend)
- `/sites` - WordPress installations (tsai2, dev, sandbox, news, autosave, demo.tsai.ai, johnturman.net, scike-news, scike-guitar, etc.)
- `/drupal` - Drupal headless content repository used by the API to manage characters, widgets, and prompts
- `/store` - Drupal 11 SaaS store (Commerce 3)

### Infrastructure Repo (tsai server)

The `infra` repo tracks server configuration files for the tsai server. Lives on that EC2 instance only.

**Locations:**

- Bare repo: `/tsai/source/infra.git`
- Working copy: `/tsai/infra/`

**Currently tracked:**

```
/tsai/infra/
├── etc/
│   ├── httpd/
│   │   ├── conf/
│   │   │   └── httpd.conf           # Main Apache config (NOT symlinked)
│   │   └── conf.d/                  # Apache vhosts, SSL configs (symlinked)
│   └── pki/
│       └── tls/
│           ├── johnturman.net/      # SSL certs
│           └── turmansolutions.ai/  # SSL certs
```

**How it works:** Most config files (`conf.d/*.conf`, SSL certs) are replaced with symlinks pointing to the repo. Backups are kept with `.bak` suffix.

**Exception — httpd.conf:** The main Apache config at `/etc/httpd/conf/httpd.conf` is tracked in git for version history but is **NOT** symlinked. A broken symlink on the main config would prevent Apache from starting at all. When changes are made, edit the repo copy, commit/push, then manually copy to the server:

```bash
cd /tsai/infra && git pull
sudo cp etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf
sudo systemctl restart httpd
```

**To add new config files:**

```bash
# SSH to tsai server
ssh -i ~/.ssh/tsai.pem ec2-user@3.147.158.171

# Create mirrored path, copy file, fix ownership
mkdir -p /tsai/infra/etc/someservice
sudo cp -a /etc/someservice/config.conf /tsai/infra/etc/someservice/
sudo chown -R ec2-user:ec2-user /tsai/infra/etc/someservice

# Backup original and create symlink
sudo mv /etc/someservice/config.conf /etc/someservice/config.conf.bak
sudo ln -s /tsai/infra/etc/someservice/config.conf /etc/someservice/config.conf

# Commit
cd /tsai/infra && git add . && git commit -m "Add someservice config" && git push
```

### Angular Projects (in /chatbot)

- **tsai** - Shared library (build first in development)
- **creator** - Chatbot builder/editor with i18n (en-US, es, fr)
- **player** - Chatbot viewer
- **sas** - Sales & Support widget
- **banner** - Embedded chat widget for WordPress

### CMS Roles

**WordPress** - Content destination for AI-generated output:

- Stores AI-generated articles and comments
- Saves chat conversations and images from the Creator app
- Each site runs the tsai-plugin for API integration

**Drupal** - Content configuration server (via JSON:API):

- **Prompts** - System prompts and templates for AI generation
- **Widgets** - Chat widget configurations
- **Characters** - AI personas that author articles and participate in chats

### Store (`/store`)

A Drupal 11 SaaS e-commerce platform using Commerce 3 modules. Allows users to purchase TSAI Chatbot subscriptions (Bronze/Silver/Gold plans) which automatically provision WordPress + Chatbot sites.

**Key Modules:**

- `commerce` - Core e-commerce functionality
- `commerce_license` - Subscription licensing
- `commerce_recurring` - Recurring payments
- `commerce_stripe` - Stripe payment processing
- `advancedqueue` - Background job processing
- `tsai_provisioning` - Custom module for site provisioning

**Provisioning Workflow:**

```
Customer Purchase → ORDER_PAID Event → ProvisioningJob Created → Queue Processing
                                                                       ↓
                                            Builder executes create-tsai-site.json
                                                                       ↓
                                            WordPress + Chatbot sites created
                                                                       ↓
                                            User sees links at /user/provisioning
```

**Domain Generation:**

- WordPress: `{slug}.turmansolutions.ai`
- Chatbot: `chat-{slug}.turmansolutions.ai`

**Key Routes:**

- `/user/provisioning` - Customer's site status page
- `/admin/content/provisioning-jobs` - Admin view of all jobs
- `/api/provisioning/status` - JSON API for status polling

**Store Settings** (`settings.php`):

```php
$settings['tsai_builder_path'] = '/path/to/builder/backend';
```

### Store Commands

```bash
cd store

# Drush (use setup-env.sh first for PHP path)
./vendor/bin/drush cr                    # Clear cache
./vendor/bin/drush cex                   # Export config
./vendor/bin/drush cim                   # Import config
./vendor/bin/drush en <module>           # Enable module
./vendor/bin/drush pm:uninstall <module> # Uninstall module

# Process provisioning queue
./vendor/bin/drush advancedqueue:queue:process provisioning

# Check provisioning queue status
./vendor/bin/drush advancedqueue:queue:list
```

### Test Payments

The store uses a **Manual** payment gateway for testing (no real payment processing).

**To test the full provisioning flow:**

1. Create an account or log in at the store
2. Add a TSAI Chatbot plan to cart
3. Go through checkout
4. Select "Test Payment" as payment method
5. Complete order
6. Visit `/user/provisioning` to see job status
7. Run queue manually: `drush advancedqueue:queue:process provisioning`

### Angular Conventions

Both Angular workspaces (`/chatbot` and `/builder/frontend`) follow these patterns:

- **Zone-free** - No zone.js; use `provideZonelessChangeDetection()`
- **Signals** - Use Angular signals extensively for reactive state management
- **Angular Material** - Use Material Design components for all UI elements

## Common Commands

### Environment Setup

**MAMP paths are centralized** in `.env` files and `setup-env.sh`. To change PHP version, update these locations:

- `api/.env` - MAMP_PHP_BIN, MAMP_MYSQL_BIN (used by FastAPI backend)
- `builder/backend/.env` - MAMP_PHP_BIN, MAMP_MYSQL_BIN (used by builder configs)
- `setup-env.sh` - MAMP_PHP_BIN, MAMP_MYSQL_BIN (used for CLI development)

```bash
source setup-env.sh  # Sets up MAMP paths (PHP 8.3, MySQL, WP-CLI) and exports MAMP_* variables
```

### Chatbot (Angular)

```bash
cd chatbot
npm run dev:all          # Build all apps in dev mode (preferred for development)
npm run tsai:build       # Build shared library only
npm run player:serve     # Serve player app
npm run creator:serve    # Serve creator app
npm run banner:serve     # Serve banner widget
npm run sas:serve        # Serve SAS widget
npm test                 # Run Karma tests
npm run lint             # ESLint
npm run lint:fix         # ESLint with auto-fix

# Build all apps (use builder backend instead of deleted build.py)
cd builder/backend
PARAM_CONFIG_TYPE=tsai python run_build.py --config configs/build-angular.json
PARAM_CONFIG_TYPE=development python run_build.py --config configs/build-angular.json
```

### API (FastAPI)

```bash
cd api
pip install -r requirements.txt
uvicorn app.main:app --reload                # Dev server (port 8000)
pytest tests/ -v                             # Run all tests
pytest tests/unit/ -v                        # Unit tests only
pytest --cov=app tests/                      # With coverage report
```

### Builder Backend

```bash
cd builder/backend
python run_build.py --config configs/build-angular.json
python run_build.py --config configs/deploy-to-tsai.json
python run_build.py --config configs/create-tsai-site.json
python run_build.py --list      # List available configs
python run_build.py --dry-run   # Validate without executing
```

### WordPress Administration

```bash
wp --info                       # WP-CLI info
wp db check                     # Check database
wp option get <option>          # Get WP option
```

## Site Configuration

Site configs are stored as `site_config` Drupal nodes (defined by the `tsai_site_config` module at `drupal/web/modules/custom/tsai_site_config/`) and loaded by `SiteConfigService` (in `/api/app/services/site_config_service.py`). On startup the service fetches all nodes via JSON:API, parses them into `SiteConfig` Pydantic models, and writes a local JSON cache to `/api/app/data/site_configs_{development,tsai,scike}.json` (gitignored). The cache is a fallback only — used when Drupal is unreachable. Each site has:

- `cb_host` - Chatbot host (e.g., `chat.turmansolutions.test`)
- `wp_host` - WordPress host URL
- `wp_dir` - WordPress directory name in `/sites`
- `slug` - Short identifier (also the Drupal username of the site owner)
- `activity_level` - Article posting frequency (0 = off)
- `drupal_uid` - Drupal user UUID (site owner)
- `drupal_target_id` - Drupal user entity ID
- `drupal_node_id` - Drupal site_config node UUID

### Working with site configs

- **View/edit in Drupal:** site configs are standard Drupal content — edit them at `/admin/content` (filter by Site Config type) on the corresponding Drupal instance. Changes won't appear in a running API process until the cache reloads.
- **Refresh the API cache:** call `SiteConfigService.reload()` to force a fresh fetch from Drupal, or restart the FastAPI process. The `_sites` list and `_loaded` flag are class-level, so reload is global.
- **Add/remove sites programmatically:** use `SiteConfigService.add_site(...)` / `remove_site(cb_host)` (async). These call `DrupalService.create_site_config_node` / `delete_site_config_node` and update the in-memory list. The builder's `create-{tsai,scike}-site.json` configs invoke this flow via the API's `/site` endpoint during provisioning.
- **CLI scripts:** scripts in `/api` and `/api/scripts` (e.g., `edit_character.py`, `schedule_character.py`, `suggest_categories.py`) instantiate `SiteConfigService()` directly and look up sites with `get_site_by_slug(slug)`. They authenticate to the API with an `Origin` header matching the site's `cb_host` — no hash/token auth.
- **Local dev aliases:** `/api/app/data/dev_aliases.json` maps extra `cb_host` values (e.g., `localhost:4300`) to existing slugs so multiple ng-serve ports can resolve to the same site without creating duplicate Drupal nodes. Only loaded when `TSAI_ENV` is `development`.
- **Environment selection:** the cache file and dev-alias behavior are driven by `TSAI_ENV` (`development` | `production` | `scike_ai`). Set this in `/api/.env`.

The API identifies the calling site from the browser's `Origin` (or `Referer`) header via `app/utilities/site_auth.py::get_current_site`. CORS allowlist (in `api/app/main.py` and `drupal/web/sites/default/services.yml`) is a wildcard regex over `*.turmansolutions.{ai,test}`, `*.johnturman.{net,test}`, `*.scike.{ai,test}`, and `localhost:*` — add new domains there.

The tsai-plugin is installed in each WordPress site at `sites/{site}/wp-content/plugins/tsai-plugin/`.

## Build Output Destinations

The builder config (`build-angular.json`) outputs to:

- `chatbot/dist/{config}/` - Standalone builds (creator, player, sas, banner) where `{config}` is `tsai`, `scike`, or `development`
- `chatbot/dist/{config}/wp-shared/` - WordPress builds (player, sas, banner) with WP-specific base-href
- The tsai shared library builds to `chatbot/dist/tsai-lib/`

WordPress sites access the WP builds via symlinks:

- **Local:** `sites/{site}/wp-content/uploads/tsai/{app}` -> `chatbot/dist/{config}/wp-shared/{app}`
- **tsai server:** `sites/{site}/wp-content/uploads/tsai/{app}` -> `/tsai/cb-build/wp-shared/{app}`
- **scike server:** `sites/{site}/wp-content/uploads/tsai/{app}` -> `/data/cb-build/wp-shared/{app}`

## Environment Variables

The API requires `.env` file in `/api` with:

- `OPENAI_API_KEY` (required)
- `ANTHROPIC_API_KEY` (optional, enables Claude models)
- `LOG_LEVEL` (optional, set to DEBUG for verbose logging)

Logs are written to `/api/logs/api.log` with rotation.

## Testing

### API Tests (`/api/tests/`)

```
api/tests/
├── conftest.py              # Shared pytest fixtures
├── unit/                    # Fast, mocked unit tests
│   ├── test_chat.py         # /chat/ endpoint tests
│   ├── test_chat_lc.py      # /chat-lc/ LangChain endpoint tests
│   ├── test_date_recency.py # Date utility tests
│   ├── test_root.py         # Root endpoint tests
│   └── test_user.py         # User authentication tests
└── integration/             # Manual integration scripts (require running server)
    ├── langchain_flow_manual.py
    ├── player_flow_manual.py
    └── registration_flow_manual.py
```

**Running Tests:**

```bash
cd api
pytest tests/ -v                    # All tests
pytest tests/unit/ -v               # Unit tests only
pytest tests/unit/test_chat_lc.py   # Single file
pytest --cov=app tests/             # With coverage report
```

**Key Fixtures** (in `conftest.py`):

- `test_client` - FastAPI TestClient
- `mock_site_config` - Test site configuration
- `mock_openai_client` - Mock OpenAI for unit tests
- `mock_langchain_chat_model` - Mock LangChain model

Unit tests mock external services (OpenAI, Anthropic) to run without API keys.

### Angular Tests (`/chatbot`)

```bash
cd chatbot
npm test                    # Run Karma tests for all projects
ng test tsai                # Test shared library only
ng test creator             # Test creator app only
```