Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Local Development Guide

This guide covers configuration details and troubleshooting for local development. For the quickstart, see the root README.

Prerequisites

  • Operating System: macOS, Linux, or Windows with WSL2
  • Node.js 24 (use nvm install && nvm use so local matches .nvmrc)
  • npm (included with Node.js)
  • Docker and Docker Compose
  • 16 GB RAM or more recommended

Environment Setup

Server Configuration

Copy server/.env.example to server/.env. The example file contains all available options with documentation. Key sections:

  • Database connections: PostgreSQL, ClickHouse, ScyllaDB, Redis
  • Kafka: Broker and schema registry settings
  • External APIs: OpenAI, SendGrid, Google APIs (optional)
  • Security: Session secrets, JWT signing keys

The default values work with Docker Compose services out of the box.

Client Configuration

Copy client/.env.example to client/.env. Defaults work for local development.

Docker Services

Start all backing services:

npm run up

This starts:

ServicePortNotes
PostgreSQL5432Primary DB (with pgvector)
ClickHouse8123, 9000Analytics warehouse
ScyllaDB9042Item submission history
Redis6379Caching and job queues
Kafka29092Event streaming
Schema Registry8081Kafka schemas
Zookeeper22181Kafka coordination
HMA5000Hash Matching Algorithm service
Jaeger16686Tracing UI (opens automatically)
OTEL Collector4317Telemetry collection

Check service health:

docker ps
docker logs <container-name>

Stop services:

npm run down

Database Operations

Running Migrations

npm run db:update -- --env staging --db api-server-pg
npm run db:update -- --env staging --db clickhouse
#Creating keyspace
npm run db:create -- --env staging --db scylla
#Running migrations
npm run db:update -- --env staging --db scylla

Other Commands

npm run db:add -- --name <migration-name> --db api-server-pg
npm run db:clean    # Drop and recreate (destructive)
npm run db:create   # Create database
npm run db:drop     # Drop database

Migration Locations

.devops/migrator/src/scripts/
├── api-server-pg/    # PostgreSQL
├── clickhouse/       # ClickHouse
├── scylla/           # ScyllaDB
└── snowflake/        # Snowflake (optional)

Running the Application

All Services Together

npm run start       # Client + server + GraphQL codegen (opens browser)
npm run compile     # Same, without opening browser
# Terminal 1
npm run client:start

# Terminal 2
npm run server:start

# Terminal 3 (optional, for GraphQL schema changes)
npm run generate:watch

With Distributed Tracing

cd server && npm run start:trace

View traces at http://localhost:16686

Access Points

ServiceURL
Clienthttp://localhost:3000
API Serverhttp://localhost:8080
GraphQLhttp://localhost:8080/graphql
Jaeger UIhttp://localhost:16686

Testing

# Server
cd server
npm run test              # Watch mode
npm run test:prepush      # Single run
npm run test:integ        # Integration tests

# Client
cd client
npm run test              # Watch mode
npm run test:prepush      # Single run

# Full validation (run before pushing)
npm run check:prepush

GraphQL Development

Coop uses schema-first GraphQL with bidirectional code generation.

npm run generate          # One-time
npm run generate:watch    # Watch mode

Generated files:

  • client/src/graphql/generated.ts
  • server/graphql/generated.ts

Schema changes trigger recompilation of both client and server. If you experience regeneration loops, stop watch mode and run manually.

Troubleshooting

ScyllaDB Not Ready

ScyllaDB takes 30-60 seconds to initialize. If migrations fail immediately after npm run up, wait and retry.

ClickHouse Migration Fails

Ensure CLICKHOUSE_USERNAME and CLICKHOUSE_PASSWORD are set in your .env.

Port Conflicts

lsof -i :3000    # Client
lsof -i :8080    # Server
lsof -i :5432    # PostgreSQL

Reset Everything

npm run down
docker volume prune    # Warning: removes all Docker volumes
npm run up
npm run db:update -- --env staging --db api-server-pg
npm run db:update -- --env staging --db clickhouse
npm run create-org

Connecting to Databases Directly

# PostgreSQL
psql -h localhost -U postgres -d postgres
# Password: postgres123

# ClickHouse
clickhouse-client --host localhost --user default --password clickhouse

# Redis
redis-cli

Code Quality

npm run lint       # ESLint
npm run format     # Prettier
npm run check:prepush    # Run before pushing