development.md

Development Guide

Prerequisites

• docker desktop + docker kubernetes
• asdf-vm - easily manage third party tools (or manually install things listed in .tool-versions).
• tilt - local development orchestrator (installed via asdf).

Tilt

Tilt is the local development orchestrator. The Tiltfile configures services, dependencies and build steps.

Linters

Be sure to install the pre-commit hooks which run various linters, formatters, and tests.

just setup

A few of the linters used:

• golangci-lint - golang linters (config in .golangci.yml)
• eslint - typescript linters
• conventional commits - enforced by pre-commit hooks

Check the github actions and .pre-commit-config.yaml for the full list.

Local Debugging

I like to debug locally in VScode. For example, to debug RPC, first turn off theRPC service in Tilt. Next, run the rpc launch profile. The .env file defines environment variables that match the port forwards in Tilt.

example .vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "graph",
      "type": "go",
      "request": "launch",
      "mode": "debug",
      "program": "graph/main.go",
      "envFile": "${workspaceFolder}/.env",
      "args": ["server"]
    },
    {
      "name": "rpc",
      "type": "go",
      "request": "launch",
      "mode": "debug",
      "program": "rpc/main.go",
      "env": {
        "PORT": "50055",
        "METRIC_ADDRESS": ":9091",
        "DB_ADAPTER": "sqlite"
      },
      "envFile": "${workspaceFolder}/.env",
      "args": ["server"]
    },
    {
      "name": "temporal - fetch feed - worker",
      "type": "go",
      "request": "launch",
      "mode": "debug",
      "program": "data-pipeline/temporal/feed_fetch/worker/main.go",
      "envFile": "${workspaceFolder}/.env"
    },
    {
      "name": "temporal - generate feed - worker",
      "type": "go",
      "request": "launch",
      "mode": "debug",
      "program": "data-pipeline/temporal/generate/worker/main.go",
      "envFile": "${workspaceFolder}/.env"
    },
    {
      "name": "ui: debug client-side (Firefox)",
      "type": "firefox",
      "request": "launch",
      "url": "http://localhost:3000",
      "reAttach": true,
      "pathMappings": [
        {
          "url": "webpack://_n_e",
          "path": "${workspaceFolder}/ui"
        },
        {
          "url": "webpack://_n_e/node_modules/",
          "path": "${workspaceFolder}/ui/node_modules"
        }
      ]
    }
  ]
}

example .vscode/settings.json:

{
  // go specific
  "go.lintOnSave": "workspace",
  "go.testEnvFile": "${workspaceFolder}/.env"
    "eslint.validate": [
    "javascript",
    "typescript",
    "typescriptreact",
    "javascriptreact"
  ],
  "[go]": {
    "editor.formatOnSave": true,
    "editor.defaultFormatter": "golang.go",
    "editor.codeActionsOnSave": {
      "source.organizeImports": "explicit"
    }
  },

  // typescript specific
  "eslint.format.enable": true,
  "[typescript]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode",
    "editor.tabSize": 2,
    "editor.formatOnPaste": true,
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
      "source.fixAll": "explicit",
      "organizeImports": "explicit"
    }
  },
  "[typescriptreact]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode",
    "editor.tabSize": 2,
    "editor.formatOnPaste": true,
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
      "source.fixAll": "explicit",
      "organizeImports": "explicit"
    }
  }
}

example .env:

# FAKE_HOST=localhost:8084
FAKE_HOST=faker:8080
GRAPH_HOST="http://localhost:8082/query"
GRAPH_PORT=8082
RPC_HOST=localhost:50055
RPC_INSECURE=true
TEMPORAL_HOST=localhost:7233
MINIO_ENDPOINT=localhost:9100
MINIO_USE_SSL=false
MINIO_ACCESS_KEY=minio
MINIO_SECRET_ACCESS_KEY=minio-password
OTEL_ENABLE=true
OTEL_SERVICE_NAME=feed-worker
OTEL_EXPORTER_OTLP_INSECURE=true
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
USE_SCHEDULE=false
FEED_SCHEDULE_ID=feed-worker
FEED_WORKFLOW_ID=feed-worker
CORS_ALLOW_ORIGINS=http://localhost:3000
CORS_ALLOW_METHODS=GET,POST,PUT
CORS_ALLOW_HEADERS=Content-Type,Authorization,Origin
CORS_EXPOSE_HEADERS=Content-Length