spider/docs/plans/2026-04-14-operations-features.md

Operations Features Implementation Plan

For agentic workers: Use superpowers:subagent-driven-development to implement tasks in parallel where possible.

Goal: Transform the spider system from a development tool into a production-ready operations platform with dashboard, scheduled tasks, merchant editing, status workflow, and follow-up notes.

Architecture: 5 independent features, each adds a model (if needed), handler, store methods, and frontend page. All share existing auth middleware and router patterns.

Tech Stack: Go/Gin backend, React/Ant Design frontend, MySQL via GORM, Redis for scheduling.

---

Feature 1: Dashboard (数据看板)

Backend

• Create: internal/handler/dashboard.go — single endpoint GET /dashboard returning all stats
• Modify: internal/handler/router.go — add dashboard route

Frontend

• Create: web/src/pages/Dashboard.tsx — stats cards, level pie chart, trend chart, recent tasks
• Modify: web/src/App.tsx — add route
• Modify: web/src/components/Layout.tsx — add menu item, make it the default landing page
• Modify: web/src/api/index.ts — add getDashboard

API: GET /api/v1/dashboard

Response:

{
  "raw_total": 1035,
  "clean_total": 535,
  "valid_total": 423,
  "by_level": {"Hot": 3, "Warm": 28, "Cold": 392},
  "by_source": {"web": 1035, "tg_channel": 0, "github": 0},
  "by_industry": {"机场": 12, "VPN": 3},
  "today_added": 50,
  "week_added": 200,
  "recent_tasks": [...last 5 tasks...],
  "daily_trend": [{"date": "2026-04-07", "count": 30}, ...]
}

---

Feature 2: Scheduled Tasks (定时采集)

Backend

• Create: internal/model/schedule.go — ScheduleJob model
• Create: internal/task/scheduler.go — cron scheduler using goroutine + ticker
• Create: internal/handler/schedule.go — CRUD endpoints
• Modify: internal/handler/router.go — add schedule routes
• Modify: cmd/server/main.go — start scheduler

Model: schedule_jobs

type ScheduleJob struct {
    ID         uint   `gorm:"primaryKey"`
    Name       string `gorm:"size:100;not null"`
    PluginName string `gorm:"size:100;not null"` // web_collector / tg_collector / github_collector / clean
    CronExpr   string `gorm:"size:50;not null"`  // "0 2 * * *" = every day 2am
    Enabled    bool   `gorm:"default:true"`
    LastRunAt  *time.Time
    NextRunAt  *time.Time
    CreatedAt  time.Time
}

Frontend

• Create: web/src/pages/Schedules.tsx — list schedules, add/edit/delete/toggle
• Modify: web/src/App.tsx, Layout.tsx, api/index.ts

---

Feature 3: Merchant Editing (商户编辑)

Backend

• Create: internal/handler/merchant_edit.go — PUT /merchants/clean/:id for editing
• Modify: internal/handler/router.go — add edit route

Frontend

• Modify: web/src/pages/MerchantsClean.tsx — add edit button in action column, edit modal with form

Editable fields: merchant_name, industry_tag, website, email, phone, remark (new field)

Model change

• Modify: internal/model/merchant_clean.go — add Remark field
• Modify: cmd/server/main.go — AutoMigrate picks it up

---

Feature 4: Status Workflow (状态流转)

Backend

• Modify: internal/model/merchant_clean.go — add FollowStatus field
• Create: internal/handler/merchant_status.go — PUT /merchants/clean/:id/status
• Modify: internal/handler/router.go — add route

FollowStatus values: pending → contacted → cooperating → rejected

Display labels: 待跟进 → 已联系 → 已合作 → 已拒绝

Frontend

• Modify: web/src/pages/MerchantsClean.tsx — add follow_status column with dropdown, filter by follow_status

---

Feature 5: Follow-up Notes (跟进记录)

Backend

• Create: internal/model/merchant_note.go — MerchantNote model
• Create: internal/handler/merchant_note.go — GET/POST notes per merchant
• Modify: internal/handler/router.go — add routes
• Modify: cmd/server/main.go — AutoMigrate

Model: merchant_notes

type MerchantNote struct {
    ID           uint   `gorm:"primaryKey"`
    MerchantID   uint   `gorm:"index;not null"`
    TgUsername   string  `gorm:"size:255;index"`
    Content      string `gorm:"type:text;not null"`
    CreatedBy    string `gorm:"size:50"`   // username of the author
    CreatedAt    time.Time
}

Frontend

• Modify: web/src/pages/MerchantsClean.tsx — add notes section in detail modal