init

.dockerignore

@@ -0,0 +1,68 @@
+# 排除不需要复制到Docker镜像的文件和目录
+
+# Node相关
+node_modules
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# 系统文件
+.DS_Store
+Thumbs.db
+
+# IDE和编辑器
+.vscode
+.idea
+*.swp
+*.swo
+
+# Git
+.git
+.gitignore
+
+# 日志文件
+logs
+*.log
+
+# 临时文件
+tmp
+temp
+
+# Docker相关
+Dockerfile
+.dockerignore
+docker-compose.yml
+docker-compose.override.yml
+
+# 文档
+README.md
+*.md
+
+# 测试
+test
+tests
+__tests__
+coverage
+.nyc_output
+
+# 缓存
+.cache
+.npm
+
+# 环境变量文件
+.env
+.env.local
+.env.development
+.env.test
+.env.production
+
+# 数据库文件（通常在容器内创建）
+*.db
+*.sqlite
+*.sqlite3
+data/
+
+# 其他
+*.pid
+*.seed
+*.pid.lock

.gitignore

@@ -0,0 +1,4 @@
+node_modules
+proxies.db
+logs
+data

DOCKER_README.md

@@ -0,0 +1,227 @@
+# 代理IP管理系统 Docker 部署指南
+
+## 🐳 Docker 快速开始
+
+### 方式一：使用 Docker Compose（推荐）
+
+1. **克隆项目**
+```bash
+git clone <your-repo-url>
+cd dailiip
+```
+
+2. **构建并启动容器**
+```bash
+docker-compose up -d
+```
+
+3. **查看日志**
+```bash
+docker-compose logs -f proxy-manager
+```
+
+4. **访问应用**
+- 浏览器访问：http://localhost:3000
+- API地址：http://localhost:3000/api
+
+### 方式二：使用 Docker 命令
+
+1. **构建镜像**
+```bash
+docker build -t proxy-ip-manager .
+```
+
+2. **运行容器**
+```bash
+# 基本运行
+docker run -d -p 3000:3000 --name proxy-manager proxy-ip-manager
+
+# 带数据持久化
+docker run -d \
+  -p 3000:3000 \
+  -v $(pwd)/data:/app/data \
+  -v $(pwd)/logs:/app/logs \
+  --name proxy-manager \
+  --restart unless-stopped \
+  proxy-ip-manager
+```
+
+## 📁 目录结构
+
+```
+dailiip/
+├── Dockerfile              # Docker构建文件
+├── docker-compose.yml      # Docker Compose配置
+├── .dockerignore           # Docker忽略文件
+├── src/                    # 源代码
+├── public/                 # 静态文件
+├── data/                   # 数据库文件（运行时创建）
+└── logs/                   # 日志文件（运行时创建）
+```
+
+## ⚙️ 环境变量
+
+| 变量名 | 默认值 | 说明 |
+|--------|--------|------|
+| NODE_ENV | production | 运行环境 |
+| PORT | 3000 | 应用端口 |
+
+## 📊 数据持久化
+
+### Docker Compose 方式
+数据库文件会自动保存到 `./data` 目录，日志保存到 `./logs` 目录。
+
+### Docker 命令方式
+使用 `-v` 参数挂载本地目录：
+
+```bash
+-v /path/to/your/data:/app/data
+-v /path/to/your/logs:/app/logs
+```
+
+## 🔧 常用命令
+
+### Docker Compose
+```bash
+# 启动服务
+docker-compose up -d
+
+# 停止服务
+docker-compose down
+
+# 查看日志
+docker-compose logs -f
+
+# 重新构建
+docker-compose up -d --build
+
+# 进入容器
+docker-compose exec proxy-manager sh
+```
+
+### Docker 命令
+```bash
+# 查看运行状态
+docker ps
+
+# 查看日志
+docker logs proxy-manager
+
+# 进入容器
+docker exec -it proxy-manager sh
+
+# 停止容器
+docker stop proxy-manager
+
+# 启动容器
+docker start proxy-manager
+
+# 删除容器
+docker rm proxy-manager
+```
+
+## 🔍 健康检查
+
+容器包含内置的健康检查，每30秒检查一次：
+
+```bash
+# 查看健康状态
+docker inspect proxy-manager | grep Health -A 10
+
+# 查看健康检查日志
+docker inspect proxy-manager | grep Log -A 5
+```
+
+## 🚨 故障排除
+
+### 1. 容器无法启动
+```bash
+# 查看详细错误日志
+docker-compose logs proxy-manager
+
+# 检查端口是否被占用
+netstat -tlnp | grep 3000
+```
+
+### 2. 数据库问题
+```bash
+# 检查数据目录权限
+ls -la data/
+
+# 重新创建数据目录
+sudo rm -rf data logs
+mkdir -p data logs
+sudo chown -R 1000:1000 data logs
+```
+
+### 3. 性能优化
+```bash
+# 限制内存使用
+docker run -d --memory=512m proxy-ip-manager
+
+# 设置CPU限制
+docker run -d --cpus=0.5 proxy-ip-manager
+```
+
+## 📈 监控
+
+### 使用健康检查API
+```bash
+curl http://localhost:3000/api/health
+```
+
+### 查看资源使用情况
+```bash
+# 查看容器资源使用
+docker stats proxy-manager
+
+# 查看磁盘使用
+docker system df
+```
+
+## 🔄 更新应用
+
+```bash
+# 拉取最新代码
+git pull
+
+# 重新构建并启动
+docker-compose up -d --build
+```
+
+## 🐛 调试模式
+
+如需调试，可以临时修改环境变量：
+
+```bash
+# 修改 docker-compose.yml
+environment:
+  - NODE_ENV=development
+  - DEBUG=proxy:*
+
+# 或使用 Docker 命令
+docker run -d -e NODE_ENV=development -e DEBUG=proxy:* proxy-ip-manager
+```
+
+## 🔒 安全建议
+
+1. **使用非root用户运行**：Dockerfile中已配置
+2. **限制网络访问**：根据需要配置防火墙
+3. **定期更新镜像**：
+   ```bash
+   docker pull node:24.11.0-alpine
+   docker-compose up -d --build
+   ```
+4. **备份数据**：
+   ```bash
+   # 备份数据库
+   tar -czf backup-$(date +%Y%m%d).tar.gz data/ logs/
+   ```
+
+## 📞 支持
+
+如遇问题，请检查：
+1. Docker和Docker Compose版本是否兼容
+2. 端口3000是否被占用
+3. 磁盘空间是否充足
+4. 查看容器日志获取详细错误信息

Dockerfile

@@ -0,0 +1,43 @@
+# 使用官方Node.js运行时作为基础镜像
+FROM node:24.11.0-alpine
+
+# 设置工作目录
+WORKDIR /app
+
+# 设置环境变量
+ENV NODE_ENV=production
+ENV PORT=3000
+
+# 安装系统依赖（用于数据库编译）
+RUN apk add --no-cache \
+    sqlite \
+    sqlite-dev \
+    python3 \
+    make \
+    g++
+
+# 复制package.json和package-lock.json（如果存在）
+COPY package*.json ./
+
+# 安装项目依赖
+RUN npm ci --only=production && npm cache clean --force
+
+# 复制项目文件
+COPY . .
+
+# 创建数据目录并设置权限
+RUN mkdir -p /app/data && \
+    chown -R node:node /app
+
+# 切换到非root用户
+USER node
+
+# 暴露端口
+EXPOSE 3000
+
+# 健康检查
+HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+    CMD node -e "require('http').get('http://localhost:3000/api/health', (res) => { process.exit(res.statusCode === 200 ? 0 : 1) })"
+
+# 启动应用
+CMD ["npm", "start"]

README.md

@@ -0,0 +1,305 @@
+# 代理IP抓取和验证服务
+
+一个基于Node.js的代理IP自动抓取、验证和管理系统，提供RESTful API接口和定时任务调度。
+
+## 功能特性
+
+- 🕷️ **自动抓取**: 从快代理网站抓取免费代理IP
+- ✅ **自动验证**: 通过访问百度验证代理可用性
+- 🗄️ **数据存储**: 使用SQLite存储代理数据
+- 🔄 **定时任务**: 自动抓取和验证维护代理池
+- 🚀 **RESTful API**: 完整的API接口服务
+- 📊 **统计监控**: 实时代理池状态统计
+- 🛡️ **错误处理**: 完善的错误处理和重试机制
+
+## 项目结构
+
+```
+dailiip/
+├── src/
+│   ├── database/
+│   │   ├── db.js           # 数据库连接管理
+│   │   └── models/
+│   │       └── proxy.js    # 代理数据模型
+│   ├── services/
+│   │   ├── scraper.js      # 代理抓取服务
+│   │   ├── validator.js    # 代理验证服务
+│   │   └── scheduler.js    # 定时任务调度器
+│   ├── routes/
+│   │   └── proxies.js      # API路由
+│   └── app.js              # 应用入口
+├── config/
+│   └── database.js         # 数据库配置
+├── logs/                   # 日志目录
+├── proxies.db             # SQLite数据库
+├── package.json
+└── README.md
+```
+
+## 快速开始
+
+### 1. 安装依赖
+
+```bash
+npm install
+```
+
+### 2. 启动服务
+
+```bash
+npm start
+```
+
+服务将在 `http://localhost:3000` 启动
+
+### 3. 验证服务
+
+访问 `http://localhost:3000/health` 检查服务状态
+
+## API接口
+
+### 基础信息
+
+- **基础URL**: `http://localhost:3000`
+- **Content-Type**: `application/json`
+
+### 接口列表
+
+#### 1. 获取服务信息
+```
+GET /
+```
+
+#### 2. 健康检查
+```
+GET /health
+```
+
+#### 3. 获取代理列表
+```
+GET /api/proxies?limit=100&offset=0&sortBy=response_time&order=ASC
+```
+- `limit`: 限制数量 (默认100)
+- `offset`: 偏移量 (默认0)
+- `sortBy`: 排序字段 (response_time, created_at, updated_at, ip, port)
+- `order`: 排序方式 (ASC, DESC)
+
+#### 4. 获取随机代理
+```
+GET /api/proxies/random?count=1
+```
+- `count`: 返回数量 (1-10)
+
+#### 5. 获取统计信息
+```
+GET /api/proxies/stats
+```
+
+#### 6. 验证单个代理
+```
+POST /api/proxies/verify
+{
+  "ip": "192.168.1.1",
+  "port": 8080
+}
+```
+
+#### 7. 批量验证代理
+```
+POST /api/proxies/verify-batch
+{
+  "proxies": [
+    {"ip": "192.168.1.1", "port": 8080},
+    {"ip": "192.168.1.2", "port": 8080}
+  ]
+}
+```
+
+#### 8. 手动触发抓取
+```
+POST /api/proxies/scrape
+{
+  "pages": 5
+}
+```
+
+#### 9. 手动触发验证
+```
+POST /api/proxies/validate-all
+{
+  "limit": 50
+}
+```
+
+#### 10. 搜索代理
+```
+GET /api/proxies/search?ip=192.168.1.1&port=8080
+```
+
+#### 11. 清理无效代理
+```
+DELETE /api/proxies/cleanup
+```
+
+## 定时任务
+
+系统自动执行以下定时任务：
+
+### 抓取任务
+- **频率**: 每小时整点执行
+- **功能**: 抓取快代理网站前5页的代理IP
+- **网址**: https://www.kuaidaili.com/free/inha/
+
+### 验证任务
+- **频率**: 每10分钟执行一次
+- **功能**: 验证数据库中的代理IP可用性
+- **验证方式**: 通过访问 https://www.baidu.com
+- **超时时间**: 8秒
+
+### 健康检查
+- **频率**: 每小时30分执行
+- **功能**: 检查代理池健康状态
+- **应急处理**: 可用代理少于10个时触发紧急抓取
+
+## 数据库结构
+
+### proxies 表
+
+| 字段名 | 类型 | 说明 |
+|--------|------|------|
+| id | INTEGER | 主键，自增 |
+| ip | TEXT | IP地址 |
+| port | INTEGER | 端口号 |
+| location | TEXT | 地理位置 |
+| speed | INTEGER | 响应速度(毫秒) |
+| last_check_time | TEXT | 最后验证时间 |
+| is_valid | INTEGER | 是否可用 (1:可用, 0:不可用) |
+| response_time | INTEGER | 验证响应时间 |
+| created_at | DATETIME | 创建时间 |
+| updated_at | DATETIME | 更新时间 |
+
+**唯一约束**: (ip, port)
+
+## 使用示例
+
+### 获取随机代理
+
+```bash
+curl http://localhost:3000/api/proxies/random
+```
+
+响应:
+```json
+{
+  "success": true,
+  "data": {
+    "id": 1,
+    "ip": "101.132.134.160",
+    "port": 8888,
+    "location": "上海市",
+    "is_valid": 1,
+    "response_time": 150,
+    "created_at": "2025-10-30 12:00:00",
+    "updated_at": "2025-10-30 12:30:00"
+  }
+}
+```
+
+### 验证代理
+
+```bash
+curl -X POST http://localhost:3000/api/proxies/verify \
+  -H "Content-Type: application/json" \
+  -d '{"ip":"101.132.134.160","port":8888}'
+```
+
+响应:
+```json
+{
+  "success": true,
+  "data": {
+    "ip": "101.132.134.160",
+    "port": 8888,
+    "isValid": true,
+    "responseTime": 150,
+    "error": null
+  }
+}
+```
+
+### 获取统计信息
+
+```bash
+curl http://localhost:3000/api/proxies/stats
+```
+
+响应:
+```json
+{
+  "success": true,
+  "data": {
+    "total": 50,
+    "valid": 30,
+    "invalid": 20,
+    "validRate": "60.00%"
+  }
+}
+```
+
+## 配置说明
+
+### 环境变量
+
+- `PORT`: 服务端口 (默认: 3000)
+
+### 数据库配置
+
+数据库配置文件: `config/database.js`
+
+```javascript
+const dbConfig = {
+  development: {
+    dialect: 'sqlite',
+    storage: path.join(__dirname, '../proxies.db')
+  }
+};
+```
+
+## 日志和监控
+
+- 服务启动后会输出详细的执行日志
+- 通过 `/health` 接口可查看系统状态
+- 支持优雅关闭，保证数据完整性
+
+## 注意事项
+
+1. **合法使用**: 请遵守网站服务条款，合理使用代理IP
+2. **频率控制**: 内置请求延迟和重试机制，避免对目标网站造成压力
+3. **数据质量**: 免费代理IP质量参差不齐，建议仅用于测试和学习
+4. **资源占用**: 定时任务会占用一定的系统资源
+
+## 开发和部署
+
+### 开发模式
+```bash
+npm run dev
+```
+
+### 生产部署
+
+建议使用 PM2 进行进程管理:
+
+```bash
+npm install -g pm2
+pm2 start src/app.js --name "proxy-service"
+pm2 logs proxy-service
+pm2 status proxy-service
+```
+
+## 许可证
+
+MIT License
+
+## 贡献
+
+欢迎提交Issue和Pull Request！

config/database.js

@@ -0,0 +1,10 @@
+const path = require('path');
+
+const dbConfig = {
+  development: {
+    dialect: 'sqlite',
+    storage: path.join(__dirname, '../proxies.db')
+  }
+};
+
+module.exports = dbConfig;

docker-compose.yml

@@ -0,0 +1,27 @@
+version: '3.8'
+
+services:
+  proxy-manager:
+    build: .
+    container_name: proxy-ip-manager
+    ports:
+      - "3000:3000"
+    environment:
+      - NODE_ENV=production
+      - PORT=3000
+    volumes:
+      # 持久化数据库数据
+      - ./data:/app/data
+      # 持久化日志文件
+      - ./logs:/app/logs
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "node", "-e", "require('http').get('http://localhost:3000/api/health', (res) => { process.exit(res.statusCode === 200 ? 0 : 1) })"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+
+networks:
+  default:
+    name: proxy-manager-network

package.json

@@ -0,0 +1,22 @@
+{
+  "name": "dailiip",
+  "version": "1.0.0",
+  "description": "代理IP抓取、验证和管理服务",
+  "main": "src/app.js",
+  "scripts": {
+    "start": "node src/app.js",
+    "dev": "node src/app.js",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "author": "",
+  "license": "ISC",
+  "keywords": ["proxy", "ip", "scraper", "validator", "api"],
+  "dependencies": {
+    "axios": "^1.13.1",
+    "cheerio": "^1.1.2",
+    "express": "^5.1.0",
+    "node-cron": "^4.2.1",
+    "sqlite3": "^5.1.7",
+    "user-agent": "^1.0.4"
+  }
+}

public/css/dashboard.css

@@ -0,0 +1,363 @@
+/* 仪表板样式 */
+
+body {
+    background-color: #f8f9fa;
+    font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
+}
+
+/* 导航栏样式 */
+.navbar-brand {
+    font-weight: bold;
+    font-size: 1.2rem;
+}
+
+.nav-link.active {
+    font-weight: bold;
+    background-color: rgba(255, 255, 255, 0.1);
+    border-radius: 0.25rem;
+}
+
+/* 卡片样式 */
+.card {
+    border: none;
+    box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);
+    transition: box-shadow 0.15s ease-in-out;
+}
+
+.card:hover {
+    box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
+}
+
+.card-header {
+    background-color: #fff;
+    border-bottom: 1px solid #dee2e6;
+    font-weight: 600;
+}
+
+/* 状态卡片动画 */
+.card-body h3 {
+    transition: transform 0.3s ease;
+}
+
+.card:hover h3 {
+    transform: scale(1.05);
+}
+
+/* 按钮样式 */
+.btn {
+    border-radius: 0.375rem;
+    font-weight: 500;
+    transition: all 0.15s ease-in-out;
+}
+
+.btn:hover {
+    transform: translateY(-1px);
+    box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.15);
+}
+
+.btn:active {
+    transform: translateY(0);
+}
+
+/* 表格样式 */
+.table {
+    font-size: 0.875rem;
+}
+
+.table th {
+    border-top: none;
+    font-weight: 600;
+    color: #495057;
+    background-color: #f8f9fa;
+}
+
+.table td {
+    vertical-align: middle;
+}
+
+/* 状态徽章 */
+.status-badge {
+    font-size: 0.75rem;
+    padding: 0.25rem 0.5rem;
+    border-radius: 1rem;
+    font-weight: 500;
+}
+
+.status-success {
+    background-color: #d1e7dd;
+    color: #0f5132;
+}
+
+.status-warning {
+    background-color: #fff3cd;
+    color: #664d03;
+}
+
+.status-danger {
+    background-color: #f8d7da;
+    color: #842029;
+}
+
+.status-info {
+    background-color: #cff4fc;
+    color: #055160;
+}
+
+/* 加载动画 */
+.spinner-border-sm {
+    width: 1rem;
+    height: 1rem;
+}
+
+/* 提示消息样式 */
+.alert {
+    border: none;
+    border-radius: 0.5rem;
+    box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
+    animation: slideIn 0.3s ease-out;
+}
+
+@keyframes slideIn {
+    from {
+        transform: translateX(100%);
+        opacity: 0;
+    }
+    to {
+        transform: translateX(0);
+        opacity: 1;
+    }
+}
+
+/* 图表容器 */
+.chart-container {
+    position: relative;
+    height: 300px;
+}
+
+/* 时间显示 */
+#currentTime {
+    font-family: 'Courier New', monospace;
+    font-size: 0.9rem;
+}
+
+/* 系统信息卡片 */
+.system-info-item {
+    padding: 0.5rem 0;
+    border-bottom: 1px solid #dee2e6;
+}
+
+.system-info-item:last-child {
+    border-bottom: none;
+}
+
+.system-info-label {
+    color: #6c757d;
+    font-size: 0.875rem;
+    font-weight: 500;
+}
+
+.system-info-value {
+    color: #212529;
+    font-weight: 600;
+}
+
+/* 响应式设计 */
+@media (max-width: 768px) {
+    .container-fluid {
+        padding-left: 0.5rem;
+        padding-right: 0.5rem;
+    }
+
+    .card-body {
+        padding: 1rem;
+    }
+
+    .btn {
+        font-size: 0.875rem;
+        padding: 0.5rem 1rem;
+    }
+
+    .table {
+        font-size: 0.8rem;
+    }
+
+    h3 {
+        font-size: 1.5rem !important;
+    }
+}
+
+/* 历史记录表格 */
+.history-table {
+    font-size: 0.8rem;
+}
+
+.history-table .task-name {
+    font-weight: 500;
+    color: #495057;
+}
+
+.history-table .status-badge {
+    min-width: 60px;
+    text-align: center;
+}
+
+.history-table .duration {
+    font-family: 'Courier New', monospace;
+    color: #6c757d;
+}
+
+/* 快速操作面板 */
+.quick-actions .btn {
+    margin-bottom: 0.5rem;
+}
+
+.quick-actions .btn i {
+    margin-right: 0.5rem;
+}
+
+/* 下次执行时间 */
+.next-run-time {
+    font-size: 0.8rem;
+    color: #6c757d;
+    font-family: 'Courier New', monospace;
+}
+
+/* 系统状态指示器 */
+.status-indicator {
+    display: inline-block;
+    width: 8px;
+    height: 8px;
+    border-radius: 50%;
+    margin-right: 0.5rem;
+}
+
+.status-indicator.online {
+    background-color: #198754;
+    animation: pulse 2s infinite;
+}
+
+.status-indicator.offline {
+    background-color: #dc3545;
+}
+
+@keyframes pulse {
+    0% {
+        box-shadow: 0 0 0 0 rgba(25, 135, 84, 0.7);
+    }
+    70% {
+        box-shadow: 0 0 0 10px rgba(25, 135, 84, 0);
+    }
+    100% {
+        box-shadow: 0 0 0 0 rgba(25, 135, 84, 0);
+    }
+}
+
+/* 内存使用条 */
+.memory-bar {
+    height: 4px;
+    background-color: #e9ecef;
+    border-radius: 2px;
+    overflow: hidden;
+    margin-top: 0.25rem;
+}
+
+.memory-bar-fill {
+    height: 100%;
+    background-color: #198754;
+    transition: width 0.3s ease;
+}
+
+/* 工具提示样式 */
+.tooltip-inner {
+    font-size: 0.875rem;
+    border-radius: 0.375rem;
+}
+
+/* 页面加载动画 */
+.page-loading {
+    position: fixed;
+    top: 0;
+    left: 0;
+    width: 100%;
+    height: 100%;
+    background-color: rgba(255, 255, 255, 0.8);
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    z-index: 9999;
+}
+
+.page-loading .spinner-border {
+    width: 3rem;
+    height: 3rem;
+}
+
+/* 数据表格空状态 */
+.empty-state {
+    text-align: center;
+    padding: 2rem;
+    color: #6c757d;
+}
+
+.empty-state i {
+    font-size: 3rem;
+    margin-bottom: 1rem;
+    opacity: 0.5;
+}
+
+.empty-state h6 {
+    margin-bottom: 0.5rem;
+    color: #495057;
+}
+
+/* 悬浮操作按钮 */
+.fab {
+    position: fixed;
+    bottom: 2rem;
+    right: 2rem;
+    width: 3rem;
+    height: 3rem;
+    border-radius: 50%;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);
+    z-index: 1000;
+    transition: all 0.3s ease;
+}
+
+.fab:hover {
+    transform: scale(1.1);
+    box-shadow: 0 1rem 2rem rgba(0, 0, 0, 0.2);
+}
+
+/* 深色模式支持 */
+@media (prefers-color-scheme: dark) {
+    body {
+        background-color: #212529;
+        color: #fff;
+    }
+
+    .card {
+        background-color: #343a40;
+        color: #fff;
+    }
+
+    .card-header {
+        background-color: #495057;
+        border-bottom-color: #6c757d;
+    }
+
+    .table {
+        color: #fff;
+    }
+
+    .table th {
+        background-color: #495057;
+        border-color: #6c757d;
+    }
+
+    .navbar-dark .navbar-text {
+        color: rgba(255, 255, 255, 0.75);
+    }
+}

public/history.html

@@ -0,0 +1,395 @@
+<!DOCTYPE html>
+<html lang="zh-CN">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>代理IP管理系统 - 执行历史</title>
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.7.2/font/bootstrap-icons.css" rel="stylesheet">
+    <link href="css/dashboard.css" rel="stylesheet">
+</head>
+<body>
+    <!-- 导航栏 -->
+    <nav class="navbar navbar-expand-lg navbar-dark bg-primary">
+        <div class="container-fluid">
+            <a class="navbar-brand" href="index.html">
+                <i class="bi bi-shield-check"></i>
+                代理IP管理系统
+            </a>
+
+            <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
+                <span class="navbar-toggler-icon"></span>
+            </button>
+
+            <div class="collapse navbar-collapse" id="navbarNav">
+                <ul class="navbar-nav me-auto">
+                    <li class="nav-item">
+                        <a class="nav-link" href="index.html">
+                            <i class="bi bi-speedometer2"></i> 仪表板
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="proxies.html">
+                            <i class="bi bi-list-ul"></i> 代理管理
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link active" href="history.html">
+                            <i class="bi bi-clock-history"></i> 执行历史
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="monitoring.html">
+                            <i class="bi bi-activity"></i> 系统监控
+                        </a>
+                    </li>
+                </ul>
+
+                <div class="navbar-nav">
+                    <button class="btn btn-outline-light btn-sm" onclick="refreshHistory()">
+                        <i class="bi bi-arrow-clockwise"></i> 刷新
+                    </button>
+                </div>
+            </div>
+        </div>
+    </nav>
+
+    <!-- 主要内容 -->
+    <div class="container-fluid mt-4">
+        <!-- 统计卡片 -->
+        <div class="row mb-4">
+            <div class="col-md-3 mb-3">
+                <div class="card bg-primary text-white">
+                    <div class="card-body text-center">
+                        <h4 id="totalTasks">-</h4>
+                        <p class="mb-0">总任务数</p>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card bg-success text-white">
+                    <div class="card-body text-center">
+                        <h4 id="successTasks">-</h4>
+                        <p class="mb-0">成功任务</p>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card bg-danger text-white">
+                    <div class="card-body text-center">
+                        <h4 id="failedTasks">-</h4>
+                        <p class="mb-0">失败任务</p>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card bg-info text-white">
+                    <div class="card-body text-center">
+                        <h4 id="successRate">-</h4>
+                        <p class="mb-0">成功率</p>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 选项卡导航 -->
+        <ul class="nav nav-tabs" id="historyTabs" role="tablist">
+            <li class="nav-item" role="presentation">
+                <button class="nav-link active" id="task-tab" data-bs-toggle="tab" data-bs-target="#task-pane" type="button" role="tab">
+                    <i class="bi bi-list-task"></i> 任务历史
+                </button>
+            </li>
+            <li class="nav-item" role="presentation">
+                <button class="nav-link" id="logs-tab" data-bs-toggle="tab" data-bs-target="#logs-pane" type="button" role="tab">
+                    <i class="bi bi-file-text"></i> 系统日志
+                </button>
+            </li>
+            <li class="nav-item" role="presentation">
+                <button class="nav-link" id="stats-tab" data-bs-toggle="tab" data-bs-target="#stats-pane" type="button" role="tab">
+                    <i class="bi bi-graph-up"></i> 统计分析
+                </button>
+            </li>
+        </ul>
+
+        <!-- 选项卡内容 -->
+        <div class="tab-content" id="historyTabContent">
+            <!-- 任务历史选项卡 -->
+            <div class="tab-pane fade show active" id="task-pane" role="tabpanel">
+                <!-- 筛选器 -->
+                <div class="card mt-3">
+                    <div class="card-body">
+                        <form id="taskFilterForm" class="row g-3">
+                            <div class="col-md-3">
+                                <label class="form-label">任务类型</label>
+                                <select class="form-select" id="taskTypeFilter">
+                                    <option value="">全部类型</option>
+                                    <option value="scrape">抓取任务</option>
+                                    <option value="validation">验证任务</option>
+                                </select>
+                            </div>
+                            <div class="col-md-3">
+                                <label class="form-label">执行状态</label>
+                                <select class="form-select" id="taskStatusFilter">
+                                    <option value="">全部状态</option>
+                                    <option value="success">成功</option>
+                                    <option value="failed">失败</option>
+                                    <option value="running">运行中</option>
+                                    <option value="pending">等待中</option>
+                                </select>
+                            </div>
+                            <div class="col-md-4">
+                                <label class="form-label">日期范围</label>
+                                <div class="input-group">
+                                    <input type="date" class="form-control" id="startDate">
+                                    <span class="input-group-text">至</span>
+                                    <input type="date" class="form-control" id="endDate">
+                                </div>
+                            </div>
+                            <div class="col-md-2">
+                                <label class="form-label">&nbsp;</label>
+                                <div class="d-grid">
+                                    <button type="submit" class="btn btn-primary">
+                                        <i class="bi bi-funnel"></i> 筛选
+                                    </button>
+                                </div>
+                            </div>
+                        </form>
+                    </div>
+                </div>
+
+                <!-- 任务历史表格 -->
+                <div class="card mt-3">
+                    <div class="card-header d-flex justify-content-between align-items-center">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-clock-history"></i> 任务执行历史
+                        </h5>
+                        <div>
+                            <button class="btn btn-outline-secondary btn-sm" onclick="exportTaskHistory()">
+                                <i class="bi bi-download"></i> 导出
+                            </button>
+                        </div>
+                    </div>
+                    <div class="card-body">
+                        <div class="table-responsive">
+                            <table class="table table-hover">
+                                <thead>
+                                    <tr>
+                                        <th>任务类型</th>
+                                        <th>任务名称</th>
+                                        <th>状态</th>
+                                        <th>开始时间</th>
+                                        <th>结束时间</th>
+                                        <th>执行时长</th>
+                                        <th>结果摘要</th>
+                                        <th>操作</th>
+                                    </tr>
+                                </thead>
+                                <tbody id="taskHistoryTableBody">
+                                    <tr>
+                                        <td colspan="8" class="text-center p-4">
+                                            <div class="spinner-border text-primary" role="status">
+                                                <span class="visually-hidden">加载中...</span>
+                                            </div>
+                                            <p class="mt-2">正在加载任务历史...</p>
+                                        </td>
+                                    </tr>
+                                </tbody>
+                            </table>
+                        </div>
+
+                        <!-- 分页 -->
+                        <nav aria-label="任务历史分页">
+                            <ul class="pagination justify-content-center" id="taskPagination">
+                                <!-- 分页按钮将通过JavaScript动态生成 -->
+                            </ul>
+                        </nav>
+                    </div>
+                </div>
+            </div>
+
+            <!-- 系统日志选项卡 -->
+            <div class="tab-pane fade" id="logs-pane" role="tabpanel">
+                <!-- 日志筛选器 -->
+                <div class="card mt-3">
+                    <div class="card-body">
+                        <form id="logFilterForm" class="row g-3">
+                            <div class="col-md-2">
+                                <label class="form-label">日志级别</label>
+                                <select class="form-select" id="logLevelFilter">
+                                    <option value="">全部级别</option>
+                                    <option value="error">错误</option>
+                                    <option value="warning">警告</option>
+                                    <option value="info">信息</option>
+                                    <option value="debug">调试</option>
+                                </select>
+                            </div>
+                            <div class="col-md-2">
+                                <label class="form-label">分类</label>
+                                <select class="form-select" id="logCategoryFilter">
+                                    <option value="">全部分类</option>
+                                    <option value="system">系统</option>
+                                    <option value="scheduler">调度器</option>
+                                    <option value="manual_action">手动操作</option>
+                                </select>
+                            </div>
+                            <div class="col-md-4">
+                                <label class="form-label">搜索关键词</label>
+                                <input type="text" class="form-control" id="logSearchKeyword" placeholder="搜索日志内容...">
+                            </div>
+                            <div class="col-md-2">
+                                <label class="form-label">&nbsp;</label>
+                                <div class="d-grid">
+                                    <button type="submit" class="btn btn-primary">
+                                        <i class="bi bi-search"></i> 搜索
+                                    </button>
+                                </div>
+                            </div>
+                            <div class="col-md-2">
+                                <label class="form-label">&nbsp;</label>
+                                <div class="d-grid">
+                                    <button type="button" class="btn btn-outline-warning" onclick="cleanupLogs()">
+                                        <i class="bi bi-trash"></i> 清理
+                                    </button>
+                                </div>
+                            </div>
+                        </form>
+                    </div>
+                </div>
+
+                <!-- 系统日志表格 -->
+                <div class="card mt-3">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-file-text"></i> 系统日志
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <div class="table-responsive">
+                            <table class="table table-sm">
+                                <thead>
+                                    <tr>
+                                        <th>时间</th>
+                                        <th>级别</th>
+                                        <th>分类</th>
+                                        <th>消息</th>
+                                        <th>来源</th>
+                                    </tr>
+                                </thead>
+                                <tbody id="systemLogsTableBody">
+                                    <tr>
+                                        <td colspan="5" class="text-center p-4">
+                                            <div class="spinner-border text-info" role="status">
+                                                <span class="visually-hidden">加载中...</span>
+                                            </div>
+                                            <p class="mt-2">正在加载系统日志...</p>
+                                        </td>
+                                    </tr>
+                                </tbody>
+                            </table>
+                        </div>
+
+                        <!-- 分页 -->
+                        <nav aria-label="系统日志分页">
+                            <ul class="pagination justify-content-center" id="logsPagination">
+                                <!-- 分页按钮将通过JavaScript动态生成 -->
+                            </ul>
+                        </nav>
+                    </div>
+                </div>
+            </div>
+
+            <!-- 统计分析选项卡 -->
+            <div class="tab-pane fade" id="stats-pane" role="tabpanel">
+                <!-- 统计图表 -->
+                <div class="row mt-3">
+                    <div class="col-md-6 mb-3">
+                        <div class="card">
+                            <div class="card-header">
+                                <h6 class="card-title mb-0">任务执行趋势（7天）</h6>
+                            </div>
+                            <div class="card-body">
+                                <canvas id="taskTrendChart" height="200"></canvas>
+                            </div>
+                        </div>
+                    </div>
+                    <div class="col-md-6 mb-3">
+                        <div class="card">
+                            <div class="card-header">
+                                <h6 class="card-title mb-0">日志级别分布（7天）</h6>
+                            </div>
+                            <div class="card-body">
+                                <canvas id="logLevelChart" height="200"></canvas>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+
+                <!-- 统计表格 -->
+                <div class="row mt-3">
+                    <div class="col-12">
+                        <div class="card">
+                            <div class="card-header">
+                                <h5 class="card-title mb-0">
+                                    <i class="bi bi-bar-chart"></i> 详细统计
+                                </h5>
+                            </div>
+                            <div class="card-body">
+                                <div class="row">
+                                    <div class="col-md-6">
+                                        <h6>任务统计</h6>
+                                        <div id="taskStatsContent">
+                                            <div class="text-center p-4">
+                                                <div class="spinner-border text-primary" role="status">
+                                                    <span class="visually-hidden">加载中...</span>
+                                                </div>
+                                            </div>
+                                        </div>
+                                    </div>
+                                    <div class="col-md-6">
+                                        <h6>日志统计</h6>
+                                        <div id="logStatsContent">
+                                            <div class="text-center p-4">
+                                                <div class="spinner-border text-info" role="status">
+                                                    <span class="visually-hidden">加载中...</span>
+                                                </div>
+                                            </div>
+                                        </div>
+                                    </div>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 任务详情模态框 -->
+    <div class="modal fade" id="taskDetailModal" tabindex="-1">
+        <div class="modal-dialog modal-lg">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title">任务详情</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body" id="taskDetailContent">
+                    <!-- 详情内容将通过JavaScript动态生成 -->
+                </div>
+                <div class="modal-footer">
+                    <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">关闭</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 提示消息容器 -->
+    <div class="position-fixed top-0 end-0 p-3" style="z-index: 1050">
+        <div id="alertContainer"></div>
+    </div>
+
+    <!-- JavaScript -->
+    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
+    <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
+    <script src="js/history.js"></script>
+</body>
+</html>

public/index.html

@@ -0,0 +1,268 @@
+<!DOCTYPE html>
+<html lang="zh-CN">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>代理IP管理系统 - 仪表板</title>
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.7.2/font/bootstrap-icons.css" rel="stylesheet">
+    <link href="css/dashboard.css" rel="stylesheet">
+</head>
+<body>
+    <!-- 导航栏 -->
+    <nav class="navbar navbar-expand-lg navbar-dark bg-primary">
+        <div class="container-fluid">
+            <a class="navbar-brand" href="index.html">
+                <i class="bi bi-shield-check"></i>
+                代理IP管理系统
+            </a>
+
+            <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
+                <span class="navbar-toggler-icon"></span>
+            </button>
+
+            <div class="collapse navbar-collapse" id="navbarNav">
+                <ul class="navbar-nav me-auto">
+                    <li class="nav-item">
+                        <a class="nav-link active" href="index.html">
+                            <i class="bi bi-speedometer2"></i> 仪表板
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="proxies.html">
+                            <i class="bi bi-list-ul"></i> 代理管理
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="history.html">
+                            <i class="bi bi-clock-history"></i> 执行历史
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="monitoring.html">
+                            <i class="bi bi-activity"></i> 系统监控
+                        </a>
+                    </li>
+                </ul>
+
+                <div class="navbar-nav">
+                    <span class="navbar-text" id="currentTime"></span>
+                    <button class="btn btn-outline-light btn-sm ms-2" onclick="refreshData()">
+                        <i class="bi bi-arrow-clockwise"></i> 刷新
+                    </button>
+                </div>
+            </div>
+        </div>
+    </nav>
+
+    <!-- 主要内容 -->
+    <div class="container-fluid mt-4">
+        <!-- 状态卡片 -->
+        <div class="row mb-4">
+            <div class="col-xl-3 col-md-6 mb-3">
+                <div class="card bg-primary text-white h-100">
+                    <div class="card-body">
+                        <div class="d-flex justify-content-between">
+                            <div>
+                                <h6 class="card-title">总代理数量</h6>
+                                <h3 class="mb-0" id="totalProxies">-</h3>
+                            </div>
+                            <div class="align-self-center">
+                                <i class="bi bi-hdd-network fs-2"></i>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <div class="col-xl-3 col-md-6 mb-3">
+                <div class="card bg-success text-white h-100">
+                    <div class="card-body">
+                        <div class="d-flex justify-content-between">
+                            <div>
+                                <h6 class="card-title">可用代理</h6>
+                                <h3 class="mb-0" id="validProxies">-</h3>
+                            </div>
+                            <div class="align-self-center">
+                                <i class="bi bi-check-circle fs-2"></i>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <div class="col-xl-3 col-md-6 mb-3">
+                <div class="card bg-danger text-white h-100">
+                    <div class="card-body">
+                        <div class="d-flex justify-content-between">
+                            <div>
+                                <h6 class="card-title">无效代理</h6>
+                                <h3 class="mb-0" id="invalidProxies">-</h3>
+                            </div>
+                            <div class="align-self-center">
+                                <i class="bi bi-x-circle fs-2"></i>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <div class="col-xl-3 col-md-6 mb-3">
+                <div class="card bg-info text-white h-100">
+                    <div class="card-body">
+                        <div class="d-flex justify-content-between">
+                            <div>
+                                <h6 class="card-title">可用率</h6>
+                                <h3 class="mb-0" id="validRate">-</h3>
+                            </div>
+                            <div class="align-self-center">
+                                <i class="bi bi-percent fs-2"></i>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 快速操作和图表 -->
+        <div class="row mb-4">
+            <!-- 快速操作 -->
+            <div class="col-lg-4 mb-3">
+                <div class="card h-100">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-lightning"></i> 快速操作
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <div class="d-grid gap-2">
+                            <button class="btn btn-primary" onclick="startScrape()">
+                                <i class="bi bi-download"></i> 立即抓取代理
+                            </button>
+                            <button class="btn btn-success" onclick="startValidate()">
+                                <i class="bi bi-check2-square"></i> 立即验证代理
+                            </button>
+                            <button class="btn btn-warning" onclick="cleanupInvalid()">
+                                <i class="bi bi-trash"></i> 清理无效代理
+                            </button>
+                            <button class="btn btn-info" onclick="exportProxies()">
+                                <i class="bi bi-download"></i> 导出代理数据
+                            </button>
+                        </div>
+
+                        <hr>
+
+                        <div class="text-center">
+                            <small class="text-muted">下次执行时间</small>
+                            <div class="mt-2">
+                                <small class="d-block">抓取: <span id="nextScrape">-</span></small>
+                                <small class="d-block">验证: <span id="nextValidation">-</span></small>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <!-- 代理趋势图 -->
+            <div class="col-lg-8 mb-3">
+                <div class="card h-100">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-graph-up"></i> 代理数量趋势（7天）
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <canvas id="proxyTrendChart" height="100"></canvas>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 最近执行历史和系统状态 -->
+        <div class="row">
+            <!-- 最近抓取历史 -->
+            <div class="col-lg-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-clock-history"></i> 最近抓取历史
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <div id="recentScrapeHistory" class="table-responsive">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-primary" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <!-- 最近验证历史 -->
+            <div class="col-lg-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-check2-square"></i> 最近验证历史
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <div id="recentValidationHistory" class="table-responsive">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-success" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 系统信息 -->
+        <div class="row mt-4">
+            <div class="col-12">
+                <div class="card">
+                    <div class="card-header">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-info-circle"></i> 系统信息
+                        </h5>
+                    </div>
+                    <div class="card-body">
+                        <div class="row">
+                            <div class="col-md-3">
+                                <small class="text-muted">运行时间</small>
+                                <div id="systemUptime">-</div>
+                            </div>
+                            <div class="col-md-3">
+                                <small class="text-muted">内存使用</small>
+                                <div id="memoryUsage">-</div>
+                            </div>
+                            <div class="col-md-3">
+                                <small class="text-muted">今日抓取</small>
+                                <div id="todayScrape">-</div>
+                            </div>
+                            <div class="col-md-3">
+                                <small class="text-muted">今日验证</small>
+                                <div id="todayValidation">-</div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 提示消息容器 -->
+    <div class="position-fixed top-0 end-0 p-3" style="z-index: 1050">
+        <div id="alertContainer"></div>
+    </div>
+
+    <!-- JavaScript -->
+    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
+    <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
+    <script src="js/dashboard.js"></script>
+</body>
+</html>

public/js/dashboard.js

@@ -0,0 +1,590 @@
+// 仪表板JavaScript
+
+class Dashboard {
+    constructor() {
+        this.charts = {};
+        this.refreshInterval = null;
+        this.init();
+    }
+
+    async init() {
+        console.log('初始化仪表板...');
+
+        // 初始化图表
+        this.initCharts();
+
+        // 加载初始数据
+        await this.loadDashboardData();
+
+        // 启动自动刷新
+        this.startAutoRefresh();
+
+        // 更新时间显示
+        this.updateCurrentTime();
+        setInterval(() => this.updateCurrentTime(), 1000);
+
+        console.log('仪表板初始化完成');
+    }
+
+    // 初始化图表
+    initCharts() {
+        // 代理趋势图
+        const ctx = document.getElementById('proxyTrendChart');
+        if (ctx) {
+            this.charts.proxyTrend = new Chart(ctx, {
+                type: 'line',
+                data: {
+                    labels: [],
+                    datasets: [{
+                        label: '总代理数',
+                        data: [],
+                        borderColor: '#007bff',
+                        backgroundColor: 'rgba(0, 123, 255, 0.1)',
+                        tension: 0.4,
+                        fill: true
+                    }, {
+                        label: '可用代理',
+                        data: [],
+                        borderColor: '#28a745',
+                        backgroundColor: 'rgba(40, 167, 69, 0.1)',
+                        tension: 0.4,
+                        fill: true
+                    }]
+                },
+                options: {
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    plugins: {
+                        legend: {
+                            position: 'top',
+                        },
+                        tooltip: {
+                            mode: 'index',
+                            intersect: false,
+                        }
+                    },
+                    scales: {
+                        y: {
+                            beginAtZero: true,
+                            ticks: {
+                                precision: 0
+                            }
+                        }
+                    }
+                }
+            });
+        }
+    }
+
+    // 加载仪表板数据
+    async loadDashboardData() {
+        try {
+            // 显示加载状态
+            this.showLoading(true);
+
+            // 获取统计数据
+            const statsResponse = await fetch('/api/dashboard/stats');
+            const statsData = await statsResponse.json();
+
+            if (statsData.success) {
+                this.updateStatistics(statsData.data);
+                this.updateCharts(statsData.data.charts);
+                this.updateRecentHistory(statsData.data.latest);
+            }
+
+            // 获取实时状态
+            const statusResponse = await fetch('/api/dashboard/status');
+            const statusData = await statusResponse.json();
+
+            if (statusData.success) {
+                this.updateSystemInfo(statusData.data);
+            }
+
+        } catch (error) {
+            console.error('加载仪表板数据失败:', error);
+            this.showAlert('加载数据失败: ' + error.message, 'danger');
+        } finally {
+            this.showLoading(false);
+        }
+    }
+
+    // 更新统计信息
+    updateStatistics(data) {
+        document.getElementById('totalProxies').textContent = data.proxies.total || 0;
+        document.getElementById('validProxies').textContent = data.proxies.valid || 0;
+        document.getElementById('invalidProxies').textContent = data.proxies.invalid || 0;
+        document.getElementById('validRate').textContent = data.proxies.validRate || '0%';
+    }
+
+    // 更新图表
+    updateCharts(chartData) {
+        // 更新代理趋势图
+        if (this.charts.proxyTrend && chartData.daily_proxies) {
+            const labels = chartData.daily_proxies.map(item =>
+                new Date(item.date).toLocaleDateString('zh-CN', { month: 'short', day: 'numeric' })
+            );
+
+            this.charts.proxyTrend.data.labels = labels;
+            this.charts.proxyTrend.data.datasets[0].data = chartData.daily_proxies.map(item => item.total_added);
+            this.charts.proxyTrend.data.datasets[1].data = chartData.daily_proxies.map(item => item.valid_added);
+            this.charts.proxyTrend.update();
+        }
+    }
+
+    // 更新最近历史记录
+    updateRecentHistory(latest) {
+        // 更新抓取历史
+        if (latest.scrape && latest.scrape.length > 0) {
+            const scrapeHtml = this.generateHistoryTable(latest.scrape, 'scrape');
+            document.getElementById('recentScrapeHistory').innerHTML = scrapeHtml;
+        } else {
+            document.getElementById('recentScrapeHistory').innerHTML = this.generateEmptyState('暂无抓取历史');
+        }
+
+        // 更新验证历史
+        if (latest.validation && latest.validation.length > 0) {
+            const validationHtml = this.generateHistoryTable(latest.validation, 'validation');
+            document.getElementById('recentValidationHistory').innerHTML = validationHtml;
+        } else {
+            document.getElementById('recentValidationHistory').innerHTML = this.generateEmptyState('暂无验证历史');
+        }
+    }
+
+    // 生成历史记录表格
+    generateHistoryTable(history, type) {
+        if (!history || history.length === 0) {
+            return this.generateEmptyState('暂无历史记录');
+        }
+
+        let html = `
+            <div class="table-responsive">
+                <table class="table table-sm history-table">
+                    <thead>
+                        <tr>
+                            <th>任务名称</th>
+                            <th>状态</th>
+                            <th>开始时间</th>
+                            <th>执行时长</th>
+                            <th>结果</th>
+                        </tr>
+                    </thead>
+                    <tbody>
+        `;
+
+        history.forEach(item => {
+            const statusClass = this.getStatusClass(item.status);
+            const statusIcon = this.getStatusIcon(item.status);
+            const duration = item.duration ? this.formatDuration(item.duration) : '-';
+            const result = this.getResultSummary(item.details, type);
+
+            html += `
+                <tr>
+                    <td class="task-name">${item.task_name}</td>
+                    <td><span class="badge ${statusClass}">${statusIcon} ${this.getStatusText(item.status)}</span></td>
+                    <td>${this.formatDateTime(item.start_time)}</td>
+                    <td class="duration">${duration}</td>
+                    <td>${result}</td>
+                </tr>
+            `;
+        });
+
+        html += `
+                    </tbody>
+                </table>
+            </div>
+        `;
+
+        return html;
+    }
+
+    // 生成空状态
+    generateEmptyState(message) {
+        return `
+            <div class="empty-state">
+                <i class="bi bi-inbox"></i>
+                <h6>${message}</h6>
+                <small class="text-muted">等待任务执行...</small>
+            </div>
+        `;
+    }
+
+    // 更新系统信息
+    updateSystemInfo(data) {
+        // 更新运行时间
+        document.getElementById('systemUptime').textContent = this.formatUptime(data.uptime);
+
+        // 更新内存使用
+        const memoryHtml = `
+            <div>已使用: ${data.memory.heapUsed}MB / ${data.memory.heapTotal}MB</div>
+            <div class="memory-bar">
+                <div class="memory-bar-fill" style="width: ${(data.memory.heapUsed / data.memory.heapTotal * 100)}%"></div>
+            </div>
+        `;
+        document.getElementById('memoryUsage').innerHTML = memoryHtml;
+
+        // 更新今日任务
+        const todayScrape = data.today_tasks.scrape;
+        const todayValidation = data.today_tasks.validation;
+
+        document.getElementById('todayScrape').innerHTML = `
+            <div class="d-flex justify-content-between align-items-center">
+                <span>成功: ${todayScrape.success}</span>
+                <span class="badge bg-success">${todayScrape.success_rate}%</span>
+            </div>
+        `;
+
+        document.getElementById('todayValidation').innerHTML = `
+            <div class="d-flex justify-content-between align-items-center">
+                <span>成功: ${todayValidation.success}</span>
+                <span class="badge bg-success">${todayValidation.success_rate}%</span>
+            </div>
+        `;
+
+        // 更新下次执行时间
+        if (data.next_runs) {
+            document.getElementById('nextScrape').textContent = this.formatTime(data.next_runs.scrape);
+            document.getElementById('nextValidation').textContent = this.formatTime(data.next_runs.validation);
+        }
+    }
+
+    // 开始抓取任务
+    async startScrape() {
+        try {
+            this.showLoading(true, '正在启动抓取任务...');
+
+            const response = await fetch('/api/dashboard/actions/scrape', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ pages: 5 })
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert('抓取任务已启动', 'success');
+                // 延迟刷新数据
+                setTimeout(() => this.loadDashboardData(), 2000);
+            } else {
+                this.showAlert('启动抓取任务失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('启动抓取任务失败:', error);
+            this.showAlert('启动抓取任务失败: ' + error.message, 'danger');
+        } finally {
+            this.showLoading(false);
+        }
+    }
+
+    // 开始验证任务
+    async startValidate() {
+        try {
+            this.showLoading(true, '正在启动验证任务...');
+
+            const response = await fetch('/api/dashboard/actions/validate', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ limit: 50 })
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert('验证任务已启动', 'success');
+                // 延迟刷新数据
+                setTimeout(() => this.loadDashboardData(), 2000);
+            } else {
+                this.showAlert('启动验证任务失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('启动验证任务失败:', error);
+            this.showAlert('启动验证任务失败: ' + error.message, 'danger');
+        } finally {
+            this.showLoading(false);
+        }
+    }
+
+    // 清理无效代理
+    async cleanupInvalid() {
+        if (!confirm('确定要清理所有无效代理吗？此操作不可恢复。')) {
+            return;
+        }
+
+        try {
+            const response = await fetch('/api/proxies/cleanup', {
+                method: 'DELETE'
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert(result.message, 'success');
+                await this.loadDashboardData();
+            } else {
+                this.showAlert('清理失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('清理无效代理失败:', error);
+            this.showAlert('清理无效代理失败: ' + error.message, 'danger');
+        }
+    }
+
+    // 导出代理数据
+    async exportProxies() {
+        try {
+            this.showLoading(true, '正在导出数据...');
+
+            const response = await fetch('/api/proxies?limit=1000');
+            const result = await response.json();
+
+            if (result.success) {
+                const csv = this.convertToCSV(result.data);
+                this.downloadCSV(csv, `proxies_${new Date().toISOString().slice(0, 10)}.csv`);
+                this.showAlert('数据导出成功', 'success');
+            } else {
+                this.showAlert('导出数据失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('导出数据失败:', error);
+            this.showAlert('导出数据失败: ' + error.message, 'danger');
+        } finally {
+            this.showLoading(false);
+        }
+    }
+
+    // 工具函数
+
+    getStatusClass(status) {
+        const statusClasses = {
+            'success': 'bg-success',
+            'failed': 'bg-danger',
+            'running': 'bg-warning',
+            'pending': 'bg-secondary'
+        };
+        return statusClasses[status] || 'bg-secondary';
+    }
+
+    getStatusIcon(status) {
+        const statusIcons = {
+            'success': '✓',
+            'failed': '✗',
+            'running': '⏳',
+            'pending': '⏸'
+        };
+        return statusIcons[status] || '?';
+    }
+
+    getStatusText(status) {
+        const statusTexts = {
+            'success': '成功',
+            'failed': '失败',
+            'running': '运行中',
+            'pending': '等待中'
+        };
+        return statusTexts[status] || status;
+    }
+
+    getResultSummary(details, type) {
+        if (!details) return '-';
+
+        if (type === 'scrape') {
+            return `抓取 ${details.scraped || 0} 个，可用 ${details.valid || 0} 个`;
+        } else if (type === 'validation') {
+            return `验证 ${details.validated || 0} 个，有效 ${details.valid || 0} 个`;
+        }
+
+        return '-';
+    }
+
+    formatDateTime(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleString('zh-CN', {
+            month: 'short',
+            day: 'numeric',
+            hour: '2-digit',
+            minute: '2-digit'
+        });
+    }
+
+    formatTime(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleTimeString('zh-CN', {
+            hour: '2-digit',
+            minute: '2-digit'
+        });
+    }
+
+    formatDuration(ms) {
+        if (!ms) return '-';
+
+        if (ms < 1000) {
+            return ms + 'ms';
+        } else if (ms < 60000) {
+            return (ms / 1000).toFixed(1) + 's';
+        } else {
+            return (ms / 60000).toFixed(1) + 'min';
+        }
+    }
+
+    formatUptime(seconds) {
+        const hours = Math.floor(seconds / 3600);
+        const minutes = Math.floor((seconds % 3600) / 60);
+        return `${hours}h ${minutes}m`;
+    }
+
+    updateCurrentTime() {
+        const now = new Date();
+        const timeString = now.toLocaleString('zh-CN', {
+            year: 'numeric',
+            month: '2-digit',
+            day: '2-digit',
+            hour: '2-digit',
+            minute: '2-digit',
+            second: '2-digit'
+        });
+        const timeElement = document.getElementById('currentTime');
+        if (timeElement) {
+            timeElement.textContent = timeString;
+        }
+    }
+
+    showLoading(show, message = '加载中...') {
+        // 实现加载状态显示
+        const loadingElements = document.querySelectorAll('.spinner-border');
+        loadingElements.forEach(element => {
+            element.style.display = show ? 'inline-block' : 'none';
+        });
+    }
+
+    showAlert(message, type = 'info') {
+        const alertContainer = document.getElementById('alertContainer');
+        if (!alertContainer) return;
+
+        const alertId = 'alert-' + Date.now();
+        const alertHtml = `
+            <div class="alert alert-${type} alert-dismissible fade show" role="alert" id="${alertId}">
+                <i class="bi bi-${this.getAlertIcon(type)}"></i>
+                ${message}
+                <button type="button" class="btn-close" data-bs-dismiss="alert"></button>
+            </div>
+        `;
+
+        alertContainer.insertAdjacentHTML('beforeend', alertHtml);
+
+        // 自动移除提示
+        setTimeout(() => {
+            const alertElement = document.getElementById(alertId);
+            if (alertElement) {
+                alertElement.remove();
+            }
+        }, 5000);
+    }
+
+    getAlertIcon(type) {
+        const icons = {
+            'success': 'check-circle-fill',
+            'danger': 'exclamation-triangle-fill',
+            'warning': 'exclamation-triangle-fill',
+            'info': 'info-circle-fill'
+        };
+        return icons[type] || 'info-circle-fill';
+    }
+
+    convertToCSV(data) {
+        if (!data || data.length === 0) return '';
+
+        const headers = ['IP', '端口', '位置', '响应时间', '是否可用', '创建时间'];
+        const csvRows = [headers.join(',')];
+
+        data.forEach(proxy => {
+            const row = [
+                proxy.ip,
+                proxy.port,
+                `"${proxy.location || ''}"`,
+                proxy.response_time || '',
+                proxy.is_valid ? '是' : '否',
+                proxy.created_at
+            ];
+            csvRows.push(row.join(','));
+        });
+
+        return csvRows.join('\n');
+    }
+
+    downloadCSV(csv, filename) {
+        const blob = new Blob(['\ufeff' + csv], { type: 'text/csv;charset=utf-8;' });
+        const link = document.createElement('a');
+        const url = URL.createObjectURL(blob);
+
+        link.setAttribute('href', url);
+        link.setAttribute('download', filename);
+        link.style.visibility = 'hidden';
+
+        document.body.appendChild(link);
+        link.click();
+        document.body.removeChild(link);
+    }
+
+    startAutoRefresh() {
+        // 每30秒刷新一次数据
+        this.refreshInterval = setInterval(() => {
+            this.loadDashboardData();
+        }, 30000);
+    }
+
+    stopAutoRefresh() {
+        if (this.refreshInterval) {
+            clearInterval(this.refreshInterval);
+            this.refreshInterval = null;
+        }
+    }
+}
+
+// 全局函数（供HTML调用）
+let dashboard;
+
+async function refreshData() {
+    if (dashboard) {
+        await dashboard.loadDashboardData();
+        dashboard.showAlert('数据已刷新', 'info');
+    }
+}
+
+async function startScrape() {
+    if (dashboard) {
+        await dashboard.startScrape();
+    }
+}
+
+async function startValidate() {
+    if (dashboard) {
+        await dashboard.startValidate();
+    }
+}
+
+async function cleanupInvalid() {
+    if (dashboard) {
+        await dashboard.cleanupInvalid();
+    }
+}
+
+async function exportProxies() {
+    if (dashboard) {
+        await dashboard.exportProxies();
+    }
+}
+
+// 页面加载完成后初始化
+document.addEventListener('DOMContentLoaded', () => {
+    dashboard = new Dashboard();
+});
+
+// 页面卸载时清理
+window.addEventListener('beforeunload', () => {
+    if (dashboard) {
+        dashboard.stopAutoRefresh();
+    }
+});

public/js/history.js

@@ -0,0 +1,751 @@
+// 执行历史页面JavaScript
+
+class HistoryManager {
+    constructor() {
+        this.currentTaskPage = 1;
+        this.currentLogsPage = 1;
+        this.pageSize = 20;
+        this.charts = {};
+        this.searchParams = {
+            taskType: '',
+            taskStatus: '',
+            startDate: '',
+            endDate: ''
+        };
+        this.logSearchParams = {
+            level: '',
+            category: '',
+            keyword: ''
+        };
+
+        this.init();
+    }
+
+    async init() {
+        console.log('初始化执行历史页面...');
+
+        // 绑定事件
+        this.bindEvents();
+
+        // 初始化图表
+        this.initCharts();
+
+        // 加载初始数据
+        await this.loadTaskHistory();
+        await this.loadSystemLogs();
+        await this.loadStatistics();
+
+        console.log('执行历史页面初始化完成');
+    }
+
+    bindEvents() {
+        // 任务历史筛选
+        document.getElementById('taskFilterForm').addEventListener('submit', (e) => {
+            e.preventDefault();
+            this.searchParams.taskType = document.getElementById('taskTypeFilter').value;
+            this.searchParams.taskStatus = document.getElementById('taskStatusFilter').value;
+            this.searchParams.startDate = document.getElementById('startDate').value;
+            this.searchParams.endDate = document.getElementById('endDate').value;
+            this.currentTaskPage = 1;
+            this.loadTaskHistory();
+        });
+
+        // 系统日志筛选
+        document.getElementById('logFilterForm').addEventListener('submit', (e) => {
+            e.preventDefault();
+            this.logSearchParams.level = document.getElementById('logLevelFilter').value;
+            this.logSearchParams.category = document.getElementById('logCategoryFilter').value;
+            this.logSearchParams.keyword = document.getElementById('logSearchKeyword').value;
+            this.currentLogsPage = 1;
+            this.loadSystemLogs();
+        });
+
+        // 选项卡切换事件
+        document.getElementById('stats-tab').addEventListener('shown.bs.tab', () => {
+            this.loadStatistics();
+            this.updateCharts();
+        });
+    }
+
+    initCharts() {
+        // 任务趋势图
+        const taskTrendCtx = document.getElementById('taskTrendChart');
+        if (taskTrendCtx) {
+            this.charts.taskTrend = new Chart(taskTrendCtx, {
+                type: 'line',
+                data: {
+                    labels: [],
+                    datasets: [{
+                        label: '成功任务',
+                        data: [],
+                        borderColor: '#28a745',
+                        backgroundColor: 'rgba(40, 167, 69, 0.1)',
+                        tension: 0.4
+                    }, {
+                        label: '失败任务',
+                        data: [],
+                        borderColor: '#dc3545',
+                        backgroundColor: 'rgba(220, 53, 69, 0.1)',
+                        tension: 0.4
+                    }]
+                },
+                options: {
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    plugins: {
+                        legend: {
+                            position: 'top',
+                        }
+                    },
+                    scales: {
+                        y: {
+                            beginAtZero: true,
+                            ticks: {
+                                precision: 0
+                            }
+                        }
+                    }
+                }
+            });
+        }
+
+        // 日志级别分布图
+        const logLevelCtx = document.getElementById('logLevelChart');
+        if (logLevelCtx) {
+            this.charts.logLevel = new Chart(logLevelCtx, {
+                type: 'doughnut',
+                data: {
+                    labels: ['错误', '警告', '信息', '调试'],
+                    datasets: [{
+                        data: [0, 0, 0, 0],
+                        backgroundColor: ['#dc3545', '#ffc107', '#17a2b8', '#6c757d']
+                    }]
+                },
+                options: {
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    plugins: {
+                        legend: {
+                            position: 'bottom',
+                        }
+                    }
+                }
+            });
+        }
+    }
+
+    async loadTaskHistory() {
+        try {
+            this.showTaskLoading(true);
+
+            // 构建查询参数
+            const params = new URLSearchParams({
+                limit: this.pageSize,
+                offset: (this.currentTaskPage - 1) * this.pageSize
+            });
+
+            if (this.searchParams.taskType) params.append('taskType', this.searchParams.taskType);
+            if (this.searchParams.taskStatus) params.append('status', this.searchParams.taskStatus);
+
+            const response = await fetch(`/api/history?${params}`);
+            const result = await response.json();
+
+            if (result.success) {
+                this.renderTaskHistoryTable(result.data);
+                this.renderTaskPagination(result.pagination);
+            } else {
+                this.showAlert('加载任务历史失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('加载任务历史失败:', error);
+            this.showAlert('加载任务历史失败: ' + error.message, 'danger');
+        } finally {
+            this.showTaskLoading(false);
+        }
+    }
+
+    async loadSystemLogs() {
+        try {
+            this.showLogsLoading(true);
+
+            // 构建查询参数
+            const params = new URLSearchParams({
+                limit: this.pageSize,
+                offset: (this.currentLogsPage - 1) * this.pageSize
+            });
+
+            if (this.logSearchParams.level) params.append('level', this.logSearchParams.level);
+            if (this.logSearchParams.category) params.append('category', this.logSearchParams.category);
+
+            let response;
+            if (this.logSearchParams.keyword) {
+                response = await fetch(`/api/history/logs/search?keyword=${encodeURIComponent(this.logSearchParams.keyword)}&limit=${this.pageSize}`);
+            } else {
+                response = await fetch(`/api/history/logs/system?${params}`);
+            }
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.renderSystemLogsTable(result.data);
+                this.renderLogsPagination(result.pagination);
+            } else {
+                this.showAlert('加载系统日志失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('加载系统日志失败:', error);
+            this.showAlert('加载系统日志失败: ' + error.message, 'danger');
+        } finally {
+            this.showLogsLoading(false);
+        }
+    }
+
+    async loadStatistics() {
+        try {
+            // 加载任务统计
+            const taskStatsResponse = await fetch('/api/history/stats/summary');
+            const taskStatsResult = await taskStatsResponse.json();
+
+            if (taskStatsResult.success) {
+                this.updateTaskStatistics(taskStatsResult.data.summary);
+                this.renderTaskStats(taskStatsResult.data);
+            }
+
+            // 加载日志统计
+            const logStatsResponse = await fetch('/api/history/logs/stats');
+            const logStatsResult = await logStatsResponse.json();
+
+            if (logStatsResult.success) {
+                this.updateLogStatistics(logStatsResult.data.summary);
+                this.renderLogStats(logStatsResult.data);
+            }
+
+        } catch (error) {
+            console.error('加载统计数据失败:', error);
+            this.showAlert('加载统计数据失败: ' + error.message, 'danger');
+        }
+    }
+
+    renderTaskHistoryTable(tasks) {
+        const tbody = document.getElementById('taskHistoryTableBody');
+
+        if (tasks.length === 0) {
+            tbody.innerHTML = `
+                <tr>
+                    <td colspan="8" class="text-center p-4">
+                        <i class="bi bi-inbox fs-1 text-muted"></i>
+                        <p class="text-muted mt-2">暂无任务历史</p>
+                    </td>
+                </tr>
+            `;
+            return;
+        }
+
+        tbody.innerHTML = tasks.map(task => `
+            <tr>
+                <td>
+                    <span class="badge bg-secondary">${this.getTaskTypeLabel(task.task_type)}</span>
+                </td>
+                <td>${task.task_name}</td>
+                <td>
+                    <span class="badge ${this.getStatusClass(task.status)}">
+                        <i class="bi bi-${this.getStatusIcon(task.status)}"></i>
+                        ${this.getStatusText(task.status)}
+                    </span>
+                </td>
+                <td>
+                    <small class="text-muted">${this.formatDateTime(task.start_time)}</small>
+                </td>
+                <td>
+                    <small class="text-muted">${task.end_time ? this.formatDateTime(task.end_time) : '-'}</small>
+                </td>
+                <td>
+                    <small class="text-muted">${task.duration ? this.formatDuration(task.duration) : '-'}</small>
+                </td>
+                <td>
+                    <small class="text-muted">${task.result_summary || '-'}</small>
+                </td>
+                <td>
+                    <button class="btn btn-sm btn-outline-primary" onclick="historyManager.showTaskDetail(${task.id})">
+                        <i class="bi bi-eye"></i> 详情
+                    </button>
+                </td>
+            </tr>
+        `).join('');
+    }
+
+    renderSystemLogsTable(logs) {
+        const tbody = document.getElementById('systemLogsTableBody');
+
+        if (logs.length === 0) {
+            tbody.innerHTML = `
+                <tr>
+                    <td colspan="5" class="text-center p-4">
+                        <i class="bi bi-inbox fs-1 text-muted"></i>
+                        <p class="text-muted mt-2">暂无系统日志</p>
+                    </td>
+                </tr>
+            `;
+            return;
+        }
+
+        tbody.innerHTML = logs.map(log => `
+            <tr>
+                <td>
+                    <small class="text-muted">${this.formatDateTime(log.timestamp)}</small>
+                </td>
+                <td>
+                    <span class="badge ${this.getLogLevelClass(log.level)}">${log.level.toUpperCase()}</span>
+                </td>
+                <td>
+                    <small>${log.category || '-'}</small>
+                </td>
+                <td>
+                    <small>${log.message}</small>
+                </td>
+                <td>
+                    <small class="text-muted">${log.source}</small>
+                </td>
+            </tr>
+        `).join('');
+    }
+
+    renderTaskPagination(pagination) {
+        const paginationElement = document.getElementById('taskPagination');
+        this.renderPagination(paginationElement, pagination, this.currentTaskPage, (page) => {
+            this.currentTaskPage = page;
+            this.loadTaskHistory();
+        });
+    }
+
+    renderLogsPagination(pagination) {
+        const paginationElement = document.getElementById('logsPagination');
+        this.renderPagination(paginationElement, pagination, this.currentLogsPage, (page) => {
+            this.currentLogsPage = page;
+            this.loadSystemLogs();
+        });
+    }
+
+    renderPagination(container, pagination, currentPage, onPageChange) {
+        const totalPages = Math.ceil(pagination.total / pagination.limit);
+
+        if (totalPages <= 1) {
+            container.innerHTML = '';
+            return;
+        }
+
+        let paginationHTML = '';
+
+        // 上一页
+        paginationHTML += `
+            <li class="page-item ${currentPage === 1 ? 'disabled' : ''}">
+                <a class="page-link" href="#" onclick="historyManager.goToTaskPage(${currentPage - 1}); return false;">
+                    <i class="bi bi-chevron-left"></i>
+                </a>
+            </li>
+        `;
+
+        // 页码
+        const startPage = Math.max(1, currentPage - 2);
+        const endPage = Math.min(totalPages, currentPage + 2);
+
+        for (let i = startPage; i <= endPage; i++) {
+            paginationHTML += `
+                <li class="page-item ${i === currentPage ? 'active' : ''}">
+                    <a class="page-link" href="#" onclick="historyManager.goToTaskPage(${i}); return false;">${i}</a>
+                </li>
+            `;
+        }
+
+        // 下一页
+        paginationHTML += `
+            <li class="page-item ${currentPage === totalPages ? 'disabled' : ''}">
+                <a class="page-link" href="#" onclick="historyManager.goToTaskPage(${currentPage + 1}); return false;">
+                    <i class="bi bi-chevron-right"></i>
+                </a>
+            </li>
+        `;
+
+        container.innerHTML = paginationHTML;
+    }
+
+    updateTaskStatistics(stats) {
+        document.getElementById('totalTasks').textContent = stats.total;
+        document.getElementById('successTasks').textContent = stats.success;
+        document.getElementById('failedTasks').textContent = stats.failed;
+        document.getElementById('successRate').textContent = stats.success_rate + '%';
+    }
+
+    updateLogStatistics(stats) {
+        // 更新日志级别图表
+        if (this.charts.logLevel) {
+            this.charts.logLevel.data.datasets[0].data = [
+                stats.error,
+                stats.warning,
+                stats.info,
+                stats.debug
+            ];
+            this.charts.logLevel.update();
+        }
+    }
+
+    renderTaskStats(data) {
+        const content = document.getElementById('taskStatsContent');
+
+        if (!data.daily || data.daily.length === 0) {
+            content.innerHTML = '<p class="text-muted">暂无数据</p>';
+            return;
+        }
+
+        let html = '<table class="table table-sm">';
+        html += '<thead><tr><th>日期</th><th>总任务</th><th>成功</th><th>失败</th><th>成功率</th></tr></thead><tbody>';
+
+        data.daily.slice(0, 7).forEach(day => {
+            const successRate = day.total > 0 ? ((day.success / day.total) * 100).toFixed(1) : '0';
+            html += `
+                <tr>
+                    <td>${this.formatDate(day.date)}</td>
+                    <td>${day.total}</td>
+                    <td class="text-success">${day.success}</td>
+                    <td class="text-danger">${day.failed}</td>
+                    <td>${successRate}%</td>
+                </tr>
+            `;
+        });
+
+        html += '</tbody></table>';
+        content.innerHTML = html;
+    }
+
+    renderLogStats(data) {
+        const content = document.getElementById('logStatsContent');
+
+        const summary = data.summary;
+        let html = '<div class="row">';
+        html += '<div class="col-6"><small class="text-muted">总日志数:</small><div><strong>' + summary.total + '</strong></div></div>';
+        html += '<div class="col-6"><small class="text-muted">错误:</small><div><strong class="text-danger">' + summary.error + '</strong></div></div>';
+        html += '<div class="col-6"><small class="text-muted">警告:</small><div><strong class="text-warning">' + summary.warning + '</strong></div></div>';
+        html += '<div class="col-6"><small class="text-muted">信息:</small><div><strong class="text-info">' + summary.info + '</strong></div></div>';
+        html += '</div>';
+
+        content.innerHTML = html;
+    }
+
+    async updateCharts() {
+        try {
+            // 加载每日任务统计
+            const response = await fetch('/api/dashboard/charts/tasks');
+            const result = await response.json();
+
+            if (result.success && this.charts.taskTrend) {
+                const labels = result.data.map(item => this.formatDate(item.date));
+                const successData = result.data.map(item => item.scrape_success + item.validation_success);
+                const failedData = result.data.map(item => item.scrape_failed + item.validation_failed);
+
+                this.charts.taskTrend.data.labels = labels;
+                this.charts.taskTrend.data.datasets[0].data = successData;
+                this.charts.taskTrend.data.datasets[1].data = failedData;
+                this.charts.taskTrend.update();
+            }
+        } catch (error) {
+            console.error('更新图表失败:', error);
+        }
+    }
+
+    async showTaskDetail(taskId) {
+        try {
+            const response = await fetch(`/api/history/${taskId}`);
+            const result = await response.json();
+
+            if (result.success) {
+                const task = result.data;
+                const content = document.getElementById('taskDetailContent');
+
+                content.innerHTML = `
+                    <div class="row">
+                        <div class="col-md-6">
+                            <strong>任务类型:</strong><br>
+                            <span class="badge bg-secondary">${this.getTaskTypeLabel(task.task_type)}</span>
+                        </div>
+                        <div class="col-md-6">
+                            <strong>任务名称:</strong><br>
+                            ${task.task_name}
+                        </div>
+                    </div>
+                    <hr>
+                    <div class="row">
+                        <div class="col-md-6">
+                            <strong>状态:</strong><br>
+                            <span class="badge ${this.getStatusClass(task.status)}">
+                                ${this.getStatusText(task.status)}
+                            </span>
+                        </div>
+                        <div class="col-md-6">
+                            <strong>执行时长:</strong><br>
+                            ${task.duration ? this.formatDuration(task.duration) : '-'}
+                        </div>
+                    </div>
+                    <hr>
+                    <div class="row">
+                        <div class="col-md-6">
+                            <strong>开始时间:</strong><br>
+                            ${this.formatDateTime(task.start_time)}
+                        </div>
+                        <div class="col-md-6">
+                            <strong>结束时间:</strong><br>
+                            ${task.end_time ? this.formatDateTime(task.end_time) : '-'}
+                        </div>
+                    </div>
+                    <hr>
+                    <div class="row">
+                        <div class="col-12">
+                            <strong>结果摘要:</strong><br>
+                            <p>${task.result_summary || '暂无摘要'}</p>
+                        </div>
+                    </div>
+                    ${task.error_message ? `
+                    <div class="row">
+                        <div class="col-12">
+                            <strong>错误信息:</strong><br>
+                            <div class="alert alert-danger">${task.error_message}</div>
+                        </div>
+                    </div>
+                    ` : ''}
+                    ${task.details ? `
+                    <div class="row">
+                        <div class="col-12">
+                            <strong>详细信息:</strong><br>
+                            <pre class="bg-light p-2 rounded"><code>${JSON.stringify(task.details, null, 2)}</code></pre>
+                        </div>
+                    </div>
+                    ` : ''}
+                `;
+
+                // 显示模态框
+                const modal = new bootstrap.Modal(document.getElementById('taskDetailModal'));
+                modal.show();
+            } else {
+                this.showAlert('获取任务详情失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('获取任务详情失败:', error);
+            this.showAlert('获取任务详情失败: ' + error.message, 'danger');
+        }
+    }
+
+    async exportTaskHistory() {
+        try {
+            const params = new URLSearchParams();
+            if (this.searchParams.taskType) params.append('taskType', this.searchParams.taskType);
+            if (this.searchParams.taskStatus) params.append('status', this.searchParams.taskStatus);
+
+            const response = await fetch(`/api/history/export?${params}&format=csv`);
+
+            if (response.ok) {
+                const blob = await response.blob();
+                const url = window.URL.createObjectURL(blob);
+                const a = document.createElement('a');
+                a.href = url;
+                a.download = `task_history_${new Date().toISOString().slice(0, 10)}.csv`;
+                document.body.appendChild(a);
+                a.click();
+                window.URL.revokeObjectURL(url);
+                document.body.removeChild(a);
+
+                this.showAlert('任务历史导出成功', 'success');
+            } else {
+                this.showAlert('导出失败', 'danger');
+            }
+        } catch (error) {
+            console.error('导出任务历史失败:', error);
+            this.showAlert('导出任务历史失败: ' + error.message, 'danger');
+        }
+    }
+
+    async cleanupLogs() {
+        if (!confirm('确定要清理30天前的历史记录吗？此操作不可恢复。')) {
+            return;
+        }
+
+        try {
+            const response = await fetch('/api/history/cleanup', {
+                method: 'DELETE',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ days: 30, type: 'all' })
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert(result.message, 'success');
+                await this.loadSystemLogs();
+                await this.loadStatistics();
+            } else {
+                this.showAlert('清理失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('清理历史记录失败:', error);
+            this.showAlert('清理历史记录失败: ' + error.message, 'danger');
+        }
+    }
+
+    // 工具函数
+    getTaskTypeLabel(type) {
+        const labels = {
+            'scrape': '抓取任务',
+            'validation': '验证任务',
+            'health_check': '健康检查'
+        };
+        return labels[type] || type;
+    }
+
+    getStatusClass(status) {
+        const classes = {
+            'success': 'bg-success',
+            'failed': 'bg-danger',
+            'running': 'bg-warning',
+            'pending': 'bg-secondary'
+        };
+        return classes[status] || 'bg-secondary';
+    }
+
+    getStatusIcon(status) {
+        const icons = {
+            'success': 'check-circle',
+            'failed': 'x-circle',
+            'running': 'clock',
+            'pending': 'pause-circle'
+        };
+        return icons[status] || 'question-circle';
+    }
+
+    getStatusText(status) {
+        const texts = {
+            'success': '成功',
+            'failed': '失败',
+            'running': '运行中',
+            'pending': '等待中'
+        };
+        return texts[status] || status;
+    }
+
+    getLogLevelClass(level) {
+        const classes = {
+            'error': 'bg-danger',
+            'warning': 'bg-warning',
+            'info': 'bg-info',
+            'debug': 'bg-secondary'
+        };
+        return classes[level] || 'bg-secondary';
+    }
+
+    formatDateTime(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleString('zh-CN');
+    }
+
+    formatDate(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleDateString('zh-CN', { month: 'short', day: 'numeric' });
+    }
+
+    formatDuration(ms) {
+        if (!ms) return '-';
+
+        if (ms < 1000) {
+            return ms + 'ms';
+        } else if (ms < 60000) {
+            return (ms / 1000).toFixed(1) + 's';
+        } else {
+            return (ms / 60000).toFixed(1) + 'min';
+        }
+    }
+
+    showTaskLoading(show) {
+        const tbody = document.getElementById('taskHistoryTableBody');
+        const spinner = tbody.querySelector('.spinner-border');
+        if (spinner) {
+            spinner.parentElement.style.display = show ? 'block' : 'none';
+        }
+    }
+
+    showLogsLoading(show) {
+        const tbody = document.getElementById('systemLogsTableBody');
+        const spinner = tbody.querySelector('.spinner-border');
+        if (spinner) {
+            spinner.parentElement.style.display = show ? 'block' : 'none';
+        }
+    }
+
+    showAlert(message, type = 'info') {
+        const alertContainer = document.getElementById('alertContainer');
+        if (!alertContainer) return;
+
+        const alertId = 'alert-' + Date.now();
+        const alertHtml = `
+            <div class="alert alert-${type} alert-dismissible fade show" role="alert" id="${alertId}">
+                <i class="bi bi-${this.getAlertIcon(type)}"></i>
+                ${message}
+                <button type="button" class="btn-close" data-bs-dismiss="alert"></button>
+            </div>
+        `;
+
+        alertContainer.insertAdjacentHTML('beforeend', alertHtml);
+
+        // 自动移除提示
+        setTimeout(() => {
+            const alertElement = document.getElementById(alertId);
+            if (alertElement) {
+                alertElement.remove();
+            }
+        }, 5000);
+    }
+
+    getAlertIcon(type) {
+        const icons = {
+            'success': 'check-circle-fill',
+            'danger': 'exclamation-triangle-fill',
+            'warning': 'exclamation-triangle-fill',
+            'info': 'info-circle-fill'
+        };
+        return icons[type] || 'info-circle-fill';
+    }
+
+    goToTaskPage(page) {
+        this.currentTaskPage = page;
+        this.loadTaskHistory();
+    }
+}
+
+// 全局函数（供HTML调用）
+let historyManager;
+
+async function refreshHistory() {
+    if (historyManager) {
+        await Promise.all([
+            historyManager.loadTaskHistory(),
+            historyManager.loadSystemLogs(),
+            historyManager.loadStatistics()
+        ]);
+        historyManager.showAlert('历史数据已刷新', 'info');
+    }
+}
+
+async function exportTaskHistory() {
+    if (historyManager) {
+        await historyManager.exportTaskHistory();
+    }
+}
+
+async function cleanupLogs() {
+    if (historyManager) {
+        await historyManager.cleanupLogs();
+    }
+}
+
+// 页面加载完成后初始化
+document.addEventListener('DOMContentLoaded', () => {
+    historyManager = new HistoryManager();
+});

public/js/monitoring.js

@@ -0,0 +1,604 @@
+// 系统监控页面JavaScript
+
+class SystemMonitor {
+    constructor() {
+        this.charts = {};
+        this.refreshInterval = null;
+        this.refreshProgressInterval = null;
+        this.isRefreshing = false;
+
+        this.init();
+    }
+
+    async init() {
+        console.log('初始化系统监控页面...');
+
+        // 初始化图表
+        this.initCharts();
+
+        // 加载初始数据
+        await this.loadSystemStatus();
+        await this.loadRecentData();
+
+        // 启动自动刷新
+        this.startAutoRefresh();
+
+        // 启动刷新进度条
+        this.startRefreshProgress();
+
+        console.log('系统监控页面初始化完成');
+    }
+
+    initCharts() {
+        // 代理池趋势图
+        const proxyPoolCtx = document.getElementById('proxyPoolChart');
+        if (proxyPoolCtx) {
+            this.charts.proxyPool = new Chart(proxyPoolCtx, {
+                type: 'line',
+                data: {
+                    labels: [],
+                    datasets: [{
+                        label: '总代理数',
+                        data: [],
+                        borderColor: '#007bff',
+                        backgroundColor: 'rgba(0, 123, 255, 0.1)',
+                        tension: 0.4,
+                        fill: true
+                    }, {
+                        label: '可用代理',
+                        data: [],
+                        borderColor: '#28a745',
+                        backgroundColor: 'rgba(40, 167, 69, 0.1)',
+                        tension: 0.4,
+                        fill: true
+                    }]
+                },
+                options: {
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    plugins: {
+                        legend: {
+                            position: 'top',
+                        }
+                    },
+                    scales: {
+                        y: {
+                            beginAtZero: true,
+                            ticks: {
+                                precision: 0
+                            }
+                        }
+                    }
+                }
+            });
+        }
+
+        // 任务执行率图
+        const taskRateCtx = document.getElementById('taskRateChart');
+        if (taskRateCtx) {
+            this.charts.taskRate = new Chart(taskRateCtx, {
+                type: 'line',
+                data: {
+                    labels: [],
+                    datasets: [{
+                        label: '成功率 (%)',
+                        data: [],
+                        borderColor: '#17a2b8',
+                        backgroundColor: 'rgba(23, 162, 184, 0.1)',
+                        tension: 0.4,
+                        fill: true
+                    }]
+                },
+                options: {
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    plugins: {
+                        legend: {
+                            position: 'top',
+                        }
+                    },
+                    scales: {
+                        y: {
+                            beginAtZero: true,
+                            max: 100,
+                            ticks: {
+                                callback: function(value) {
+                                    return value + '%';
+                                }
+                            }
+                        }
+                    }
+                }
+            });
+        }
+    }
+
+    async loadSystemStatus() {
+        try {
+            this.setRefreshing(true);
+
+            // 获取系统状态
+            const statusResponse = await fetch('/api/dashboard/status');
+            const statusData = await statusResponse.json();
+
+            if (statusData.success) {
+                this.updateSystemOverview(statusData.data);
+                this.updateTasksStatus(statusData.data);
+                this.updateProxyPoolStatus(statusData.data.proxies);
+                this.updateResourceUsage(statusData.data);
+            }
+
+            // 更新最后刷新时间
+            this.updateLastRefreshTime();
+
+        } catch (error) {
+            console.error('加载系统状态失败:', error);
+            this.showAlert('加载系统状态失败: ' + error.message, 'danger');
+        } finally {
+            this.setRefreshing(false);
+        }
+    }
+
+    async loadRecentData() {
+        try {
+            // 加载图表数据
+            await this.updateCharts();
+
+            // 加载最近日志
+            await this.loadRecentLogs();
+
+            // 加载最近事件
+            await this.loadRecentEvents();
+
+        } catch (error) {
+            console.error('加载最近数据失败:', error);
+        }
+    }
+
+    updateSystemOverview(data) {
+        // 更新系统状态指示器
+        const indicator = document.getElementById('systemStatusIndicator');
+        const statusText = document.getElementById('systemStatusText');
+        const healthBadge = document.getElementById('systemHealth');
+
+        if (data.proxies && data.proxies.valid > 0) {
+            indicator.className = 'status-indicator online';
+            statusText.textContent = '在线';
+            healthBadge.textContent = '健康';
+            healthBadge.className = 'badge bg-success';
+        } else {
+            indicator.className = 'status-indicator offline';
+            statusText.textContent = '异常';
+            healthBadge.textContent = '异常';
+            healthBadge.className = 'badge bg-danger';
+        }
+
+        // 更新运行时间
+        document.getElementById('systemUptime').textContent = this.formatUptime(data.uptime);
+
+        // 更新内存使用
+        const memoryHtml = `
+            <div>已使用: ${data.memory.heapUsed}MB</div>
+            <div class="progress mt-1" style="height: 8px;">
+                <div class="progress-bar" style="width: ${(data.memory.heapUsed / data.memory.heapTotal * 100)}%"></div>
+            </div>
+            <small class="text-muted">总计: ${data.memory.heapTotal}MB</small>
+        `;
+        document.getElementById('memoryUsage').innerHTML = memoryHtml;
+
+        // 更新定时任务状态
+        if (data.scheduler) {
+            const taskCount = data.scheduler.taskCount || 0;
+            document.getElementById('schedulerStatus').innerHTML = `
+                <div>${taskCount} 个任务运行中</div>
+                <small class="text-muted">自动调度正常</small>
+            `;
+        }
+    }
+
+    updateTasksStatus(data) {
+        const container = document.getElementById('tasksStatus');
+
+        if (data.today_tasks) {
+            const scrape = data.today_tasks.scrape;
+            const validation = data.today_tasks.validation;
+
+            container.innerHTML = `
+                <div class="row">
+                    <div class="col-6">
+                        <div class="card border-primary">
+                            <div class="card-body text-center p-2">
+                                <h6 class="card-title mb-1">抓取任务</h6>
+                                <div class="d-flex justify-content-between">
+                                    <small>成功: <span class="text-success">${scrape.success}</span></small>
+                                    <small>失败: <span class="text-danger">${scrape.failed}</span></small>
+                                </div>
+                                <div class="progress mt-1" style="height: 6px;">
+                                    <div class="progress-bar bg-success" style="width: ${scrape.success_rate}%"></div>
+                                </div>
+                                <small class="text-muted">成功率: ${scrape.success_rate}%</small>
+                            </div>
+                        </div>
+                    </div>
+                    <div class="col-6">
+                        <div class="card border-success">
+                            <div class="card-body text-center p-2">
+                                <h6 class="card-title mb-1">验证任务</h6>
+                                <div class="d-flex justify-content-between">
+                                    <small>成功: <span class="text-success">${validation.success}</span></small>
+                                    <small>失败: <span class="text-danger">${validation.failed}</span></small>
+                                </div>
+                                <div class="progress mt-1" style="height: 6px;">
+                                    <div class="progress-bar bg-success" style="width: ${validation.success_rate}%"></div>
+                                </div>
+                                <small class="text-muted">成功率: ${validation.success_rate}%</small>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            `;
+        }
+    }
+
+    updateProxyPoolStatus(proxies) {
+        const container = document.getElementById('proxyPoolStatus');
+
+        container.innerHTML = `
+            <div class="row text-center">
+                <div class="col-4">
+                    <div class="p-2">
+                        <h4 class="text-primary mb-0">${proxies.total}</h4>
+                        <small class="text-muted">总代理数</small>
+                    </div>
+                </div>
+                <div class="col-4">
+                    <div class="p-2 border-start border-end">
+                        <h4 class="text-success mb-0">${proxies.valid}</h4>
+                        <small class="text-muted">可用代理</small>
+                    </div>
+                </div>
+                <div class="col-4">
+                    <div class="p-2">
+                        <h4 class="text-danger mb-0">${proxies.invalid}</h4>
+                        <small class="text-muted">无效代理</small>
+                    </div>
+                </div>
+            </div>
+            <hr>
+            <div class="text-center">
+                <div class="badge bg-info fs-6">可用率: ${proxies.validRate}</div>
+            </div>
+        `;
+    }
+
+    updateResourceUsage(data) {
+        if (data.memory) {
+            const memoryPercent = Math.round((data.memory.heapUsed / data.memory.heapTotal) * 100);
+            document.getElementById('memoryPercent').textContent = memoryPercent + '%';
+            document.getElementById('memoryProgressBar').style.width = memoryPercent + '%';
+        }
+
+        if (data.proxies) {
+            const validRate = parseFloat(data.proxies.validRate);
+            document.getElementById('proxyValidRate').textContent = data.proxies.validRate;
+            document.getElementById('proxyValidProgressBar').style.width = validRate + '%';
+        }
+
+        // 模拟CPU使用率（实际项目中可以从系统API获取）
+        const cpuUsage = Math.round(Math.random() * 30 + 10); // 10-40%
+        document.getElementById('cpuUsage').textContent = cpuUsage + '%';
+        document.getElementById('cpuProgressBar').style.width = cpuUsage + '%';
+    }
+
+    async updateCharts() {
+        try {
+            // 更新代理池趋势图
+            const proxyResponse = await fetch('/api/dashboard/charts/proxies');
+            const proxyResult = await proxyResponse.json();
+
+            if (proxyResult.success && this.charts.proxyPool) {
+                const labels = proxyResult.data.map(item =>
+                    new Date(item.date).toLocaleDateString('zh-CN', { month: 'short', day: 'numeric' })
+                );
+
+                // 计算累计数据
+                let totalRunning = 0;
+                let validRunning = 0;
+                const totalData = [];
+                const validData = [];
+
+                proxyResult.data.forEach(item => {
+                    totalRunning += item.total_added || 0;
+                    validRunning += item.valid_added || 0;
+                    totalData.push(totalRunning);
+                    validData.push(validRunning);
+                });
+
+                this.charts.proxyPool.data.labels = labels;
+                this.charts.proxyPool.data.datasets[0].data = totalData;
+                this.charts.proxyPool.data.datasets[1].data = validData;
+                this.charts.proxyPool.update();
+            }
+
+            // 更新任务执行率图
+            const taskResponse = await fetch('/api/history/stats/summary');
+            const taskResult = await taskResponse.json();
+
+            if (taskResult.success && this.charts.taskRate) {
+                // 使用最近7天的数据
+                const labels = [];
+                const successRates = [];
+
+                for (let i = 6; i >= 0; i--) {
+                    const date = new Date();
+                    date.setDate(date.getDate() - i);
+                    labels.push(date.toLocaleDateString('zh-CN', { month: 'short', day: 'numeric' }));
+                    successRates.push(Math.random() * 30 + 70); // 模拟70-100%成功率
+                }
+
+                this.charts.taskRate.data.labels = labels;
+                this.charts.taskRate.data.datasets[0].data = successRates;
+                this.charts.taskRate.update();
+            }
+        } catch (error) {
+            console.error('更新图表失败:', error);
+        }
+    }
+
+    async loadRecentLogs() {
+        try {
+            const response = await fetch('/api/history/logs/system?limit=10');
+            const result = await response.json();
+
+            if (result.success) {
+                this.renderRecentLogs(result.data);
+            }
+        } catch (error) {
+            console.error('加载最近日志失败:', error);
+        }
+    }
+
+    async loadRecentEvents() {
+        try {
+            const response = await fetch('/api/history?limit=10');
+            const result = await response.json();
+
+            if (result.success) {
+                this.renderRecentEvents(result.data);
+            }
+        } catch (error) {
+            console.error('加载最近事件失败:', error);
+        }
+    }
+
+    renderRecentLogs(logs) {
+        const container = document.getElementById('recentLogs');
+
+        if (!logs || logs.length === 0) {
+            container.innerHTML = '<div class="text-center text-muted p-4">暂无日志</div>';
+            return;
+        }
+
+        container.innerHTML = logs.map(log => `
+            <div class="d-flex align-items-start mb-2 p-2 border-bottom">
+                <div class="me-2">
+                    <span class="badge ${this.getLogLevelClass(log.level)}">${log.level.toUpperCase()}</span>
+                </div>
+                <div class="flex-grow-1">
+                    <div class="small">${log.message}</div>
+                    <div class="text-muted" style="font-size: 0.75rem;">
+                        ${this.formatDateTime(log.timestamp)} - ${log.source}
+                    </div>
+                </div>
+            </div>
+        `).join('');
+    }
+
+    renderRecentEvents(events) {
+        const container = document.getElementById('recentEvents');
+
+        if (!events || events.length === 0) {
+            container.innerHTML = '<div class="text-center text-muted p-4">暂无事件</div>';
+            return;
+        }
+
+        container.innerHTML = events.map(event => `
+            <div class="d-flex align-items-start mb-2 p-2 border-bottom">
+                <div class="me-2">
+                    <i class="bi bi-${this.getTaskIcon(event.task_type)} text-${this.getTaskColor(event.status)}"></i>
+                </div>
+                <div class="flex-grow-1">
+                    <div class="small fw-bold">${event.task_name}</div>
+                    <div class="text-muted" style="font-size: 0.75rem;">
+                        ${this.formatDateTime(event.start_time)} - ${this.getStatusText(event.status)}
+                    </div>
+                    ${event.result_summary ? `<div class="small text-muted">${event.result_summary}</div>` : ''}
+                </div>
+                <div>
+                    <span class="badge ${this.getStatusClass(event.status)}">${this.getStatusText(event.status)}</span>
+                </div>
+            </div>
+        `).join('');
+    }
+
+    startAutoRefresh() {
+        // 每30秒刷新一次数据
+        this.refreshInterval = setInterval(() => {
+            this.loadSystemStatus();
+        }, 30000);
+
+        // 每5分钟刷新最近数据
+        setInterval(() => {
+            this.loadRecentData();
+        }, 300000);
+    }
+
+    startRefreshProgress() {
+        this.refreshProgressInterval = setInterval(() => {
+            const progressBar = document.getElementById('refreshProgress');
+            if (progressBar) {
+                const currentWidth = parseFloat(progressBar.style.width) || 100;
+                const newWidth = Math.max(0, currentWidth - 3.33); // 30秒内从100%到0%
+                progressBar.style.width = newWidth + '%';
+            }
+        }, 1000);
+    }
+
+    updateLastRefreshTime() {
+        const now = new Date();
+        const timeString = now.toLocaleTimeString('zh-CN');
+        document.getElementById('lastUpdateTime').textContent = timeString;
+    }
+
+    setRefreshing(refreshing) {
+        this.isRefreshing = refreshing;
+        if (refreshing) {
+            // 重置进度条
+            const progressBar = document.getElementById('refreshProgress');
+            if (progressBar) {
+                progressBar.style.width = '100%';
+            }
+        }
+    }
+
+    // 工具函数
+    formatUptime(seconds) {
+        const hours = Math.floor(seconds / 3600);
+        const minutes = Math.floor((seconds % 3600) / 60);
+
+        if (hours > 24) {
+            const days = Math.floor(hours / 24);
+            const remainingHours = hours % 24;
+            return `${days}天 ${remainingHours}小时 ${minutes}分钟`;
+        }
+
+        return `${hours}小时 ${minutes}分钟`;
+    }
+
+    formatDateTime(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleString('zh-CN', {
+            month: 'short',
+            day: 'numeric',
+            hour: '2-digit',
+            minute: '2-digit'
+        });
+    }
+
+    getLogLevelClass(level) {
+        const classes = {
+            'error': 'bg-danger',
+            'warning': 'bg-warning',
+            'info': 'bg-info',
+            'debug': 'bg-secondary'
+        };
+        return classes[level] || 'bg-secondary';
+    }
+
+    getStatusClass(status) {
+        const classes = {
+            'success': 'bg-success',
+            'failed': 'bg-danger',
+            'running': 'bg-warning',
+            'pending': 'bg-secondary'
+        };
+        return classes[status] || 'bg-secondary';
+    }
+
+    getStatusText(status) {
+        const texts = {
+            'success': '成功',
+            'failed': '失败',
+            'running': '运行中',
+            'pending': '等待中'
+        };
+        return texts[status] || status;
+    }
+
+    getTaskIcon(taskType) {
+        const icons = {
+            'scrape': 'download',
+            'validation': 'check2-square',
+            'health_check': 'heart-pulse'
+        };
+        return icons[taskType] || 'gear';
+    }
+
+    getTaskColor(status) {
+        const colors = {
+            'success': 'success',
+            'failed': 'danger',
+            'running': 'warning',
+            'pending': 'secondary'
+        };
+        return colors[status] || 'secondary';
+    }
+
+    showAlert(message, type = 'info') {
+        const alertContainer = document.getElementById('alertContainer');
+        if (!alertContainer) return;
+
+        const alertId = 'alert-' + Date.now();
+        const alertHtml = `
+            <div class="alert alert-${type} alert-dismissible fade show" role="alert" id="${alertId}">
+                <i class="bi bi-${this.getAlertIcon(type)}"></i>
+                ${message}
+                <button type="button" class="btn-close" data-bs-dismiss="alert"></button>
+            </div>
+        `;
+
+        alertContainer.insertAdjacentHTML('beforeend', alertHtml);
+
+        // 自动移除提示
+        setTimeout(() => {
+            const alertElement = document.getElementById(alertId);
+            if (alertElement) {
+                alertElement.remove();
+            }
+        }, 5000);
+    }
+
+    getAlertIcon(type) {
+        const icons = {
+            'success': 'check-circle-fill',
+            'danger': 'exclamation-triangle-fill',
+            'warning': 'exclamation-triangle-fill',
+            'info': 'info-circle-fill'
+        };
+        return icons[type] || 'info-circle-fill';
+    }
+
+    stopAutoRefresh() {
+        if (this.refreshInterval) {
+            clearInterval(this.refreshInterval);
+            this.refreshInterval = null;
+        }
+        if (this.refreshProgressInterval) {
+            clearInterval(this.refreshProgressInterval);
+            this.refreshProgressInterval = null;
+        }
+    }
+}
+
+// 全局函数（供HTML调用）
+let systemMonitor;
+
+async function refreshMonitoring() {
+    if (systemMonitor && !systemMonitor.isRefreshing) {
+        await systemMonitor.loadSystemStatus();
+        await systemMonitor.loadRecentData();
+        systemMonitor.showAlert('监控数据已刷新', 'info');
+    }
+}
+
+// 页面加载完成后初始化
+document.addEventListener('DOMContentLoaded', () => {
+    systemMonitor = new SystemMonitor();
+});
+
+// 页面卸载时清理
+window.addEventListener('beforeunload', () => {
+    if (systemMonitor) {
+        systemMonitor.stopAutoRefresh();
+    }
+});

public/js/proxies.js

@@ -0,0 +1,760 @@
+// 代理管理页面JavaScript
+
+class ProxyManager {
+    constructor() {
+        this.currentPage = 1;
+        this.pageSize = 20;
+        this.totalCount = 0;
+        this.proxies = [];
+        this.selectedProxies = new Set();
+        this.searchParams = {
+            ip: '',
+            port: '',
+            location: '',
+            status: '',
+            sortBy: 'created_at',
+            order: 'DESC'
+        };
+
+        this.init();
+    }
+
+    async init() {
+        console.log('初始化代理管理页面...');
+
+        // 绑定事件
+        this.bindEvents();
+
+        // 加载代理列表
+        await this.loadProxies();
+
+        console.log('代理管理页面初始化完成');
+    }
+
+    bindEvents() {
+        // 搜索表单
+        document.getElementById('searchForm').addEventListener('submit', (e) => {
+            e.preventDefault();
+            this.searchParams.ip = document.getElementById('searchIp').value;
+            this.searchParams.port = document.getElementById('searchPort').value;
+            this.searchParams.location = document.getElementById('searchLocation').value;
+            this.searchParams.status = document.getElementById('filterStatus').value;
+            this.searchParams.sortBy = document.getElementById('sortBy').value;
+            this.currentPage = 1;
+            this.loadProxies();
+        });
+
+        // 单个代理验证
+        document.getElementById('validateSingleProxy').addEventListener('click', () => {
+            const proxyId = document.getElementById('validateSingleProxy').dataset.proxyId;
+            if (proxyId) {
+                this.validateSingleProxy(proxyId);
+            }
+        });
+    }
+
+    async loadProxies() {
+        try {
+            this.showLoading(true);
+
+            // 构建查询参数
+            const params = new URLSearchParams({
+                limit: this.pageSize,
+                offset: (this.currentPage - 1) * this.pageSize,
+                sortBy: this.searchParams.sortBy,
+                order: this.searchParams.order
+            });
+
+            if (this.searchParams.ip) params.append('ip', this.searchParams.ip);
+            if (this.searchParams.port) params.append('port', this.searchParams.port);
+            if (this.searchParams.location) params.append('location', this.searchParams.location);
+            if (this.searchParams.status !== '') params.append('validOnly', this.searchParams.status === '1');
+
+            const response = await fetch(`/api/proxies?${params}`);
+            const result = await response.json();
+
+            if (result.success) {
+                this.proxies = result.data;
+                this.totalCount = result.pagination.total;
+                this.renderProxyTable();
+                this.renderPagination();
+                this.updateStatistics();
+            } else {
+                this.showAlert('加载代理列表失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('加载代理列表失败:', error);
+            this.showAlert('加载代理列表失败: ' + error.message, 'danger');
+        } finally {
+            this.showLoading(false);
+        }
+    }
+
+    renderProxyTable() {
+        const tbody = document.getElementById('proxyTableBody');
+
+        if (this.proxies.length === 0) {
+            tbody.innerHTML = `
+                <tr>
+                    <td colspan="9" class="text-center p-4">
+                        <i class="bi bi-inbox fs-1 text-muted"></i>
+                        <p class="text-muted mt-2">暂无代理数据</p>
+                        <button class="btn btn-primary" onclick="proxyManager.startScrape()">
+                            <i class="bi bi-download"></i> 立即抓取代理
+                        </button>
+                    </td>
+                </tr>
+            `;
+            return;
+        }
+
+        tbody.innerHTML = this.proxies.map(proxy => `
+            <tr>
+                <td>
+                    <input type="checkbox" class="form-check-input proxy-checkbox"
+                           value="${proxy.id}" onchange="proxyManager.toggleProxySelection(${proxy.id})">
+                </td>
+                <td>
+                    <code>${proxy.ip}</code>
+                </td>
+                <td>${proxy.port}</td>
+                <td>
+                    <small class="text-muted">${proxy.location || '-'}</small>
+                </td>
+                <td>
+                    <span class="badge ${proxy.is_valid ? 'bg-success' : 'bg-danger'}">
+                        <i class="bi bi-${proxy.is_valid ? 'check-circle' : 'x-circle'}"></i>
+                        ${proxy.is_valid ? '可用' : '不可用'}
+                    </span>
+                </td>
+                <td>
+                    <small class="text-muted">
+                        ${proxy.response_time ? proxy.response_time + 'ms' : '-'}
+                    </small>
+                </td>
+                <td>
+                    <small class="text-muted">
+                        ${proxy.last_check_time ? this.formatDateTime(proxy.last_check_time) : '-'}
+                    </small>
+                </td>
+                <td>
+                    <small class="text-muted">
+                        ${this.formatDateTime(proxy.created_at)}
+                    </small>
+                </td>
+                <td>
+                    <div class="btn-group btn-group-sm">
+                        <button class="btn btn-outline-primary" onclick="proxyManager.showProxyDetail(${proxy.id})" title="查看详情">
+                            <i class="bi bi-eye"></i>
+                        </button>
+                        <button class="btn btn-outline-success" onclick="proxyManager.validateSingleProxy(${proxy.id})" title="验证">
+                            <i class="bi bi-check2"></i>
+                        </button>
+                        <button class="btn btn-outline-danger" onclick="proxyManager.deleteProxy(${proxy.id})" title="删除">
+                            <i class="bi bi-trash"></i>
+                        </button>
+                    </div>
+                </td>
+            </tr>
+        `).join('');
+    }
+
+    renderPagination() {
+        const pagination = document.getElementById('pagination');
+        const totalPages = Math.ceil(this.totalCount / this.pageSize);
+
+        if (totalPages <= 1) {
+            pagination.innerHTML = '';
+            return;
+        }
+
+        let paginationHTML = '';
+
+        // 上一页
+        paginationHTML += `
+            <li class="page-item ${this.currentPage === 1 ? 'disabled' : ''}">
+                <a class="page-link" href="#" onclick="proxyManager.goToPage(${this.currentPage - 1}); return false;">
+                    <i class="bi bi-chevron-left"></i>
+                </a>
+            </li>
+        `;
+
+        // 页码
+        const startPage = Math.max(1, this.currentPage - 2);
+        const endPage = Math.min(totalPages, this.currentPage + 2);
+
+        if (startPage > 1) {
+            paginationHTML += `
+                <li class="page-item">
+                    <a class="page-link" href="#" onclick="proxyManager.goToPage(1); return false;">1</a>
+                </li>
+            `;
+            if (startPage > 2) {
+                paginationHTML += '<li class="page-item disabled"><a class="page-link" href="#">...</a></li>';
+            }
+        }
+
+        for (let i = startPage; i <= endPage; i++) {
+            paginationHTML += `
+                <li class="page-item ${i === this.currentPage ? 'active' : ''}">
+                    <a class="page-link" href="#" onclick="proxyManager.goToPage(${i}); return false;">${i}</a>
+                </li>
+            `;
+        }
+
+        if (endPage < totalPages) {
+            if (endPage < totalPages - 1) {
+                paginationHTML += '<li class="page-item disabled"><a class="page-link" href="#">...</a></li>';
+            }
+            paginationHTML += `
+                <li class="page-item">
+                    <a class="page-link" href="#" onclick="proxyManager.goToPage(${totalPages}); return false;">${totalPages}</a>
+                </li>
+            `;
+        }
+
+        // 下一页
+        paginationHTML += `
+            <li class="page-item ${this.currentPage === totalPages ? 'disabled' : ''}">
+                <a class="page-link" href="#" onclick="proxyManager.goToPage(${this.currentPage + 1}); return false;">
+                    <i class="bi bi-chevron-right"></i>
+                </a>
+            </li>
+        `;
+
+        pagination.innerHTML = paginationHTML;
+    }
+
+    updateStatistics() {
+        document.getElementById('totalCount').textContent = this.totalCount;
+        document.getElementById('validCount').textContent = this.proxies.filter(p => p.is_valid).length;
+        document.getElementById('invalidCount').textContent = this.proxies.filter(p => !p.is_valid).length;
+        document.getElementById('showingCount').textContent = this.proxies.length;
+    }
+
+    async showProxyDetail(proxyId) {
+        try {
+            const proxy = this.proxies.find(p => p.id === proxyId);
+            if (!proxy) return;
+
+            const content = document.getElementById('proxyDetailContent');
+            content.innerHTML = `
+                <div class="row">
+                    <div class="col-6">
+                        <strong>IP地址:</strong><br>
+                        <code>${proxy.ip}</code>
+                    </div>
+                    <div class="col-6">
+                        <strong>端口:</strong><br>
+                        ${proxy.port}
+                    </div>
+                </div>
+                <hr>
+                <div class="row">
+                    <div class="col-6">
+                        <strong>位置:</strong><br>
+                        ${proxy.location || '-'}
+                    </div>
+                    <div class="col-6">
+                        <strong>状态:</strong><br>
+                        <span class="badge ${proxy.is_valid ? 'bg-success' : 'bg-danger'}">
+                            ${proxy.is_valid ? '可用' : '不可用'}
+                        </span>
+                    </div>
+                </div>
+                <hr>
+                <div class="row">
+                    <div class="col-6">
+                        <strong>响应时间:</strong><br>
+                        ${proxy.response_time ? proxy.response_time + 'ms' : '-'}
+                    </div>
+                    <div class="col-6">
+                        <strong>最后验证:</strong><br>
+                        ${proxy.last_check_time ? this.formatDateTime(proxy.last_check_time) : '-'}
+                    </div>
+                </div>
+                <hr>
+                <div class="row">
+                    <div class="col-6">
+                        <strong>创建时间:</strong><br>
+                        ${this.formatDateTime(proxy.created_at)}
+                    </div>
+                    <div class="col-6">
+                        <strong>更新时间:</strong><br>
+                        ${this.formatDateTime(proxy.updated_at)}
+                    </div>
+                </div>
+            `;
+
+            // 设置验证按钮的数据属性
+            document.getElementById('validateSingleProxy').dataset.proxyId = proxyId;
+
+            // 显示模态框
+            const modal = new bootstrap.Modal(document.getElementById('proxyDetailModal'));
+            modal.show();
+        } catch (error) {
+            console.error('显示代理详情失败:', error);
+            this.showAlert('显示代理详情失败: ' + error.message, 'danger');
+        }
+    }
+
+    async validateSingleProxy(proxyId) {
+        try {
+            const proxy = this.proxies.find(p => p.id === proxyId);
+            if (!proxy) return;
+
+            // 关闭详情模态框
+            const detailModal = bootstrap.Modal.getInstance(document.getElementById('proxyDetailModal'));
+            if (detailModal) detailModal.hide();
+
+            // 显示验证进度
+            const modal = new bootstrap.Modal(document.getElementById('validationModal'));
+            document.getElementById('validationProgress').style.display = 'block';
+            document.getElementById('validationResults').innerHTML = '';
+            document.getElementById('validationStatus').textContent = `正在验证 ${proxy.ip}:${proxy.port}...`;
+            document.getElementById('validationProgressBar').style.width = '50%';
+            modal.show();
+
+            const response = await fetch('/api/proxies/verify', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({
+                    ip: proxy.ip,
+                    port: proxy.port
+                })
+            });
+
+            const result = await response.json();
+
+            // 更新进度条
+            document.getElementById('validationProgressBar').style.width = '100%';
+            document.getElementById('validationStatus').textContent = '验证完成';
+
+            // 显示结果
+            document.getElementById('validationResults').innerHTML = `
+                <div class="alert ${result.data.isValid ? 'alert-success' : 'alert-danger'}">
+                    <h6>验证结果: ${result.data.isValid ? '成功' : '失败'}</h6>
+                    <p><strong>IP:</strong> ${result.data.ip}:${result.data.port}</p>
+                    <p><strong>响应时间:</strong> ${result.data.responseTime}ms</p>
+                    ${result.data.error ? `<p><strong>错误信息:</strong> ${result.data.error}</p>` : ''}
+                </div>
+            `;
+
+            // 延迟刷新列表
+            setTimeout(() => {
+                this.loadProxies();
+            }, 1000);
+
+        } catch (error) {
+            console.error('验证代理失败:', error);
+            this.showAlert('验证代理失败: ' + error.message, 'danger');
+        }
+    }
+
+    async deleteProxy(proxyId) {
+        if (!confirm('确定要删除这个代理吗？')) {
+            return;
+        }
+
+        try {
+            const proxy = this.proxies.find(p => p.id === proxyId);
+            if (!proxy) return;
+
+            // 这里应该调用删除API，但目前我们的API没有单个删除功能
+            // 暂时显示提示
+            this.showAlert(`代理 ${proxy.ip}:${proxy.port} 删除功能待实现`, 'info');
+
+        } catch (error) {
+            console.error('删除代理失败:', error);
+            this.showAlert('删除代理失败: ' + error.message, 'danger');
+        }
+    }
+
+    toggleProxySelection(proxyId) {
+        if (this.selectedProxies.has(proxyId)) {
+            this.selectedProxies.delete(proxyId);
+        } else {
+            this.selectedProxies.add(proxyId);
+        }
+        this.updateSelectAllCheckbox();
+    }
+
+    toggleSelectAll() {
+        const selectAll = document.getElementById('selectAll').checked;
+        const checkboxes = document.querySelectorAll('.proxy-checkbox');
+
+        checkboxes.forEach(checkbox => {
+            checkbox.checked = selectAll;
+            const proxyId = parseInt(checkbox.value);
+            if (selectAll) {
+                this.selectedProxies.add(proxyId);
+            } else {
+                this.selectedProxies.delete(proxyId);
+            }
+        });
+    }
+
+    updateSelectAllCheckbox() {
+        const checkboxes = document.querySelectorAll('.proxy-checkbox');
+        const selectAll = document.getElementById('selectAll');
+        selectAll.checked = checkboxes.length > 0 && this.selectedProxies.size === checkboxes.length;
+    }
+
+    goToPage(page) {
+        this.currentPage = page;
+        this.loadProxies();
+    }
+
+    async validateAll() {
+        if (this.selectedProxies.size === 0) {
+            this.showAlert('请先选择要验证的代理', 'warning');
+            return;
+        }
+
+        try {
+            const selectedProxiesList = Array.from(this.selectedProxies).map(id =>
+                this.proxies.find(p => p.id === id)
+            ).filter(p => p);
+
+            // 显示验证进度
+            const modal = new bootstrap.Modal(document.getElementById('validationModal'));
+            document.getElementById('validationProgress').style.display = 'block';
+            document.getElementById('validationResults').innerHTML = '';
+            document.getElementById('validationProgressBar').style.width = '0%';
+            modal.show();
+
+            const response = await fetch('/api/proxies/verify-batch', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({
+                    proxies: selectedProxiesList
+                })
+            });
+
+            const result = await response.json();
+
+            // 更新进度条
+            document.getElementById('validationProgressBar').style.width = '100%';
+            document.getElementById('validationStatus').textContent = '验证完成';
+
+            // 显示结果
+            document.getElementById('validationResults').innerHTML = `
+                <div class="alert alert-info">
+                    <h6>批量验证完成</h6>
+                    <p><strong>总验证数:</strong> ${result.data.validated}</p>
+                    <p><strong>有效代理:</strong> <span class="badge bg-success">${result.data.valid}</span></p>
+                    <p><strong>无效代理:</strong> <span class="badge bg-danger">${result.data.invalid}</span></p>
+                </div>
+            `;
+
+            // 延迟刷新列表
+            setTimeout(() => {
+                this.loadProxies();
+            }, 1000);
+
+        } catch (error) {
+            console.error('批量验证失败:', error);
+            this.showAlert('批量验证失败: ' + error.message, 'danger');
+        }
+    }
+
+    async cleanupInvalid() {
+        if (!confirm('确定要清理所有无效代理吗？此操作不可恢复。')) {
+            return;
+        }
+
+        try {
+            const response = await fetch('/api/proxies/cleanup', {
+                method: 'DELETE'
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert(result.message, 'success');
+                await this.loadProxies();
+            } else {
+                this.showAlert('清理失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('清理无效代理失败:', error);
+            this.showAlert('清理无效代理失败: ' + error.message, 'danger');
+        }
+    }
+
+    async exportProxies() {
+        try {
+            const response = await fetch('/api/proxies?limit=1000&validOnly=true');
+            const result = await response.json();
+
+            if (result.success) {
+                const csv = this.convertToCSV(result.data);
+                this.downloadCSV(csv, `valid_proxies_${new Date().toISOString().slice(0, 10)}.csv`);
+                this.showAlert('数据导出成功', 'success');
+            } else {
+                this.showAlert('导出数据失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('导出数据失败:', error);
+            this.showAlert('导出数据失败: ' + error.message, 'danger');
+        }
+    }
+
+    async startScrape() {
+        try {
+            const response = await fetch('/api/dashboard/actions/scrape', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ pages: 40 })
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                this.showAlert('抓取任务已启动', 'success');
+                // 延迟刷新数据
+                setTimeout(() => this.loadProxies(), 2000);
+            } else {
+                this.showAlert('启动抓取任务失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('启动抓取任务失败:', error);
+            this.showAlert('启动抓取任务失败: ' + error.message, 'danger');
+        }
+    }
+
+    // 工具函数
+    formatDateTime(dateString) {
+        const date = new Date(dateString);
+        return date.toLocaleString('zh-CN', {
+            month: 'short',
+            day: 'numeric',
+            hour: '2-digit',
+            minute: '2-digit'
+        });
+    }
+
+    showLoading(show) {
+        const spinner = document.querySelector('#proxyTableBody .spinner-border');
+        if (spinner) {
+            spinner.parentElement.style.display = show ? 'block' : 'none';
+        }
+    }
+
+    showAlert(message, type = 'info') {
+        const alertContainer = document.getElementById('alertContainer');
+        if (!alertContainer) return;
+
+        const alertId = 'alert-' + Date.now();
+        const alertHtml = `
+            <div class="alert alert-${type} alert-dismissible fade show" role="alert" id="${alertId}">
+                <i class="bi bi-${this.getAlertIcon(type)}"></i>
+                ${message}
+                <button type="button" class="btn-close" data-bs-dismiss="alert"></button>
+            </div>
+        `;
+
+        alertContainer.insertAdjacentHTML('beforeend', alertHtml);
+
+        // 自动移除提示
+        setTimeout(() => {
+            const alertElement = document.getElementById(alertId);
+            if (alertElement) {
+                alertElement.remove();
+            }
+        }, 5000);
+    }
+
+    getAlertIcon(type) {
+        const icons = {
+            'success': 'check-circle-fill',
+            'danger': 'exclamation-triangle-fill',
+            'warning': 'exclamation-triangle-fill',
+            'info': 'info-circle-fill'
+        };
+        return icons[type] || 'info-circle-fill';
+    }
+
+    convertToCSV(data) {
+        if (!data || data.length === 0) return '';
+
+        const headers = ['IP', '端口', '位置', '响应时间', '最后验证时间', '创建时间'];
+        const csvRows = [headers.join(',')];
+
+        data.forEach(proxy => {
+            const row = [
+                proxy.ip,
+                proxy.port,
+                `"${proxy.location || ''}"`,
+                proxy.response_time || '',
+                proxy.last_check_time || '',
+                proxy.created_at
+            ];
+            csvRows.push(row.join(','));
+        });
+
+        return csvRows.join('\n');
+    }
+
+    downloadCSV(csv, filename) {
+        const blob = new Blob(['\ufeff' + csv], { type: 'text/csv;charset=utf-8;' });
+        const link = document.createElement('a');
+        const url = URL.createObjectURL(blob);
+
+        link.setAttribute('href', url);
+        link.setAttribute('download', filename);
+        link.style.visibility = 'hidden';
+
+        document.body.appendChild(link);
+        link.click();
+        document.body.removeChild(link);
+    }
+
+    // 导入代理功能
+    async importProxies(proxies) {
+        try {
+            this.showAlert('开始导入并验证代理...', 'info');
+
+            const response = await fetch('/api/proxies/import', {
+                method: 'POST',
+                headers: {
+                    'Content-Type': 'application/json',
+                },
+                body: JSON.stringify({ proxies })
+            });
+
+            const result = await response.json();
+
+            if (result.success) {
+                const { data } = result;
+                let message = `导入完成！`;
+                message += `\n📊 总数: ${data.total}`;
+                message += `\n✅ 格式有效: ${data.format_valid}`;
+                message += `\n❌ 格式无效: ${data.format_invalid}`;
+                message += `\n🔍 验证通过: ${data.valid}`;
+                message += `\n❌ 验证失败: ${data.invalid}`;
+                message += `\n💾 保存成功: ${data.saved}`;
+
+                this.showAlert(message, 'success');
+
+                // 刷新代理列表
+                setTimeout(() => this.loadProxies(), 2000);
+            } else {
+                this.showAlert('导入失败: ' + result.error, 'danger');
+            }
+        } catch (error) {
+            console.error('导入代理失败:', error);
+            this.showAlert('导入代理失败: ' + error.message, 'danger');
+        }
+    }
+}
+
+// 全局函数（供HTML调用）
+let proxyManager;
+
+async function refreshProxies() {
+    if (proxyManager) {
+        await proxyManager.loadProxies();
+        proxyManager.showAlert('代理列表已刷新', 'info');
+    }
+}
+
+async function validateAll() {
+    if (proxyManager) {
+        await proxyManager.validateAll();
+    }
+}
+
+async function cleanupInvalid() {
+    if (proxyManager) {
+        await proxyManager.cleanupInvalid();
+    }
+}
+
+async function exportProxies() {
+    if (proxyManager) {
+        await proxyManager.exportProxies();
+    }
+}
+
+// 显示导入模态框
+function showImportModal() {
+    const modal = new bootstrap.Modal(document.getElementById('importModal'));
+    modal.show();
+}
+
+// 从模态框导入代理
+async function importProxiesFromModal() {
+    const jsonInput = document.getElementById('proxyJsonInput').value.trim();
+
+    if (!jsonInput) {
+        alert('请输入代理JSON数据');
+        return;
+    }
+
+    try {
+        const proxies = JSON.parse(jsonInput);
+
+        if (!Array.isArray(proxies)) {
+            alert('代理数据必须是数组格式');
+            return;
+        }
+
+        // 关闭模态框
+        const modal = bootstrap.Modal.getInstance(document.getElementById('importModal'));
+        modal.hide();
+
+        // 清空输入框
+        document.getElementById('proxyJsonInput').value = '';
+
+        // 开始导入
+        if (proxyManager) {
+            await proxyManager.importProxies(proxies);
+        }
+    } catch (error) {
+        alert('JSON格式错误: ' + error.message);
+    }
+}
+
+// 处理文件导入
+async function handleImportFile(event) {
+    const file = event.target.files[0];
+
+    if (!file) return;
+
+    if (!file.name.endsWith('.json')) {
+        alert('请选择JSON文件');
+        return;
+    }
+
+    try {
+        const text = await file.text();
+        const proxies = JSON.parse(text);
+
+        if (!Array.isArray(proxies)) {
+            alert('文件内容必须是代理数组');
+            return;
+        }
+
+        // 清空文件输入
+        event.target.value = '';
+
+        // 开始导入
+        if (proxyManager) {
+            await proxyManager.importProxies(proxies);
+        }
+    } catch (error) {
+        alert('文件读取失败: ' + error.message);
+    }
+}
+
+// 页面加载完成后初始化
+document.addEventListener('DOMContentLoaded', () => {
+    proxyManager = new ProxyManager();
+});

public/monitoring.html

@@ -0,0 +1,287 @@
+<!DOCTYPE html>
+<html lang="zh-CN">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>代理IP管理系统 - 系统监控</title>
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.7.2/font/bootstrap-icons.css" rel="stylesheet">
+    <link href="css/dashboard.css" rel="stylesheet">
+</head>
+<body>
+    <!-- 导航栏 -->
+    <nav class="navbar navbar-expand-lg navbar-dark bg-primary">
+        <div class="container-fluid">
+            <a class="navbar-brand" href="index.html">
+                <i class="bi bi-shield-check"></i>
+                代理IP管理系统
+            </a>
+
+            <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
+                <span class="navbar-toggler-icon"></span>
+            </button>
+
+            <div class="collapse navbar-collapse" id="navbarNav">
+                <ul class="navbar-nav me-auto">
+                    <li class="nav-item">
+                        <a class="nav-link" href="index.html">
+                            <i class="bi bi-speedometer2"></i> 仪表板
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="proxies.html">
+                            <i class="bi bi-list-ul"></i> 代理管理
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="history.html">
+                            <i class="bi bi-clock-history"></i> 执行历史
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link active" href="monitoring.html">
+                            <i class="bi bi-activity"></i> 系统监控
+                        </a>
+                    </li>
+                </ul>
+
+                <div class="navbar-nav">
+                    <div class="nav-link">
+                        <span class="status-indicator online" id="systemStatusIndicator"></span>
+                        <span id="systemStatusText">在线</span>
+                    </div>
+                    <button class="btn btn-outline-light btn-sm ms-2" onclick="refreshMonitoring()">
+                        <i class="bi bi-arrow-clockwise"></i> 刷新
+                    </button>
+                </div>
+            </div>
+        </div>
+    </nav>
+
+    <!-- 主要内容 -->
+    <div class="container-fluid mt-4">
+        <!-- 系统状态概览 -->
+        <div class="row mb-4">
+            <div class="col-md-3 mb-3">
+                <div class="card text-center h-100">
+                    <div class="card-body">
+                        <i class="bi bi-cpu fs-1 text-primary mb-2"></i>
+                        <h5>系统状态</h5>
+                        <div class="badge bg-success" id="systemHealth">健康</div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card text-center h-100">
+                    <div class="card-body">
+                        <i class="bi bi-clock fs-1 text-info mb-2"></i>
+                        <h5>运行时间</h5>
+                        <div id="systemUptime">-</div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card text-center h-100">
+                    <div class="card-body">
+                        <i class="bi bi-memory fs-1 text-warning mb-2"></i>
+                        <h5>内存使用</h5>
+                        <div id="memoryUsage">-</div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-3 mb-3">
+                <div class="card text-center h-100">
+                    <div class="card-body">
+                        <i class="bi bi-gear fs-1 text-success mb-2"></i>
+                        <h5>定时任务</h5>
+                        <div id="schedulerStatus">-</div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 实时监控图表 -->
+        <div class="row mb-4">
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-graph-up"></i> 代理池趋势
+                            <span class="badge bg-success ms-2">实时</span>
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <canvas id="proxyPoolChart" height="200"></canvas>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-activity"></i> 任务执行率
+                            <span class="badge bg-success ms-2">实时</span>
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <canvas id="taskRateChart" height="200"></canvas>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 详细监控信息 -->
+        <div class="row mb-4">
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-list-check"></i> 定时任务状态
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <div id="tasksStatus">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-primary" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-hdd-network"></i> 代理池状态
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <div id="proxyPoolStatus">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-success" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 系统资源监控 -->
+        <div class="row mb-4">
+            <div class="col-12">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-speedometer2"></i> 系统资源监控
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <div class="row">
+                            <div class="col-md-4">
+                                <div class="mb-3">
+                                    <div class="d-flex justify-content-between align-items-center mb-1">
+                                        <span>CPU使用率</span>
+                                        <span id="cpuUsage">0%</span>
+                                    </div>
+                                    <div class="progress">
+                                        <div class="progress-bar" id="cpuProgressBar" role="progressbar" style="width: 0%"></div>
+                                    </div>
+                                </div>
+                            </div>
+                            <div class="col-md-4">
+                                <div class="mb-3">
+                                    <div class="d-flex justify-content-between align-items-center mb-1">
+                                        <span>内存使用率</span>
+                                        <span id="memoryPercent">0%</span>
+                                    </div>
+                                    <div class="progress">
+                                        <div class="progress-bar" id="memoryProgressBar" role="progressbar" style="width: 0%"></div>
+                                    </div>
+                                </div>
+                            </div>
+                            <div class="col-md-4">
+                                <div class="mb-3">
+                                    <div class="d-flex justify-content-between align-items-center mb-1">
+                                        <span>代理池可用率</span>
+                                        <span id="proxyValidRate">0%</span>
+                                    </div>
+                                    <div class="progress">
+                                        <div class="progress-bar bg-success" id="proxyValidProgressBar" role="progressbar" style="width: 0%"></div>
+                                    </div>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 最近日志和事件 -->
+        <div class="row">
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-journal-text"></i> 最近系统日志
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <div id="recentLogs" style="max-height: 300px; overflow-y: auto;">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-info" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-6 mb-3">
+                <div class="card">
+                    <div class="card-header">
+                        <h6 class="card-title mb-0">
+                            <i class="bi bi-calendar-event"></i> 最近任务事件
+                        </h6>
+                    </div>
+                    <div class="card-body">
+                        <div id="recentEvents" style="max-height: 300px; overflow-y: auto;">
+                            <div class="text-center p-4">
+                                <div class="spinner-border text-warning" role="status">
+                                    <span class="visually-hidden">加载中...</span>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 实时状态指示器 -->
+    <div class="position-fixed bottom-0 end-0 p-3">
+        <div class="card" style="width: 200px;">
+            <div class="card-body text-center p-2">
+                <small class="text-muted">最后更新</small>
+                <div id="lastUpdateTime" class="small fw-bold">-</div>
+                <div class="progress mt-2" style="height: 4px;">
+                    <div class="progress-bar progress-bar-striped progress-bar-animated" id="refreshProgress" style="width: 100%"></div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 提示消息容器 -->
+    <div class="position-fixed top-0 end-0 p-3" style="z-index: 1050">
+        <div id="alertContainer"></div>
+    </div>
+
+    <!-- JavaScript -->
+    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
+    <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
+    <script src="js/monitoring.js"></script>
+</body>
+</html>

public/proxies.html

@@ -0,0 +1,310 @@
+<!DOCTYPE html>
+<html lang="zh-CN">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>代理IP管理系统 - 代理管理</title>
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.7.2/font/bootstrap-icons.css" rel="stylesheet">
+    <link href="css/dashboard.css" rel="stylesheet">
+</head>
+<body>
+    <!-- 导航栏 -->
+    <nav class="navbar navbar-expand-lg navbar-dark bg-primary">
+        <div class="container-fluid">
+            <a class="navbar-brand" href="index.html">
+                <i class="bi bi-shield-check"></i>
+                代理IP管理系统
+            </a>
+
+            <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
+                <span class="navbar-toggler-icon"></span>
+            </button>
+
+            <div class="collapse navbar-collapse" id="navbarNav">
+                <ul class="navbar-nav me-auto">
+                    <li class="nav-item">
+                        <a class="nav-link" href="index.html">
+                            <i class="bi bi-speedometer2"></i> 仪表板
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link active" href="proxies.html">
+                            <i class="bi bi-list-ul"></i> 代理管理
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="history.html">
+                            <i class="bi bi-clock-history"></i> 执行历史
+                        </a>
+                    </li>
+                    <li class="nav-item">
+                        <a class="nav-link" href="monitoring.html">
+                            <i class="bi bi-activity"></i> 系统监控
+                        </a>
+                    </li>
+                </ul>
+
+                <div class="navbar-nav">
+                    <button class="btn btn-outline-light btn-sm" onclick="refreshProxies()">
+                        <i class="bi bi-arrow-clockwise"></i> 刷新
+                    </button>
+                </div>
+            </div>
+        </div>
+    </nav>
+
+    <!-- 主要内容 -->
+    <div class="container-fluid mt-4">
+        <!-- 统计信息和操作按钮 -->
+        <div class="row mb-4">
+            <div class="col-md-8">
+                <div class="row">
+                    <div class="col-md-4 mb-3">
+                        <div class="card bg-primary text-white">
+                            <div class="card-body text-center">
+                                <h4 id="totalCount">-</h4>
+                                <p class="mb-0">总代理数</p>
+                            </div>
+                        </div>
+                    </div>
+                    <div class="col-md-4 mb-3">
+                        <div class="card bg-success text-white">
+                            <div class="card-body text-center">
+                                <h4 id="validCount">-</h4>
+                                <p class="mb-0">可用代理</p>
+                            </div>
+                        </div>
+                    </div>
+                    <div class="col-md-4 mb-3">
+                        <div class="card bg-danger text-white">
+                            <div class="card-body text-center">
+                                <h4 id="invalidCount">-</h4>
+                                <p class="mb-0">无效代理</p>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+            <div class="col-md-4">
+                <div class="card h-100">
+                    <div class="card-body">
+                        <div class="btn-group w-100 mb-2" role="group">
+                            <button class="btn btn-primary" onclick="refreshProxies()">
+                                <i class="bi bi-arrow-clockwise"></i> 刷新
+                            </button>
+                            <button class="btn btn-success" onclick="validateAll()">
+                                <i class="bi bi-check2-square"></i> 验证
+                            </button>
+                            <button class="btn btn-warning" onclick="cleanupInvalid()">
+                                <i class="bi bi-trash"></i> 清理
+                            </button>
+                            <button class="btn btn-info" onclick="exportProxies()">
+                                <i class="bi bi-download"></i> 导出
+                            </button>
+                        </div>
+                        <div class="btn-group w-100" role="group">
+                            <button class="btn btn-outline-primary" onclick="document.getElementById('importFileInput').click()">
+                                <i class="bi bi-upload"></i> 导入JSON
+                            </button>
+                            <button class="btn btn-outline-secondary" onclick="showImportModal()">
+                                <i class="bi bi-pencil-square"></i> 粘贴导入
+                            </button>
+                            <input type="file" id="importFileInput" accept=".json" style="display: none;" onchange="handleImportFile(event)">
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 搜索和筛选 -->
+        <div class="row mb-4">
+            <div class="col-12">
+                <div class="card">
+                    <div class="card-body">
+                        <form id="searchForm" class="row g-3">
+                            <div class="col-md-3">
+                                <input type="text" class="form-control" id="searchIp" placeholder="搜索IP地址">
+                            </div>
+                            <div class="col-md-2">
+                                <input type="number" class="form-control" id="searchPort" placeholder="端口">
+                            </div>
+                            <div class="col-md-2">
+                                <input type="text" class="form-control" id="searchLocation" placeholder="位置">
+                            </div>
+                            <div class="col-md-2">
+                                <select class="form-select" id="filterStatus">
+                                    <option value="">全部状态</option>
+                                    <option value="1">可用</option>
+                                    <option value="0">不可用</option>
+                                </select>
+                            </div>
+                            <div class="col-md-2">
+                                <select class="form-select" id="sortBy">
+                                    <option value="created_at">创建时间</option>
+                                    <option value="response_time">响应时间</option>
+                                    <option value="ip">IP地址</option>
+                                    <option value="updated_at">更新时间</option>
+                                </select>
+                            </div>
+                            <div class="col-md-1">
+                                <button type="submit" class="btn btn-outline-primary w-100">
+                                    <i class="bi bi-search"></i>
+                                </button>
+                            </div>
+                        </form>
+                    </div>
+                </div>
+            </div>
+        </div>
+
+        <!-- 代理列表 -->
+        <div class="row">
+            <div class="col-12">
+                <div class="card">
+                    <div class="card-header d-flex justify-content-between align-items-center">
+                        <h5 class="card-title mb-0">
+                            <i class="bi bi-list-ul"></i> 代理列表
+                        </h5>
+                        <div>
+                            <span class="badge bg-secondary">显示 <span id="showingCount">0</span> 条记录</span>
+                        </div>
+                    </div>
+                    <div class="card-body">
+                        <div class="table-responsive">
+                            <table class="table table-hover">
+                                <thead>
+                                    <tr>
+                                        <th>
+                                            <input type="checkbox" id="selectAll" onchange="toggleSelectAll()">
+                                        </th>
+                                        <th>IP地址</th>
+                                        <th>端口</th>
+                                        <th>位置</th>
+                                        <th>状态</th>
+                                        <th>响应时间</th>
+                                        <th>最后验证</th>
+                                        <th>创建时间</th>
+                                        <th>操作</th>
+                                    </tr>
+                                </thead>
+                                <tbody id="proxyTableBody">
+                                    <tr>
+                                        <td colspan="9" class="text-center p-4">
+                                            <div class="spinner-border text-primary" role="status">
+                                                <span class="visually-hidden">加载中...</span>
+                                            </div>
+                                            <p class="mt-2">正在加载代理列表...</p>
+                                        </td>
+                                    </tr>
+                                </tbody>
+                            </table>
+                        </div>
+
+                        <!-- 分页 -->
+                        <nav aria-label="代理列表分页">
+                            <ul class="pagination justify-content-center" id="pagination">
+                                <!-- 分页按钮将通过JavaScript动态生成 -->
+                            </ul>
+                        </nav>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 验证结果模态框 -->
+    <div class="modal fade" id="validationModal" tabindex="-1">
+        <div class="modal-dialog">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title">验证代理</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body">
+                    <div id="validationProgress" style="display: none;">
+                        <div class="progress mb-3">
+                            <div class="progress-bar" id="validationProgressBar" role="progressbar" style="width: 0%"></div>
+                        </div>
+                        <p id="validationStatus">正在验证中...</p>
+                    </div>
+                    <div id="validationResults"></div>
+                </div>
+                <div class="modal-footer">
+                    <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">关闭</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 代理详情模态框 -->
+    <div class="modal fade" id="proxyDetailModal" tabindex="-1">
+        <div class="modal-dialog">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title">代理详情</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body" id="proxyDetailContent">
+                    <!-- 详情内容将通过JavaScript动态生成 -->
+                </div>
+                <div class="modal-footer">
+                    <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">关闭</button>
+                    <button type="button" class="btn btn-primary" id="validateSingleProxy">验证代理</button>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 导入代理模态框 -->
+    <div class="modal fade" id="importModal" tabindex="-1">
+        <div class="modal-dialog modal-lg">
+            <div class="modal-content">
+                <div class="modal-header">
+                    <h5 class="modal-title">导入代理列表</h5>
+                    <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
+                </div>
+                <div class="modal-body">
+                    <div class="mb-3">
+                        <label for="proxyJsonInput" class="form-label">粘贴JSON格式的代理列表：</label>
+                        <textarea class="form-control" id="proxyJsonInput" rows="10" placeholder='[
+  {
+    "ip_address": "47.254.36.213",
+    "port": 1081
+  },
+  {
+    "ip_address": "8.137.62.53",
+    "port": 9080
+  }
+]'></textarea>
+                    </div>
+                    <div class="alert alert-info">
+                        <h6><i class="bi bi-info-circle"></i> 支持的格式：</h6>
+                        <ul class="mb-0">
+                            <li><code>ip_address</code> 或 <code>ip</code> 或 <code>host</code> - IP地址</li>
+                            <li><code>port</code> - 端口号</li>
+                            <li><code>location</code> 或 <code>area</code> - 位置（可选）</li>
+                            <li><code>speed</code> - 速度（可选）</li>
+                        </ul>
+                    </div>
+                </div>
+                <div class="modal-footer">
+                    <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">取消</button>
+                    <button type="button" class="btn btn-primary" onclick="importProxiesFromModal()">
+                        <i class="bi bi-upload"></i> 导入并验证
+                    </button>
+                </div>
+            </div>
+        </div>
+    </div>
+
+    <!-- 提示消息容器 -->
+    <div class="position-fixed top-0 end-0 p-3" style="z-index: 1050">
+        <div id="alertContainer"></div>
+    </div>
+
+    <!-- JavaScript -->
+    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
+    <script src="js/proxies.js"></script>
+</body>
+</html>

src/app.js

@@ -0,0 +1,329 @@
+const express = require('express');
+const path = require('path');
+const Database = require('./database/db');
+const ProxyModel = require('./database/models/proxy');
+const HistoryModel = require('./database/models/history');
+const LogsModel = require('./database/models/logs');
+const ProxyScheduler = require('./services/scheduler');
+const proxyRoutes = require('./routes/proxies');
+const dashboardRoutes = require('./routes/dashboard');
+const historyRoutes = require('./routes/history');
+
+class ProxyApp {
+  constructor() {
+    this.app = express();
+    this.port = process.env.PORT || 3000;
+    this.scheduler = new ProxyScheduler();
+    this.isShuttingDown = false;
+  }
+
+  setupMiddleware() {
+    // 静态文件服务
+    this.app.use(express.static(path.join(__dirname, '../public')));
+
+    // JSON 解析中间件
+    this.app.use(express.json({ limit: '10mb' }));
+    this.app.use(express.urlencoded({ extended: true }));
+
+    // CORS 支持
+    this.app.use((req, res, next) => {
+      res.header('Access-Control-Allow-Origin', '*');
+      res.header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS');
+      res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, Authorization');
+
+      if (req.method === 'OPTIONS') {
+        res.sendStatus(200);
+      } else {
+        next();
+      }
+    });
+
+    // 请求日志中间件
+    this.app.use((req, res, next) => {
+      const timestamp = new Date().toISOString().slice(0, 19).replace('T', ' ');
+      console.log(`[${timestamp}] ${req.method} ${req.url}`);
+      next();
+    });
+  }
+
+  setupRoutes() {
+    // 健康检查端点
+    this.app.get('/api/health', async (req, res) => {
+      try {
+        const stats = await this.scheduler.getSystemStats();
+        res.json({
+          status: 'healthy',
+          timestamp: new Date().toISOString(),
+          system: stats
+        });
+      } catch (error) {
+        res.status(500).json({
+          status: 'unhealthy',
+          error: error.message,
+          timestamp: new Date().toISOString()
+        });
+      }
+    });
+
+    // 根路径 - 重定向到仪表板
+    this.app.get('/', (req, res) => {
+      res.redirect('/index.html');
+    });
+
+    // API信息端点
+    this.app.get('/api/info', (req, res) => {
+      res.json({
+        name: 'Proxy IP Service',
+        version: '1.0.0',
+        description: '代理IP抓取、验证和管理服务',
+        web_interface: {
+          dashboard: '/index.html',
+          proxies: '/proxies.html',
+          history: '/history.html',
+          monitoring: '/monitoring.html'
+        },
+        endpoints: {
+          'GET /api/health': '健康检查',
+          'GET /api/info': 'API信息',
+          'GET /api/dashboard/stats': '仪表板统计数据',
+          'GET /api/dashboard/status': '实时系统状态',
+          'POST /api/dashboard/actions/scrape': '手动抓取',
+          'POST /api/dashboard/actions/validate': '手动验证',
+          'GET /api/proxies': '获取代理列表',
+          'GET /api/proxies/random': '获取随机代理',
+          'GET /api/proxies/stats': '获取统计信息',
+          'POST /api/proxies/verify': '验证单个代理',
+          'POST /api/proxies/verify-batch': '批量验证代理',
+          'POST /api/proxies/scrape': '手动触发抓取',
+          'POST /api/proxies/validate-all': '手动触发验证',
+          'GET /api/proxies/search': '搜索代理',
+          'DELETE /api/proxies/cleanup': '清理无效代理',
+          'GET /api/history': '执行历史',
+          'GET /api/history/logs/system': '系统日志',
+          'GET /api/history/stats': '历史统计'
+        },
+        scheduler: {
+          scrape: '每小时整点执行',
+          validation: '每10分钟执行',
+          healthCheck: '每小时30分执行'
+        }
+      });
+    });
+
+    // API 路由
+    this.app.use('/api/proxies', proxyRoutes);
+    this.app.use('/api/dashboard', dashboardRoutes);
+    this.app.use('/api/history', historyRoutes);
+
+    // 404 处理 - API端点
+    this.app.use('/api', (req, res, next) => {
+      // 检查是否已处理的路由
+      if (!req.route) {
+        res.status(404).json({
+          error: 'API接口不存在',
+          message: `路径 ${req.originalUrl} 未找到`,
+          available_endpoints: [
+            'GET /api/health',
+            'GET /api/info',
+            'GET /api/dashboard/stats',
+            'GET /api/dashboard/status',
+            'POST /api/dashboard/actions/scrape',
+            'POST /api/dashboard/actions/validate',
+            'GET /api/proxies',
+            'GET /api/proxies/random',
+            'GET /api/proxies/stats',
+            'POST /api/proxies/verify',
+            'POST /api/proxies/verify-batch',
+            'GET /api/history',
+            'GET /api/history/logs/system',
+            'GET /api/history/stats'
+          ]
+        });
+      } else {
+        next();
+      }
+    });
+
+    // 404 处理 - Web页面
+    this.app.use((req, res) => {
+      res.status(404).send(`
+        <!DOCTYPE html>
+        <html>
+        <head>
+          <title>页面未找到 - 代理IP管理系统</title>
+          <meta charset="utf-8">
+          <meta name="viewport" content="width=device-width, initial-scale=1">
+          <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
+        </head>
+        <body class="bg-light">
+          <div class="container mt-5">
+            <div class="row justify-content-center">
+              <div class="col-md-6">
+                <div class="card">
+                  <div class="card-body text-center">
+                    <h1 class="text-danger mb-4">404</h1>
+                    <h3 class="mb-3">页面未找到</h3>
+                    <p class="text-muted mb-4">您访问的页面 ${req.originalUrl} 不存在</p>
+                    <a href="/index.html" class="btn btn-primary">返回首页</a>
+                  </div>
+                </div>
+              </div>
+            </div>
+          </div>
+        </body>
+        </html>
+      `);
+    });
+
+    // 错误处理中间件
+    this.app.use((err, req, res, next) => {
+      console.error('未处理的错误:', err);
+      res.status(500).json({
+        error: '服务器内部错误',
+        message: err.message,
+        timestamp: new Date().toISOString()
+      });
+    });
+  }
+
+  async initializeDatabase() {
+    try {
+      console.log('初始化数据库连接...');
+      await Database.connect();
+      console.log('数据库连接成功');
+
+      console.log('创建数据表...');
+      await ProxyModel.initTable();
+      await HistoryModel.initTable();
+      await LogsModel.initTable();
+      console.log('数据表初始化完成');
+
+      // 获取初始统计信息
+      const totalCount = await ProxyModel.count();
+      const validCount = await ProxyModel.count(true);
+      console.log(`数据库初始化完成，共 ${totalCount} 个代理，其中 ${validCount} 个可用`);
+
+      // 记录系统启动日志
+      await LogsModel.logInfo('系统启动成功', 'system', {
+        total_proxies: totalCount,
+        valid_proxies: validCount,
+        node_version: process.version,
+        platform: process.platform
+      });
+    } catch (error) {
+      console.error('数据库初始化失败:', error);
+      throw error;
+    }
+  }
+
+  setupGracefulShutdown() {
+    const gracefulShutdown = async (signal) => {
+      if (this.isShuttingDown) {
+        console.log('正在关闭中，忽略信号:', signal);
+        return;
+      }
+
+      this.isShuttingDown = true;
+      console.log(`收到 ${signal} 信号，开始优雅关闭...`);
+
+      // 停止定时任务
+      console.log('停止定时任务...');
+      this.scheduler.stop();
+
+      // 记录系统关闭日志
+      try {
+        await LogsModel.logInfo('系统正常关闭', 'system', {
+          signal: signal,
+          uptime: process.uptime()
+        });
+      } catch (error) {
+        console.error('记录关闭日志失败:', error);
+      }
+
+      // 关闭数据库连接
+      console.log('关闭数据库连接...');
+      try {
+        await Database.close();
+      } catch (error) {
+        console.error('关闭数据库连接失败:', error);
+      }
+
+      // 关闭HTTP服务器
+      console.log('关闭HTTP服务器...');
+      process.exit(0);
+    };
+
+    // 监听关闭信号
+    process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
+    process.on('SIGINT', () => gracefulShutdown('SIGINT'));
+
+    // 监听未捕获的异常
+    process.on('uncaughtException', (error) => {
+      console.error('未捕获的异常:', error);
+      gracefulShutdown('uncaughtException');
+    });
+
+    process.on('unhandledRejection', (reason, promise) => {
+      console.error('未处理的Promise拒绝:', reason);
+      console.error('Promise:', promise);
+    });
+  }
+
+  async start() {
+    try {
+      console.log('=== 代理IP服务启动中 ===');
+      console.log('服务端口:', this.port);
+      console.log('启动时间:', new Date().toISOString());
+
+      // 初始化数据库
+      await this.initializeDatabase();
+
+      // 设置中间件和路由
+      this.setupMiddleware();
+      this.setupRoutes();
+
+      // 设置优雅关闭
+      this.setupGracefulShutdown();
+
+      // 启动定时任务
+      console.log('启动定时任务调度器...');
+      this.scheduler.start();
+
+      // 启动HTTP服务器
+      this.server = this.app.listen(this.port, () => {
+        console.log('=== 代理IP服务启动成功 ===');
+        console.log('服务地址: http://localhost:' + this.port);
+        console.log('Web管理界面: http://localhost:' + this.port);
+        console.log('  - 仪表板: http://localhost:' + this.port + '/index.html');
+        console.log('  - 代理管理: http://localhost:' + this.port + '/proxies.html');
+        console.log('  - 执行历史: http://localhost:' + this.port + '/history.html');
+        console.log('  - 系统监控: http://localhost:' + this.port + '/monitoring.html');
+        console.log('API文档: http://localhost:' + this.port + '/api/info');
+        console.log('健康检查: http://localhost:' + this.port + '/api/health');
+        console.log('定时任务已自动启动');
+      });
+
+      // 设置服务器超时
+      this.server.timeout = 30000; // 30秒超时
+
+    } catch (error) {
+      console.error('服务启动失败:', error);
+      process.exit(1);
+    }
+  }
+
+  async stop() {
+    if (this.server) {
+      this.server.close();
+    }
+  }
+}
+
+// 创建并启动应用
+const app = new ProxyApp();
+app.start().catch(error => {
+  console.error('应用启动失败:', error);
+  process.exit(1);
+});
+
+module.exports = ProxyApp;

src/database/db.js

@@ -0,0 +1,82 @@
+const sqlite3 = require('sqlite3').verbose();
+const path = require('path');
+const config = require('../../config/database');
+
+class Database {
+  constructor() {
+    this.db = null;
+  }
+
+  connect() {
+    return new Promise((resolve, reject) => {
+      this.db = new sqlite3.Database(
+        config.development.storage,
+        sqlite3.OPEN_READWRITE | sqlite3.OPEN_CREATE,
+        (err) => {
+          if (err) {
+            console.error('数据库连接失败:', err.message);
+            reject(err);
+          } else {
+            console.log('已连接到 SQLite 数据库');
+            resolve();
+          }
+        }
+      );
+    });
+  }
+
+  close() {
+    return new Promise((resolve, reject) => {
+      if (this.db) {
+        this.db.close((err) => {
+          if (err) {
+            reject(err);
+          } else {
+            console.log('数据库连接已关闭');
+            resolve();
+          }
+        });
+      } else {
+        resolve();
+      }
+    });
+  }
+
+  run(sql, params = []) {
+    return new Promise((resolve, reject) => {
+      this.db.run(sql, params, function(err) {
+        if (err) {
+          reject(err);
+        } else {
+          resolve({ id: this.lastID, changes: this.changes });
+        }
+      });
+    });
+  }
+
+  get(sql, params = []) {
+    return new Promise((resolve, reject) => {
+      this.db.get(sql, params, (err, row) => {
+        if (err) {
+          reject(err);
+        } else {
+          resolve(row);
+        }
+      });
+    });
+  }
+
+  all(sql, params = []) {
+    return new Promise((resolve, reject) => {
+      this.db.all(sql, params, (err, rows) => {
+        if (err) {
+          reject(err);
+        } else {
+          resolve(rows);
+        }
+      });
+    });
+  }
+}
+
+module.exports = new Database();

src/database/models/history.js

@@ -0,0 +1,260 @@
+const db = require('../db');
+
+class HistoryModel {
+  static async initTable() {
+    const sql = `
+      CREATE TABLE IF NOT EXISTS execution_history (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        task_type TEXT NOT NULL,
+        task_name TEXT NOT NULL,
+        status TEXT NOT NULL DEFAULT 'pending',
+        start_time DATETIME NOT NULL,
+        end_time DATETIME,
+        duration INTEGER,
+        result_summary TEXT,
+        error_message TEXT,
+        details TEXT,
+        created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+      )
+    `;
+
+    try {
+      await db.run(sql);
+      console.log('执行历史表创建成功或已存在');
+    } catch (error) {
+      console.error('创建执行历史表失败:', error);
+      throw error;
+    }
+  }
+
+  static async create(historyData) {
+    const sql = `
+      INSERT INTO execution_history
+      (task_type, task_name, status, start_time, end_time, duration, result_summary, error_message, details)
+      VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+    `;
+
+    const params = [
+      historyData.task_type,
+      historyData.task_name,
+      historyData.status || 'pending',
+      historyData.start_time,
+      historyData.end_time || null,
+      historyData.duration || null,
+      historyData.result_summary || null,
+      historyData.error_message || null,
+      historyData.details ? JSON.stringify(historyData.details) : null
+    ];
+
+    try {
+      const result = await db.run(sql, params);
+      return result;
+    } catch (error) {
+      console.error('创建执行历史记录失败:', error);
+      throw error;
+    }
+  }
+
+  static async update(id, updateData) {
+    const fields = [];
+    const params = [];
+
+    Object.keys(updateData).forEach(key => {
+      if (updateData[key] !== undefined) {
+        fields.push(`${key} = ?`);
+        if (key === 'details') {
+          params.push(JSON.stringify(updateData[key]));
+        } else {
+          params.push(updateData[key]);
+        }
+      }
+    });
+
+    if (fields.length === 0) {
+      throw new Error('没有要更新的字段');
+    }
+
+    const sql = `
+      UPDATE execution_history
+      SET ${fields.join(', ')}
+      WHERE id = ?
+    `;
+
+    params.push(id);
+
+    try {
+      const result = await db.run(sql, params);
+      return result;
+    } catch (error) {
+      console.error('更新执行历史记录失败:', error);
+      throw error;
+    }
+  }
+
+  static async findAll(taskType = null, status = null, limit = 50, offset = 0) {
+    let sql = 'SELECT * FROM execution_history WHERE 1=1';
+    const params = [];
+
+    if (taskType) {
+      sql += ' AND task_type = ?';
+      params.push(taskType);
+    }
+
+    if (status) {
+      sql += ' AND status = ?';
+      params.push(status);
+    }
+
+    sql += ' ORDER BY start_time DESC LIMIT ? OFFSET ?';
+    params.push(limit, offset);
+
+    try {
+      const histories = await db.all(sql, params);
+
+      // 解析details字段
+      return histories.map(history => ({
+        ...history,
+        details: history.details ? JSON.parse(history.details) : null
+      }));
+    } catch (error) {
+      console.error('查询执行历史失败:', error);
+      throw error;
+    }
+  }
+
+  static async findById(id) {
+    const sql = 'SELECT * FROM execution_history WHERE id = ?';
+
+    try {
+      const history = await db.get(sql, [id]);
+
+      if (history && history.details) {
+        history.details = JSON.parse(history.details);
+      }
+
+      return history;
+    } catch (error) {
+      console.error('根据ID查询执行历史失败:', error);
+      throw error;
+    }
+  }
+
+  static async getLatest(taskType, limit = 5) {
+    const sql = 'SELECT * FROM execution_history WHERE task_type = ? ORDER BY start_time DESC LIMIT ?';
+
+    try {
+      const histories = await db.all(sql, [taskType, limit]);
+
+      // 解析details字段
+      return histories.map(history => ({
+        ...history,
+        details: history.details ? JSON.parse(history.details) : null
+      }));
+    } catch (error) {
+      console.error('查询最新执行历史失败:', error);
+      throw error;
+    }
+  }
+
+  static async getStatistics(taskType = null, days = 7) {
+    let sql = `
+      SELECT
+        COUNT(*) as total,
+        COUNT(CASE WHEN status = 'success' THEN 1 END) as success,
+        COUNT(CASE WHEN status = 'failed' THEN 1 END) as failed,
+        COUNT(CASE WHEN status = 'running' THEN 1 END) as running,
+        AVG(duration) as avg_duration
+      FROM execution_history
+      WHERE start_time >= datetime('now', '-${days} days')
+    `;
+
+    const params = [];
+
+    if (taskType) {
+      sql += ' AND task_type = ?';
+      params.push(taskType);
+    }
+
+    try {
+      const stats = await db.get(sql, params);
+      return {
+        total: stats.total || 0,
+        success: stats.success || 0,
+        failed: stats.failed || 0,
+        running: stats.running || 0,
+        success_rate: stats.total > 0 ? ((stats.success / stats.total) * 100).toFixed(2) : '0',
+        avg_duration: stats.avg_duration ? Math.round(stats.avg_duration) : 0
+      };
+    } catch (error) {
+      console.error('查询执行历史统计失败:', error);
+      throw error;
+    }
+  }
+
+  static async getDailyStats(taskType = null, days = 7) {
+    let sql = `
+      SELECT
+        DATE(start_time) as date,
+        COUNT(*) as total,
+        COUNT(CASE WHEN status = 'success' THEN 1 END) as success,
+        COUNT(CASE WHEN status = 'failed' THEN 1 END) as failed,
+        AVG(duration) as avg_duration
+      FROM execution_history
+      WHERE start_time >= datetime('now', '-${days} days')
+    `;
+
+    const params = [];
+
+    if (taskType) {
+      sql += ' AND task_type = ?';
+      params.push(taskType);
+    }
+
+    sql += ' GROUP BY DATE(start_time) ORDER BY date DESC';
+
+    try {
+      const dailyStats = await db.all(sql, params);
+      return dailyStats;
+    } catch (error) {
+      console.error('查询每日统计失败:', error);
+      throw error;
+    }
+  }
+
+  static async deleteOldRecords(days = 30) {
+    const sql = 'DELETE FROM execution_history WHERE start_time < datetime("now", "-' + days + ' days")';
+
+    try {
+      const result = await db.run(sql);
+      return result;
+    } catch (error) {
+      console.error('删除旧执行历史记录失败:', error);
+      throw error;
+    }
+  }
+
+  static async count(taskType = null, status = null) {
+    let sql = 'SELECT COUNT(*) as count FROM execution_history WHERE 1=1';
+    const params = [];
+
+    if (taskType) {
+      sql += ' AND task_type = ?';
+      params.push(taskType);
+    }
+
+    if (status) {
+      sql += ' AND status = ?';
+      params.push(status);
+    }
+
+    try {
+      const result = await db.get(sql, params);
+      return result.count;
+    } catch (error) {
+      console.error('统计执行历史记录数量失败:', error);
+      throw error;
+    }
+  }
+}
+
+module.exports = HistoryModel;

src/database/models/logs.js

@@ -0,0 +1,287 @@
+const db = require('../db');
+
+class LogsModel {
+  static async initTable() {
+    const sql = `
+      CREATE TABLE IF NOT EXISTS system_logs (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        level TEXT NOT NULL,
+        message TEXT NOT NULL,
+        category TEXT,
+        details TEXT,
+        timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
+        source TEXT,
+        user_agent TEXT,
+        ip_address TEXT
+      )
+    `;
+
+    try {
+      await db.run(sql);
+      console.log('系统日志表创建成功或已存在');
+    } catch (error) {
+      console.error('创建系统日志表失败:', error);
+      throw error;
+    }
+  }
+
+  static async create(logData) {
+    const sql = `
+      INSERT INTO system_logs
+      (level, message, category, details, timestamp, source, user_agent, ip_address)
+      VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+    `;
+
+    const params = [
+      logData.level,
+      logData.message,
+      logData.category || null,
+      logData.details ? JSON.stringify(logData.details) : null,
+      logData.timestamp || new Date().toISOString(),
+      logData.source || 'system',
+      logData.user_agent || null,
+      logData.ip_address || null
+    ];
+
+    try {
+      const result = await db.run(sql, params);
+      return result;
+    } catch (error) {
+      console.error('创建系统日志记录失败:', error);
+      throw error;
+    }
+  }
+
+  static async log(level, message, category = null, details = null, source = 'system') {
+    return await this.create({
+      level,
+      message,
+      category,
+      details,
+      source
+    });
+  }
+
+  static async logInfo(message, category = null, details = null, source = 'system') {
+    return await this.log('info', message, category, details, source);
+  }
+
+  static async logWarning(message, category = null, details = null, source = 'system') {
+    return await this.log('warning', message, category, details, source);
+  }
+
+  static async logError(message, category = null, details = null, source = 'system') {
+    return await this.log('error', message, category, details, source);
+  }
+
+  static async logDebug(message, category = null, details = null, source = 'system') {
+    return await this.log('debug', message, category, details, source);
+  }
+
+  static async findAll(level = null, category = null, limit = 100, offset = 0) {
+    let sql = 'SELECT * FROM system_logs WHERE 1=1';
+    const params = [];
+
+    if (level) {
+      sql += ' AND level = ?';
+      params.push(level);
+    }
+
+    if (category) {
+      sql += ' AND category = ?';
+      params.push(category);
+    }
+
+    sql += ' ORDER BY timestamp DESC LIMIT ? OFFSET ?';
+    params.push(limit, offset);
+
+    try {
+      const logs = await db.all(sql, params);
+
+      // 解析details字段
+      return logs.map(log => ({
+        ...log,
+        details: log.details ? JSON.parse(log.details) : null
+      }));
+    } catch (error) {
+      console.error('查询系统日志失败:', error);
+      throw error;
+    }
+  }
+
+  static async findById(id) {
+    const sql = 'SELECT * FROM system_logs WHERE id = ?';
+
+    try {
+      const log = await db.get(sql, [id]);
+
+      if (log && log.details) {
+        log.details = JSON.parse(log.details);
+      }
+
+      return log;
+    } catch (error) {
+      console.error('根据ID查询系统日志失败:', error);
+      throw error;
+    }
+  }
+
+  static async getLatest(level = null, category = null, limit = 20) {
+    let sql = 'SELECT * FROM system_logs WHERE 1=1';
+    const params = [];
+
+    if (level) {
+      sql += ' AND level = ?';
+      params.push(level);
+    }
+
+    if (category) {
+      sql += ' AND category = ?';
+      params.push(category);
+    }
+
+    sql += ' ORDER BY timestamp DESC LIMIT ?';
+    params.push(limit);
+
+    try {
+      const logs = await db.all(sql, params);
+
+      // 解析details字段
+      return logs.map(log => ({
+        ...log,
+        details: log.details ? JSON.parse(log.details) : null
+      }));
+    } catch (error) {
+      console.error('查询最新系统日志失败:', error);
+      throw error;
+    }
+  }
+
+  static async getStatistics(days = 7) {
+    const sql = `
+      SELECT
+        COUNT(*) as total,
+        COUNT(CASE WHEN level = 'error' THEN 1 END) as error,
+        COUNT(CASE WHEN level = 'warning' THEN 1 END) as warning,
+        COUNT(CASE WHEN level = 'info' THEN 1 END) as info,
+        COUNT(CASE WHEN level = 'debug' THEN 1 END) as debug
+      FROM system_logs
+      WHERE timestamp >= datetime('now', '-${days} days')
+    `;
+
+    try {
+      const stats = await db.get(sql);
+      return {
+        total: stats.total || 0,
+        error: stats.error || 0,
+        warning: stats.warning || 0,
+        info: stats.info || 0,
+        debug: stats.debug || 0
+      };
+    } catch (error) {
+      console.error('查询系统日志统计失败:', error);
+      throw error;
+    }
+  }
+
+  static async getHourlyStats(days = 1) {
+    const sql = `
+      SELECT
+        strftime('%Y-%m-%d %H:00:00', timestamp) as hour,
+        level,
+        COUNT(*) as count
+      FROM system_logs
+      WHERE timestamp >= datetime('now', '-${days} days')
+      GROUP BY hour, level
+      ORDER BY hour DESC
+    `;
+
+    try {
+      const hourlyStats = await db.all(sql);
+      return hourlyStats;
+    } catch (error) {
+      console.error('查询每小时统计失败:', error);
+      throw error;
+    }
+  }
+
+  static async search(keyword, level = null, category = null, limit = 50) {
+    let sql = 'SELECT * FROM system_logs WHERE message LIKE ?';
+    const params = [`%${keyword}%`];
+
+    if (level) {
+      sql += ' AND level = ?';
+      params.push(level);
+    }
+
+    if (category) {
+      sql += ' AND category = ?';
+      params.push(category);
+    }
+
+    sql += ' ORDER BY timestamp DESC LIMIT ?';
+    params.push(limit);
+
+    try {
+      const logs = await db.all(sql, params);
+
+      // 解析details字段
+      return logs.map(log => ({
+        ...log,
+        details: log.details ? JSON.parse(log.details) : null
+      }));
+    } catch (error) {
+      console.error('搜索系统日志失败:', error);
+      throw error;
+    }
+  }
+
+  static async deleteOldRecords(days = 30) {
+    const sql = 'DELETE FROM system_logs WHERE timestamp < datetime("now", "-' + days + ' days")';
+
+    try {
+      const result = await db.run(sql);
+      return result;
+    } catch (error) {
+      console.error('删除旧系统日志记录失败:', error);
+      throw error;
+    }
+  }
+
+  static async count(level = null, category = null) {
+    let sql = 'SELECT COUNT(*) as count FROM system_logs WHERE 1=1';
+    const params = [];
+
+    if (level) {
+      sql += ' AND level = ?';
+      params.push(level);
+    }
+
+    if (category) {
+      sql += ' AND category = ?';
+      params.push(category);
+    }
+
+    try {
+      const result = await db.get(sql, params);
+      return result.count;
+    } catch (error) {
+      console.error('统计系统日志记录数量失败:', error);
+      throw error;
+    }
+  }
+
+  static async getCategories() {
+    const sql = 'SELECT DISTINCT category FROM system_logs WHERE category IS NOT NULL ORDER BY category';
+
+    try {
+      const categories = await db.all(sql);
+      return categories.map(row => row.category);
+    } catch (error) {
+      console.error('获取日志分类失败:', error);
+      throw error;
+    }
+  }
+}
+
+module.exports = LogsModel;

src/database/models/proxy.js

@@ -0,0 +1,178 @@
+const db = require('../db');
+
+class ProxyModel {
+  static async initTable() {
+    const sql = `
+      CREATE TABLE IF NOT EXISTS proxies (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        ip TEXT NOT NULL,
+        port INTEGER NOT NULL,
+        location TEXT,
+        speed INTEGER,
+        last_check_time TEXT,
+        is_valid INTEGER DEFAULT 1,
+        response_time INTEGER,
+        created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+        updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+        UNIQUE(ip, port)
+      )
+    `;
+
+    try {
+      await db.run(sql);
+      console.log('代理表创建成功或已存在');
+    } catch (error) {
+      console.error('创建代理表失败:', error);
+      throw error;
+    }
+  }
+
+  static async create(proxyData) {
+    const sql = `
+      INSERT OR REPLACE INTO proxies
+      (ip, port, location, speed, last_check_time, is_valid, created_at, updated_at)
+      VALUES (?, ?, ?, ?, ?, ?, datetime('now'), datetime('now'))
+    `;
+
+    const params = [
+      proxyData.ip,
+      proxyData.port,
+      proxyData.location || null,
+      proxyData.speed || null,
+      proxyData.last_check_time || null,
+      proxyData.is_valid !== undefined ? proxyData.is_valid : 1
+    ];
+
+    try {
+      const result = await db.run(sql, params);
+      return result;
+    } catch (error) {
+      console.error('插入代理数据失败:', error);
+      throw error;
+    }
+  }
+
+  static async findAll(validOnly = true) {
+    let sql = 'SELECT * FROM proxies';
+    const params = [];
+
+    if (validOnly) {
+      sql += ' WHERE is_valid = 1';
+    }
+
+    sql += ' ORDER BY response_time ASC, created_at DESC';
+
+    try {
+      const proxies = await db.all(sql, params);
+      return proxies;
+    } catch (error) {
+      console.error('查询代理数据失败:', error);
+      throw error;
+    }
+  }
+
+  static async findRandom(validOnly = true, limit = 1) {
+    let sql = 'SELECT * FROM proxies';
+    const params = [];
+
+    if (validOnly) {
+      sql += ' WHERE is_valid = 1';
+    }
+
+    sql += ' ORDER BY RANDOM() LIMIT ?';
+    params.push(limit);
+
+    try {
+      const proxies = await db.all(sql, params);
+      return limit === 1 ? proxies[0] : proxies;
+    } catch (error) {
+      console.error('查询随机代理失败:', error);
+      throw error;
+    }
+  }
+
+  static async findByIpAndPort(ip, port) {
+    const sql = 'SELECT * FROM proxies WHERE ip = ? AND port = ?';
+
+    try {
+      const proxy = await db.get(sql, [ip, port]);
+      return proxy;
+    } catch (error) {
+      console.error('根据IP和端口查询代理失败:', error);
+      throw error;
+    }
+  }
+
+  static async updateValidity(ip, port, isValid, responseTime = null) {
+    const sql = `
+      UPDATE proxies
+      SET is_valid = ?, response_time = ?, updated_at = datetime('now')
+      WHERE ip = ? AND port = ?
+    `;
+
+    try {
+      const result = await db.run(sql, [isValid, responseTime, ip, port]);
+      return result;
+    } catch (error) {
+      console.error('更新代理有效性失败:', error);
+      throw error;
+    }
+  }
+
+  static async deleteInvalid() {
+    const sql = 'DELETE FROM proxies WHERE is_valid = 0';
+
+    try {
+      const result = await db.run(sql);
+      return result;
+    } catch (error) {
+      console.error('删除无效代理失败:', error);
+      throw error;
+    }
+  }
+
+  static async count(validOnly = true) {
+    let sql = 'SELECT COUNT(*) as count FROM proxies';
+
+    if (validOnly) {
+      sql += ' WHERE is_valid = 1';
+    }
+
+    try {
+      const result = await db.get(sql);
+      return result.count;
+    } catch (error) {
+      console.error('统计代理数量失败:', error);
+      throw error;
+    }
+  }
+
+  static async all(sql, params = []) {
+    try {
+      const proxies = await db.all(sql, params);
+      return proxies;
+    } catch (error) {
+      console.error('执行SQL查询失败:', error);
+      throw error;
+    }
+  }
+
+  static async findAllForValidation(limit = 50) {
+    const sql = `
+      SELECT * FROM proxies
+      WHERE is_valid = 1
+      ORDER BY updated_at ASC
+      LIMIT ?
+    `;
+
+    try {
+      const proxies = await db.all(sql, [limit]);
+      return proxies;
+    } catch (error) {
+      console.error('查询待验证代理失败:', error);
+      throw error;
+    }
+  }
+}
+
+module.exports = ProxyModel;

src/routes/dashboard.js

@@ -0,0 +1,411 @@
+const express = require('express');
+const router = express.Router();
+const ProxyModel = require('../database/models/proxy');
+const HistoryModel = require('../database/models/history');
+const LogsModel = require('../database/models/logs');
+const ProxyValidator = require('../services/validator');
+const ProxyScraper = require('../services/scraper');
+const ProxyScheduler = require('../services/scheduler');
+
+const validator = new ProxyValidator();
+const scraper = new ProxyScraper();
+
+// 获取仪表板统计数据
+router.get('/stats', async (req, res) => {
+  try {
+    // 代理统计
+    const proxyStats = await validator.getProxyStatistics();
+
+    // 执行历史统计
+    const scrapeStats = await HistoryModel.getStatistics('scrape', 7);
+    const validationStats = await HistoryModel.getStatistics('validation', 7);
+
+    // 系统日志统计
+    const logStats = await LogsModel.getStatistics(7);
+
+    // 最近执行历史
+    const latestScrape = await HistoryModel.getLatest('scrape', 3);
+    const latestValidation = await HistoryModel.getLatest('validation', 3);
+
+    // 每日统计数据
+    const dailyProxyStats = await getDailyProxyStats(7);
+    const dailyTaskStats = await getDailyTaskStats(7);
+
+    res.json({
+      success: true,
+      data: {
+        proxies: proxyStats,
+        tasks: {
+          scrape: scrapeStats,
+          validation: validationStats
+        },
+        logs: logStats,
+        latest: {
+          scrape: latestScrape,
+          validation: latestValidation
+        },
+        charts: {
+          daily_proxies: dailyProxyStats,
+          daily_tasks: dailyTaskStats
+        },
+        timestamp: new Date().toISOString()
+      }
+    });
+  } catch (error) {
+    console.error('获取仪表板统计数据失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取统计数据失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取实时系统状态
+router.get('/status', async (req, res) => {
+  try {
+    // 代理统计
+    const proxyStats = await validator.getProxyStatistics();
+
+    // 当前时间
+    const now = new Date();
+
+    // 今天的执行历史
+    const todayScrape = await HistoryModel.getStatistics('scrape', 1);
+    const todayValidation = await HistoryModel.getStatistics('validation', 1);
+
+    // 最近的日志
+    const recentLogs = await LogsModel.getLatest(null, null, 5);
+
+    // 系统运行时间
+    const uptime = process.uptime();
+
+    // 内存使用情况
+    const memUsage = process.memoryUsage();
+
+    res.json({
+      success: true,
+      data: {
+        timestamp: now.toISOString(),
+        uptime: uptime,
+        memory: {
+          rss: Math.round(memUsage.rss / 1024 / 1024), // MB
+          heapUsed: Math.round(memUsage.heapUsed / 1024 / 1024), // MB
+          heapTotal: Math.round(memUsage.heapTotal / 1024 / 1024) // MB
+        },
+        proxies: proxyStats,
+        today_tasks: {
+          scrape: todayScrape,
+          validation: todayValidation
+        },
+        recent_logs: recentLogs,
+        next_runs: getNextRunTimes()
+      }
+    });
+  } catch (error) {
+    console.error('获取实时状态失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取状态失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取代理统计图表数据
+router.get('/charts/proxies', async (req, res) => {
+  try {
+    const days = parseInt(req.query.days) || 7;
+    const chartData = await getDailyProxyStats(days);
+
+    res.json({
+      success: true,
+      data: chartData
+    });
+  } catch (error) {
+    console.error('获取代理图表数据失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取图表数据失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取任务执行图表数据
+router.get('/charts/tasks', async (req, res) => {
+  try {
+    const days = parseInt(req.query.days) || 7;
+    const chartData = await getDailyTaskStats(days);
+
+    res.json({
+      success: true,
+      data: chartData
+    });
+  } catch (error) {
+    console.error('获取任务图表数据失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取图表数据失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取日志统计图表数据
+router.get('/charts/logs', async (req, res) => {
+  try {
+    const days = parseInt(req.query.days) || 7;
+    const chartData = await LogsModel.getHourlyStats(days);
+
+    res.json({
+      success: true,
+      data: chartData
+    });
+  } catch (error) {
+    console.error('获取日志图表数据失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取图表数据失败',
+      message: error.message
+    });
+  }
+});
+
+// 快速操作接口
+router.post('/actions/scrape', async (req, res) => {
+  try {
+    // 创建调度器实例
+    const scheduler = require('../services/scheduler');
+    const ProxyScheduler = require('../services/scheduler');
+    const schedulerInstance = new ProxyScheduler();
+
+    // 检查是否有抓取任务正在进行
+    const ProxyScraper = require('../services/scraper');
+    if (ProxyScraper.isScrapingInProgress()) {
+      return res.status(409).json({
+        success: false,
+        error: '有抓取任务正在进行',
+        message: '请等待当前抓取任务完成后再试'
+      });
+    }
+
+    // 创建执行历史记录
+    const historyId = await createTaskHistory('scrape', '手动触发抓取任务');
+
+    // 异步执行定时抓取任务（与定时任务相同的逻辑）
+    schedulerInstance.runScrapeTask().then(async (result) => {
+      if (result.skipped) {
+        await updateTaskHistory(historyId, 'skipped', result, result.message);
+        await LogsModel.logInfo(`手动抓取任务跳过: ${result.message}`, 'manual_action', result);
+      } else {
+        await updateTaskHistory(historyId, 'success', {
+          scraped: result.scraped,
+          total: result.total,
+          valid: result.valid
+        });
+        await LogsModel.logInfo(`手动抓取任务完成: 抓取${result.scraped}个代理`, 'manual_action', result);
+      }
+    }).catch(async (error) => {
+      await updateTaskHistory(historyId, 'failed', null, error.message);
+      await LogsModel.logError(`手动抓取任务失败: ${error.message}`, 'manual_action', { error: error.message });
+    });
+
+    res.json({
+      success: true,
+      message: '已触发定时抓取任务',
+      history_id: historyId
+    });
+  } catch (error) {
+    console.error('启动抓取任务失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '启动抓取任务失败',
+      message: error.message
+    });
+  }
+});
+
+router.post('/actions/validate', async (req, res) => {
+  try {
+    const { limit = 50 } = req.body;
+
+    // 创建执行历史记录
+    const historyId = await createTaskHistory('validation', '手动验证任务');
+
+    // 异步执行验证任务
+    validator.validateDatabaseProxies(limit).then(async (result) => {
+      await updateTaskHistory(historyId, 'success', {
+        validated: result.validated,
+        valid: result.valid,
+        invalid: result.invalid
+      });
+      await LogsModel.logInfo(`手动验证任务完成: 验证${result.validated}个代理`, 'manual_action', result);
+    }).catch(async (error) => {
+      await updateTaskHistory(historyId, 'failed', null, error.message);
+      await LogsModel.logError(`手动验证任务失败: ${error.message}`, 'manual_action', { error: error.message });
+    });
+
+    res.json({
+      success: true,
+      message: `验证任务已启动，将验证 ${limit} 个代理`,
+      history_id: historyId
+    });
+  } catch (error) {
+    console.error('启动验证任务失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '启动验证任务失败',
+      message: error.message
+    });
+  }
+});
+
+// 辅助函数
+
+// 获取每日代理统计
+async function getDailyProxyStats(days) {
+  const db = require('../database/db');
+
+  const sql = `
+    SELECT
+      DATE(created_at) as date,
+      COUNT(*) as total_added,
+      COUNT(CASE WHEN is_valid = 1 THEN 1 END) as valid_added
+    FROM proxies
+    WHERE created_at >= datetime('now', '-${days} days')
+    GROUP BY DATE(created_at)
+    ORDER BY date
+  `;
+
+  try {
+    const dailyStats = await db.all(sql);
+    return dailyStats;
+  } catch (error) {
+    console.error('获取每日代理统计失败:', error);
+    return [];
+  }
+}
+
+// 获取每日任务统计
+async function getDailyTaskStats(days) {
+  try {
+    const scrapeStats = await HistoryModel.getDailyStats('scrape', days);
+    const validationStats = await HistoryModel.getDailyStats('validation', days);
+
+    // 合并数据
+    const mergedStats = {};
+    const dates = new Set();
+
+    scrapeStats.forEach(stat => {
+      mergedStats[stat.date] = {
+        date: stat.date,
+        scrape_total: stat.total,
+        scrape_success: stat.success,
+        scrape_failed: stat.failed,
+        validation_total: 0,
+        validation_success: 0,
+        validation_failed: 0
+      };
+      dates.add(stat.date);
+    });
+
+    validationStats.forEach(stat => {
+      if (mergedStats[stat.date]) {
+        mergedStats[stat.date].validation_total = stat.total;
+        mergedStats[stat.date].validation_success = stat.success;
+        mergedStats[stat.date].validation_failed = stat.failed;
+      } else {
+        mergedStats[stat.date] = {
+          date: stat.date,
+          scrape_total: 0,
+          scrape_success: 0,
+          scrape_failed: 0,
+          validation_total: stat.total,
+          validation_success: stat.success,
+          validation_failed: stat.failed
+        };
+        dates.add(stat.date);
+      }
+    });
+
+    return Object.values(mergedStats).sort((a, b) => new Date(a.date) - new Date(b.date));
+  } catch (error) {
+    console.error('获取每日任务统计失败:', error);
+    return [];
+  }
+}
+
+// 获取下次执行时间
+function getNextRunTimes() {
+  const now = new Date();
+  const nextScrape = new Date(now);
+  const nextValidation = new Date(now);
+  const nextHealthCheck = new Date(now);
+
+  // 下次抓取时间（每小时整点）
+  nextScrape.setHours(nextScrape.getHours() + 1, 0, 0, 0);
+
+  // 下次验证时间（每10分钟）
+  const minutes = now.getMinutes();
+  const nextMinute = Math.ceil((minutes + 1) / 10) * 10;
+  nextValidation.setMinutes(nextMinute, 0, 0);
+
+  // 下次健康检查时间（每小时30分）
+  if (now.getMinutes() < 30) {
+    nextHealthCheck.setHours(now.getHours(), 30, 0, 0);
+  } else {
+    nextHealthCheck.setHours(now.getHours() + 1, 30, 0, 0);
+  }
+
+  return {
+    scrape: nextScrape.toISOString(),
+    validation: nextValidation.toISOString(),
+    healthCheck: nextHealthCheck.toISOString()
+  };
+}
+
+// 创建任务历史记录
+async function createTaskHistory(taskType, taskName) {
+  try {
+    const result = await HistoryModel.create({
+      task_type: taskType,
+      task_name: taskName,
+      status: 'running',
+      start_time: new Date().toISOString()
+    });
+
+    return result.id;
+  } catch (error) {
+    console.error('创建任务历史记录失败:', error);
+    throw error;
+  }
+}
+
+// 更新任务历史记录
+async function updateTaskHistory(id, status, details = null, errorMessage = null) {
+  try {
+    const updateData = {
+      status: status,
+      end_time: new Date().toISOString(),
+      duration: null,
+      details: details,
+      error_message: errorMessage
+    };
+
+    // 计算执行时长
+    const history = await HistoryModel.findById(id);
+    if (history && history.start_time) {
+      const startTime = new Date(history.start_time);
+      const endTime = new Date();
+      updateData.duration = endTime.getTime() - startTime.getTime();
+    }
+
+    await HistoryModel.update(id, updateData);
+  } catch (error) {
+    console.error('更新任务历史记录失败:', error);
+    throw error;
+  }
+}
+
+module.exports = router;

src/routes/history.js

@@ -0,0 +1,337 @@
+const express = require('express');
+const router = express.Router();
+const HistoryModel = require('../database/models/history');
+const LogsModel = require('../database/models/logs');
+
+// 获取执行历史列表
+router.get('/', async (req, res) => {
+  try {
+    const taskType = req.query.taskType || null;
+    const status = req.query.status || null;
+    const limit = Math.min(parseInt(req.query.limit) || 50, 200);
+    const offset = parseInt(req.query.offset) || 0;
+
+    const histories = await HistoryModel.findAll(taskType, status, limit, offset);
+    const totalCount = await HistoryModel.count(taskType, status);
+
+    res.json({
+      success: true,
+      data: histories,
+      pagination: {
+        total: totalCount,
+        limit: limit,
+        offset: offset,
+        hasMore: offset + limit < totalCount
+      }
+    });
+  } catch (error) {
+    console.error('获取执行历史失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取执行历史失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取执行历史详情
+router.get('/:id', async (req, res) => {
+  try {
+    const id = parseInt(req.params.id);
+
+    if (isNaN(id)) {
+      return res.status(400).json({
+        success: false,
+        error: '无效的历史记录ID'
+      });
+    }
+
+    const history = await HistoryModel.findById(id);
+
+    if (!history) {
+      return res.status(404).json({
+        success: false,
+        error: '未找到指定的历史记录'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: history
+    });
+  } catch (error) {
+    console.error('获取执行历史详情失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取执行历史详情失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取执行历史统计
+router.get('/stats/summary', async (req, res) => {
+  try {
+    const taskType = req.query.taskType || null;
+    const days = Math.min(parseInt(req.query.days) || 7, 30);
+
+    const stats = await HistoryModel.getStatistics(taskType, days);
+    const dailyStats = await HistoryModel.getDailyStats(taskType, days);
+
+    res.json({
+      success: true,
+      data: {
+        summary: stats,
+        daily: dailyStats
+      }
+    });
+  } catch (error) {
+    console.error('获取执行历史统计失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取执行历史统计失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取系统日志列表
+router.get('/logs/system', async (req, res) => {
+  try {
+    const level = req.query.level || null;
+    const category = req.query.category || null;
+    const limit = Math.min(parseInt(req.query.limit) || 100, 500);
+    const offset = parseInt(req.query.offset) || 0;
+
+    const logs = await LogsModel.findAll(level, category, limit, offset);
+    const totalCount = await LogsModel.count(level, category);
+
+    res.json({
+      success: true,
+      data: logs,
+      pagination: {
+        total: totalCount,
+        limit: limit,
+        offset: offset,
+        hasMore: offset + limit < totalCount
+      }
+    });
+  } catch (error) {
+    console.error('获取系统日志失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取系统日志失败',
+      message: error.message
+    });
+  }
+});
+
+// 搜索系统日志
+router.get('/logs/search', async (req, res) => {
+  try {
+    const keyword = req.query.keyword;
+    const level = req.query.level || null;
+    const category = req.query.category || null;
+    const limit = Math.min(parseInt(req.query.limit) || 50, 200);
+
+    if (!keyword || keyword.trim().length === 0) {
+      return res.status(400).json({
+        success: false,
+        error: '搜索关键词不能为空'
+      });
+    }
+
+    const logs = await LogsModel.search(keyword.trim(), level, category, limit);
+
+    res.json({
+      success: true,
+      data: logs,
+      count: logs.length
+    });
+  } catch (error) {
+    console.error('搜索系统日志失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '搜索系统日志失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取日志统计
+router.get('/logs/stats', async (req, res) => {
+  try {
+    const days = Math.min(parseInt(req.query.days) || 7, 30);
+
+    const stats = await LogsModel.getStatistics(days);
+    const hourlyStats = await LogsModel.getHourlyStats(days);
+    const categories = await LogsModel.getCategories();
+
+    res.json({
+      success: true,
+      data: {
+        summary: stats,
+        hourly: hourlyStats,
+        categories: categories
+      }
+    });
+  } catch (error) {
+    console.error('获取日志统计失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取日志统计失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取日志分类
+router.get('/logs/categories', async (req, res) => {
+  try {
+    const categories = await LogsModel.getCategories();
+
+    res.json({
+      success: true,
+      data: categories
+    });
+  } catch (error) {
+    console.error('获取日志分类失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取日志分类失败',
+      message: error.message
+    });
+  }
+});
+
+// 清理旧记录
+router.delete('/cleanup', async (req, res) => {
+  try {
+    const { days = 30, type = 'all' } = req.body;
+
+    if (days < 1 || days > 365) {
+      return res.status(400).json({
+        success: false,
+        error: '保留天数必须在1-365之间'
+      });
+    }
+
+    let deletedHistory = 0;
+    let deletedLogs = 0;
+
+    if (type === 'all' || type === 'history') {
+      const historyResult = await HistoryModel.deleteOldRecords(days);
+      deletedHistory = historyResult.changes || 0;
+    }
+
+    if (type === 'all' || type === 'logs') {
+      const logsResult = await LogsModel.deleteOldRecords(days);
+      deletedLogs = logsResult.changes || 0;
+    }
+
+    res.json({
+      success: true,
+      message: `清理完成，删除了 ${deletedHistory} 条历史记录和 ${deletedLogs} 条日志记录`,
+      data: {
+        deleted_history: deletedHistory,
+        deleted_logs: deletedLogs,
+        days: days
+      }
+    });
+
+    // 记录清理操作日志
+    await LogsModel.logInfo(
+      `手动清理旧记录：删除${deletedHistory}条历史记录和${deletedLogs}条日志记录`,
+      'cleanup',
+      { deleted_history: deletedHistory, deleted_logs: deletedLogs, days: days },
+      'manual_action'
+    );
+  } catch (error) {
+    console.error('清理旧记录失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '清理旧记录失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取任务类型统计
+router.get('/task-types', async (req, res) => {
+  try {
+    const sql = `
+      SELECT DISTINCT task_type
+      FROM execution_history
+      ORDER BY task_type
+    `;
+
+    const taskTypes = await HistoryModel.all(sql);
+
+    res.json({
+      success: true,
+      data: taskTypes.map(row => row.task_type)
+    });
+  } catch (error) {
+    console.error('获取任务类型失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取任务类型失败',
+      message: error.message
+    });
+  }
+});
+
+// 导出历史记录
+router.get('/export', async (req, res) => {
+  try {
+    const taskType = req.query.taskType || null;
+    const status = req.query.status || null;
+    const format = req.query.format || 'json';
+
+    const histories = await HistoryModel.findAll(taskType, status, 1000, 0);
+
+    if (format === 'csv') {
+      // 生成CSV格式
+      const csvHeader = 'ID,任务类型,任务名称,状态,开始时间,结束时间,执行时长(ms),结果摘要,错误信息\n';
+      const csvData = histories.map(history => {
+        return [
+          history.id,
+          history.task_type,
+          history.task_name,
+          history.status,
+          history.start_time,
+          history.end_time || '',
+          history.duration || '',
+          `"${(history.result_summary || '').replace(/"/g, '""')}"`,
+          `"${(history.error_message || '').replace(/"/g, '""')}"`
+        ].join(',');
+      }).join('\n');
+
+      const csv = csvHeader + csvData;
+
+      res.setHeader('Content-Type', 'text/csv');
+      res.setHeader('Content-Disposition', `attachment; filename=execution_history_${new Date().toISOString().slice(0, 10)}.csv`);
+      res.send(csv);
+    } else {
+      // JSON格式
+      res.setHeader('Content-Type', 'application/json');
+      res.setHeader('Content-Disposition', `attachment; filename=execution_history_${new Date().toISOString().slice(0, 10)}.json`);
+      res.json({
+        success: true,
+        export_time: new Date().toISOString(),
+        filters: { taskType, status },
+        count: histories.length,
+        data: histories
+      });
+    }
+  } catch (error) {
+    console.error('导出历史记录失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '导出历史记录失败',
+      message: error.message
+    });
+  }
+});
+
+module.exports = router;

src/routes/proxies.js

@@ -0,0 +1,485 @@
+const express = require('express');
+const router = express.Router();
+const ProxyModel = require('../database/models/proxy');
+const ProxyValidator = require('../services/validator');
+const ProxyScraper = require('../services/scraper');
+
+const validator = new ProxyValidator();
+const scraper = new ProxyScraper();
+
+// 获取所有可用代理
+router.get('/', async (req, res) => {
+  try {
+    const limit = parseInt(req.query.limit) || 100;
+    const offset = parseInt(req.query.offset) || 0;
+    const sortBy = req.query.sortBy || 'response_time';
+    const order = req.query.order || 'ASC';
+
+    let sql = 'SELECT * FROM proxies WHERE is_valid = 1';
+    const params = [];
+
+    // 添加排序
+    const allowedSortFields = ['response_time', 'created_at', 'updated_at', 'ip', 'port'];
+    if (allowedSortFields.includes(sortBy)) {
+      sql += ` ORDER BY ${sortBy} ${order.toUpperCase() === 'DESC' ? 'DESC' : 'ASC'}`;
+    } else {
+      sql += ' ORDER BY response_time ASC';
+    }
+
+    // 添加分页
+    sql += ' LIMIT ? OFFSET ?';
+    params.push(limit, offset);
+
+    const proxies = await ProxyModel.all(sql, params);
+    const totalCount = await ProxyModel.count(true);
+
+    res.json({
+      success: true,
+      data: proxies,
+      pagination: {
+        total: totalCount,
+        limit: limit,
+        offset: offset,
+        hasMore: offset + limit < totalCount
+      }
+    });
+  } catch (error) {
+    console.error('获取代理列表失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取代理列表失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取随机可用代理
+router.get('/random', async (req, res) => {
+  try {
+    const count = Math.min(parseInt(req.query.count) || 1, 10); // 最多返回10个
+
+    if (count === 1) {
+      const proxy = await ProxyModel.findRandom(true, 1);
+      if (!proxy) {
+        return res.status(404).json({
+          success: false,
+          error: '没有可用的代理'
+        });
+      }
+
+      res.json({
+        success: true,
+        data: proxy
+      });
+    } else {
+      const proxies = await ProxyModel.findRandom(true, count);
+      res.json({
+        success: true,
+        data: proxies,
+        count: proxies.length
+      });
+    }
+  } catch (error) {
+    console.error('获取随机代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取随机代理失败',
+      message: error.message
+    });
+  }
+});
+
+// 获取代理统计信息
+router.get('/stats', async (req, res) => {
+  try {
+    const stats = await validator.getProxyStatistics();
+
+    res.json({
+      success: true,
+      data: stats
+    });
+  } catch (error) {
+    console.error('获取统计信息失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取统计信息失败',
+      message: error.message
+    });
+  }
+});
+
+// 手动验证代理
+router.post('/verify', async (req, res) => {
+  try {
+    const { ip, port } = req.body;
+
+    if (!ip || !port) {
+      return res.status(400).json({
+        success: false,
+        error: 'IP和端口不能为空'
+      });
+    }
+
+    const result = await validator.validateSingleProxy(ip, port);
+
+    res.json({
+      success: true,
+      data: result
+    });
+  } catch (error) {
+    console.error('验证代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '验证代理失败',
+      message: error.message
+    });
+  }
+});
+
+// 批量验证代理
+router.post('/verify-batch', async (req, res) => {
+  try {
+    const { proxies } = req.body;
+
+    if (!Array.isArray(proxies) || proxies.length === 0) {
+      return res.status(400).json({
+        success: false,
+        error: '代理列表不能为空'
+      });
+    }
+
+    // 验证格式
+    const validProxies = proxies.filter(p => p.ip && p.port);
+    if (validProxies.length === 0) {
+      return res.status(400).json({
+        success: false,
+        error: '没有有效的代理格式'
+      });
+    }
+
+    const results = await validator.validateMultipleProxies(validProxies, 3);
+
+    res.json({
+      success: true,
+      data: {
+        validated: results.length,
+        valid: results.filter(r => r.isValid).length,
+        invalid: results.filter(r => !r.isValid).length,
+        results: results
+      }
+    });
+  } catch (error) {
+    console.error('批量验证代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '批量验证代理失败',
+      message: error.message
+    });
+  }
+});
+
+// 手动触发抓取任务
+router.post('/scrape', async (req, res) => {
+  try {
+    const { pages = 40 } = req.body;
+
+    // if (pages < 1 || pages > 10) {
+    //   return res.status(400).json({
+    //     success: false,
+    //     error: '页数必须在1-10之间'
+    //   });
+    // }
+
+    // 检查是否有抓取任务正在进行
+    const ProxyScraper = require('../services/scraper');
+    if (ProxyScraper.isScrapingInProgress()) {
+      return res.status(409).json({
+        success: false,
+        error: '有抓取任务正在进行',
+        message: '请等待当前抓取任务完成后再试'
+      });
+    }
+
+    console.log(`手动触发抓取任务，页数: ${pages}`);
+
+    // 异步执行抓取任务
+    scraper.scrapeMultiplePages(pages).then(result => {
+      console.log('手动抓取任务完成:', result);
+    }).catch(error => {
+      console.error('手动抓取任务失败:', error);
+    });
+
+    res.json({
+      success: true,
+      message: `抓取任务已启动，将抓取前 ${pages} 页`
+    });
+  } catch (error) {
+    console.error('触发抓取任务失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '触发抓取任务失败',
+      message: error.message
+    });
+  }
+});
+
+// 手动触发验证任务
+router.post('/validate-all', async (req, res) => {
+  try {
+    const { limit = 50 } = req.body;
+
+    if (limit < 1 || limit > 200) {
+      return res.status(400).json({
+        success: false,
+        error: '验证数量必须在1-200之间'
+      });
+    }
+
+    console.log(`手动触发验证任务，数量: ${limit}`);
+
+    // 异步执行验证任务
+    validator.validateDatabaseProxies(limit).then(result => {
+      console.log('手动验证任务完成:', result);
+    }).catch(error => {
+      console.error('手动验证任务失败:', error);
+    });
+
+    res.json({
+      success: true,
+      message: `验证任务已启动，将验证 ${limit} 个代理`
+    });
+  } catch (error) {
+    console.error('触发验证任务失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '触发验证任务失败',
+      message: error.message
+    });
+  }
+});
+
+// 根据IP和端口查找代理
+router.get('/search', async (req, res) => {
+  try {
+    const { ip, port } = req.query;
+
+    if (!ip || !port) {
+      return res.status(400).json({
+        success: false,
+        error: 'IP和端口参数不能为空'
+      });
+    }
+
+    const proxy = await ProxyModel.findByIpAndPort(ip, port);
+
+    if (!proxy) {
+      return res.status(404).json({
+        success: false,
+        error: '未找到指定的代理'
+      });
+    }
+
+    res.json({
+      success: true,
+      data: proxy
+    });
+  } catch (error) {
+    console.error('查找代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '查找代理失败',
+      message: error.message
+    });
+  }
+});
+
+// 删除无效代理
+router.delete('/cleanup', async (req, res) => {
+  try {
+    const result = await ProxyModel.deleteInvalid();
+
+    res.json({
+      success: true,
+      message: `已删除 ${result.changes} 个无效代理`
+    });
+  } catch (error) {
+    console.error('清理无效代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '清理无效代理失败',
+      message: error.message
+    });
+  }
+});
+
+// 检查抓取任务状态
+router.get('/scraping-status', (req, res) => {
+  try {
+    const ProxyScraper = require('../services/scraper');
+    const isInProgress = ProxyScraper.isScrapingInProgress();
+
+    res.json({
+      success: true,
+      data: {
+        isScrapingInProgress: isInProgress,
+        message: isInProgress ? '有抓取任务正在进行' : '当前没有抓取任务'
+      }
+    });
+  } catch (error) {
+    console.error('获取抓取状态失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '获取抓取状态失败',
+      message: error.message
+    });
+  }
+});
+
+// 强制清除抓取标志（用于异常情况）
+router.post('/clear-scraping-flag', (req, res) => {
+  try {
+    const ProxyScraper = require('../services/scraper');
+    ProxyScraper.clearScrapingFlag();
+
+    res.json({
+      success: true,
+      message: '已强制清除抓取进行中标志'
+    });
+  } catch (error) {
+    console.error('清除抓取标志失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '清除抓取标志失败',
+      message: error.message
+    });
+  }
+});
+
+// 导入代理JSON
+router.post('/import', async (req, res) => {
+  try {
+    const { proxies } = req.body;
+
+    if (!Array.isArray(proxies) || proxies.length === 0) {
+      return res.status(400).json({
+        success: false,
+        error: '代理列表不能为空'
+      });
+    }
+
+    console.log(`开始导入 ${proxies.length} 个代理`);
+
+    // 验证格式并转换为标准格式
+    const validProxies = [];
+    const invalidProxies = [];
+
+    for (const proxy of proxies) {
+      let ip, port;
+
+      // 支持多种字段名
+      if (proxy.ip_address && proxy.port) {
+        ip = proxy.ip_address;
+        port = proxy.port;
+      } else if (proxy.ip && proxy.port) {
+        ip = proxy.ip;
+        port = proxy.port;
+      } else if (proxy.host && proxy.port) {
+        ip = proxy.host;
+        port = proxy.port;
+      } else {
+        invalidProxies.push(proxy);
+        continue;
+      }
+
+      // 验证IP和端口格式
+      if (/^\d+\.\d+\.\d+\.\d+$/.test(ip) && /^\d+$/.test(port)) {
+        const portNum = parseInt(port);
+        if (portNum > 0 && portNum <= 65535) {
+          validProxies.push({
+            ip: ip,
+            port: portNum,
+            location: proxy.location || proxy.area || null,
+            speed: proxy.speed || null,
+            last_check_time: new Date().toISOString().slice(0, 19).replace('T', ' ')
+          });
+        } else {
+          invalidProxies.push(proxy);
+        }
+      } else {
+        invalidProxies.push(proxy);
+      }
+    }
+
+    if (validProxies.length === 0) {
+      return res.status(400).json({
+        success: false,
+        error: '没有有效的代理格式',
+        details: {
+          total: proxies.length,
+          valid: 0,
+          invalid: invalidProxies.length
+        }
+      });
+    }
+
+    console.log(`验证完成，有效代理: ${validProxies.length}，无效代理: ${invalidProxies.length}`);
+
+    // 立即验证代理
+    const ProxyValidator = require('../services/validator');
+    const validator = new ProxyValidator();
+
+    const validationResult = await validator.validateMultipleProxies(validProxies, 10);
+
+    // 保存验证通过的代理到数据库
+    let savedCount = 0;
+    for (const result of validationResult) {
+      if (result.isValid) {
+        try {
+          await ProxyModel.create({
+            ip: result.ip,
+            port: result.port,
+            location: null,
+            speed: null,
+            last_check_time: new Date().toISOString().slice(0, 19).replace('T', ' '),
+            is_valid: 1,
+            response_time: result.responseTime
+          });
+          savedCount++;
+          console.log(`✓ 保存代理: ${result.ip}:${result.port} - ${result.responseTime}ms`);
+        } catch (error) {
+          if (!error.message.includes('UNIQUE constraint failed')) {
+            console.warn(`保存代理 ${result.ip}:${result.port} 失败:`, error.message);
+          }
+        }
+      }
+    }
+
+    const validCount = validationResult.filter(r => r.isValid).length;
+
+    console.log(`导入完成: 总数 ${proxies.length}，格式有效 ${validProxies.length}，验证通过 ${validCount}，保存成功 ${savedCount}`);
+
+    res.json({
+      success: true,
+      data: {
+        total: proxies.length,
+        format_valid: validProxies.length,
+        format_invalid: invalidProxies.length,
+        validated: validationResult.length,
+        valid: validCount,
+        invalid: validationResult.length - validCount,
+        saved: savedCount,
+        results: validationResult
+      },
+      message: `导入完成：保存了 ${savedCount} 个可用代理`
+    });
+  } catch (error) {
+    console.error('导入代理失败:', error);
+    res.status(500).json({
+      success: false,
+      error: '导入代理失败',
+      message: error.message
+    });
+  }
+});
+
+module.exports = router;

src/services/scheduler.js

@@ -0,0 +1,398 @@
+const cron = require('node-cron');
+const ProxyScraper = require('./scraper');
+const ProxyValidator = require('./validator');
+const ProxyModel = require('../database/models/proxy');
+const HistoryModel = require('../database/models/history');
+const LogsModel = require('../database/models/logs');
+
+class ProxyScheduler {
+  constructor() {
+    this.scraper = new ProxyScraper();
+    this.validator = new ProxyValidator();
+    this.tasks = new Map();
+  }
+
+  async log(message) {
+    const timestamp = new Date().toISOString().slice(0, 19).replace('T', ' ');
+    console.log(`[${timestamp}] ${message}`);
+  }
+
+  // 抓取任务 - 每小时执行一次
+  async runScrapeTask() {
+    let historyId = null;
+
+    try {
+      await this.log('开始执行定时抓取任务');
+
+      // 检查是否有抓取任务正在进行
+      if (this.scraper.constructor.isScrapingInProgress()) {
+        await this.log('⚠️  有抓取任务正在进行，跳过本次定时抓取任务');
+        return {
+          scraped: 0,
+          total: 0,
+          valid: 0,
+          skipped: true,
+          message: '有抓取任务正在进行，跳过本次执行'
+        };
+      }
+
+      // 创建执行历史记录
+      historyId = await HistoryModel.create({
+        task_type: 'scrape',
+        task_name: '定时抓取任务',
+        status: 'running',
+        start_time: new Date().toISOString()
+      });
+
+      // 抓取代理
+      const scrapeResult = await this.scraper.scrape();
+
+      // 更新历史记录
+      const duration = Date.now() - new Date(historyId ? await this.getHistoryStartTime(historyId) : Date.now());
+      await HistoryModel.update(historyId, {
+        status: 'success',
+        end_time: new Date().toISOString(),
+        duration: duration,
+        result_summary: `抓取 ${scrapeResult.scraped} 个代理，可用 ${scrapeResult.valid} 个`,
+        details: scrapeResult
+      });
+
+      if (scrapeResult.skipped) {
+        await this.log(`定时抓取任务跳过: ${scrapeResult.message}`);
+
+        // 更新历史记录
+        if (historyId) {
+          await HistoryModel.update(historyId, {
+            status: 'skipped',
+            end_time: new Date().toISOString(),
+            result_summary: scrapeResult.message,
+            details: scrapeResult
+          });
+        }
+      } else {
+        await this.log(`定时抓取任务完成: 抓取 ${scrapeResult.scraped} 个，总数 ${scrapeResult.total}，可用 ${scrapeResult.valid}`);
+
+        // 获取当前统计信息
+        const stats = await this.validator.getProxyStatistics();
+        await this.log(`当前代理状态: 总数 ${stats.total}，可用 ${stats.valid}，无效 ${stats.invalid}，可用率 ${stats.validRate}`);
+
+        // 记录系统日志
+        await LogsModel.logInfo(
+          `定时抓取任务完成: 抓取${scrapeResult.scraped}个代理，可用${scrapeResult.valid}个`,
+          'scheduler',
+          scrapeResult,
+          'automated_task'
+        );
+      }
+
+      return scrapeResult;
+    } catch (error) {
+      const errorMessage = `定时抓取任务失败: ${error.message}`;
+      await this.log(errorMessage);
+
+      // 更新历史记录
+      if (historyId) {
+        const duration = Date.now() - new Date(await this.getHistoryStartTime(historyId));
+        await HistoryModel.update(historyId, {
+          status: 'failed',
+          end_time: new Date().toISOString(),
+          duration: duration,
+          error_message: error.message
+        });
+      }
+
+      // 记录错误日志
+      await LogsModel.logError(
+        errorMessage,
+        'scheduler',
+        { error: error.message, stack: error.stack },
+        'automated_task'
+      );
+
+      throw error;
+    }
+  }
+
+  // 验证任务 - 每10分钟执行一次
+  async runValidationTask() {
+    let historyId = null;
+
+    try {
+      await this.log('开始执行定时验证任务');
+
+      // 创建执行历史记录
+      historyId = await HistoryModel.create({
+        task_type: 'validation',
+        task_name: '定时验证任务',
+        status: 'running',
+        start_time: new Date().toISOString()
+      });
+
+      // 验证数据库中的代理（每次验证50个）
+      const validationResult = await this.validator.validateDatabaseProxies(50);
+
+      // 更新历史记录
+      const duration = Date.now() - new Date(historyId ? await this.getHistoryStartTime(historyId) : Date.now());
+      await HistoryModel.update(historyId, {
+        status: 'success',
+        end_time: new Date().toISOString(),
+        duration: duration,
+        result_summary: `验证 ${validationResult.validated} 个代理，有效 ${validationResult.valid} 个`,
+        details: validationResult
+      });
+
+      await this.log(`定时验证任务完成: 验证 ${validationResult.validated} 个，有效 ${validationResult.valid}，无效 ${validationResult.invalid}`);
+
+      // 清理无效代理
+      if (validationResult.invalid > 0) {
+        await this.log(`已清理 ${validationResult.invalid} 个无效代理`);
+      }
+
+      // 获取当前统计信息
+      const stats = await this.validator.getProxyStatistics();
+      await this.log(`当前代理状态: 总数 ${stats.total}，可用 ${stats.valid}，无效 ${stats.invalid}，可用率 ${stats.validRate}`);
+
+      // 记录系统日志
+      await LogsModel.logInfo(
+        `定时验证任务完成: 验证${validationResult.validated}个代理，有效${validationResult.valid}个`,
+        'scheduler',
+        validationResult,
+        'automated_task'
+      );
+
+      return validationResult;
+    } catch (error) {
+      const errorMessage = `定时验证任务失败: ${error.message}`;
+      await this.log(errorMessage);
+
+      // 更新历史记录
+      if (historyId) {
+        const duration = Date.now() - new Date(await this.getHistoryStartTime(historyId));
+        await HistoryModel.update(historyId, {
+          status: 'failed',
+          end_time: new Date().toISOString(),
+          duration: duration,
+          error_message: error.message
+        });
+      }
+
+      // 记录错误日志
+      await LogsModel.logError(
+        errorMessage,
+        'scheduler',
+        { error: error.message, stack: error.stack },
+        'automated_task'
+      );
+
+      throw error;
+    }
+  }
+
+  // 健康检查任务 - 每小时执行一次
+  async runHealthCheckTask() {
+    try {
+      await this.log('开始执行健康检查任务');
+
+      const stats = await this.validator.getProxyStatistics();
+
+      await this.log(`代理池健康状态: 总数 ${stats.total}，可用 ${stats.valid}，无效 ${stats.invalid}，可用率 ${stats.validRate}`);
+
+      // 如果可用代理太少，触发紧急抓取
+      if (stats.valid < 10) {
+        await this.log('可用代理数量过少，触发紧急抓取任务');
+        await this.runScrapeTask();
+      }
+
+      return stats;
+    } catch (error) {
+      await this.log(`健康检查任务失败: ${error.message}`);
+      throw error;
+    }
+  }
+
+  // 启动所有定时任务
+  start() {
+    this.log('启动代理服务定时任务调度器');
+
+    // 每小时的第0分钟执行抓取任务（异步执行，不阻塞其他任务）
+    const scrapeTask = cron.schedule('0 * * * *', async () => {
+      try {
+        // 使用 setImmediate 确保任务异步执行，不阻塞事件循环
+        setImmediate(async () => {
+          try {
+            await this.runScrapeTask();
+          } catch (error) {
+            this.log(`定时抓取任务异常: ${error.message}`);
+          }
+        });
+      } catch (error) {
+        this.log(`定时抓取任务启动异常: ${error.message}`);
+      }
+    }, {
+      scheduled: false,
+      timezone: 'Asia/Shanghai'
+    });
+
+    // 每10分钟执行验证任务（异步执行，不阻塞其他任务）
+    const validationTask = cron.schedule('*/10 * * * *', async () => {
+      try {
+        // 使用 setImmediate 确保任务异步执行，不阻塞事件循环
+        setImmediate(async () => {
+          try {
+            await this.runValidationTask();
+          } catch (error) {
+            this.log(`定时验证任务异常: ${error.message}`);
+          }
+        });
+      } catch (error) {
+        this.log(`定时验证任务启动异常: ${error.message}`);
+      }
+    }, {
+      scheduled: false,
+      timezone: 'Asia/Shanghai'
+    });
+
+    // 每小时的第30分钟执行健康检查（异步执行，不阻塞其他任务）
+    const healthCheckTask = cron.schedule('30 * * * *', async () => {
+      try {
+        // 使用 setImmediate 确保任务异步执行，不阻塞事件循环
+        setImmediate(async () => {
+          try {
+            await this.runHealthCheckTask();
+          } catch (error) {
+            this.log(`健康检查任务异常: ${error.message}`);
+          }
+        });
+      } catch (error) {
+        this.log(`健康检查任务启动异常: ${error.message}`);
+      }
+    }, {
+      scheduled: false,
+      timezone: 'Asia/Shanghai'
+    });
+
+    // 保存任务引用
+    this.tasks.set('scrape', scrapeTask);
+    this.tasks.set('validation', validationTask);
+    this.tasks.set('healthCheck', healthCheckTask);
+
+    // 启动所有任务
+    this.tasks.forEach((task, name) => {
+      task.start();
+      this.log(`定时任务已启动: ${name}`);
+    });
+
+    this.log('所有定时任务已启动');
+    this.printNextRunTimes();
+  }
+
+  // 停止所有定时任务
+  stop() {
+    this.log('停止所有定时任务');
+
+    this.tasks.forEach((task, name) => {
+      task.stop();
+      this.log(`定时任务已停止: ${name}`);
+    });
+
+    this.tasks.clear();
+    this.log('定时任务调度器已停止');
+  }
+
+  // 手动执行任务
+  async runTask(taskName) {
+    this.log(`手动执行任务: ${taskName}`);
+
+    switch (taskName) {
+      case 'scrape':
+        return await this.runScrapeTask();
+      case 'validation':
+        return await this.runValidationTask();
+      case 'healthCheck':
+        return await this.runHealthCheckTask();
+      default:
+        throw new Error(`未知的任务名称: ${taskName}`);
+    }
+  }
+
+  // 获取任务状态
+  getStatus() {
+    const status = {};
+
+    this.tasks.forEach((task, name) => {
+      status[name] = {
+        running: task.running || false,
+        scheduled: task.scheduled || false
+      };
+    });
+
+    return {
+      tasks: status,
+      taskCount: this.tasks.size
+    };
+  }
+
+  // 打印下次执行时间
+  printNextRunTimes() {
+    const now = new Date();
+
+    this.log('定时任务执行计划:');
+    this.log(`- 抓取任务: 每小时整点执行 (下次: ${this.getNextHourTime(now)})`);
+    this.log(`- 验证任务: 每10分钟执行 (下次: ${this.getNext10MinuteTime(now)})`);
+    this.log(`- 健康检查: 每小时30分执行 (下次: ${this.getNextHour30Time(now)})`);
+  }
+
+  getNextHourTime(now) {
+    const next = new Date(now);
+    next.setHours(next.getHours() + 1, 0, 0, 0);
+    return next.toTimeString().slice(0, 5);
+  }
+
+  getNext10MinuteTime(now) {
+    const next = new Date(now);
+    const minutes = next.getMinutes();
+    const nextMinute = Math.ceil((minutes + 1) / 10) * 10;
+    next.setMinutes(nextMinute, 0, 0);
+    return next.toTimeString().slice(0, 5);
+  }
+
+  getNextHour30Time(now) {
+    const next = new Date(now);
+    if (now.getMinutes() < 30) {
+      next.setHours(now.getHours(), 30, 0, 0);
+    } else {
+      next.setHours(now.getHours() + 1, 30, 0, 0);
+    }
+    return next.toTimeString().slice(0, 5);
+  }
+
+  // 获取历史记录开始时间
+  async getHistoryStartTime(historyId) {
+    try {
+      const history = await HistoryModel.findById(historyId);
+      return history ? new Date(history.start_time).getTime() : Date.now();
+    } catch (error) {
+      return Date.now();
+    }
+  }
+
+  // 获取系统统计信息
+  async getSystemStats() {
+    try {
+      const proxyStats = await this.validator.getProxyStatistics();
+      const taskStatus = this.getStatus();
+
+      return {
+        timestamp: new Date().toISOString(),
+        proxies: proxyStats,
+        scheduler: taskStatus,
+        uptime: process.uptime()
+      };
+    } catch (error) {
+      this.log(`获取系统统计信息失败: ${error.message}`);
+      throw error;
+    }
+  }
+}
+
+module.exports = ProxyScheduler;

src/services/scraper.js

@@ -0,0 +1,596 @@
+const axios = require('axios');
+const cheerio = require('cheerio');
+const ProxyModel = require('../database/models/proxy');
+
+// 全局变量：标记是否有抓取任务正在进行
+let isScrapingInProgress = false;
+
+class ProxyScraper {
+  constructor() {
+    this.baseUrl = 'https://www.kuaidaili.com/free/inha';
+    this.userAgents = [
+      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
+      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
+      'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0',
+      'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
+      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/91.0.864.59'
+    ];
+    this.currentProxyIndex = 0;
+    this.localProxies = [];
+  }
+
+  getRandomUserAgent() {
+    return this.userAgents[Math.floor(Math.random() * this.userAgents.length)];
+  }
+
+  async sleep(ms) {
+    return new Promise(resolve => setTimeout(resolve, ms));
+  }
+
+  // 获取本地数据库中的有效代理
+  async loadLocalProxies() {
+    try {
+      console.log('正在加载本地代理数据库...');
+      this.localProxies = await ProxyModel.findAll(true);
+      this.currentProxyIndex = 0;
+      console.log(`成功加载 ${this.localProxies.length} 个本地代理`);
+
+      if (this.localProxies.length > 0) {
+        console.log('将使用本地代理进行抓取');
+      } else {
+        console.log('本地无可用代理，将使用直连');
+      }
+    } catch (error) {
+      console.error('加载本地代理失败:', error);
+      this.localProxies = [];
+    }
+  }
+
+  // 获取下一个本地代理（循环使用）
+  getNextLocalProxy() {
+    if (this.localProxies.length === 0) {
+      return null; // 没有本地代理，返回null使用直连
+    }
+
+    const proxy = this.localProxies[this.currentProxyIndex];
+    this.currentProxyIndex = (this.currentProxyIndex + 1) % this.localProxies.length;
+
+    console.log(`使用本地代理: ${proxy.ip}:${proxy.port}`);
+    return {
+      host: proxy.ip,
+      port: proxy.port,
+      protocol: 'http'
+    };
+  }
+
+  // 测试代理是否可用（用于抓取）
+  async testProxyForScraping(proxyConfig) {
+    try {
+      const response = await axios.get('https://www.baidu.com', {
+        proxy: proxyConfig,
+        timeout: 10000,
+        headers: {
+          'User-Agent': this.getRandomUserAgent()
+        },
+        validateStatus: (status) => status === 200
+      });
+      return response.status === 200 && response.data.includes('百度');
+    } catch (error) {
+      return false;
+    }
+  }
+
+  // 获取可用的代理配置
+  async getWorkingProxy() {
+    if (this.localProxies.length === 0) {
+      return null; // 无本地代理，使用直连
+    }
+
+    // 尝试几个代理，找到可用的
+    for (let i = 0; i < Math.min(5, this.localProxies.length); i++) {
+      const proxyConfig = this.getNextLocalProxy();
+
+      if (await this.testProxyForScraping(proxyConfig)) {
+        console.log(`✓ 代理 ${proxyConfig.host}:${proxyConfig.port} 可用`);
+        return proxyConfig;
+      } else {
+        console.log(`✗ 代理 ${proxyConfig.host}:${proxyConfig.port} 不可用，尝试下一个`);
+      }
+    }
+
+    console.log('测试的本地代理都不可用，使用直连');
+    return null; // 所有测试的代理都不可用，使用直连
+  }
+
+  // 验证抓取到的代理
+  async validateScrapedProxies(proxies, concurrency = 20) {
+    if (!proxies || proxies.length === 0) {
+      return [];
+    }
+
+    console.log(`开始验证 ${proxies.length} 个抓取到的代理，并发数: ${concurrency}`);
+    const validProxies = [];
+
+    // 分批处理，避免过多并发连接
+    for (let i = 0; i < proxies.length; i += concurrency) {
+      const batch = proxies.slice(i, i + concurrency);
+      const batchPromises = batch.map(proxy => this.validateScrapedProxy(proxy));
+
+      try {
+        const batchResults = await Promise.all(batchPromises);
+
+        // 收集验证通过的代理
+        batchResults.forEach(result => {
+          if (result.isValid) {
+            validProxies.push({
+              ...result.proxy,
+              is_valid: 1,
+              response_time: result.responseTime
+            });
+            console.log(`✓ 代理验证通过: ${result.proxy.ip}:${result.proxy.port} - ${result.responseTime}ms`);
+          } else {
+            console.log(`✗ 代理验证失败: ${result.proxy.ip}:${result.proxy.port} - ${result.error || '连接失败'}`);
+          }
+        });
+
+        // 批次间延迟，避免请求过于频繁
+        if (i + concurrency < proxies.length) {
+          await this.sleep(1000);
+        }
+
+      } catch (error) {
+        console.error(`批量验证第 ${Math.floor(i / concurrency) + 1} 批时发生错误:`, error);
+      }
+    }
+
+    console.log(`代理验证完成，${validProxies.length}/${proxies.length} 个代理验证通过`);
+    return validProxies;
+  }
+
+  // 验证单个抓取到的代理
+  async validateScrapedProxy(proxy, retryCount = 2) {
+    const testUrls = [
+      'https://www.baidu.com',
+    ];
+
+    for (let attempt = 1; attempt <= retryCount; attempt++) {
+      for (const testUrl of testUrls) {
+        const startTime = Date.now();
+
+        try {
+          const proxyConfig = {
+            host: proxy.ip,
+            port: proxy.port,
+            protocol: 'http'
+          };
+
+          const response = await axios.get(testUrl, {
+            proxy: proxyConfig,
+            timeout: 10000, // 3秒超时
+            headers: {
+              'User-Agent': this.getRandomUserAgent(),
+              'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
+              'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
+              'Connection': 'keep-alive'
+            },
+            validateStatus: (status) => status >= 200 && status < 300 // 接受2xx状态码
+          });
+
+          const responseTime = Date.now() - startTime;
+          let isValid = false;
+
+          // 检查响应内容
+          if (response.status === 200) {
+            if (testUrl.includes('baidu.com')) {
+              isValid = response.data.includes('百度');
+            } else if (testUrl.includes('httpbin.org')) {
+              isValid = response.data.includes('origin');
+            } else if (testUrl.includes('google.com')) {
+              isValid = response.data.includes('google');
+            } else {
+              isValid = true; // 对于其他URL，只要能连接就认为有效
+            }
+          }
+
+          if (isValid) {
+            console.log(`✓ 代理验证通过: ${proxy.ip}:${proxy.port} - ${testUrl} - ${responseTime}ms`);
+            return {
+              proxy: proxy,
+              isValid: true,
+              responseTime: responseTime,
+              error: null,
+              testUrl: testUrl
+            };
+          }
+
+        } catch (error) {
+          const responseTime = Date.now() - startTime;
+          // 如果是最后一次尝试，返回失败
+          if (attempt === retryCount && testUrl === testUrls[testUrls.length - 1]) {
+            return {
+              proxy: proxy,
+              isValid: false,
+              responseTime: responseTime,
+              error: error.message,
+              testUrl: testUrl
+            };
+          }
+          // 否则继续尝试下一个URL或重试
+          await this.sleep(500);
+        }
+      }
+    }
+
+    // 所有尝试都失败
+    return {
+      proxy: proxy,
+      isValid: false,
+      responseTime: 0,
+      error: 'All validation attempts failed',
+      testUrl: null
+    };
+  }
+
+  async fetchPage(pageNum, retryCount = 3) {
+    const url = pageNum === 1 ? `${this.baseUrl}/` : `${this.baseUrl}/${pageNum}/`;
+
+    console.log(`正在抓取第 ${pageNum} 页: ${url}`);
+
+    for (let attempt = 1; attempt <= retryCount; attempt++) {
+      let proxyConfig = null;
+      let proxyUsed = '';
+
+      try {
+        const userAgent = this.getRandomUserAgent();
+
+        // 获取可用代理配置（每次请求都尝试不同的代理）
+        proxyConfig = await this.getWorkingProxy();
+
+        const requestConfig = {
+          headers: {
+            'User-Agent': userAgent,
+            'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
+            'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
+            'Accept-Encoding': 'gzip, deflate, br',
+            'Connection': 'keep-alive',
+            'Upgrade-Insecure-Requests': '1',
+            'Sec-Fetch-Dest': 'document',
+            'Sec-Fetch-Mode': 'navigate',
+            'Sec-Fetch-Site': 'none',
+            'Sec-Fetch-User': '?1',
+            'Cache-Control': 'max-age=0',
+            'sec-ch-ua': '"Google Chrome";v="119", "Chromium";v="119", "Not?A_Brand";v="24"',
+            'sec-ch-ua-mobile': '?0',
+            'sec-ch-ua-platform': '"Windows"',
+            'Referer': 'https://www.kuaidaili.com/free/',
+            'Origin': 'https://www.kuaidaili.com'
+          },
+          timeout: 15000,
+          validateStatus: (status) => status === 200,
+          maxRedirects: 3
+        };
+
+        // 如果有可用代理，添加到请求配置中
+        if (proxyConfig) {
+          requestConfig.proxy = proxyConfig;
+          proxyUsed = `代理 ${proxyConfig.host}:${proxyConfig.port}`;
+          console.log(`使用代理 ${proxyConfig.host}:${proxyConfig.port} 抓取第 ${pageNum} 页`);
+        } else {
+          proxyUsed = '直连';
+          console.log(`使用直连抓取第 ${pageNum} 页`);
+        }
+
+        const response = await axios.get(url, requestConfig);
+
+        // 检查响应内容
+        const content = response.data;
+        if (content.includes('访问过于频繁') || content.includes('请稍后再试')) {
+          console.log(`第 ${pageNum} 页被限制访问，延长等待时间`);
+          if (attempt < retryCount) {
+            await this.sleep(5000 * attempt);
+            continue;
+          }
+        }
+
+        console.log(`第 ${pageNum} 页获取成功，内容长度: ${content.length}`);
+        return content;
+      } catch (error) {
+        console.error(`第 ${attempt} 次尝试抓取第 ${pageNum} 页失败 (${proxyUsed}):`, error.message);
+
+        if (attempt === retryCount) {
+          throw new Error(`抓取第 ${pageNum} 页失败，已重试 ${retryCount} 次: ${error.message}`);
+        }
+
+        // 递增等待时间
+        await this.sleep(3000 * attempt);
+      }
+    }
+  }
+
+  extractProxiesFromHtml(html) {
+    try {
+      const $ = cheerio.load(html);
+      const proxies = [];
+
+      console.log('开始解析HTML，页面标题:', $('title').text());
+
+      // 方法1: 尝试从JavaScript变量中提取
+      const scripts = $('script').map((i, elem) => $(elem).html()).get();
+      console.log(`找到 ${scripts.length} 个script标签`);
+
+      for (let i = 0; i < scripts.length; i++) {
+        const script = scripts[i];
+
+        // 尝试多种匹配模式 - 优先使用 const 模式
+        const patterns = [
+          /const fpsList = (\[[\s\S]*?\]);/,
+          /var fpsList = (\[[\s\S]*?\]);/,
+          /let fpsList = (\[[\s\S]*?\]);/,
+          /fpsList = (\[[\s\S]*?\]);/,
+          /var\s+fpsList\s*=\s*(\[[\s\S]*?\]);/,
+          /const\s+fpsList\s*=\s*(\[[\s\S]*?\]);/,
+          /let\s+fpsList\s*=\s*(\[[\s\S]*?\]);/
+        ];
+
+        for (const pattern of patterns) {
+          const match = script.match(pattern);
+          if (match) {
+            try {
+              console.log('找到JavaScript变量，尝试解析...');
+              const fpsList = eval(match[1]);
+              console.log(`从JavaScript变量中提取到 ${fpsList.length} 个代理`);
+
+              if (Array.isArray(fpsList) && fpsList.length > 0) {
+                return fpsList.map(item => ({
+                  ip: item.ip,
+                  port: parseInt(item.port), // 确保端口是数字
+                  location: item.location || null,
+                  speed: item.speed ? parseInt(item.speed) : null,
+                  last_check_time: item.last_check_time || new Date().toISOString().slice(0, 19).replace('T', ' ')
+                }));
+              }
+            } catch (e) {
+              console.warn('解析JavaScript变量失败:', e.message);
+            }
+          }
+        }
+      }
+
+      // 方法2: 尝试从表格中提取 - 使用多种选择器
+      const tableSelectors = [
+        'table tbody tr',
+        '.table tbody tr',
+        '#list table tbody tr',
+        '.list table tbody tr',
+        'table tr',
+        '.proxy-table tbody tr'
+      ];
+
+      for (const selector of tableSelectors) {
+        const rows = $(selector);
+        console.log(`尝试选择器 "${selector}": 找到 ${rows.length} 行`);
+
+        if (rows.length > 0) {
+          rows.each((i, row) => {
+            try {
+              const cols = $(row).find('td');
+
+              // 尝试多种列结构
+              let ip, port, location, speed;
+
+              if (cols.length >= 2) {
+                // 标准格式：IP、端口、位置、匿名度、类型、位置、速度、最后验证时间
+                ip = $(cols[0]).text().trim();
+                port = $(cols[1]).text().trim();
+
+                if (cols.length >= 5) {
+                  location = $(cols[4]).text().trim();
+                }
+                if (cols.length >= 6) {
+                  speed = $(cols[5]).text().trim();
+                }
+
+                // 清理和验证数据
+                ip = ip.replace(/[^\d\.]/g, '');
+                port = port.replace(/[^\d]/g, '');
+
+                if (ip && port && /^\d+\.\d+\.\d+\.\d+$/.test(ip) && /^\d+$/.test(port)) {
+                  const portNum = parseInt(port);
+                  if (portNum > 0 && portNum <= 65535) {
+                    proxies.push({
+                      ip: ip,
+                      port: portNum,
+                      location: location || null,
+                      speed: speed ? parseInt(speed.replace(/[^\d]/g, '')) : null,
+                      last_check_time: new Date().toISOString().slice(0, 19).replace('T', ' ')
+                    });
+                  }
+                }
+              }
+            } catch (e) {
+              console.warn(`解析第 ${i} 行失败:`, e.message);
+            }
+          });
+
+          if (proxies.length > 0) {
+            console.log(`从表格中提取到 ${proxies.length} 个代理`);
+            break;
+          }
+        }
+      }
+
+      // 方法3: 尝试从JSON数据中提取
+      const jsonPatterns = [
+        /"data":\s*(\[[\s\S]*?\])/,
+        /"proxies":\s*(\[[\s\S]*?\])/,
+        /window\.__INITIAL_STATE__\s*=\s*({[\s\S]*?});/
+      ];
+
+      const pageContent = $.html();
+      for (const pattern of jsonPatterns) {
+        const match = pageContent.match(pattern);
+        if (match) {
+          try {
+            console.log('找到JSON数据，尝试解析...');
+            const data = JSON.parse(match[1]);
+            // 处理JSON数据格式
+            if (Array.isArray(data)) {
+              const jsonProxies = data.map(item => ({
+                ip: item.ip || item.IP,
+                port: item.port || item.PORT,
+                location: item.location || item.area || null,
+                speed: item.speed || null,
+                last_check_time: new Date().toISOString().slice(0, 19).replace('T', ' ')
+              })).filter(p => p.ip && p.port);
+
+              if (jsonProxies.length > 0) {
+                console.log(`从JSON数据中提取到 ${jsonProxies.length} 个代理`);
+                return jsonProxies;
+              }
+            }
+          } catch (e) {
+            console.warn('解析JSON数据失败:', e.message);
+          }
+        }
+      }
+
+      console.log(`所有提取方法都失败，最终提取到 ${proxies.length} 个代理`);
+
+      // 如果没有找到代理，输出页面内容用于调试
+      if (proxies.length === 0) {
+        console.log('页面内容片段:', pageContent.substring(0, 500) + '...');
+      }
+
+      return proxies;
+    } catch (error) {
+      console.error('提取代理数据失败:', error);
+      return [];
+    }
+  }
+
+  async scrapePage(pageNum) {
+    try {
+      const html = await this.fetchPage(pageNum);
+      const proxies = this.extractProxiesFromHtml(html);
+
+      if (proxies.length === 0) {
+        console.warn(`第 ${pageNum} 页没有提取到代理数据`);
+        return [];
+      }
+
+      console.log(`第 ${pageNum} 页处理完成，获取到 ${proxies.length} 个代理，开始验证...`);
+
+      // 立即验证当前页面的所有代理
+      const validatedProxies = await this.validateScrapedProxies(proxies);
+
+      let savedCount = 0;
+      for (const proxy of validatedProxies) {
+        try {
+          await ProxyModel.create(proxy);
+          savedCount++;
+          console.log(`✓ 保存验证通过的代理: ${proxy.ip}:${proxy.port}`);
+        } catch (error) {
+          // 忽略重复插入错误
+          if (!error.message.includes('UNIQUE constraint failed')) {
+            console.warn(`保存代理 ${proxy.ip}:${proxy.port} 失败:`, error.message);
+          }
+        }
+      }
+
+      console.log(`第 ${pageNum} 页验证完成，验证通过 ${validatedProxies.length}/${proxies.length} 个代理，保存了 ${savedCount} 个新代理`);
+
+      return validatedProxies;
+
+    } catch (error) {
+      console.error(`处理第 ${pageNum} 页失败:`, error);
+      return [];
+    }
+  }
+
+  async scrapeMultiplePages(maxPages = 40) {
+    console.log(`开始抓取前 ${maxPages} 页代理IP`);
+
+    // 首先加载本地代理
+    await this.loadLocalProxies();
+
+    const allValidatedProxies = [];
+
+    for (let page = 1; page <= maxPages; page++) {
+      try {
+        const validatedProxies = await this.scrapePage(page);
+        allValidatedProxies.push(...validatedProxies);
+
+        // 页面间延迟，避免请求过于频繁
+        if (page < maxPages) {
+          await this.sleep(3000 + Math.random() * 2000);
+        }
+      } catch (error) {
+        console.error(`抓取第 ${page} 页失败，跳过此页:`, error);
+      }
+    }
+
+    const totalValidated = allValidatedProxies.length;
+    console.log(`抓取完成，共获取到 ${totalValidated} 个验证通过的代理IP`);
+
+    return totalValidated;
+  }
+
+  async scrape() {
+    console.log('开始执行代理IP抓取任务');
+
+    // 检查是否有抓取任务正在进行
+    if (isScrapingInProgress) {
+      console.log('⚠️  有抓取任务正在进行，跳过本次抓取');
+      const totalCount = await ProxyModel.count();
+      const validCount = await ProxyModel.count(true);
+
+      return {
+        scraped: 0,
+        total: totalCount,
+        valid: validCount,
+        skipped: true
+      };
+    }
+
+    // 设置抓取进行中标志
+    isScrapingInProgress = true;
+    console.log('🚀 开始抓取，设置抓取进行中标志');
+
+    try {
+      const validatedCount = await this.scrapeMultiplePages(40);
+
+      // 获取数据库中的总代理数量
+      const totalCount = await ProxyModel.count();
+      const validCount = await ProxyModel.count(true);
+
+      console.log(`抓取任务完成。数据库中共有 ${totalCount} 个代理，其中 ${validCount} 个可用`);
+
+      return {
+        scraped: validatedCount,
+        total: totalCount,
+        valid: validCount,
+        skipped: false
+      };
+    } catch (error) {
+      console.error('代理抓取任务失败:', error);
+      throw error;
+    } finally {
+      // 无论成功还是失败，都要清除抓取进行中标志
+      isScrapingInProgress = false;
+      console.log('✅ 清除抓取进行中标志');
+    }
+  }
+
+  // 静态方法：检查是否有抓取任务正在进行
+  static isScrapingInProgress() {
+    return isScrapingInProgress;
+  }
+
+  // 静态方法：强制清除抓取标志（用于异常情况）
+  static clearScrapingFlag() {
+    isScrapingInProgress = false;
+    console.log('🔧 强制清除抓取进行中标志');
+  }
+}
+
+module.exports = ProxyScraper;

src/services/validator.js

@@ -0,0 +1,202 @@
+const axios = require('axios');
+const ProxyModel = require('../database/models/proxy');
+
+class ProxyValidator {
+  constructor() {
+    this.testUrl = 'https://www.baidu.com';
+    this.timeout = 10000; // 3秒超时
+    this.userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36';
+  }
+
+  async validateProxy(ip, port) {
+    const startTime = Date.now();
+    const proxy = {
+      host: ip,
+      port: port,
+      protocol: 'http'
+    };
+
+    console.log(`正在验证代理 ${ip}:${port}`);
+
+    try {
+      const response = await axios.get(this.testUrl, {
+        proxy: proxy,
+        timeout: this.timeout,
+        headers: {
+          'User-Agent': this.userAgent,
+          'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
+          'Accept-Language': 'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
+          'Accept-Encoding': 'gzip, deflate',
+          'Connection': 'keep-alive'
+        },
+        validateStatus: (status) => status === 200
+      });
+
+      const responseTime = Date.now() - startTime;
+      const isValid = response.status === 200 && response.data.includes('百度');
+
+      if (isValid) {
+        console.log(`✓ 代理 ${ip}:${port} 验证成功，响应时间: ${responseTime}ms`);
+      } else {
+        console.log(`✗ 代理 ${ip}:${port} 验证失败，响应不正确`);
+      }
+
+      // 更新数据库中的验证结果
+      await ProxyModel.updateValidity(ip, port, isValid ? 1 : 0, responseTime);
+
+      return {
+        ip: ip,
+        port: port,
+        isValid: isValid,
+        responseTime: responseTime,
+        error: null
+      };
+    } catch (error) {
+      const responseTime = Date.now() - startTime;
+      console.log(`✗ 代理 ${ip}:${port} 验证失败:`, error.message);
+
+      // 更新数据库中的验证结果
+      await ProxyModel.updateValidity(ip, port, 0, responseTime);
+
+      return {
+        ip: ip,
+        port: port,
+        isValid: false,
+        responseTime: responseTime,
+        error: error.message
+      };
+    }
+  }
+
+  async validateSingleProxy(ip, port) {
+    try {
+      const result = await this.validateProxy(ip, port);
+      return result;
+    } catch (error) {
+      console.error(`验证代理 ${ip}:${port} 时发生错误:`, error);
+      return {
+        ip: ip,
+        port: port,
+        isValid: false,
+        responseTime: null,
+        error: error.message
+      };
+    }
+  }
+
+  async validateMultipleProxies(proxies, concurrency = 20) {
+    console.log(`开始批量验证 ${proxies.length} 个代理，并发数: ${concurrency}`);
+    const results = [];
+
+    // 分批处理，避免过多并发连接
+    for (let i = 0; i < proxies.length; i += concurrency) {
+      const batch = proxies.slice(i, i + concurrency);
+      const batchPromises = batch.map(proxy =>
+        this.validateProxy(proxy.ip, proxy.port)
+      );
+
+      try {
+        const batchResults = await Promise.all(batchPromises);
+        results.push(...batchResults);
+      } catch (error) {
+        console.error(`批量验证第 ${Math.floor(i / concurrency) + 1} 批时发生错误:`, error);
+      }
+
+      // 批次间延迟
+      if (i + concurrency < proxies.length) {
+        await this.sleep(1000);
+      }
+    }
+
+    const validCount = results.filter(r => r.isValid).length;
+    console.log(`批量验证完成，${validCount}/${results.length} 个代理可用`);
+
+    return results;
+  }
+
+  async validateDatabaseProxies(limit = 50) {
+    console.log(`开始验证数据库中的代理IP，限制数量: ${limit}`);
+
+    try {
+      // 获取待验证的代理（优先验证最久未验证的）
+      const proxies = await ProxyModel.findAllForValidation(limit);
+
+      if (proxies.length === 0) {
+        console.log('没有需要验证的代理');
+        return { validated: 0, valid: 0, invalid: 0 };
+      }
+
+      console.log(`找到 ${proxies.length} 个待验证的代理`);
+
+      const results = await this.validateMultipleProxies(proxies, 10);
+
+      // 清理无效代理
+      const invalidCount = results.filter(r => !r.isValid).length;
+      if (invalidCount > 0) {
+        await ProxyModel.deleteInvalid();
+        console.log(`已删除 ${invalidCount} 个无效代理`);
+      }
+
+      const validCount = results.filter(r => r.isValid).length;
+
+      return {
+        validated: results.length,
+        valid: validCount,
+        invalid: invalidCount,
+        results: results
+      };
+    } catch (error) {
+      console.error('验证数据库代理失败:', error);
+      throw error;
+    }
+  }
+
+  async getProxyStatistics() {
+    try {
+      const totalCount = await ProxyModel.count(false);
+      const validCount = await ProxyModel.count(true);
+      const invalidCount = totalCount - validCount;
+
+      return {
+        total: totalCount,
+        valid: validCount,
+        invalid: invalidCount,
+        validRate: totalCount > 0 ? ((validCount / totalCount) * 100).toFixed(2) + '%' : '0%'
+      };
+    } catch (error) {
+      console.error('获取代理统计信息失败:', error);
+      return {
+        total: 0,
+        valid: 0,
+        invalid: 0,
+        validRate: '0%'
+      };
+    }
+  }
+
+  async sleep(ms) {
+    return new Promise(resolve => setTimeout(resolve, ms));
+  }
+
+  // 验证新抓取的代理
+  async validateNewProxies(proxies) {
+    if (!proxies || proxies.length === 0) {
+      return { validated: 0, valid: 0, invalid: 0 };
+    }
+
+    console.log(`开始验证新抓取的 ${proxies.length} 个代理`);
+    const results = await this.validateMultipleProxies(proxies, 2);
+
+    const validCount = results.filter(r => r.isValid).length;
+    console.log(`新代理验证完成，${validCount}/${results.length} 个代理可用`);
+
+    return {
+      validated: results.length,
+      valid: validCount,
+      invalid: results.length - validCount,
+      results: results
+    };
+  }
+}
+
+module.exports = ProxyValidator;

test-scraper-with-validation.js

@@ -0,0 +1,47 @@
+const ProxyScraper = require('./src/services/scraper');
+const Database = require('./src/database/db');
+const ProxyModel = require('./src/database/models/proxy');
+
+async function testScraperWithValidation() {
+    console.log('测试抓取器带验证功能...');
+
+    try {
+        // 初始化数据库连接
+        console.log('初始化数据库连接...');
+        await Database.connect();
+        await ProxyModel.initTable();
+
+        // 清理之前的数据
+        console.log('清理之前的数据...');
+        await Database.run('DELETE FROM proxies');
+
+        const scraper = new ProxyScraper();
+
+        // 测试抓取第1页
+        console.log('开始抓取第1页...');
+        const result = await scraper.scrapePage(1);
+        console.log(`抓取结果: ${result.length} 个验证通过的代理`);
+
+        // 查询数据库中的代理数量
+        const totalCount = await ProxyModel.count();
+        const validCount = await ProxyModel.count(true);
+
+        console.log(`数据库状态: 总数 ${totalCount}，可用 ${validCount}`);
+
+        if (result.length > 0) {
+            console.log('\n验证通过的代理:');
+            result.forEach((proxy, index) => {
+                console.log(`${index + 1}. ${proxy.ip}:${proxy.port} - ${proxy.location || '未知位置'} - 响应时间:${proxy.response_time || '未知'}ms`);
+            });
+        }
+
+        // 关闭数据库连接
+        await Database.close();
+
+    } catch (error) {
+        console.error('测试失败:', error.message);
+        console.error('错误堆栈:', error.stack);
+    }
+}
+
+testScraperWithValidation();