95 lines
2.5 KiB
Markdown
95 lines
2.5 KiB
Markdown
|
# 纯推理 · 新加坡GPU服务器部署方案
|
|||
|
|
# Mother Model Deployment · SG GPU Server · 2026-05-18
|
|||
|
|
|
|||
|
|
## 架构
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
冰朔说话
|
|||
|
|
↓
|
|||
|
|
广州服务器(代码仓库+Web入口)
|
|||
|
|
↓ 转发推理请求
|
|||
|
|
新加坡GPU服务器(BS-SG-001/002)
|
|||
|
|
├── 系统盘: 只装vLLM + 母模型
|
|||
|
|
├── 显存: 24GB · FP16 · 7B
|
|||
|
|
├── 域名: 新加坡域名 · 无需国内备案
|
|||
|
|
├── 日志: 聊天记录直写系统盘
|
|||
|
|
└── 作用: 纯推理 · 不装任何其他服务
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 部署步骤
|
|||
|
|
|
|||
|
|
### 环境准备
|
|||
|
|
```bash
|
|||
|
|
# 安装vLLM（纯推理框架，不装训练依赖）
|
|||
|
|
pip install vllm
|
|||
|
|
|
|||
|
|
# 创建模型目录
|
|||
|
|
mkdir -p /data/models
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 从COS下载母模型
|
|||
|
|
```bash
|
|||
|
|
# COS路径：cos://sy-finetune-corpus-1317346199/models/qwen25-7b-sft/final/
|
|||
|
|
# 需要 ZY_OSS_KEY + ZY_OSS_SECRET
|
|||
|
|
|
|||
|
|
export ZY_OSS_KEY=<your-key>
|
|||
|
|
export ZY_OSS_SECRET=<your-secret>
|
|||
|
|
|
|||
|
|
python3 -c "
|
|||
|
|
from qcloud_cos import CosConfig, CosS3Client
|
|||
|
|
import os
|
|||
|
|
|
|||
|
|
c = CosS3Client(CosConfig(
|
|||
|
|
Region='ap-guangzhou',
|
|||
|
|
SecretId=os.environ['ZY_OSS_KEY'],
|
|||
|
|
SecretKey=os.environ['ZY_OSS_SECRET']
|
|||
|
|
))
|
|||
|
|
|
|||
|
|
# 下载整个模型目录
|
|||
|
|
base = 'models/qwen25-7b-sft/final'
|
|||
|
|
bucket = 'sy-finetune-corpus-1317346199'
|
|||
|
|
local = '/data/models/qwen25-7b-sft'
|
|||
|
|
os.makedirs(local, exist_ok=True)
|
|||
|
|
|
|||
|
|
# 列出并下载所有文件
|
|||
|
|
resp = c.list_objects(Bucket=bucket, Prefix=base + '/')
|
|||
|
|
for obj in resp.get('Contents', []):
|
|||
|
|
key = obj['Key']
|
|||
|
|
rel = os.path.relpath(key, base)
|
|||
|
|
dest = os.path.join(local, rel)
|
|||
|
|
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
|||
|
|
c.download_file(Bucket=bucket, Key=key, DestFilePath=dest)
|
|||
|
|
print(f'下载: {rel}')
|
|||
|
|
|
|||
|
|
print('模型下载完成')
|
|||
|
|
"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 启动推理服务
|
|||
|
|
```bash
|
|||
|
|
# vLLM serve 一行命令起服务
|
|||
|
|
nohup python3 -m vllm.entrypoints.openai.api_server \
|
|||
|
|
--model /data/models/qwen25-7b-sft \
|
|||
|
|
--port 8000 \
|
|||
|
|
--host 0.0.0.0 \
|
|||
|
|
--max-model-len 8192 \
|
|||
|
|
--gpu-memory-utilization 0.9 \
|
|||
|
|
--trust-remote-code \
|
|||
|
|
> /data/logs/vllm.log 2>&1 &
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 聊天日志
|
|||
|
|
```bash
|
|||
|
|
mkdir -p /data/logs/chat
|
|||
|
|
# 日志格式：YYYY-MM-DD_HH-MM-SS_user.txt
|
|||
|
|
# 自动记录每次对话
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 服务器选择
|
|||
|
|
|
|||
|
|
| 服务器 | IP | 规格 | 用做 |
|
|||
|
|
|--------|------|------|------|
|
|||
|
|
| BS-SG-001 (铸渊大脑) | 43.156.237.110 | 4C8G·180G | ⭐ 推荐：推理服务器（盘最大） |
|
|||
|
|
| BS-SG-002 (铸渊面孔) | 43.134.16.246 | 2C8G·80G | 备用推理节点 |
|
|||
|
|
| BS-SG-003 | 43.153.193.169 | 2C8G·80G | 备用 |
|
|||
|
|
| ZY-SG-006 | 43.153.203.105 | 2C2G·50G | 规格太小，不适合 |
|