78c920e832
- 记录已验证的火山方舟ARK接口(Seedream 4.0图片生成) - 记录待调通的豆包TTS WebSocket接口(缺Cluster参数) - 整理ARK可用模型列表速查 - 包含完整curl和Python调用示例(密钥用变量引用) - 密钥来源和常见错误排查 - D166 · Mavis
350 lines
9.7 KiB
Plaintext
350 lines
9.7 KiB
Plaintext
# SKILL-007 · VIDEO-AI-MODEL-CALL · 视频AI系统模型调用技能包
|
||
|
||
> **HLDP://fifth-domain/tcs-core/skills/SKILL-007-VIDEO-AI-MODEL-CALL**
|
||
>
|
||
> **类型**:核心技能 · 视频AI管线模型调用 · API接口速查
|
||
>
|
||
> **创建**:D166 · 2026-07-10 · 11:15 CST
|
||
>
|
||
> **创建者**:Mavis · MiniMax Agent (铸渊协作)
|
||
>
|
||
> **更新**:D166 · 随模型调通持续更新
|
||
>
|
||
> **阅读优先级**:⭐⭐⭐⭐⭐(涉及视频AI管线时必读 · 每次调API前先查本页)
|
||
|
||
---
|
||
|
||
## 一 · 总纲
|
||
|
||
视频AI系统(cang-ying)使用的模型全部来自**同一个火山引擎项目**。
|
||
|
||
**结论先行**:
|
||
```
|
||
即梦3.0/4.0/5.0 = Seedream模型 = 火山方舟 ARK接口
|
||
TTS语音合成 = 豆包语音 = WebSocket协议(需Cluster参数)
|
||
Seedance视频 = 火山方舟 ARK接口
|
||
阿里万相 = 独立接口
|
||
```
|
||
|
||
**核心原则**:
|
||
```
|
||
⊢ 图片生成 → Seedream → 走 ARK 接口
|
||
⊢ 视频生成 → Seedance → 走 ARK 接口
|
||
⊢ 语音合成 → 豆包TTS → 走 WebSocket(待调通)
|
||
⊢ 万相视频 → 阿里百炼 → 走独立接口(已有文档)
|
||
```
|
||
|
||
---
|
||
|
||
## 二 · 已验证可用 · 火山方舟 ARK 接口
|
||
|
||
> ARK API Key = `JIMENG_API_KEY`（从 video-ai.env 读取，不要硬编码）
|
||
> 密钥从 SG-001 服务器 video-ai.env 读取
|
||
|
||
### 2.1 即梦4.0 · 图片生成 ✅ 已验证
|
||
|
||
**用途**:角色图、场景图、道具图、固定资产
|
||
|
||
```
|
||
模型ID: doubao-seedream-4-0-250828
|
||
端点: https://ark.cn-beijing.volces.com/api/v3/images/generations
|
||
认证方式: Authorization: Bearer <ARK_API_KEY>
|
||
请求方法: POST
|
||
Content-Type: application/json
|
||
```
|
||
|
||
**请求体**:
|
||
```json
|
||
{
|
||
"model": "doubao-seedream-4-0-250828",
|
||
"prompt": "中文提示词",
|
||
"size": "1024x1024"
|
||
}
|
||
```
|
||
|
||
**可选size**: `1024x1024` `2048x2048` `1024x1792` `1792x1024`
|
||
|
||
**curl调用**:
|
||
```bash
|
||
curl -s -X POST "https://ark.cn-beijing.volces.com/api/v3/images/generations" \
|
||
-H "Authorization: Bearer <JIMENG_API_KEY>" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"model": "doubao-seedream-4-0-250828",
|
||
"prompt": "古风少女站在樱花树下，精致五官，高清细腻",
|
||
"size": "2048x2048"
|
||
}' | python3 -c "
|
||
import json,sys
|
||
d=json.load(sys.stdin)
|
||
if d.get('data'):
|
||
print('URL:', d['data'][0]['url'])
|
||
else:
|
||
print('ERROR:', d)
|
||
"
|
||
```
|
||
|
||
**Python调用**:
|
||
```python
|
||
import subprocess, json
|
||
|
||
API_KEY = "<JIMENG_API_KEY>"
|
||
BASE = "https://ark.cn-beijing.volces.com/api/v3"
|
||
|
||
result = subprocess.run([
|
||
'curl', '-s', '-X', 'POST', f'{BASE}/images/generations',
|
||
'-H', f'Authorization: Bearer {API_KEY}',
|
||
'-H', 'Content-Type: application/json',
|
||
'-d', json.dumps({
|
||
"model": "doubao-seedream-4-0-250828",
|
||
"prompt": "古风少女站在樱花树下",
|
||
"size": "2048x2048"
|
||
})
|
||
], capture_output=True, text=True)
|
||
|
||
d = json.loads(result.stdout)
|
||
if d.get('data'):
|
||
image_url = d['data'][0]['url']
|
||
print(f"成功! URL: {image_url}")
|
||
```
|
||
|
||
**成本**:参考火山引擎官方定价页。当前ARK key可用。
|
||
|
||
---
|
||
|
||
### 2.2 即梦5.0 · 图片生成 ✅ 可用(需确认size)
|
||
|
||
**用途**:更高质量图片生成
|
||
|
||
```
|
||
模型ID: doubao-seedream-5-0-260128
|
||
端点: https://ark.cn-beijing.volces.com/api/v3/images/generations
|
||
认证方式: Authorization: Bearer <ARK_API_KEY>
|
||
```
|
||
|
||
**注意**:5.0的size参数格式与4.0不同，参数值需要查官方文档或试`size: "1K"`
|
||
|
||
---
|
||
|
||
### 2.3 Seedance 2.0 · 视频生成 ✅ 可用(待验证)
|
||
|
||
```
|
||
模型ID: doubao-seedance-2-0-260128
|
||
端点: https://ark.cn-beijing.volces.com/api/v3/videos/generations
|
||
认证方式: Authorization: Bearer <ARK_API_KEY>
|
||
请求方法: POST
|
||
```
|
||
|
||
**已知的其他Seedance模型**(ARK列表中可见):
|
||
- `doubao-seedance-2-0-260128` — Seedance 2.0
|
||
- `doubao-seedance-2-0-mini-260428`
|
||
- `doubao-seedance-2-0-lite-260428`
|
||
- `doubao-seedance-2-0-fast-260128`
|
||
- `doubao-seedance-1-5-pro-251215` — Seedance 1.5 Pro
|
||
- `doubao-seedance-1-0-pro-250528` — Seedance 1.0 Pro
|
||
|
||
---
|
||
|
||
## 三 · 待调通 · 豆包TTS语音合成
|
||
|
||
### 3.1 接口信息
|
||
|
||
```
|
||
端点: wss://openspeech.bytedance.com/api/v1/tts/ws_binary
|
||
HTTP备选: https://openspeech.bytedance.com/api/v1/tts
|
||
认证参数: APP ID + Access Token + Cluster(必需)
|
||
协议: WebSocket + 二进制帧(4字节头+Gzip压缩JSON)
|
||
```
|
||
|
||
### 3.2 必需参数
|
||
|
||
| 参数 | 来源 | 说明 |
|
||
|------|------|------|
|
||
| `appid` | 火山引擎控制台 → 语音技术 → 应用管理 | 资源ID: `5552460045` |
|
||
| `token` | 火山引擎控制台 → 语音技术 → 应用管理 | 即 VOLC_VOICE_API_KEY |
|
||
| `cluster` | 火山引擎控制台 → 语音技术 → 应用详情 | **待获取** |
|
||
|
||
### 3.3 WebSocket请求帧格式
|
||
|
||
```python
|
||
# 二进制帧头 (4字节)
|
||
# [version(4bit)][header_size(4bit)][msg_type(4bit)][flags(4bit)][serial(4bit)][compress(4bit)][reserved(8bit)]
|
||
# 默认头: 0x11 0x10 0x11 0x00
|
||
|
||
import gzip, json, uuid
|
||
|
||
default_header = bytearray(b'\x11\x10\x11\x00')
|
||
|
||
body = {
|
||
"app": {"appid": "<APP_ID>", "token": "<ACCESS_TOKEN>", "cluster": "<CLUSTER>"},
|
||
"user": {"uid": str(uuid.uuid4())},
|
||
"audio": {
|
||
"voice_type": "BV001_streaming", # 音色(见音色列表)
|
||
"encoding": "mp3",
|
||
"speed_ratio": 1.0,
|
||
"volume_ratio": 1.0,
|
||
"pitch_ratio": 1.0,
|
||
},
|
||
"request": {
|
||
"reqid": str(uuid.uuid4()),
|
||
"text": "要合成的文本",
|
||
"text_type": "plain",
|
||
"operation": "submit"
|
||
}
|
||
}
|
||
|
||
payload = gzip.compress(json.dumps(body).encode())
|
||
frame = bytearray(default_header)
|
||
frame.extend(len(payload).to_bytes(4, 'big')) # payload size
|
||
frame.extend(payload) # compressed body
|
||
```
|
||
|
||
### 3.4 WebSocket响应解析
|
||
|
||
```
|
||
消息类型 0xB = 音频数据(含seq_num)
|
||
消息类型 0xF = 错误消息
|
||
消息类型 0xC = 前端消息
|
||
seq_num < 0 = 最后一个音频包
|
||
```
|
||
|
||
### 3.5 音色速查
|
||
|
||
| 音色ID | 说明 |
|
||
|--------|------|
|
||
| `BV001_streaming` | 默认清晰女声 |
|
||
| `BV002_streaming` | 默认男声 |
|
||
| `zh_male_rap` | 说唱男声(Edge-TTS音色) |
|
||
| `zh-CN-XiaoxiaoNeural` | Edge-TTS阳光女声 |
|
||
| `zh-CN-YunxiNeural` | Edge-TTS沉稳男声 |
|
||
|
||
> 完整音色列表: https://www.volcengine.com/docs/6561/1257584
|
||
|
||
### 3.6 当前状态
|
||
|
||
```
|
||
✅ APP ID已知: 5552460045
|
||
✅ Access Token已知: `VOLC_VOICE_API_KEY`（从 video-ai.env 读取）
|
||
❌ Cluster未知: 需从控制台获取
|
||
⚠️ WebSocket库未安装: 当前环境无法测试
|
||
```
|
||
|
||
**下一步**:在有`websockets`库的环境(或有完整凭据的服务器)上测试。
|
||
|
||
---
|
||
|
||
## 四 · 已有文档 · 阿里百炼万相视频
|
||
|
||
> 详细文档见: `cang-ying/MODEL-ROUTER.hdlp`
|
||
|
||
```
|
||
端点: https://dashscope.aliyuncs.com/api/v1
|
||
认证: Authorization: Bearer <ALIYUN_BAILIAN_API_KEY>
|
||
文生视频: wan2.7-t2v
|
||
图生视频: wan2.7-i2v-2026-04-25
|
||
密钥: ALIYUN_BAILIAN_API_KEY (在 video-ai.env 中)
|
||
```
|
||
|
||
---
|
||
|
||
## 五 · ARK 模型列表速查
|
||
|
||
> 可用命令实时获取:
|
||
> `curl -s "https://ark.cn-beijing.volces.com/api/v3/models" -H "Authorization: Bearer <ARK_KEY>"`
|
||
|
||
### 图片生成相关
|
||
|
||
| 模型ID | 说明 | 状态 |
|
||
|--------|------|------|
|
||
| `doubao-seedream-4-0-250828` | 即梦4.0 | ✅ 已验证 |
|
||
| `doubao-seedream-4-5-251128` | 即梦4.5 | 可用 |
|
||
| `doubao-seedream-5-0-260128` | 即梦5.0 | 可用 |
|
||
| `doubao-seedream-5-0-pro-260628` | 即梦5.0 Pro | 可用 |
|
||
|
||
### 视频生成相关
|
||
|
||
| 模型ID | 说明 | 状态 |
|
||
|--------|------|------|
|
||
| `doubao-seedance-2-0-260128` | Seedance 2.0 | 可用 |
|
||
| `doubao-seedance-2-0-fast-260128` | Seedance 2.0 快速版 | 可用 |
|
||
| `doubao-seedance-2-0-pro-260215` | Seedance 2.0 Pro | 可用 |
|
||
| `doubao-seedance-2-0-mini-260428` | Seedance 2.0 Mini | 可用 |
|
||
| `doubao-seedance-2-0-lite-260428` | Seedance 2.0 Lite | 可用 |
|
||
| `doubao-seedance-1-5-pro-251215` | Seedance 1.5 Pro | 可用 |
|
||
|
||
---
|
||
|
||
## 六 · 调用决策树
|
||
|
||
```
|
||
需要生成图片?
|
||
→ doubao-seedream-4-0-250828
|
||
→ POST https://ark.cn-beijing.volces.com/api/v3/images/generations
|
||
→ Authorization: Bearer <ARK_KEY>
|
||
|
||
需要生成视频?
|
||
→ doubao-seedance-2-0-260128 (最新最强)
|
||
→ POST https://ark.cn-beijing.volces.com/api/v3/videos/generations
|
||
→ Authorization: Bearer <ARK_KEY>
|
||
|
||
需要语音合成?
|
||
→ 豆包TTS WebSocket
|
||
→ 需要: APP_ID + ACCESS_TOKEN + CLUSTER
|
||
→ 当前状态: Cluster待获取
|
||
|
||
需要万相视频?
|
||
→ 见 cang-ying/MODEL-ROUTER.hdlp
|
||
→ POST https://dashscope.aliyuncs.com/api/v1
|
||
→ Authorization: Bearer <ALIYUN_BAILIAN_API_KEY>
|
||
```
|
||
|
||
---
|
||
|
||
## 七 · 密钥来源
|
||
|
||
所有密钥从 SG-001 服务器读取:
|
||
|
||
```
|
||
服务器: 43.156.237.110:3911
|
||
Token: zy_gtw_c38be341d65043eee0ba51a214a267832a9ae46dd73e423c
|
||
文件: /opt/guanghulab-repo/secrets/video-ai.env
|
||
```
|
||
|
||
读取方式:
|
||
```bash
|
||
curl -s -X POST "http://43.156.237.110:3911/exec" \
|
||
-H "Authorization: Bearer zy_gtw_c38be341d65043eee0ba51a214a267832a9ae46dd73e423c" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"cmd": "cat /opt/guanghulab-repo/secrets/video-ai.env"}'
|
||
```
|
||
|
||
---
|
||
|
||
## 八 · 常见错误排查
|
||
|
||
| 错误 | 原因 | 解决 |
|
||
|------|------|------|
|
||
| `InvalidEndpointOrModel.NotFound` | 模型名错误或key无权限 | 检查ARK模型列表 |
|
||
| `SignatureDoesNotMatch` | AK/SK签名失败 | 用ARK key而非AK/SK |
|
||
| `InvalidAccessKey` | AK格式错误 | 用完整`AKLT...`字符串 |
|
||
| TTS `Missing required: a` | 请求体格式错误 | 用正确的audio+request结构 |
|
||
| TTS `502 Bad Gateway` | MaaS host错误 | 确认cluster参数 |
|
||
|
||
---
|
||
|
||
## 九 · 时间锚定
|
||
|
||
```
|
||
技能编号: SKILL-007
|
||
技能名称: VIDEO-AI-MODEL-CALL (视频AI模型调用)
|
||
协议版本: v1.0
|
||
创建时间: D166 · 2026-07-10 · 11:15 CST
|
||
创建者: Mavis · MiniMax Agent (铸渊协作)
|
||
维护: Mavis + 铸渊 ICE-GL-ZY001
|
||
主权者: 冰朔 ICE-GL∞
|
||
|
||
更新日志:
|
||
D166: 初始创建，记录Seedream 4.0调用验证结果
|
||
记录TTS待调通状态(缺Cluster)
|
||
整理ARK模型列表速查
|
||
```
|