qupengwei/pallet_storage_detection_system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

The first commit

Browse Source
This commit is contained in:
qupengwei
2026-01-04 16:51:58 +08:00
parent 684a923cda
commit 9b2a6bf423
107 changed files with 35063 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
.gitignore vendored Normal file
View File

@@ -0,0 +1,74 @@
# Build directories
# 忽略所有build目录（包括根目录和子目录）
build/
# 注意：项目只在 image_capture/build/ 目录下构建
# 根目录的 build/ 文件夹应该被忽略，如果存在可以安全删除
bin/
lib/
!camport3/lib/
image_capture/src/images_template/
image_capture/build_debug
# CMake generated files
CMakeCache.txt
CMakeFiles/
cmake_install.cmake
Makefile
*.cmake
# Visual Studio files
.vs/
*.vcxproj
*.vcxproj.filters
*.vcxproj.user
*.sln
*.suo
*.user
*.sdf
*.opensdf
# Qt autogen files
*_autogen/
.qt/
ui_*.h
moc_*.cpp
qrc_*.cpp
# Compiled files
*.o
*.obj
*.exe
*.a
*.lib
!image_capture/camera_sdk/lib/**/*.lib
# Saved images
*.png
*.jpg
*.jpeg
*.ply
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~
# Temporary files
*.tmp
*.temp
*.log
compile_commands.json.tmp*
# OS files
.DS_Store
Thumbs.db
*.gif
.cache/
!image_capture/camera_sdk/
!image_capture/camera_sdk/lib/
!image_capture/cmake/*.cmake

221
docs/cmake_configuration_summary.md Normal file
View File

@@ -0,0 +1,221 @@
# CMake 配置文档
本文档总结了 `image_capture` 项目的 CMake 构建系统配置。
---
## 目录结构
```
image_capture/
├── CMakeLists.txt # 主构建配置文件
└── cmake/ # CMake 模块目录
├── CompilerOptions.cmake # 编译器选项配置
├── Dependencies.cmake # 外部依赖管理
└── PercipioSDK.cmake # 相机 SDK 配置
```
---
## 主配置文件：[CMakeLists.txt](file:///d:/Git/stereo_warehouse_inspection/image_capture/CMakeLists.txt)
### 基本信息
- **CMake 最低版本**: 3.10
- **项目名称**: `image_capture`
- **编程语言**: C++
- **构建生成器**: Visual Studio 17 2022 (MSVC)
### 输出目录
```cmake
CMAKE_RUNTIME_OUTPUT_DIRECTORY = ${CMAKE_BINARY_DIR}/bin/Release # 可执行文件
CMAKE_LIBRARY_OUTPUT_DIRECTORY = ${CMAKE_BINARY_DIR}/lib/Release # 动态库
CMAKE_ARCHIVE_OUTPUT_DIRECTORY = ${CMAKE_BINARY_DIR}/lib/Release # 静态库
```
### 模块化设计
项目采用模块化的 CMake 配置，通过 `cmake/` 目录下的三个模块文件组织：
1. **CompilerOptions.cmake** - 编译器和全局设置
2. **Dependencies.cmake** - Qt6、OpenCV、Open3D 依赖
3. **PercipioSDK.cmake** - 图漾相机 SDK 配置
### 库和可执行文件
#### 1. Algorithm Library (`algorithm_lib`)
**类型**: 静态库
**源文件**:
- `src/algorithm/core/detection_base.cpp`
- `src/algorithm/core/detection_result.cpp`
- `src/algorithm/utils/image_processor.cpp`
- `src/algorithm/detections/slot_occupancy_detection.cpp`
- `src/algorithm/detections/pallet_offset_detection.cpp`
- `src/algorithm/detections/beam_rack_deflection_detection.cpp`
- `src/algorithm/detections/visual_inventory_detection.cpp`
- `src/algorithm/detections/visual_inventory_end_detection.cpp`
**包含路径**:
- `src`
- `third_party/percipio/common` (修复 json11.hpp 引用)
**依赖**: OpenCV, Open3D
#### 2. Main Executable (`image_capture`)
**类型**: 可执行文件
**主要源文件**:
- `src/main.cpp`
- `src/camera/ty_multi_camera_capture.cpp`
- `src/camera/mvs_multi_camera_capture.cpp`
- `src/device/device_manager.cpp`
- `src/redis/redis_communicator.cpp`
- `src/task/task_manager.cpp`
- `src/vision/vision_controller.cpp`
- `src/common/log_manager.cpp`
- `src/common/config_manager.cpp`
- `src/gui/mainwindow.cpp` / `.h` / `.ui`
**链接的库**:
- `algorithm_lib` (项目内部算法库)
- `cpp_api_lib` (相机 SDK C++ API 封装)
- `tycam` (相机 SDK 动态库)
- `${OpenCV_LIBS}` (OpenCV 库)
- `Open3D::Open3D` (Open3D 库)
- `Qt6::Core` 和 `Qt6::Widgets` (Qt 框架)
- `MvCameraControl.lib` (海康 MVS SDK)
### 测试配置
- **选项**: `BUILD_TESTS` (默认 ON)
- **测试目录**: `tests/` (通过 `add_subdirectory` 添加)
---
## CMake 模块详解
### 1. [CompilerOptions.cmake](file:///d:/Git/stereo_warehouse_inspection/image_capture/cmake/CompilerOptions.cmake)
#### C++ 标准
- **标准**: C++17
- **要求**: 必须支持
#### Qt 自动化工具
```cmake
CMAKE_AUTOMOC ON # 自动 Meta-Object Compiler
CMAKE_AUTORCC ON # 自动 Resource Compiler
CMAKE_AUTOUIC ON # 自动 UI Compiler
```
#### 编译器优化选项 (MSVC)
**Release 模式** (默认):
```cmake
/O2 # 优化速度
/Ob2 # 内联任何合适的函数
/Oi # 启用内建函数
/Ot # 代码速度优先
/Oy # 省略帧指针
/GL # 全局程序优化
```
**Debug 模式**:
```cmake
/Od # 禁用优化
/Zi # 生成完整调试信息
```
#### 其他设置
- **定义**: `OPENCV_DEPENDENCIES`
- **compile_commands.json**: 自动生成（用于 IDE 智能提示）
---
### 2. [Dependencies.cmake](file:///d:/Git/stereo_warehouse_inspection/image_capture/cmake/Dependencies.cmake)
#### Qt6 配置
```cmake
find_package(Qt6 REQUIRED COMPONENTS Widgets)
```
#### OpenCV 配置
```cmake
find_package(OpenCV REQUIRED)
```
#### Open3D 配置
```cmake
find_package(Open3D REQUIRED)
```
用于点云处理和算法运算。
---
### 3. [PercipioSDK.cmake](file:///d:/Git/stereo_warehouse_inspection/image_capture/cmake/PercipioSDK.cmake)
#### 相机 SDK 路径配置
```cmake
CAMPORT3_ROOT = ${CMAKE_CURRENT_SOURCE_DIR}/camera_sdk
CAMPORT3_LIB_DIR = ${CAMPORT3_ROOT}/lib/win/x64
```
#### 导入 tycam 动态库
```cmake
add_library(tycam SHARED IMPORTED)
```
#### C++ API 封装库 (`cpp_api_lib`)
**类型**: 静态库
**源文件**:
`camera_sdk/sample_v2/cpp/*`, `camera_sdk/common/*`
**依赖**: OpenCV
---
## 构建流程
### 配置项目
```bash
cd image_capture/build
cmake ..
```
可选参数:
```bash
-DOpenCV_DIR=<path> # 指定 OpenCV 路径
-DQt6_DIR=<path> # 指定 Qt6 路径
-DOpen3D_DIR=<path> # 指定 Open3D 路径
```
### 编译项目
```bash
cmake --build . --config Release
# 或
cmake --build . --config Debug
```
---
## 依赖项总结
| 依赖项 | 版本要求 | 用途 |
|--------|---------|------|
| CMake | ≥ 3.10 | 构建系统 |
| C++ | C++17 | 编程语言标准 |
| Qt6 | Widgets 组件 | GUI 框架 |
| OpenCV | 4.x | 图像处理 |
| Open3D | 0.17+ | 3D点云处理 |
| Percipio SDK | tycam.dll | 相机驱动 |
| MSVC | VS2022 (v143) | 编译器 |
---
## 维护建议
1. **环境一致性**: 确保所有依赖项（Qt, OpenCV, Open3D）都是使用 MSVC 编译的 x64 版本。
2. **DLL 管理**: 运行时确保所有必要的 DLL 都在可执行文件目录下。
3. **版本检测**: 保持 Open3D 和 OpenCV 版本的一致性，避免 ABI 冲突。
---
*文档更新时间: 2025-12-19*

306
docs/project_architecture.md Normal file
View File

@@ -0,0 +1,306 @@
# 项目架构及调用关系文档
## 1. 系统概述
本系统是一个基于立体视觉的仓库巡检图像采集与处理系统。它集成了图漾(Percipio)工业相机SDK和海康(MVS)相机SDK进行多相机图像采集，使用OpenCV进行图像处理，Qt6作为用户界面框架，并通过Redis与外部系统（如机器人控制系统）进行通信和任务调度。
系统主要功能包括：
- 多相机同步采集（深度图与彩色图）
- 实时图像预览与状态监控
- 基于Redis的任务触发与结果上报
- 多种检测算法（货位占用、横梁/立柱变形、托盘偏差等）
- 系统配置管理与日志记录
## 2. 目录结构说明
```text
scripts/ # 批处理脚本 (数据库配置、模拟任务等)
image_capture/
└── src/
├── algorithm/ # 核心算法库
│ ├── core/ # 算法基类与结果定义 (DetectionBase, DetectionResult)
│ ├── detections/ # 具体检测算法实现 (SlotOccupancy, BeamRackDeflection等)
│ └── utils/ # 图像处理工具 (ImageProcessor)
├── camera/ # 相机驱动层
│ ├── ty_multi_camera_capture.cpp/h # 图漾(Percipio) 3D相机封装
│ └── mvs_multi_camera_capture.cpp/h # 海康(MVS) 2D相机封装
├── common/ # 通用设施
│ ├── config_manager.cpp/h # 配置管理单例
│ ├── log_manager.cpp/h # 日志管理
│ └── log_streambuf.h # std::cout重定向到GUI
├── device/ # 硬件设备管理
│ └── device_manager.cpp/h # 相机设备单例管理
├── gui/ # 用户界面
│ └── mainwindow.cpp/h/ui # 主窗口实现 (集成Settings Tab)
├── redis/ # 通信模块
│ └── redis_communicator.cpp/h # Redis客户端封装
├── task/ # 任务调度
│ └── task_manager.cpp/h # 任务分发与执行逻辑
├── vision/ # 系统控制
│ └── vision_controller.cpp/h # 顶层控制器，协调Redis与Task
├── common_types.h # 通用数据类型 (Point3D, CameraIntrinsics)
└── main.cpp # 程序入口
```
## 3. 核心架构设计
系统采用分层架构设计，各模块职责明确：
- **展示层 (GUI)**: `MainWindow` 负责界面显示、手动控制、参数配置及日志展示。
- **控制层 (Controller)**: `VisionController` 作为系统级控制器，负责服务的启动/停止，协调 `RedisCommunicator` 和 `TaskManager`。
- **业务逻辑层 (Task/Manager)**: `TaskManager` 解析任务指令，`DeviceManager` 管理硬件资源。
- **算法层 (Algorithm)**: 提供具体的视觉检测功能，继承自 `DetectionBase`。
- **驱动层 (Driver)**: `CameraCapture` 封装底层SDK调用。
### 系统分层架构图
```mermaid
graph TB
subgraph Presentation ["展示层 (Presentation)"]
direction TB
GUI[MainWindow]
end
subgraph Control ["控制层 (Control)"]
VC[VisionController]
end
subgraph Business ["业务逻辑层 (Business Logic)"]
direction TB
TM[TaskManager]
DM[DeviceManager]
end
subgraph Algorithm ["算法层 (Algorithm)"]
direction TB
DB[DetectionBase]
Det[Concrete Detections<br/>(Slot, Beam, etc.)]
end
subgraph Infrastructure ["基础设施层 (Infrastructure)"]
direction TB
Cam[CameraCapture]
Redis[RedisCommunicator]
Conf[ConfigManager]
end
%% 层级调用关系
GUI --> VC
VC --> TM
VC --> Redis
TM --> DM
TM --> DB
DB <|-- Det
DM --> Cam
DM --> MVS[MvsMultiCameraCapture]
%% 跨层辅助调用
GUI -.-> Conf
TM -.-> Conf
style Presentation fill:#e1f5fe,stroke:#01579b
style Control fill:#e8f5e9,stroke:#2e7d32
style Business fill:#fff3e0,stroke:#ef6c00
style Algorithm fill:#f3e5f5,stroke:#7b1fa2
style Infrastructure fill:#eceff1,stroke:#455a64
```
### 系统类图
```mermaid
classDiagram
class MainWindow {
+VisionController visionController_
+updateImage()
+onSaveSettings()
}
class VisionController {
+RedisCommunicator redis_comm_
+TaskManager task_manager_
+start()
+stop()
}
class DeviceManager {
<<Singleton>>
+CameraCapture camera_capture_
+initialize()
+computePointCloud()
}
class TaskManager {
+executeTask()
-algorithms_ map
}
class CameraCapture {
+getLatestImages()
+computePointCloud()
+start()
-captureThreadFunc()
}
class RedisCommunicator {
+connect()
+listenForTasks()
+publishResult()
}
class ConfigManager {
<<Singleton>>
+loadConfig()
+saveConfig()
}
MainWindow --> VisionController : 只有与管理
VisionController --> RedisCommunicator : 使用
VisionController --> TaskManager : 使用
VisionController ..> DeviceManager : 依赖(全局)
TaskManager ..> DeviceManager : 获取图像/点云
DeviceManager --> CameraCapture : 拥有
MainWindow ..> ConfigManager : 读写配置
TaskManager ..> ConfigManager : 读取参数
```
## 4. 关键模块详解
### 4.1 GUI与主入口 (MainWindow)
- **职责**: 程序的主要入口，负责UI渲染、用户交互、参数配置及系统状态反馈。
- **调用关系**:
- 初始化时创建 `VisionController`。
- 通过 `QTimer` 定期从 `DeviceManager` 获取图像更新界面。
- **Settings Tab**: 直接在 `MainWindow` 中实现，提供 "Beam/Rack Deflection", "Pallet Offset" 等算法参数配置界面。
- 通过 `ConfigManager` 加载和保存配置项，包括ROI点坐标和各类阈值。
### 4.2 视觉控制器 (VisionController)
- **职责**: 系统的"大脑"，不依赖于GUI运行（设计上支持无头模式）。
- **流程**:
1. `initialize()`: 连接Redis。
2. `start()`: 启动Redis监听线程。
3. `onTaskReceived()`: 当Redis收到任务时，转发给 `TaskManager`。
### 4.3 任务管理 (TaskManager)
- **职责**: 解析Redis下发的JSON指令，选择合适的算法执行。
- **工作流**:
1. 接收任务ID和参数。
2. 从 `DeviceManager` 获取当前最新的一帧图像（深度+彩色）。
3. **点云生成**: 对于需要3D数据的任务（Flag 2/3），调用 `DeviceManager::computePointCloud()` 生成点云。
4. 根据任务类型实例化或调用相应的 `DetectionBase` 子类。
5. 执行 `detect()`，传入图像和点云数据。
6. 将结果打包为JSON，通过回调或直接通过 `RedisCommunicator` 返回。
### 4.4 设备管理 (DeviceManager)
- **职责**: 硬件资源的全局访问点（单例模式）。
- **封装**: 内部持有 `CameraCapture` 实例，确保相机资源全生命周期只被初始化一次。
- **功能**:
- 提供线程安全的图像获取接口 `getLatestImages()`。
- 提供点云计算接口 `computePointCloud()`，利用SDK内部参数生成高精度点云。
### 4.5 相机驱动 (CameraCapture)
- **实现**: `ty_multi_camera_capture.cpp`
- **机制**:
- 为每个相机开启独立采集线程。
- 维护内部帧缓冲区。
- 将SDK的 `TYImage` 转换为 OpenCV `cv::Mat`。
- **点云优化**: 内部集成 `TYMapDepthImageToPoint3d`，利用相机标定参数直接计算3D点云，消除畸变。
### 4.6 配置管理 (ConfigManager)
- **职责**: 管理 `config.json` 文件，集中管理系统配置。
- **管理内容**:
- Redis 连接信息。
- 算法阈值 (Beam/Rack, Pallet Offset 等)。
- ROI (Region of Interest) 坐标点。
- 系统通用参数 (最小/最大深度等)。
- **特性**: 单例模式，支持热加载（部分参数）和持久化保存。程序启动时由 `MainWindow` 加载，确保算法使用持久化的用户设置。GUI中的Settings Tab直接操作此模块。
## 5. 系统执行与数据流
### 5.1 初始化流程
1. `main()` 启动 `QApplication`。
2. `MainWindow` 构造：
- 初始化UI。
- **调用 `ConfigManager::getInstance().loadConfig()` 加载本地配置。**
- 调用 `DeviceManager::getInstance().initialize()` 初始化相机。
- 创建并初始化 `VisionController`（连接 Redis，但暂不启动监听）。
- 启动定时器调用 `updateImage()` 刷新界面显示。
3. 启动设备采集：调用 `DeviceManager::startAll()`。
4. 设备启动成功后再调用 `VisionController::start()` 开启 Redis 监听，确保任务到来时设备已就绪。
### 5.2 自动任务执行流 (Redis触发)
```mermaid
sequenceDiagram
participant Redis
participant RC as RedisCommunicator
participant VC as VisionController
participant TM as TaskManager
participant DM as DeviceManager
participant Algo as DetectionAlgorithm
Redis->>RC: Publish Task (JSON)
RC->>VC: onTaskReceived(data)
VC->>TM: executeTask(data)
activate TM
TM->>DM: getLatestImages()
DM-->>TM: depth_img, color_img
TM->>DM: computePointCloud(depth)
DM-->>TM: point_cloud (vector<Point3D>)
TM->>Algo: execute(images, point_cloud)
activate Algo
Algo-->>TM: DetectionResult
deactivate Algo
TM->>TM: processResult()
TM->>RC: writeString(key, value)
RC->>Redis: Set Key-Value
deactivate TM
```
1. **外部触发**: Redis 发布任务消息。
2. **接收**: `RedisCommunicator` 监听到消息，触发回调。
3. **调度**: `VisionController` 调用 `TaskManager::executeTask()`。
4. **获取数据**: `TaskManager` 从 `DeviceManager` 获取最新帧。
5. **算法处理**: 调用相应算法（如 `SlotOccupancyDetection::detect`）。
6. **结果反馈**: 结果封装成JSON，通过 `RedisCommunicator` 写入Redis结果队列。
7. **任务复位**: 结果写入完成后，将 `vision_task_flag` 置 `0`、`vision_task_side`/`vision_task_time` 置空，避免程序重启后被旧任务自动触发。
### 5.3 实时监控执行流 (GUI)
```mermaid
sequenceDiagram
participant Timer as QTimer
participant MainWin as MainWindow
participant DM as DeviceManager
Timer->>MainWin: timeout()
activate MainWin
MainWin->>DM: getLatestImages()
DM-->>MainWin: depth_img, color_img
MainWin->>MainWin: Convert to QImage
MainWin->>MainWin: update QLabel
deactivate MainWin
```
1. **定时刷新**: `MainWindow` 的 `QTimer` 触发 `updateImage()`。
2. **数据拉取**: 调用 `DeviceManager::getInstance().getLatestImages()`。
3. **渲染**: 将OpenCV Mat 转换为 QImage 并显示在 `QLabel` 上。
- 深度图进行伪彩色处理以便观察。
- 自适应窗口大小缩放。
## 6. 异常处理与日志
- **日志**: 使用 `LogManager` 和 `spdlog` (如果集成) 或标准输出。
- **重定向**: `LogStreamBuf` 将 `std::cout/cerr` 重定向到GUI的日志窗口，方便现场调试。
- **错误恢复**: 相机掉线重连机制（在驱动层实现或计划中）。
## 7. 编译与构建
- **工具**: CMake
- **依赖**: Qt6, OpenCV 4.x, Percipio SDK, (Redis库通常被封装或作为源码包含)
- **平台**: Windows (MSVC/MinGW)

133
docs/project_class_interaction.md Normal file
View File

@@ -0,0 +1,133 @@
# 项目功能类调用关系说明 (Project Class Interaction Documentation)
本主要介绍 `image_capture` 项目核心功能类之间的调用关系、数据流向以及模块划分。
## 1. 核心模块概览 (Core Modules Overview)
系统主要由以下几个核心模块组成：
* **GUI 模块 (`MainWindow`)**: 程序的入口与界面显示，负责系统初始化。
* **Vision 控制器 (`VisionController`)**: 系统的核心中枢，协调通信与任务管理。
* **任务管理 (`TaskManager`)**: 负责具体的业务逻辑执行、算法调度和结果处理。
* **设备管理 (`DeviceManager`)**: 负责相机等硬件设备的统一管理（单例模式）。
* **通信模块 (`RedisCommunicator`)**: 负责与外部系统（如 WMS）通过 Redis 交互。
* **算法模块 (`DetectionBase` 及其子类)**: 具体的图像处理算法。
## 2. 类调用关系图 (Class Interaction Diagram)
```mermaid
classDiagram
class MainWindow {
+VisionController vision_controller
+init()
}
class VisionController {
-shared_ptr<RedisCommunicator> redis_comm
-shared_ptr<TaskManager> task_manager
+start()
+stop()
-onTaskReceived()
}
class RedisCommunicator {
+startListening()
+writeDetectionResult()
+setTaskCallback()
}
class TaskManager {
-queue<RedisTaskData> task_queue
-map detectors
+handleTask()
-executeDetectionTask()
-getDetector(flag)
}
class DeviceManager {
<<Singleton>>
+getInstance()
+getLatestImages()
+startAll()
}
class DetectionBase {
<<Abstract>>
+execute(depth, color, ...)
}
class ConcreteDetection {
+execute()
}
MainWindow --> VisionController : 拥有并管理
VisionController --> RedisCommunicator : 管理 (监听/发送)
VisionController --> TaskManager : 分发任务
RedisCommunicator --> VisionController : 回调通知 (Callback)
TaskManager ..> DeviceManager : 获取图像数据 (Dependency)
TaskManager --> DetectionBase : 调用算法
DetectionBase <|-- ConcreteDetection : 继承
```
## 3. 详细调用流程 (Detailed Call Flow)
### 3.1 系统初始化与启动 (Initialization & Startup)
1. **Entry Point**: `main.cpp` 创建 `QApplication` 并实例化 `MainWindow`。
2. **MainWindow**:
* 构造函数中初始化界面。
* 调用 `DeviceManager::getInstance().initialize()` 扫描并初始化相机设备。
* 实例化 `VisionController` 成员变量。
* 调用 `VisionController::initialize()`，配置 Redis 连接参数。
* 调用 `VisionController::start()` 启动后台服务。
3. **VisionController**:
* 在 `start()` 中调用 `RedisCommunicator::startListening()` 开启监听线程。
### 3.2 任务触发与执行 (Task Trigger & Execution)
当 Redis 中 `vision_task_flag` 发生变化时，流程如下：
1. **RedisCommunicator**:
* 监听线程检测到 Flag 变化。
* 通过回调函数 `VisionController::onTaskReceived` 通知控制器。
2. **VisionController**:
* `onTaskReceived` 将接收到的 `RedisTaskData` 传递给 `TaskManager::handleTask`。
3. **TaskManager**:
* `handleTask` 将任务推入内部的任务队列 `task_queue_`。
* 工作线程 `taskExecutionThreadFunc` 从队列中取出任务。
* **获取图像**: 调用 `DeviceManager::getInstance().getLatestImages(...)` 获取当前最新的深度图和彩色图。
* **选择算法**: 根据任务 Flag 调用 `getDetector(flag)` 获取对应的算法实例（如 `PalletOffsetDetection`）。
* **执行算法**: 调用 `detector->execute(depth_img, color_img, ...)` 进行计算。
* **结果封装**: 将算法返回的数据填充到 `DetectionResult` 结构体中。
### 3.3 结果处理 (Result Handling)
算法执行完成后：
1. **TaskManager**:
* 调用 `processResult(result)`。
* 该函数会格式化结果为 JSON 字符串，并计算报警/警告状态。
* 调用 `redis_result_comm_->writeDetectionResult(json)` 将结果写入 Redis。
2. **RedisCommunicator**:
* 执行 Redis SET 命令，将 JSON 数据写入指定的 Key。
## 4. 关键类说明 (Key Class Descriptions)
### VisionController (`src/vision/vision_controller.h`)
* **职责**: 作为系统的外观（Facade），对外提供统一的 start/stop 接口，对内协调 Redis 和 TaskManager。
* **特点**: 它是 MainWindow 唯一直接交互的非 GUI 业务类。
### DeviceManager (`src/device/device_manager.h`)
* **职责**: 屏蔽底层相机 SDK（Percipio / MVS）的差异，提供统一的图像获取接口。
* **模式**: 单例模式 (Singleton)。确保系统中只有一份硬件控制实例。
### TaskManager (`src/task/task_manager.h`)
* **职责**: 真正的“大脑”。负责任务的缓冲（队列）、图像获取、算法调度和结果回传。
* **并发**: 拥有独立的执行线程，避免阻塞 Redis 监听线程或 GUI 线程。
### RedisCommunicator (`src/redis/redis_communicator.h`)
* **职责**: 封装 Redis 的底层 socket 操作，提供易用的读写接口和异步监听机制。
### DetectionBase (`src/algorithm/core/detection_base.h`)
* **职责**: 定义所有检测算法的统一接口 `execute`。
* **扩展**: 新增算法只需继承此类并在 `TaskManager` 中注册即可。
---
*文档生成时间: 2025-12-29*

247
image_capture/CMakeLists.txt Normal file
View File

@@ -0,0 +1,247 @@
cmake_minimum_required(VERSION 3.10)
# 支持 MSVC
# 注意：配置 CMake 时请选择合适的生成器（例如 "Visual Studio 17 2022" ）
project(image_capture LANGUAGES CXX)
if(NOT MSVC)
message(FATAL_ERROR "This project requires MSVC (Visual Studio) compiler. Please use a Visual Studio generator (e.g., -G \"Visual Studio 17 2022\").")
endif()
# ============================================================================
# 输出目录
# ============================================================================
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
# 生成 compile_commands.json 文件，供 IntelliSense 使用
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# ============================================================================
# CMake 模块路径
# ============================================================================
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
include(CompilerOptions)
# ============================================================================
# 依赖项 (Qt6, OpenCV)
# ============================================================================
include(Dependencies)
# ============================================================================
# 相机 SDK 配置
# ============================================================================
include(PercipioSDK)
# ============================================================================
# 算法库
# ============================================================================
add_library(algorithm_lib STATIC
src/algorithm/core/detection_base.cpp
src/algorithm/core/detection_result.cpp
src/algorithm/utils/image_processor.cpp
src/algorithm/detections/slot_occupancy/slot_occupancy_detection.cpp
src/algorithm/detections/pallet_offset/pallet_offset_detection.cpp
src/algorithm/detections/beam_rack_deflection/beam_rack_deflection_detection.cpp
src/algorithm/detections/visual_inventory/visual_inventory_detection.cpp
)
target_link_libraries(algorithm_lib PUBLIC
${OpenCV_LIBS}
Open3D::Open3D
Qt6::Core
${HALCON_LIBRARIES}
)
target_include_directories(algorithm_lib PUBLIC
${HALCON_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
src
${CMAKE_CURRENT_SOURCE_DIR}/third_party/percipio/common
)
target_link_directories(algorithm_lib PUBLIC ${OpenCV_LIB_DIRS})
# ============================================================================
# 主可执行文件
# ============================================================================
set(SOURCES
src/main.cpp
src/camera/ty_multi_camera_capture.cpp
src/camera/mvs_multi_camera_capture.cpp
src/device/device_manager.cpp
src/redis/redis_communicator.cpp
src/task/task_manager.cpp
src/vision/vision_controller.cpp
src/common/log_manager.cpp
src/common/config_manager.cpp
src/gui/mainwindow.cpp
src/gui/mainwindow.h
src/gui/mainwindow.ui
src/gui/settings_widget.cpp
src/gui/settings_widget.h
)
add_executable(${PROJECT_NAME} WIN32 ${SOURCES})
target_include_directories(${PROJECT_NAME} PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src
${CMAKE_CURRENT_SOURCE_DIR}/third_party/mvs/Includes
${OpenCV_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR} # Qt AUTOUIC 生成的头文件
)
target_link_libraries(${PROJECT_NAME} PRIVATE
algorithm_lib
cpp_api_lib
tycam
${OpenCV_LIBS}
Qt6::Core
Qt6::Widgets
ws2_32
${CMAKE_CURRENT_SOURCE_DIR}/third_party/mvs/Libraries/win64/MvCameraControl.lib
)
target_link_directories(${PROJECT_NAME} PRIVATE ${OpenCV_LIB_DIRS})
if(Open3D_RUNTIME_DLLS)
foreach(DLL_FILE ${Open3D_RUNTIME_DLLS})
get_filename_component(DLL_NAME "${DLL_FILE}" NAME)
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${DLL_FILE}"
"$<TARGET_FILE_DIR:${PROJECT_NAME}>"
COMMENT "Copying runtime dependency: ${DLL_NAME}"
)
endforeach()
endif()
# Copy tycam.dll to executable directory
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${CAMPORT3_LIB_DIR}/tycam.dll"
"$<TARGET_FILE_DIR:${PROJECT_NAME}>"
COMMENT "Copying tycam.dll to executable directory"
)
# Copy Halcon DLLs
if(HALCON_ROOT)
set(HALCON_BIN_DIR "${HALCON_ROOT}/bin/x64-win64")
# Verify directory exists
if(EXISTS "${HALCON_BIN_DIR}")
set(HALCON_DLLS "halcon.dll" "halconcpp.dll")
foreach(DLL_NAME ${HALCON_DLLS})
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${HALCON_BIN_DIR}/${DLL_NAME}"
"$<TARGET_FILE_DIR:${PROJECT_NAME}>"
COMMENT "Copying Halcon DLL: ${DLL_NAME}"
)
endforeach()
else()
message(WARNING "Halcon bin directory not found at: ${HALCON_BIN_DIR}. DLLs will not be copied.")
endif()
endif()
# ============================================================================
# 工具链
# ============================================================================
add_executable(slot_algo_tuner WIN32
src/tools/slot_algo_tuner/main.cpp
src/tools/slot_algo_tuner/tuner_widget.cpp
src/tools/slot_algo_tuner/tuner_widget.h
)
target_include_directories(slot_algo_tuner PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src
${OpenCV_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
)
target_link_libraries(slot_algo_tuner PRIVATE
${OpenCV_LIBS}
Qt6::Core
Qt6::Widgets
)
target_link_directories(slot_algo_tuner PRIVATE ${OpenCV_LIB_DIRS})
add_executable(calibration_tool WIN32
src/tools/calibration_tool/main.cpp
src/tools/calibration_tool/calibration_widget.cpp
src/tools/calibration_tool/calibration_widget.h
)
target_include_directories(calibration_tool PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src
${OpenCV_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/percipio/include
${CMAKE_CURRENT_SOURCE_DIR}/third_party/mvs/Includes
)
target_link_libraries(calibration_tool PRIVATE
${OpenCV_LIBS}
Qt6::Core
Qt6::Widgets
Open3D::Open3D
tycam
)
target_compile_definitions(calibration_tool PRIVATE NOMINMAX)
target_link_directories(calibration_tool PRIVATE ${OpenCV_LIB_DIRS})
# Intrinsic Dumper Tool
add_executable(intrinsic_dumper
src/tools/intrinsic_dumper/main.cpp
)
target_include_directories(intrinsic_dumper PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src
${OpenCV_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/percipio/include
)
target_link_libraries(intrinsic_dumper PRIVATE
Qt6::Core
tycam
)
# Reference Generator (Teach Tool)
add_executable(generate_reference
src/tools/generate_reference/main.cpp
src/device/device_manager.cpp
src/camera/ty_multi_camera_capture.cpp
src/camera/mvs_multi_camera_capture.cpp
src/common/log_manager.cpp
src/common/config_manager.cpp
)
target_include_directories(generate_reference PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src
${OpenCV_INCLUDE_DIRS}
${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/percipio/include
${CMAKE_CURRENT_SOURCE_DIR}/third_party/mvs/Includes
)
target_link_libraries(generate_reference PRIVATE
algorithm_lib
cpp_api_lib
${OpenCV_LIBS}
Qt6::Core
Qt6::Widgets
tycam
${CMAKE_CURRENT_SOURCE_DIR}/third_party/mvs/Libraries/win64/MvCameraControl.lib
)
target_link_directories(generate_reference PRIVATE ${OpenCV_LIB_DIRS})

32
image_capture/cmake/CompilerOptions.cmake Normal file
View File

@@ -0,0 +1,32 @@
# C++ Standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
# Output Directories
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
# Generate compile_commands.json
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# Definitions
add_definitions(-DOPENCV_DEPENDENCIES)
# Qt6 Setup (Global)
set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTORCC ON)
set(CMAKE_AUTOUIC ON)
# Compiler Specific Options
if(MSVC)
# MSVC specific options
add_compile_options(/utf-8) # Fix C4819 encoding warning
add_compile_options(/W3) # Warning level 3
add_compile_options(/MP) # Multi-processor compilation
add_definitions(-D_CRT_SECURE_NO_WARNINGS) # Suppress C4996 deprecated warnings
add_compile_options($<$<CONFIG:Release>:/O2>) # Maximize speed
add_compile_options($<$<CONFIG:Release>:/Ob2>) # Inline function expansion
endif()

129
image_capture/cmake/Dependencies.cmake Normal file
View File

@@ -0,0 +1,129 @@
# Qt6
if(NOT Qt6_DIR AND NOT ENV{Qt6_DIR} AND NOT CMAKE_PREFIX_PATH)
message(WARNING "Qt6 not found in environment. Please set CMAKE_PREFIX_PATH or Qt6_DIR.")
endif()
find_package(Qt6 REQUIRED COMPONENTS Widgets)
# OpenCV
if(DEFINED ENV{OpenCV_DIR})
set(OpenCV_DIR $ENV{OpenCV_DIR})
message(STATUS "Using OpenCV_DIR from environment: ${OpenCV_DIR}")
elseif(NOT OpenCV_DIR)
message(STATUS "OpenCV_DIR not set, trying to find OpenCV in standard locations...")
set(LEGACY_OPENCV_PATH "D:/enviroments/OPencv4.55/OPencv4.55_MSVC/opencv/build/x64/vc15/lib")
if(EXISTS ${LEGACY_OPENCV_PATH})
set(OpenCV_DIR ${LEGACY_OPENCV_PATH})
message(STATUS "Found legacy OpenCV path: ${OpenCV_DIR}")
endif()
endif()
find_package(OpenCV REQUIRED)
message(STATUS "OpenCV found: ${OpenCV_VERSION}")
message(STATUS "OpenCV libraries: ${OpenCV_LIBS}")
message(STATUS "OpenCV include dirs: ${OpenCV_INCLUDE_DIRS}")
# Open3D
# Open3D
if(DEFINED ENV{Open3D_DIR})
set(Open3D_DIR $ENV{Open3D_DIR})
message(STATUS "Using Open3D_DIR from environment: ${Open3D_DIR}")
elseif(NOT Open3D_DIR)
# Default to 0.18 Release
set(DEFAULT_OPEN3D_PATH "D:/enviroments/Open3d/open3d-devel-windows-amd64-0.18.0-release/CMake")
# Debug path: D:/enviroments/Open3d/open3d-devel-windows-amd64-0.18.0-debug/CMake
if(EXISTS ${DEFAULT_OPEN3D_PATH})
set(Open3D_DIR ${DEFAULT_OPEN3D_PATH})
message(STATUS "Using default Open3D path: ${Open3D_DIR}")
endif()
endif()
find_package(Open3D REQUIRED)
message(STATUS "Open3D found: ${Open3D_VERSION}")
message(STATUS "Open3D DIR: ${Open3D_DIR}")
# Find Open3D DLL and dependencies (TBB)
# Adjust ROOT calculation based on where Config is found.
get_filename_component(DIR_NAME "${Open3D_DIR}" NAME)
if("${DIR_NAME}" STREQUAL "CMake")
# Structure: root/CMake/Open3DConfig.cmake -> root is up one level
get_filename_component(Open3D_ROOT "${Open3D_DIR}/.." ABSOLUTE)
else()
# Assume standard install: root/lib/cmake/Open3D/Open3DConfig.cmake -> root is up 3 levels
get_filename_component(Open3D_ROOT "${Open3D_DIR}/../../.." ABSOLUTE)
endif()
set(Open3D_BIN_DIR "${Open3D_ROOT}/bin")
set(Open3D_RUNTIME_DLLS "")
find_file(Open3D_DLL NAMES Open3D.dll PATHS ${Open3D_BIN_DIR} NO_DEFAULT_PATH)
if(Open3D_DLL)
list(APPEND Open3D_RUNTIME_DLLS ${Open3D_DLL})
message(STATUS "Found Open3D DLL: ${Open3D_DLL}")
else()
message(WARNING "Open3D DLL not found in ${Open3D_BIN_DIR}. You might need to add it to your PATH manually.")
endif()
# Find TBB DLLs (tbb.dll or tbb12_debug.dll etc)
# We glob for tbb*.dll but filter based on build type to avoid mixing runtimes
file(GLOB TBB_ALL_DLLS "${Open3D_BIN_DIR}/tbb*.dll")
set(TBB_DLLS ${TBB_ALL_DLLS})
# Filter out debug DLLs (ending in _debug.dll or d.dll)
list(FILTER TBB_DLLS EXCLUDE REGEX ".*(_debug|d)\\.dll$")
if(NOT TBB_DLLS)
# If no release DLLs found, check if we only have debug ones
if(TBB_ALL_DLLS)
message(WARNING "Only Debug TBB DLLs found in ${Open3D_BIN_DIR}. Release build might crash due to ABI mismatch!")
# Fallback: copy everything (dangerous but better than nothing?)
set(TBB_DLLS ${TBB_ALL_DLLS})
else()
message(WARNING "No TBB DLLs found in ${Open3D_BIN_DIR}.")
endif()
endif()
if(TBB_DLLS)
list(APPEND Open3D_RUNTIME_DLLS ${TBB_DLLS})
message(STATUS "Found TBB DLLs: ${TBB_DLLS}")
endif()
# Halcon
# Force usage of the user known path if possible, or fallback to environment
set(USER_PROVIDED_HALCON_ROOT "C:/Users/cve/AppData/Local/Programs/MVTec/HALCON-23.11-Progress")
if(EXISTS "${USER_PROVIDED_HALCON_ROOT}")
set(HALCON_ROOT "${USER_PROVIDED_HALCON_ROOT}")
message(STATUS "Using user provided HALCON_ROOT: ${HALCON_ROOT}")
elseif(DEFINED ENV{HALCONROOT})
set(HALCON_ROOT $ENV{HALCONROOT})
file(TO_CMAKE_PATH "${HALCON_ROOT}" HALCON_ROOT)
message(STATUS "Using HALCON_ROOT from environment: ${HALCON_ROOT}")
else()
message(WARNING "HALCONROOT not found.")
endif()
if(HALCON_ROOT)
set(HALCON_INCLUDE_DIRS
"${HALCON_ROOT}/include"
"${HALCON_ROOT}/include/halconcpp"
)
if(WIN32)
set(HALCON_LIB_DIR "${HALCON_ROOT}/lib/x64-win64")
if(NOT EXISTS "${HALCON_LIB_DIR}")
set(HALCON_LIB_DIR "${HALCON_ROOT}/lib")
endif()
set(HALCON_LIBRARIES
"${HALCON_LIB_DIR}/halcon.lib"
"${HALCON_LIB_DIR}/halconcpp.lib"
)
endif()
message(STATUS "Halcon include: ${HALCON_INCLUDE_DIRS}")
message(STATUS "Halcon libs: ${HALCON_LIBRARIES}")
endif()

55
image_capture/cmake/PercipioSDK.cmake Normal file
View File

@@ -0,0 +1,55 @@
# Camera SDK Paths
set(CAMPORT3_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/third_party/percipio)
set(CAMPORT3_LIB_DIR ${CAMPORT3_ROOT}/lib/win/x64)
# Import tycam library (MinGW)
add_library(tycam SHARED IMPORTED)
if(EXISTS ${CAMPORT3_LIB_DIR}/libtycam.dll.a)
set_target_properties(tycam PROPERTIES
IMPORTED_LOCATION ${CAMPORT3_LIB_DIR}/tycam.dll
IMPORTED_IMPLIB ${CAMPORT3_LIB_DIR}/libtycam.dll.a
)
message(STATUS "Using libtycam.dll.a (MinGW compatible)")
elseif(EXISTS ${CAMPORT3_LIB_DIR}/tycam.lib)
set_target_properties(tycam PROPERTIES
IMPORTED_LOCATION ${CAMPORT3_LIB_DIR}/tycam.dll
IMPORTED_IMPLIB ${CAMPORT3_LIB_DIR}/tycam.lib
)
message(STATUS "Using tycam.lib (may require conversion to .dll.a if linking fails)")
else()
message(FATAL_ERROR "Neither libtycam.dll.a nor tycam.lib found in ${CAMPORT3_LIB_DIR}")
endif()
# Static API Library Sources
set(CPP_API_SOURCES
${CAMPORT3_ROOT}/sample_v2/cpp/Device.cpp
${CAMPORT3_ROOT}/sample_v2/cpp/Frame.cpp
${CAMPORT3_ROOT}/common/MatViewer.cpp
${CAMPORT3_ROOT}/common/TYThread.cpp
${CAMPORT3_ROOT}/common/crc32.cpp
${CAMPORT3_ROOT}/common/json11.cpp
${CAMPORT3_ROOT}/common/ParametersParse.cpp
${CAMPORT3_ROOT}/common/huffman.cpp
${CAMPORT3_ROOT}/common/ImageSpeckleFilter.cpp
${CAMPORT3_ROOT}/common/DepthInpainter.cpp
)
add_library(cpp_api_lib STATIC ${CPP_API_SOURCES})
target_include_directories(cpp_api_lib PUBLIC
${CAMPORT3_ROOT}/include
${CAMPORT3_ROOT}/sample_v2/hpp
${CAMPORT3_ROOT}/common
${OpenCV_INCLUDE_DIRS}
)
# Fix for MinGW: Ensure standard C++ headers are found
if(MINGW)
target_include_directories(cpp_api_lib SYSTEM PUBLIC
${CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES}
)
endif()
target_link_libraries(cpp_api_lib PUBLIC ${OpenCV_LIBS})
target_link_directories(cpp_api_lib PUBLIC ${OpenCV_LIB_DIRS})

130
image_capture/config.json Normal file
View File

@@ -0,0 +1,130 @@
{
"redis": {
"host": "127.0.0.1",
"port": 6379,
"db": 0
},
"cameras": {
"depth_enabled": true,
"color_enabled": true,
"mapping": [
{
"id": "camera_0",
"index": 0
},
{
"id": "camera_1",
"index": 1
},
{
"id": "camera_2",
"index": 2
},
{
"id": "camera_3",
"index": 3
}
]
},
"vision": {
"save_path": "./images",
"log_level": 1
},
"algorithms": {
"beam_rack_deflection": {
"beam_roi_points": [
{
"x": 100,
"y": 50
},
{
"x": 540,
"y": 80
},
{
"x": 540,
"y": 280
},
{
"x": 100,
"y": 280
}
],
"rack_roi_points": [
{
"x": 50,
"y": 50
},
{
"x": 150,
"y": 50
},
{
"x": 150,
"y": 430
},
{
"x": 50,
"y": 430
}
],
"beam_thresholds": {
"A": -10.0,
"B": -5.0,
"C": 5.0,
"D": 10.0
},
"rack_thresholds": {
"A": -6.0,
"B": -3.0,
"C": 3.0,
"D": 6.0
}
},
"pallet_offset": {
"offset_lat_mm_thresholds": {
"A": -20.0,
"B": -10.0,
"C": 10.0,
"D": 20.0
},
"offset_lon_mm_thresholds": {
"A": -20.0,
"B": -10.0,
"C": 10.0,
"D": 20.0
},
"rotation_angle_thresholds": {
"A": -5.0,
"B": -2.5,
"C": 2.5,
"D": 5.0
},
"hole_def_mm_left_thresholds": {
"A": -8.0,
"B": -4.0,
"C": 4.0,
"D": 8.0
},
"hole_def_mm_right_thresholds": {
"A": -8.0,
"B": -4.0,
"C": 4.0,
"D": 8.0
}
},
"slot_occupancy": {
"depth_threshold_mm": 100.0,
"confidence_threshold": 0.8
},
"visual_inventory": {
"barcode_confidence_threshold": 0.7,
"roi_enabled": true
},
"general": {
"min_depth_mm": 800.0,
"max_depth_mm": 3000.0,
"sample_points": 50
}
}
}

6
image_capture/note.md Normal file
View File

@@ -0,0 +1,6 @@
# 确保在 image_capture 目录下
cd d:\Git\stereo_warehouse_inspection\image_capture
Remove-Item -Recurse -Force build
# 使用 Visual Studio 生成器重新配置项目： 指定 -G "Visual Studio 17 2022" (根据你的VS版本调整，通常是 16 2019 或 17 2022)。
cmake -G "Visual Studio 17 2022" -A x64 -B build
cmake --build build --config Release

0
README.md → image_capture/run_log.txt
View File

172
image_capture/src/algorithm/core/detection_base.cpp Normal file
View File

@@ -0,0 +1,172 @@
#include "detection_base.h"
#include "../detections/beam_rack_deflection/beam_rack_deflection_detection.h"
#include "../detections/pallet_offset/pallet_offset_detection.h"
#include "../detections/slot_occupancy/slot_occupancy_detection.h"
#include "../detections/visual_inventory/visual_inventory_detection.h"
#include "detection_result.h"
#include <chrono>
#include <ctime>
#include <iomanip>
#include <opencv2/opencv.hpp>
#include <sstream>
/**
* @brief 获取当前时间戳字符串
*/
static std::string getCurrentTimeString() {
auto now = std::chrono::system_clock::now();
auto time_t = std::chrono::system_clock::to_time_t(now);
std::tm *tm = std::localtime(&time_t);
std::stringstream ss;
ss << std::put_time(tm, "%Y-%m-%d %H:%M:%S");
return ss.str();
}
// ========== SlotOccupancyDetection ==========
bool SlotOccupancyDetection::execute(const cv::Mat &depth_img,
const cv::Mat &color_img,
const std::string &side,
DetectionResult &result,
const std::vector<Point3D> *point_cloud,
int beam_length) {
result.result_type = 1;
result.result_status = "fail";
// 调用算法进行检测
SlotOccupancyResult algo_result;
if (!SlotOccupancyAlgorithm::detect(depth_img, color_img, side,
algo_result)) {
std::cout
<< "[Detection] SlotOccupancy: Detection failed (Algorithm error)."
<< std::endl;
result.result_status = "fail";
result.last_update_time = getCurrentTimeString();
return false;
}
// 将算法结果填充到 DetectionResult
result.slot_occupied = algo_result.slot_occupied;
result.result_status = algo_result.success ? "success" : "fail";
result.last_update_time = getCurrentTimeString();
// 日志输出到界面 (UI Log)
std::cout << "[Detection] SlotOccupancy Result: "
<< (result.slot_occupied ? "Occupied (有货)" : "Empty (无货)")
<< std::endl;
return algo_result.success;
}
// ========== PalletOffsetDetection ==========
bool PalletOffsetDetection::execute(const cv::Mat &depth_img,
const cv::Mat &color_img,
const std::string &side,
DetectionResult &result,
const std::vector<Point3D> *point_cloud,
int beam_length) {
result.result_type = 2;
result.result_status = "fail";
// 调用算法进行检测
PalletOffsetResult algo_result;
if (!PalletOffsetAlgorithm::detect(depth_img, color_img, side, algo_result,
point_cloud)) {
result.result_status = "fail";
result.last_update_time = getCurrentTimeString();
return false;
}
// 将算法结果填充到 DetectionResult
result.offset_lat_mm_value = algo_result.offset_lat_mm_value;
result.offset_lon_mm_value = algo_result.offset_lon_mm_value;
result.rotation_angle_value = algo_result.rotation_angle_value;
result.hole_def_mm_left_value = algo_result.hole_def_mm_left_value;
result.hole_def_mm_right_value = algo_result.hole_def_mm_right_value;
result.offset_lat_mm_threshold = algo_result.offset_lat_mm_threshold;
result.offset_lon_mm_threshold = algo_result.offset_lon_mm_threshold;
result.rotation_angle_threshold = algo_result.rotation_angle_threshold;
result.hole_def_mm_left_threshold = algo_result.hole_def_mm_left_threshold;
result.hole_def_mm_right_threshold = algo_result.hole_def_mm_right_threshold;
result.offset_lat_mm_warning_alarm = algo_result.offset_lat_mm_warning_alarm;
result.offset_lon_mm_warning_alarm = algo_result.offset_lon_mm_warning_alarm;
result.rotation_angle_warning_alarm =
algo_result.rotation_angle_warning_alarm;
result.hole_def_mm_left_warning_alarm =
algo_result.hole_def_mm_left_warning_alarm;
result.hole_def_mm_right_warning_alarm =
algo_result.hole_def_mm_right_warning_alarm;
result.result_status = algo_result.success ? "success" : "fail";
result.last_update_time = getCurrentTimeString();
return algo_result.success;
}
// ========== BeamRackDeflectionDetection ==========
bool BeamRackDeflectionDetection::execute(
const cv::Mat &depth_img, const cv::Mat &color_img, const std::string &side,
DetectionResult &result, const std::vector<Point3D> *point_cloud,
int beam_length) {
result.result_type = 3;
result.result_status = "fail";
// Select ROI based on beam_length
std::vector<cv::Point2i> beam_roi;
if (beam_length == 2180) {
beam_roi = BeamRackDeflectionAlgorithm::BEAM_ROI_2180;
} else if (beam_length == 1380) {
beam_roi = BeamRackDeflectionAlgorithm::BEAM_ROI_1380;
}
// 调用算法进行检测
BeamRackDeflectionResult algo_result;
if (!BeamRackDeflectionAlgorithm::detect(
depth_img, color_img, side, algo_result, point_cloud, beam_roi)) {
result.result_status = "fail";
result.last_update_time = getCurrentTimeString();
return false;
}
// 将算法结果填充到 DetectionResult
result.beam_def_mm_value = algo_result.beam_def_mm_value;
result.rack_def_mm_value = algo_result.rack_def_mm_value;
result.beam_def_mm_threshold = algo_result.beam_def_mm_threshold;
result.rack_def_mm_threshold = algo_result.rack_def_mm_threshold;
result.beam_def_mm_warning_alarm = algo_result.beam_def_mm_warning_alarm;
result.rack_def_mm_warning_alarm = algo_result.rack_def_mm_warning_alarm;
result.result_status = algo_result.success ? "success" : "fail";
result.last_update_time = getCurrentTimeString();
return algo_result.success;
}
// ========== VisualInventoryDetection ==========
bool VisualInventoryDetection::execute(const cv::Mat &depth_img,
const cv::Mat &color_img,
const std::string &side,
DetectionResult &result,
const std::vector<Point3D> *point_cloud,
int beam_length) {
result.result_type = 4;
result.result_status = "fail";
// 调用算法进行检测
VisualInventoryResult algo_result;
if (!VisualInventoryAlgorithm::detect(depth_img, color_img, side,
algo_result)) {
result.result_status = "fail";
result.last_update_time = getCurrentTimeString();
return false;
}
// 将算法结果填充到 DetectionResult
result.result_barcodes = algo_result.result_barcodes;
result.result_status = algo_result.success ? "success" : "fail";
result.last_update_time = getCurrentTimeString();
return algo_result.success;
}

116
image_capture/src/algorithm/core/detection_base.h Normal file
View File

@@ -0,0 +1,116 @@
#pragma once
#include <string>
struct DetectionResult;
namespace cv {
class Mat;
}
#include "../../common_types.h"
/**
* @brief 检测任务基类
*
* 所有检测任务都继承自此类，实现统一的接口
*/
class DetectionBase {
public:
DetectionBase() {}
virtual ~DetectionBase() {}
/**
* 执行检测任务
* @param depth_img 深度图像（可选）
* @param color_img 彩色图像（可选）
* @param side 货架侧（"left"或"right"）
* @param result [输出] 检测结果
* @param point_cloud [可选] 点云数据
* @return 是否检测成功
*/
virtual bool execute(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, DetectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
int beam_length = 0) = 0;
/**
* 获取任务类型（对应flag值）
*/
virtual int getTaskType() const = 0;
/**
* 获取任务名称
*/
virtual std::string getTaskName() const = 0;
};
/**
* @brief Task 1: 货位有无检测
*/
class SlotOccupancyDetection : public DetectionBase {
public:
SlotOccupancyDetection() {}
virtual ~SlotOccupancyDetection() {}
bool execute(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, DetectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
int beam_length = 0) override;
int getTaskType() const override { return 1; }
std::string getTaskName() const override { return "SlotOccupancyDetection"; }
};
/**
* @brief Task 2: 托盘位置偏移检测 - 插孔变形检测（取货时）
*/
class PalletOffsetDetection : public DetectionBase {
public:
PalletOffsetDetection() {}
virtual ~PalletOffsetDetection() {}
bool execute(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, DetectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
int beam_length = 0) override;
int getTaskType() const override { return 2; }
std::string getTaskName() const override { return "PalletOffsetDetection"; }
};
/**
* @brief Task 3: 横梁变形检测 - 货架立柱变形检测（放货时）
*/
class BeamRackDeflectionDetection : public DetectionBase {
public:
BeamRackDeflectionDetection() {}
virtual ~BeamRackDeflectionDetection() {}
bool execute(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, DetectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
int beam_length = 0) override;
int getTaskType() const override { return 3; }
std::string getTaskName() const override {
return "BeamRackDeflectionDetection";
}
};
/**
* @brief Task 4: 视觉盘点（扫码）
*/
class VisualInventoryDetection : public DetectionBase {
public:
VisualInventoryDetection() {}
virtual ~VisualInventoryDetection() {}
bool execute(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, DetectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
int beam_length = 0) override;
int getTaskType() const override { return 4; }
std::string getTaskName() const override {
return "VisualInventoryDetection";
}
};

184
image_capture/src/algorithm/core/detection_result.cpp Normal file
View File

@@ -0,0 +1,184 @@
#include "detection_result.h"
#include <iostream>
#include <sstream>
std::string DetectionResult::toJson() const {
// TODO: 使用JSON库（如nlohmann/json）生成JSON字符串
// 当前使用简单的字符串拼接方式
std::ostringstream oss;
oss << "{";
// 基础字段
oss << "\"result_status\":\"" << result_status << "\",";
oss << "\"result_type\":" << result_type << ",";
oss << "\"last_update_time\":\"" << last_update_time << "\"";
// Flag 1
if (result_type == 1) {
oss << ",\"slot_occupied\":" << (slot_occupied ? "true" : "false");
}
// Flag 2
if (result_type == 2) {
oss << ",\"offset_lat_mm_value\":" << offset_lat_mm_value;
if (!offset_lat_mm_threshold.empty()) {
oss << ",\"offset_lat_mm_threshold\":" << offset_lat_mm_threshold;
}
if (!offset_lat_mm_warning_alarm.empty()) {
oss << ",\"offset_lat_mm_warning_alarm\":" << offset_lat_mm_warning_alarm;
}
oss << ",\"offset_lon_mm_value\":" << offset_lon_mm_value;
if (!offset_lon_mm_threshold.empty()) {
oss << ",\"offset_lon_mm_threshold\":" << offset_lon_mm_threshold;
}
if (!offset_lon_mm_warning_alarm.empty()) {
oss << ",\"offset_lon_mm_warning_alarm\":" << offset_lon_mm_warning_alarm;
}
oss << ",\"hole_def_mm_left_value\":" << hole_def_mm_left_value;
if (!hole_def_mm_left_threshold.empty()) {
oss << ",\"hole_def_mm_left_threshold\":" << hole_def_mm_left_threshold;
}
if (!hole_def_mm_left_warning_alarm.empty()) {
oss << ",\"hole_def_mm_left_warning_alarm\":"
<< hole_def_mm_left_warning_alarm;
}
oss << ",\"hole_def_mm_right_value\":" << hole_def_mm_right_value;
if (!hole_def_mm_right_threshold.empty()) {
oss << ",\"hole_def_mm_right_threshold\":" << hole_def_mm_right_threshold;
}
if (!hole_def_mm_right_warning_alarm.empty()) {
oss << ",\"hole_def_mm_right_warning_alarm\":"
<< hole_def_mm_right_warning_alarm;
}
oss << ",\"rotation_angle_value\":" << rotation_angle_value;
if (!rotation_angle_threshold.empty()) {
oss << ",\"rotation_angle_threshold\":" << rotation_angle_threshold;
}
if (!rotation_angle_warning_alarm.empty()) {
oss << ",\"rotation_angle_warning_alarm\":"
<< rotation_angle_warning_alarm;
}
}
// Flag 3
if (result_type == 3) {
oss << ",\"beam_def_mm_value\":" << beam_def_mm_value;
if (!beam_def_mm_threshold.empty()) {
oss << ",\"beam_def_mm_threshold\":" << beam_def_mm_threshold;
}
if (!beam_def_mm_warning_alarm.empty()) {
oss << ",\"beam_def_mm_warning_alarm\":" << beam_def_mm_warning_alarm;
}
oss << ",\"rack_def_mm_value\":" << rack_def_mm_value;
if (!rack_def_mm_threshold.empty()) {
oss << ",\"rack_def_mm_threshold\":" << rack_def_mm_threshold;
}
if (!rack_def_mm_warning_alarm.empty()) {
oss << ",\"rack_def_mm_warning_alarm\":" << rack_def_mm_warning_alarm;
}
}
// Flag 4 & 5
if (result_type == 4 || result_type == 5) {
if (!result_barcodes.empty()) {
oss << ",\"result_barcodes\":" << result_barcodes;
}
}
oss << "}";
return oss.str();
}
bool DetectionResult::fromJson(const std::string &json_str) {
// TODO: 使用JSON库解析JSON字符串
// 当前实现为占位符
std::cerr << "[DetectionResult] TODO: Implement JSON parsing" << std::endl;
return false;
}
std::map<std::string, std::string> DetectionResult::toMap() const {
std::map<std::string, std::string> m;
// Helper to convert float to string
auto floatToStr = [](float val) { return std::to_string(val); };
// Helper to convert bool to string "true"/"false"
auto boolToStr = [](bool val) { return val ? "true" : "false"; };
// 基础字段 (总是写入)
m["result_status"] = result_status;
m["result_type"] = std::to_string(result_type);
m["last_update_time"] = last_update_time;
// Flag 1: 货位有无
if (result_type == 1) {
m["slot_occupied"] = boolToStr(slot_occupied);
}
// Flag 2: 托盘检测
if (result_type == 2) {
m["offset_lat_mm_value"] = floatToStr(offset_lat_mm_value);
m["offset_lat_mm_threshold"] =
offset_lat_mm_threshold.empty() ? "{}" : offset_lat_mm_threshold;
m["offset_lat_mm_warning_alarm"] = offset_lat_mm_warning_alarm.empty()
? "{}"
: offset_lat_mm_warning_alarm;
m["offset_lon_mm_value"] = floatToStr(offset_lon_mm_value);
m["offset_lon_mm_threshold"] =
offset_lon_mm_threshold.empty() ? "{}" : offset_lon_mm_threshold;
m["offset_lon_mm_warning_alarm"] = offset_lon_mm_warning_alarm.empty()
? "{}"
: offset_lon_mm_warning_alarm;
m["hole_def_mm_left_value"] = floatToStr(hole_def_mm_left_value);
m["hole_def_mm_left_threshold"] =
hole_def_mm_left_threshold.empty() ? "{}" : hole_def_mm_left_threshold;
m["hole_def_mm_left_warning_alarm"] = hole_def_mm_left_warning_alarm.empty()
? "{}"
: hole_def_mm_left_warning_alarm;
m["hole_def_mm_right_value"] = floatToStr(hole_def_mm_right_value);
m["hole_def_mm_right_threshold"] = hole_def_mm_right_threshold.empty()
? "{}"
: hole_def_mm_right_threshold;
m["hole_def_mm_right_warning_alarm"] =
hole_def_mm_right_warning_alarm.empty()
? "{}"
: hole_def_mm_right_warning_alarm;
m["rotation_angle_value"] = floatToStr(rotation_angle_value);
m["rotation_angle_threshold"] =
rotation_angle_threshold.empty() ? "{}" : rotation_angle_threshold;
m["rotation_angle_warning_alarm"] = rotation_angle_warning_alarm.empty()
? "{}"
: rotation_angle_warning_alarm;
}
// Flag 3: 横梁/立柱检测
if (result_type == 3) {
m["beam_def_mm_value"] = floatToStr(beam_def_mm_value);
m["beam_def_mm_threshold"] =
beam_def_mm_threshold.empty() ? "{}" : beam_def_mm_threshold;
m["beam_def_mm_warning_alarm"] =
beam_def_mm_warning_alarm.empty() ? "{}" : beam_def_mm_warning_alarm;
m["rack_def_mm_value"] = floatToStr(rack_def_mm_value);
m["rack_def_mm_threshold"] =
rack_def_mm_threshold.empty() ? "{}" : rack_def_mm_threshold;
m["rack_def_mm_warning_alarm"] =
rack_def_mm_warning_alarm.empty() ? "{}" : rack_def_mm_warning_alarm;
}
// Flag 4 & 5: 视觉盘点 & 结束
if (result_type == 4 || result_type == 5) {
m["result_barcodes"] = result_barcodes.empty() ? "{}" : result_barcodes;
}
return m;
}

96
image_capture/src/algorithm/core/detection_result.h Normal file
View File

@@ -0,0 +1,96 @@
#pragma once
#include <string>
#include <map>
// TODO: 添加nlohmann/json库依赖
// 临时使用简单的JSON字符串表示，后续替换为nlohmann::json
// 为了简化，这里使用std::string存储JSON字符串
// 实际实现时应该使用nlohmann::json或类似的JSON库
using JsonValue = std::string; // 临时定义，实际应使用nlohmann::json
/**
* @brief 检测结果数据结构
*
* 包含所有检测任务的结果数据，根据任务类型（flag）填充相应字段
*/
struct DetectionResult {
// 基础字段
std::string result_status; // "success" 或 "fail"
int result_type; // 对应 vision_task_flag（1~5）
std::string last_update_time; // "YYYY-MM-DD HH:MM:SS"
// Flag 1: 货位有无检测
bool slot_occupied; // 货位是否有托盘/货物
// Flag 2: 托盘位置偏移检测 - 插孔变形检测（取货时）
// 左右偏移量
float offset_lat_mm_value;
JsonValue offset_lat_mm_threshold; // {"A": -5.0, "B": -3.0, "C": 3.0, "D": 5.0}
JsonValue offset_lat_mm_warning_alarm; // {"warning": false, "alarm": false}
// 前后偏移量
float offset_lon_mm_value;
JsonValue offset_lon_mm_threshold;
JsonValue offset_lon_mm_warning_alarm;
// 左侧插孔变形
float hole_def_mm_left_value;
JsonValue hole_def_mm_left_threshold;
JsonValue hole_def_mm_left_warning_alarm;
// 右侧插孔变形
float hole_def_mm_right_value;
JsonValue hole_def_mm_right_threshold;
JsonValue hole_def_mm_right_warning_alarm;
// 托盘整体旋转角度
float rotation_angle_value;
JsonValue rotation_angle_threshold;
JsonValue rotation_angle_warning_alarm;
// Flag 3: 横梁变形检测 - 货架立柱变形检测（放货时）
// 横梁弯曲量
float beam_def_mm_value;
JsonValue beam_def_mm_threshold;
JsonValue beam_def_mm_warning_alarm;
// 立柱弯曲量
float rack_def_mm_value;
JsonValue rack_def_mm_threshold;
JsonValue rack_def_mm_warning_alarm;
// Flag 4: 视觉盘点（扫码）
JsonValue result_barcodes; // {"A01":["BOX111","BOX112"], "A02":["BOX210"]}
DetectionResult()
: result_status("fail")
, result_type(0)
, slot_occupied(false)
, offset_lat_mm_value(0.0f)
, offset_lon_mm_value(0.0f)
, hole_def_mm_left_value(0.0f)
, hole_def_mm_right_value(0.0f)
, rotation_angle_value(0.0f)
, beam_def_mm_value(0.0f)
, rack_def_mm_value(0.0f)
{
}
/**
* 转换为JSON字符串
*/
std::string toJson() const;
/**
* 从JSON字符串解析
*/
bool fromJson(const std::string& json_str);
/**
* 转换为Key-Value Map
* 用于分别写入Redis各个Key
*/
std::map<std::string, std::string> toMap() const;
};

932
image_capture/src/algorithm/detections/beam_rack_deflection/beam_rack_deflection_detection.cpp Normal file
View File

@@ -0,0 +1,932 @@
#include "beam_rack_deflection_detection.h"
#include "../../../common/config_manager.h"
#define DEBUG_ROI_SELECTION // 启用交互式ROI选择（调试模式）
#include <algorithm>
#include <cmath>
#include <iostream>
#include <string>
#include <vector>
#include <Eigen/Dense>
#include <QCoreApplication>
#include <QDebug>
#include <QDir>
#include <QFile>
#include <QJsonArray>
#include <QJsonDocument>
#include <QJsonObject>
//====================
// 步骤1：默认ROI点定义
//====================
// 定义默认ROI点（四个点：左上、右上、右下、左下）
// 横梁ROI默认点（示例值，可根据实际场景调整）
const std::vector<cv::Point2i>
BeamRackDeflectionAlgorithm::DEFAULT_BEAM_ROI_POINTS = {
cv::Point2i(100, 50), // 左上
cv::Point2i(540, 80), // 右上
cv::Point2i(540, 280), // 右下
cv::Point2i(100, 280) // 左下
};
// 2180mm 横梁 ROI (Placeholder - Same as Default for now)
const std::vector<cv::Point2i> BeamRackDeflectionAlgorithm::BEAM_ROI_2180 = {
cv::Point2i(100, 50), cv::Point2i(540, 80), cv::Point2i(540, 280),
cv::Point2i(100, 280)};
// 1380mm 横梁 ROI (Placeholder - Same as Default for now)
const std::vector<cv::Point2i> BeamRackDeflectionAlgorithm::BEAM_ROI_1380 = {
cv::Point2i(100, 50), cv::Point2i(540, 80), cv::Point2i(540, 280),
cv::Point2i(100, 280)};
//====================
// 步骤2：立柱ROI默认点定义
//====================
// 立柱ROI默认点（示例值，可根据实际场景调整）
const std::vector<cv::Point2i>
BeamRackDeflectionAlgorithm::DEFAULT_RACK_ROI_POINTS = {
cv::Point2i(50, 50), // 左上
cv::Point2i(150, 50), // 右上
cv::Point2i(150, 430), // 右下
cv::Point2i(50, 430) // 左下
};
//====================
// 步骤3：横梁阈值默认值定义
//====================
// 定义默认阈值（四个值：A负方向报警, B负方向警告, C正方向警告, D正方向报警）
// 横梁阈值默认值（示例值，可根据实际需求调整）
const std::vector<float> BeamRackDeflectionAlgorithm::DEFAULT_BEAM_THRESHOLDS =
{
-50.0f, // A: 负方向报警阈值 (横梁Y+方向忽略)
-30.0f, // B: 负方向警告阈值 (横梁Y+方向忽略)
30.0f, // C: 正方向警告阈值 (>30mm)
50.0f // D: 正方向报警阈值 (>50mm)
};
//====================
// 步骤4：立柱阈值默认值定义
//====================
// 立柱阈值默认值（示例值，可根据实际需求调整）
const std::vector<float> BeamRackDeflectionAlgorithm::DEFAULT_RACK_THRESHOLDS =
{
-50.0f, // A: 负方向报警阈值 (对称参考)
-30.0f, // B: 负方向警告阈值 (对称参考)
30.0f, // C: 正方向警告阈值 (绝对值 > 30mm)
50.0f // D: 正方向报警阈值 (绝对值 > 50mm)
};
//====================
// 步骤5：加载标定参数
//====================
bool BeamRackDeflectionAlgorithm::loadCalibration(Eigen::Matrix4d &transform) {
// 在当前目录查找 calibration_result_*.json 文件
QDir dir = QDir::current();
QStringList filters;
filters << "calibration_result_*.json";
dir.setNameFilters(filters);
QFileInfoList list = dir.entryInfoList(QDir::Files, QDir::Time); // 按时间排序
if (list.empty()) {
std::cerr << "[BeamRackDeflectionAlgorithm] Warning: No calibration file "
"found. Using Identity."
<< std::endl;
transform = Eigen::Matrix4d::Identity();
return false;
}
// 使用最新的文件
QString filePath = list.first().absoluteFilePath();
std::cout << "[BeamRackDeflectionAlgorithm] Loading calibration from: "
<< filePath.toStdString() << std::endl;
QFile file(filePath);
if (!file.open(QIODevice::ReadOnly)) {
std::cerr << "[BeamRackDeflectionAlgorithm] Error: Could not open file."
<< std::endl;
transform = Eigen::Matrix4d::Identity();
return false;
}
QByteArray data = file.readAll();
QJsonDocument doc = QJsonDocument::fromJson(data);
if (doc.isNull()) {
std::cerr << "[BeamRackDeflectionAlgorithm] Error: Invalid JSON."
<< std::endl;
transform = Eigen::Matrix4d::Identity();
return false;
}
QJsonObject root = doc.object();
if (root.contains("transformation_matrix")) {
QJsonArray arr = root["transformation_matrix"].toArray();
if (arr.size() == 16) {
for (int i = 0; i < 4; ++i) {
for (int j = 0; j < 4; ++j) {
transform(i, j) = arr[i * 4 + j].toDouble();
}
}
return true;
}
}
std::cerr << "[BeamRackDeflectionAlgorithm] Error: transformation_matrix "
"missing or invalid."
<< std::endl;
transform = Eigen::Matrix4d::Identity();
return false;
}
//====================
// 步骤6：横梁和立柱变形检测主函数
//====================
bool BeamRackDeflectionAlgorithm::detect(
const cv::Mat &depth_img, const cv::Mat &color_img, const std::string &side,
BeamRackDeflectionResult &result, const std::vector<Point3D> *point_cloud,
const std::vector<cv::Point2i> &beam_roi_points,
const std::vector<cv::Point2i> &rack_roi_points,
const std::vector<float> &beam_thresholds,
const std::vector<float> &rack_thresholds) {
// 算法启用开关
const bool USE_ALGORITHM = true;
if (USE_ALGORITHM) {
// --- 真实算法逻辑 ---
// 6.1 初始化结果
result.success = false;
result.beam_def_mm_value = 0.0f;
result.rack_def_mm_value = 0.0f;
// 6.2 验证深度图
if (depth_img.empty()) {
std::cerr << "[BeamRackDeflectionAlgorithm] ERROR: Depth image empty!"
<< std::endl;
return false;
}
// 6.3 检查点云
if (!point_cloud || point_cloud->empty()) {
std::cerr
<< "[BeamRackDeflectionAlgorithm] ERROR: Point cloud empty or null!"
<< std::endl;
return false;
}
// 6.4 加载标定参数
Eigen::Matrix4d transform;
loadCalibration(transform);
// 6.5 转换点云并按ROI组织
// 注意：假设点云与深度图分辨率匹配（行优先）
// 如果点云只是有效点的列表而没有结构，我们无法轻松映射2D ROI
// 但通常标准会保持 size = width * height
if (point_cloud->size() != depth_img.cols * depth_img.rows) {
std::cerr << "[BeamRackDeflectionAlgorithm] Warning: Point cloud size "
"mismatch. Assuming organized."
<< std::endl;
}
int width = depth_img.cols;
int height = depth_img.rows;
std::vector<Eigen::Vector3d> beam_points_3d;
std::vector<Eigen::Vector3d> rack_points_3d;
// 6.6 辅助函数：检查点是否在ROI内
auto isInRoi = [](const std::vector<cv::Point2i> &roi, int x, int y) {
if (roi.size() < 3)
return false;
return cv::pointPolygonTest(roi, cv::Point2f((float)x, (float)y),
false) >= 0;
};
// 6.7 确定实际使用的ROI（使用默认值或自定义值）
std::vector<cv::Point2i> actual_beam_roi =
beam_roi_points.empty() ? DEFAULT_BEAM_ROI_POINTS : beam_roi_points;
std::vector<cv::Point2i> actual_rack_roi =
rack_roi_points.empty() ? DEFAULT_RACK_ROI_POINTS : rack_roi_points;
// 6.8 交互式ROI选择（调试模式）
#ifdef DEBUG_ROI_SELECTION
// 辅助lambda函数：用于4点ROI选择
auto selectPolygonROI =
[&](const std::string &winName,
const cv::Mat &bg_img) -> std::vector<cv::Point2i> {
std::vector<cv::Point> clicks;
std::string fullWinName = winName + " (Click 4 points)";
cv::namedWindow(fullWinName, cv::WINDOW_AUTOSIZE);
cv::setMouseCallback(
fullWinName,
[](int event, int x, int y, int flags, void *userdata) {
auto *points = static_cast<std::vector<cv::Point> *>(userdata);
if (event == cv::EVENT_LBUTTONDOWN) {
if (points->size() < 4) {
points->push_back(cv::Point(x, y));
std::cout << "Clicked: (" << x << ", " << y << ")" << std::endl;
}
}
},
&clicks);
while (clicks.size() < 4) {
cv::Mat display = bg_img.clone();
for (size_t i = 0; i < clicks.size(); ++i) {
cv::circle(display, clicks[i], 4, cv::Scalar(0, 0, 255), -1);
if (i > 0)
cv::line(display, clicks[i - 1], clicks[i], cv::Scalar(0, 255, 0),
2);
}
cv::imshow(fullWinName, display);
int key = cv::waitKey(10);
if (key == 27)
return {}; // ESC键取消
}
// 闭合多边形可视化
cv::Mat final_display = bg_img.clone();
for (size_t i = 0; i < clicks.size(); ++i) {
cv::circle(final_display, clicks[i], 4, cv::Scalar(0, 0, 255), -1);
if (i > 0)
cv::line(final_display, clicks[i - 1], clicks[i],
cv::Scalar(0, 255, 0), 2);
}
cv::line(final_display, clicks.back(), clicks.front(),
cv::Scalar(0, 255, 0), 2);
cv::imshow(fullWinName, final_display);
cv::waitKey(500); // Show for a bit
cv::destroyWindow(fullWinName);
// Convert to Point2i
std::vector<cv::Point2i> result;
for (const auto &p : clicks)
result.push_back(p);
return result;
};
static bool showed_debug_warning = false;
if (!showed_debug_warning) {
std::cout << "[BeamRackDeflectionAlgorithm] DEBUG INFO: Interactive "
"Rectified ROI Selection Enabled."
<< std::endl;
showed_debug_warning = true;
}
if (!depth_img.empty()) {
// --- 矫正逻辑 ---
cv::Mat display_img;
cv::normalize(depth_img, display_img, 0, 255, cv::NORM_MINMAX, CV_8U);
cv::cvtColor(display_img, display_img, cv::COLOR_GRAY2BGR);
// 尝试加载内参以进行矫正
cv::Mat H = cv::Mat::eye(3, 3, CV_64F);
bool can_rectify = false;
QDir dir_curr = QDir::current();
QStringList filters;
filters << "intrinsics_*.json";
dir_curr.setNameFilters(filters);
QFileInfoList list = dir_curr.entryInfoList(QDir::Files, QDir::Time);
if (!list.empty()) {
QFile i_file(list.first().absoluteFilePath());
if (i_file.open(QIODevice::ReadOnly)) {
QJsonDocument i_doc = QJsonDocument::fromJson(i_file.readAll());
if (!i_doc.isNull() && i_doc.object().contains("depth")) {
QJsonObject d_obj = i_doc.object()["depth"].toObject();
if (d_obj.contains("intrinsic")) {
QJsonArray i_arr = d_obj["intrinsic"].toArray();
if (i_arr.size() >= 9) {
double fx = i_arr[0].toDouble();
double fy = i_arr[4].toDouble();
double cx = i_arr[2].toDouble();
double cy = i_arr[5].toDouble();
Eigen::Matrix3d K;
K << fx, 0, cx, 0, fy, cy, 0, 0, 1;
Eigen::Matrix3d R = transform.block<3, 3>(0, 0);
// 单应性矩阵 H = K * R * K_inv
// 这将图像变换为仿佛相机已按 R 旋转
Eigen::Matrix3d H_eig = K * R * K.inverse();
for (int r = 0; r < 3; ++r)
for (int c = 0; c < 3; ++c)
H.at<double>(r, c) = H_eig(r, c);
can_rectify = true;
std::cout << "[BeamRackDeflectionAlgorithm] Intrinsics loaded. "
"Rectification enabled."
<< std::endl;
}
}
}
}
}
cv::Mat warp_img;
cv::Mat H_final = H.clone(); // 复制原始 H 以开始
if (can_rectify) {
// 1. 计算变换后的角点以找到新的边界框
std::vector<cv::Point2f> corners = {
cv::Point2f(0, 0), cv::Point2f((float)width, 0),
cv::Point2f((float)width, (float)height),
cv::Point2f(0, (float)height)};
std::vector<cv::Point2f> warped_corners;
cv::perspectiveTransform(corners, warped_corners, H);
cv::Rect bbox = cv::boundingRect(warped_corners);
// 2. 创建平移矩阵以将图像移入视野
cv::Mat T = cv::Mat::eye(3, 3, CV_64F);
T.at<double>(0, 2) = -bbox.x;
T.at<double>(1, 2) = -bbox.y;
// 3. 更新单应性矩阵
H_final = T * H;
// 4. 使用新尺寸和 H 进行变换
cv::warpPerspective(display_img, warp_img, H_final, bbox.size());
std::cout << "[BeamRackDeflectionAlgorithm] Rectified Image Size: "
<< bbox.width << "x" << bbox.height << std::endl;
} else {
std::cout << "[BeamRackDeflectionAlgorithm] Warning: Intrinsics not "
"found. Showing unrectified image."
<< std::endl;
warp_img = display_img.clone();
}
// --- 选择横梁 ROI ---
std::cout << "[BeamRackDeflectionAlgorithm] Please click 4 points for "
"BEAM ROI..."
<< std::endl;
auto beam_poly_visual = selectPolygonROI("Select BEAM", warp_img);
// 如果已矫正，则映射回原始坐标
if (beam_poly_visual.size() == 4) {
if (can_rectify) {
std::vector<cv::Point2f> src, dst;
for (auto p : beam_poly_visual)
src.push_back(cv::Point2f(p.x, p.y));
cv::perspectiveTransform(src, dst,
H_final.inv()); // Use H_final.inv()
actual_beam_roi.clear();
for (auto p : dst)
actual_beam_roi.push_back(
cv::Point2i(std::round(p.x), std::round(p.y)));
} else {
actual_beam_roi = beam_poly_visual;
}
std::cout << "[BeamRackDeflectionAlgorithm] Beam ROI Updated."
<< std::endl;
}
// --- 选择立柱 ROI ---
std::cout << "[BeamRackDeflectionAlgorithm] Please click 4 points for "
"RACK ROI..."
<< std::endl;
auto rack_poly_visual = selectPolygonROI("Select RACK", warp_img);
if (rack_poly_visual.size() == 4) {
if (can_rectify) {
std::vector<cv::Point2f> src, dst;
for (auto p : rack_poly_visual)
src.push_back(cv::Point2f(p.x, p.y));
cv::perspectiveTransform(src, dst,
H_final.inv()); // Use H_final.inv()
actual_rack_roi.clear();
for (auto p : dst)
actual_rack_roi.push_back(
cv::Point2i(std::round(p.x), std::round(p.y)));
} else {
actual_rack_roi = rack_poly_visual;
}
std::cout << "[BeamRackDeflectionAlgorithm] Rack ROI Updated."
<< std::endl;
}
}
#endif
// ============================================
cv::Rect beam_bbox = cv::boundingRect(actual_beam_roi);
cv::Rect rack_bbox = cv::boundingRect(actual_rack_roi);
// 处理横梁 ROI 区域
float max_beam_deflection = 0.0f;
float max_rack_deflection = 0.0f;
auto process_roi = [&](const cv::Rect &bbox,
const std::vector<cv::Point2i> &poly,
std::vector<Eigen::Vector3d> &out_pts) {
int start_x = std::max(0, bbox.x);
int end_x = std::min(width, bbox.x + bbox.width);
int start_y = std::max(0, bbox.y);
int end_y = std::min(height, bbox.y + bbox.height);
for (int y = start_y; y < end_y; ++y) {
for (int x = start_x; x < end_x; ++x) {
if (!isInRoi(poly, x, y))
continue;
int idx = y * width + x;
if (idx >= point_cloud->size())
continue;
const Point3D &pt = (*point_cloud)[idx];
if (pt.z <= 0.0f || std::isnan(pt.x))
continue;
// Transform
Eigen::Vector4d p(pt.x, pt.y, pt.z, 1.0);
Eigen::Vector4d p_trans = transform * p;
out_pts.emplace_back(p_trans.head<3>());
}
}
};
process_roi(beam_bbox, actual_beam_roi, beam_points_3d);
process_roi(rack_bbox, actual_rack_roi, rack_points_3d);
// ===========================================
// FIX: 自动旋转矫正 (PCA)
// 解决 "基准线不水平" 的问题，确保横梁水平，立柱垂直
// 通过将数据旋转到水平/垂直，基准线（连接端点）将变为水平/垂直。
// 从而使变形量（点到线的距离）等于 Y 轴（横梁）或 X 轴（立柱）的偏差。
// ===========================================
auto correctRotation = [](std::vector<Eigen::Vector3d> &points,
bool is_beam) {
if (points.size() < 10)
return;
// 1. Convert to cv::Mat for PCA (Only use X, Y)
int n = points.size();
cv::Mat data(n, 2, CV_64F);
for (int i = 0; i < n; ++i) {
data.at<double>(i, 0) = points[i].x();
data.at<double>(i, 1) = points[i].y();
}
// 2. Perform PCA
cv::PCA pca(data, cv::Mat(), cv::PCA::DATA_AS_ROW);
// 3. Get primary eigenvector (direction of max variance)
// Eigenvectors are stored in rows. Row 0 is the primary vector.
cv::Point2d eigen_vec(pca.eigenvectors.at<double>(0, 0),
pca.eigenvectors.at<double>(0, 1));
// 4. Calculate angle relative to desired axis
// Beam (is_beam=true): Should align with X-axis (1, 0)
// Rack (is_beam=false): Should align with Y-axis (0, 1)
double angle = std::atan2(eigen_vec.y, eigen_vec.x); // Angle of the data
double rotation_angle = 0.0;
if (is_beam) {
// Target: Horizontal (0 degrees)
rotation_angle = -angle;
} else {
// Target: Vertical (90 degrees or PI/2)
rotation_angle = (CV_PI / 2.0) - angle;
}
// Normalize to -PI ~ PI
while (rotation_angle > CV_PI)
rotation_angle -= 2 * CV_PI;
while (rotation_angle < -CV_PI)
rotation_angle += 2 * CV_PI;
// Safety check: Don't rotate if angle is suspicious huge (> 45 deg)
// unless confident For now, we trust PCA for standard slight tilts (< 30
// deg).
std::cout << "[BeamRackDeflectionAlgorithm] Correcting "
<< (is_beam ? "Beam" : "Rack")
<< " Rotation: " << rotation_angle * 180.0 / CV_PI << " deg."
<< std::endl;
// 5. Apply Rotation
double c = std::cos(rotation_angle);
double s = std::sin(rotation_angle);
// Center of rotation: PCA mean
double cx = pca.mean.at<double>(0);
double cy = pca.mean.at<double>(1);
for (int i = 0; i < n; ++i) {
double x = points[i].x() - cx;
double y = points[i].y() - cy;
double x_new = x * c - y * s;
double y_new = x * s + y * c;
points[i].x() = x_new + cx;
points[i].y() = y_new + cy;
// Z unchanged
}
};
// Apply corrections
correctRotation(beam_points_3d, true);
correctRotation(rack_points_3d, false);
// ===========================================
// 6.9 计算变形量
// 分箱（切片）方法辅助函数
auto calculate_deflection_binned = [&](std::vector<Eigen::Vector3d> &points,
bool is_beam_y_check,
const std::string &label) -> float {
if (points.empty())
return 0.0f;
// 1. 沿主轴排序点
std::sort(points.begin(), points.end(),
[is_beam_y_check](const Eigen::Vector3d &a,
const Eigen::Vector3d &b) {
return is_beam_y_check ? (a.x() < b.x()) : (a.y() < b.y());
});
// 2. 分箱
int num_bins = 50;
if (points.size() < 100)
num_bins = 10; // Reduce bins for small sets
double min_u = is_beam_y_check ? points.front().x() : points.front().y();
double max_u = is_beam_y_check ? points.back().x() : points.back().y();
// 可视化辅助
#ifdef DEBUG_ROI_SELECTION
int viz_w = 800;
int viz_h = 400;
cv::Mat viz_img = cv::Mat::zeros(viz_h, viz_w, CV_8UC3);
double disp_min_u = min_u;
double disp_max_u = max_u;
double min_v = 1e9, max_v = -1e9;
auto map_u = [&](double u) -> int {
return (int)((u - disp_min_u) / (disp_max_u - disp_min_u) *
(viz_w - 40) +
20);
};
// Will define map_v later after range finding
#endif
std::vector<Eigen::Vector3d> raw_centroids;
std::vector<int> counts;
double range_min = min_u;
double range_max = max_u;
double bin_step = (range_max - range_min) / num_bins;
if (bin_step < 1.0)
return 0.0f;
auto it = points.begin();
double avg_pts_per_bin = 0;
int filled_bins = 0;
for (int i = 0; i < num_bins; ++i) {
double bin_start = range_min + i * bin_step;
double bin_end = bin_start + bin_step;
std::vector<Eigen::Vector3d> bin_pts;
while (it != points.end()) {
double val = is_beam_y_check ? it->x() : it->y();
// double val_v = is_beam_y_check ? it->y() : it->x();
if (val > bin_end)
break;
bin_pts.push_back(*it);
++it;
}
if (!bin_pts.empty()) {
// Robust Centroid (Trimmed Mean)
std::sort(bin_pts.begin(), bin_pts.end(),
[is_beam_y_check](const Eigen::Vector3d &a,
const Eigen::Vector3d &b) {
double val_a = is_beam_y_check ? a.y() : a.x(); // V axis
double val_b = is_beam_y_check ? b.y() : b.x();
return val_a < val_b;
});
size_t n = bin_pts.size();
size_t start = (size_t)(n * 0.25);
size_t end = (size_t)(n * 0.75);
if (end <= start) {
start = 0;
end = n;
}
Eigen::Vector3d sum(0, 0, 0);
int count = 0;
for (size_t k = start; k < end; ++k) {
sum += bin_pts[k];
count++;
}
if (count > 0) {
raw_centroids.push_back(sum / count);
counts.push_back(bin_pts.size());
avg_pts_per_bin += bin_pts.size();
filled_bins++;
}
}
}
if (filled_bins < 2)
return 0.0f;
avg_pts_per_bin /= filled_bins;
// --- 2.1 Bin Filtering (Remove Noise) ---
// Filter out bins with significantly low density (e.g. < 20% of average)
std::vector<Eigen::Vector3d> bin_centroids;
for (size_t i = 0; i < raw_centroids.size(); ++i) {
if (counts[i] > avg_pts_per_bin * 0.2) {
bin_centroids.push_back(raw_centroids[i]);
// Track V range for filtered points
double v =
is_beam_y_check ? raw_centroids[i].y() : raw_centroids[i].x();
#ifdef DEBUG_ROI_SELECTION
if (v < min_v)
min_v = v;
if (v > max_v)
max_v = v;
#endif
}
}
if (bin_centroids.size() < 2) {
std::cerr << "[BeamRack] Filtered bins too few." << std::endl;
return 0.0f;
}
#ifdef DEBUG_ROI_SELECTION
// Adjust V range
double v_range = max_v - min_v;
if (v_range < 1.0)
v_range = 10.0;
min_v -= v_range * 0.5; // More margin
max_v += v_range * 0.5;
auto map_v = [&](double v) -> int {
return (int)((v - min_v) / (max_v - min_v) * (viz_h - 40) + 20);
};
// Draw Points
for (size_t i = 0; i < bin_centroids.size(); ++i) {
double u =
is_beam_y_check ? bin_centroids[i].x() : bin_centroids[i].y();
double v =
is_beam_y_check ? bin_centroids[i].y() : bin_centroids[i].x();
cv::circle(viz_img, cv::Point(map_u(u), map_v(v)), 3,
cv::Scalar(255, 255, 0), -1); // Cyan Centroids
}
#endif
// --- 3. Robust Baseline Fitting (Support Line) ---
// Instead of simple endpoints, fit a line to "valid support regions"
// Support Regions: First 15% and Last 15% of VALID centroids.
std::vector<Eigen::Vector3d> support_points;
int support_count = (int)(bin_centroids.size() * 0.15);
if (support_count < 2)
support_count = 2; // At least 2 points at each end
if (support_count * 2 > bin_centroids.size())
support_count = bin_centroids.size() / 2;
for (int i = 0; i < support_count; ++i)
support_points.push_back(bin_centroids[i]);
for (int i = 0; i < support_count; ++i)
support_points.push_back(bin_centroids[bin_centroids.size() - 1 - i]);
// Fit Line to Support Points (Least Squares)
// Model: v = m * u + c (since rotated, m should be close to 0)
double sum_u = 0, sum_v = 0, sum_uv = 0, sum_uu = 0;
int N = support_points.size();
for (const auto &p : support_points) {
double u = is_beam_y_check ? p.x() : p.y();
double v = is_beam_y_check ? p.y() : p.x();
sum_u += u;
sum_v += v;
sum_uv += u * v;
sum_uu += u * u;
}
double slope = 0, intercept = 0;
double denom = N * sum_uu - sum_u * sum_u;
if (std::abs(denom) > 1e-6) {
slope = (N * sum_uv - sum_u * sum_v) / denom;
intercept = (sum_v - slope * sum_u) / N;
} else {
// Vertical line? Should not happen after rotation. Fallback average.
slope = 0;
intercept = sum_v / N;
}
std::cout << "[BeamRack] Baseline Fit: slope=" << slope
<< ", intercept=" << intercept << " (Support Pts: " << N << ")"
<< std::endl;
// --- 4. Calculate Max Deflection ---
double max_def = 0.0;
Eigen::Vector3d max_pt;
double max_theoretical_v = 0;
for (const auto &p : bin_centroids) {
double u = is_beam_y_check ? p.x() : p.y();
double v = is_beam_y_check ? p.y() : p.x();
double theoretical_v = slope * u + intercept;
double def = 0;
if (is_beam_y_check) {
// Beam: Y+ is down. Deflection = ActualY - TheoreticalY
// We rotated data, so Y+ might still be relevant if rotation was just
// alignment. Assuming standard coords: Sag (Down) is Y decreasing? Or
// increasing? In Camera Coords: Y is DOWN. So Sag is INCREASING Y.
// Deflection = v - theoretical_v. Positive = Down.
def = v - theoretical_v;
} else {
// Rack: Deflection is absolute distance
def = std::abs(v - theoretical_v);
}
if (def > max_def) {
max_def = def;
max_pt = p;
max_theoretical_v = theoretical_v;
}
}
// Robust Average of Max Region (Top 3)
// ... (Simplified: use raw max for now, or implement top-k avg if
// preferred) Sticking to Max for simplicity as requested, but previous
// code used Average. Let's reimplement Top 3 Average roughly around max
// peak? Actually, just returning max_def is cleaner for "maximum sag".
#ifdef DEBUG_ROI_SELECTION
// Draw Baseline
double u_start = disp_min_u;
double v_start = slope * u_start + intercept;
double u_end = disp_max_u;
double v_end = slope * u_end + intercept;
cv::line(viz_img, cv::Point(map_u(u_start), map_v(v_start)),
cv::Point(map_u(u_end), map_v(v_end)), cv::Scalar(0, 255, 0), 2);
// Draw Max Deflection
if (max_def != 0.0) {
double u_d = is_beam_y_check ? max_pt.x() : max_pt.y();
double v_d = is_beam_y_check ? max_pt.y() : max_pt.x();
cv::line(viz_img, cv::Point(map_u(u_d), map_v(v_d)),
cv::Point(map_u(u_d), map_v(max_theoretical_v)),
cv::Scalar(0, 0, 255), 2);
cv::putText(viz_img, "Max: " + std::to_string(max_def),
cv::Point(viz_w / 2, 50), cv::FONT_HERSHEY_SIMPLEX, 0.8,
cv::Scalar(0, 0, 255), 2);
}
cv::imshow("Robust Deflection: " + label, viz_img);
cv::waitKey(100);
#endif
return (float)max_def;
};
// 6.10 Run Calculation logic
// --- 横梁变形（Y+ 方向）---
max_beam_deflection =
calculate_deflection_binned(beam_points_3d, true, "Beam");
// --- 立柱变形（X 方向）---
max_rack_deflection =
calculate_deflection_binned(rack_points_3d, false, "Rack");
// 存储结果
result.beam_def_mm_value = max_beam_deflection;
result.rack_def_mm_value = max_rack_deflection;
std::cout << "[BeamRackDeflectionAlgorithm] Results: Beam="
<< max_beam_deflection << "mm, Rack=" << max_rack_deflection
<< "mm"
<< " (Beam Points: " << beam_points_3d.size()
<< ", Rack Points: " << rack_points_3d.size() << ")" << std::endl;
// 使用默认或提供的阈值
// std::vector<float> actual_beam_thresh = beam_thresholds.empty() ?
// DEFAULT_BEAM_THRESHOLDS : beam_thresholds; // OLD std::vector<float>
// actual_rack_thresh = rack_thresholds.empty() ? DEFAULT_RACK_THRESHOLDS :
// rack_thresholds; // OLD
// NEW: Use ConfigManager
std::vector<float> actual_beam_thresh =
ConfigManager::getInstance().getBeamThresholds();
std::vector<float> actual_rack_thresh =
ConfigManager::getInstance().getRackThresholds();
// Fallback if empty (should not happen with getBeamThresholds defaults)
if (actual_beam_thresh.size() < 4)
actual_beam_thresh = DEFAULT_BEAM_THRESHOLDS;
if (actual_rack_thresh.size() < 4)
actual_rack_thresh = DEFAULT_RACK_THRESHOLDS;
// 制作 json 阈值字符串的辅助函数
auto make_json_thresh = [](const std::vector<float> &t) {
return "{\"A\":" + std::to_string(t[0]) +
",\"B\":" + std::to_string(t[1]) +
",\"C\":" + std::to_string(t[2]) +
",\"D\":" + std::to_string(t[3]) + "}";
};
if (actual_beam_thresh.size() >= 4) {
result.beam_def_mm_threshold = make_json_thresh(actual_beam_thresh);
}
if (actual_rack_thresh.size() >= 4) {
result.rack_def_mm_threshold = make_json_thresh(actual_rack_thresh);
}
// 检查状态
// 横梁：正值为向下（Y+）。检查 C 和 D。
// 负值为向上（Y-）。检查 A 和 B？
// 用户要求：“横梁由于货物仅向下弯曲...（Y 正方向）”
// 所以我们主要使用 max_beam_deflection检查 C（警告）和 D（报警）（这是
// >0）。 遗留阈值具有负值，可能用于范围检查。 我们将假设标准 [A(neg),
// B(neg), C(pos), D(pos)] 格式。
bool beam_warn = (max_beam_deflection >= actual_beam_thresh[2]); // > C
bool beam_alrm = (max_beam_deflection >= actual_beam_thresh[3]); // > D
// Rack: Bends Left or Right. We took Abs() -> always positive.
// So we check against C and D.
bool rack_warn = (max_rack_deflection >= actual_rack_thresh[2]);
bool rack_alrm = (max_rack_deflection >= actual_rack_thresh[3]);
auto make_json_status = [](bool w, bool a) {
return "{\"warning\":" + std::string(w ? "true" : "false") +
",\"alarm\":" + std::string(a ? "true" : "false") + "}";
};
result.beam_def_mm_warning_alarm = make_json_status(beam_warn, beam_alrm);
result.rack_def_mm_warning_alarm = make_json_status(rack_warn, rack_alrm);
// 标记为成功
result.success = true;
#ifdef DEBUG_ROI_SELECTION
std::cout << "[BeamRackDeflectionAlgorithm] Press ANY KEY to close graphs "
"and continue..."
<< std::endl;
cv::waitKey(0);
cv::destroyAllWindows();
#endif
return result.success;
} else {
// --- 模拟数据逻辑 ---
std::cout << "[BeamRackDeflectionAlgorithm] Using FAKE DATA implementation "
"(Switch OFF)."
<< std::endl;
result.beam_def_mm_value = 5.5f; // 模拟横梁弯曲
result.rack_def_mm_value = 2.2f; // 模拟立柱弯曲
result.success = true;
// 设置模拟阈值
// std::vector<float> actual_beam_thresh = beam_thresholds.empty() ?
// DEFAULT_BEAM_THRESHOLDS : beam_thresholds; std::vector<float>
// actual_rack_thresh = rack_thresholds.empty() ? DEFAULT_RACK_THRESHOLDS :
// rack_thresholds;
std::vector<float> actual_beam_thresh =
ConfigManager::getInstance().getBeamThresholds();
std::vector<float> actual_rack_thresh =
ConfigManager::getInstance().getRackThresholds();
if (actual_beam_thresh.size() < 4)
actual_beam_thresh = DEFAULT_BEAM_THRESHOLDS;
if (actual_rack_thresh.size() < 4)
actual_rack_thresh = DEFAULT_RACK_THRESHOLDS;
auto make_json_thresh = [](const std::vector<float> &t) {
return "{\"A\":" + std::to_string(t[0]) +
",\"B\":" + std::to_string(t[1]) +
",\"C\":" + std::to_string(t[2]) +
",\"D\":" + std::to_string(t[3]) + "}";
};
if (actual_beam_thresh.size() >= 4)
result.beam_def_mm_threshold = make_json_thresh(actual_beam_thresh);
if (actual_rack_thresh.size() >= 4)
result.rack_def_mm_threshold = make_json_thresh(actual_rack_thresh);
// 设置模拟警告/报警状态
result.beam_def_mm_warning_alarm = "{\"warning\":false,\"alarm\":false}";
result.rack_def_mm_warning_alarm = "{\"warning\":false,\"alarm\":false}";
return result.success;
}
}

134
image_capture/src/algorithm/detections/beam_rack_deflection/beam_rack_deflection_detection.h Normal file
View File

@@ -0,0 +1,134 @@
#pragma once
#include "../../../common_types.h"
#include <Eigen/Dense>
#include <opencv2/opencv.hpp>
#include <string>
/**
* @brief 四边形ROI结构（四个点定义）
*/
struct QuadrilateralROI {
cv::Point2i points[4]; // 四个点：左上、右上、右下、左下（按顺序）
QuadrilateralROI() {
for (int i = 0; i < 4; ++i) {
points[i] = cv::Point2i(0, 0);
}
}
QuadrilateralROI(const cv::Point2i &pt0, const cv::Point2i &pt1,
const cv::Point2i &pt2, const cv::Point2i &pt3) {
points[0] = pt0;
points[1] = pt1;
points[2] = pt2;
points[3] = pt3;
}
bool isValid() const {
// 检查是否有有效的点（不全为0）
for (int i = 0; i < 4; ++i) {
if (points[i].x > 0 || points[i].y > 0) {
return true;
}
}
return false;
}
cv::Rect getBoundingRect() const {
if (!isValid()) {
return cv::Rect();
}
int min_x = points[0].x, max_x = points[0].x;
int min_y = points[0].y, max_y = points[0].y;
for (int i = 1; i < 4; ++i) {
min_x = std::min(min_x, points[i].x);
max_x = std::max(max_x, points[i].x);
min_y = std::min(min_y, points[i].y);
max_y = std::max(max_y, points[i].y);
}
return cv::Rect(min_x, min_y, max_x - min_x, max_y - min_y);
}
};
/**
* @brief 横梁变形检测算法结果
*/
struct BeamRackDeflectionResult {
// 变形量
float beam_def_mm_value; // 横梁弯曲量（mm）
float rack_def_mm_value; // 立柱弯曲量（mm）
// 阈值（JSON字符串）
std::string beam_def_mm_threshold;
std::string rack_def_mm_threshold;
// 警告和报警信号（JSON字符串）
std::string beam_def_mm_warning_alarm;
std::string rack_def_mm_warning_alarm;
bool success; // 算法是否执行成功
BeamRackDeflectionResult()
: beam_def_mm_value(0.0f), rack_def_mm_value(0.0f), success(false) {}
};
/**
* @brief 横梁变形检测算法
*
* 检测横梁和货架立柱的变形
*/
class BeamRackDeflectionAlgorithm {
public:
// 默认ROI点定义（四个点：左上、右上、右下、左下）
// 横梁ROI默认点
static const std::vector<cv::Point2i> DEFAULT_BEAM_ROI_POINTS;
// 立柱ROI默认点
static const std::vector<cv::Point2i> DEFAULT_RACK_ROI_POINTS;
// 默认阈值定义（四个值：A负方向报警, B负方向警告, C正方向警告, D正方向报警）
// 横梁阈值默认值
static const std::vector<float> DEFAULT_BEAM_THRESHOLDS;
// 立柱阈值默认值
static const std::vector<float> DEFAULT_RACK_THRESHOLDS;
// 2180mm 横梁 ROI (Placeholder)
static const std::vector<cv::Point2i> BEAM_ROI_2180;
// 1380mm 横梁 ROI (Placeholder)
static const std::vector<cv::Point2i> BEAM_ROI_1380;
// ... (keep class definition)
/**
* 执行横梁变形检测（使用深度图方案）
* @param depth_img 深度图像
* @param color_img 彩色图像
* @param side 货架侧（"left"或"right"）
* @param result [输出] 检测结果
* @param point_cloud [可选] 点云数据
* @param beam_roi_points
* 横梁ROI的四个点（左上、右上、右下、左下），为空时使用默认值
* @param rack_roi_points
* 立柱ROI的四个点（左上、右上、右下、左下），为空时使用默认值
* @param beam_thresholds 横梁阈值四个值[A,B,C,D]，为空时使用默认值
* @param rack_thresholds 立柱阈值四个值[A,B,C,D]，为空时使用默认值
* @return 是否检测成功
*/
static bool
detect(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, BeamRackDeflectionResult &result,
const std::vector<Point3D> *point_cloud = nullptr,
const std::vector<cv::Point2i> &beam_roi_points =
std::vector<cv::Point2i>(),
const std::vector<cv::Point2i> &rack_roi_points =
std::vector<cv::Point2i>(),
const std::vector<float> &beam_thresholds = std::vector<float>(),
const std::vector<float> &rack_thresholds = std::vector<float>());
private:
static bool loadCalibration(Eigen::Matrix4d &transform);
};

1396
image_capture/src/algorithm/detections/pallet_offset/pallet_offset_detection.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

104
image_capture/src/algorithm/detections/pallet_offset/pallet_offset_detection.h Normal file
View File

@@ -0,0 +1,104 @@
#pragma once
#include <string>
#include <vector>
#include <opencv2/core.hpp>
#include <Eigen/Dense>
#include "../../../common_types.h"
namespace cv { class Mat; }
/**
* @brief 托盘位置偏移检测算法结果
*/
struct PalletOffsetResult {
// 位置偏移量 (相对于参考得出的世界坐标系下的差异)
float offset_lat_mm_value; // 左右偏移量（mm）- X轴 World
float offset_lon_mm_value; // 前后偏移量（mm）- Z轴 World
float rotation_angle_value; // 旋转角度（度）- 绕 Y轴 World
// 插孔变形
float hole_def_mm_left_value; // 左侧插孔变形（mm）
float hole_def_mm_right_value; // 右侧插孔变形（mm）
// 绝对坐标 (用于生成参考模板)
float abs_x;
float abs_y;
float abs_z;
// 个体插孔坐标 (可选，用于调试或Reference生成)
Point3D left_hole_pos;
Point3D right_hole_pos;
// 阈值（JSON字符串）
std::string offset_lat_mm_threshold;
std::string offset_lon_mm_threshold;
std::string rotation_angle_threshold;
std::string hole_def_mm_left_threshold;
std::string hole_def_mm_right_threshold;
// 警告和报警信号（JSON字符串）
std::string offset_lat_mm_warning_alarm;
std::string offset_lon_mm_warning_alarm;
std::string rotation_angle_warning_alarm;
std::string hole_def_mm_left_warning_alarm;
std::string hole_def_mm_right_warning_alarm;
bool success; // 算法是否执行成功
PalletOffsetResult()
: offset_lat_mm_value(0.0f)
, offset_lon_mm_value(0.0f)
, rotation_angle_value(0.0f)
, hole_def_mm_left_value(0.0f)
, hole_def_mm_right_value(0.0f)
, abs_x(0.0f), abs_y(0.0f), abs_z(0.0f)
, success(false)
{}
};
/**
* @brief 托盘位置偏移检测算法
*
* 检测托盘位置偏移和插孔变形
* 核心逻辑：
* 1. 纯深度图输入，不依赖点云。
* 2. 交互式 ROI 选择（当未提供 ROI 时）。
* 3. 2D 特征检测 + 稀疏 3D 转换（利用内参和标定矩阵）。
* 4. 世界坐标系下的偏移与变形计算。
*/
class PalletOffsetAlgorithm {
public:
/**
* @brief 执行托盘位置偏移检测
*
* @param depth_img 深度图像 (CV_16U or CV_32F)
* @param color_img 彩色图像 (仅用于显示，可选)
* @param side 货架侧（"left"或"right"）
* @param result [输出] 检测结果
* @param point_cloud [可选] 点云数据 (若为空，则使用 depth + intrinsics 计算)
* @param roi_points [可选] ROI区域，若为空则触发交互式选择
* @param intrinsics [可选] 相机内参，用于 2D->3D 转换。若为0则尝试自动获取。
* @return 是否检测成功
*/
static bool detect(const cv::Mat& depth_img,
const cv::Mat& color_img,
const std::string& side,
PalletOffsetResult& result,
const std::vector<Point3D>* point_cloud = nullptr,
const std::vector<cv::Point2i>& roi_points = {},
const CameraIntrinsics& intrinsics = CameraIntrinsics(),
const cv::Mat* calib_mat_override = nullptr);
private:
/**
* @brief 从 JSON 文件加载标定矩阵
*/
static bool loadCalibration(Eigen::Matrix4d& transform);
/**
* @brief 交互式 ROI 选择
*/
static std::vector<cv::Point2i> selectPolygonROI(const cv::Mat& visual_img);
};

261
image_capture/src/algorithm/detections/slot_occupancy/slot_occupancy_detection.cpp Normal file
View File

@@ -0,0 +1,261 @@
#include "slot_occupancy_detection.h"
#include <iostream>
#include <mutex>
#include <opencv2/opencv.hpp>
#include <vector>
//====================
// 步骤1：配置参数
//====================
namespace Config {
// 基准图文件相对路径列表 (按顺序尝试)
const std::vector<std::string> TEMPLATE_PATHS = {
"src\\images_template\\temp.bmp",
"..\\src\\images_template\\temp.bmp",
"..\\..\\src\\images_template\\temp.bmp",
"..\\..\\..\\src\\images_template\\temp.bmp",
"..\\..\\..\\..\\src\\images_template\\temp.bmp",
"d:\\Git\\stereo_warehouse_inspection\\image_capture\\src\\images_"
"template\\temp.bmp"};
// 差异阈值：当前像素与背景像素相差多少算“有变化” (0-255)
// 建议：如果环境光稳定，设为 20-30；如果光照波动大，设为 40-50
const int DIFF_THRESHOLD = 28;
// 面积阈值：差异像素总数超过多少算“有货”
// 建议：根据 ROI 大小调整，通常设为 ROI 面积的 5% - 10%
const int AREA_THRESHOLD = 1000000;
// 高斯模糊核大小 (必须是奇数)
const int BLUR_SIZE = 7;
// 目标工作分辨率 (相机分辨率)
const cv::Size TARGET_SIZE(4024, 3036);
// ROI (感兴趣区域) - 默认值
const cv::Rect ROI_DEFAULT(1400, 600, 1200, 1800);
} // namespace Config
//====================
// 步骤2：算法上下文（用于管理静态资源）
//====================
class SlotAlgoContext {
public:
SlotAlgoContext() : initialized_(false) {}
// 初始化：加载并预处理基准图
// 返回值：是否初始化成功
bool ensureInitialized() {
std::lock_guard<std::mutex> lock(mutex_);
if (initialized_)
return true;
cv::Mat raw_ref;
// 1. 尝试加载基准图
for (const auto &path : Config::TEMPLATE_PATHS) {
raw_ref = cv::imread(path, cv::IMREAD_GRAYSCALE);
if (!raw_ref.empty()) {
std::cout << "[SlotAlgo] Loaded template from: " << path << std::endl;
break;
}
}
if (raw_ref.empty()) {
std::cerr << "[SlotAlgo] CRITICAL: Failed to load template image."
<< std::endl;
return false;
}
// 2. 尺寸对齐 (Resize)
// 只有当尺寸不匹配时才执行 Resize，确保 ref_img_processed_ 始终是
// TARGET_SIZE
if (raw_ref.size() != Config::TARGET_SIZE) {
std::cout << "[SlotAlgo] Resizing template from " << raw_ref.cols << "x"
<< raw_ref.rows << " to " << Config::TARGET_SIZE.width << "x"
<< Config::TARGET_SIZE.height << std::endl;
cv::resize(raw_ref, ref_img_processed_, Config::TARGET_SIZE);
} else {
ref_img_processed_ = raw_ref;
}
// 3. 预处理：高斯模糊
// 提前对整张基准图进行模糊，避免每帧对 ROI 进行模糊，减少计算量
// (注：如果内存紧张，可以只存原图，但为了速度建议存模糊后的图)
cv::GaussianBlur(ref_img_processed_, ref_img_processed_,
cv::Size(Config::BLUR_SIZE, Config::BLUR_SIZE), 0);
// 4. 初始化形态学核
morph_kernel_ = cv::getStructuringElement(cv::MORPH_RECT, cv::Size(5, 5));
std::cout << "[SlotAlgo] Initialization complete. Reference size: "
<< ref_img_processed_.cols << "x" << ref_img_processed_.rows
<< std::endl;
initialized_ = true;
return true;
}
// 获取处理后的基准图 (只读引用)
const cv::Mat &getRefImage() const { return ref_img_processed_; }
// 获取形态学核
const cv::Mat &getMorphKernel() const { return morph_kernel_; }
bool isInitialized() const { return initialized_; }
private:
std::mutex mutex_;
bool initialized_;
cv::Mat ref_img_processed_; // 存储 Resize 并 Blur 后的基准图
cv::Mat morph_kernel_;
};
// 全局静态上下文实例
static SlotAlgoContext g_algo_context;
//====================
// 步骤3：辅助函数
//====================
static cv::Rect getSafeROI(const cv::Rect &request_roi, int img_width,
int img_height) {
cv::Rect roi = request_roi;
roi.x = std::max(0, roi.x);
roi.y = std::max(0, roi.y);
roi.width = std::min(roi.width, img_width - roi.x);
roi.height = std::min(roi.height, img_height - roi.y);
return roi;
}
//====================
// 步骤4：核心算法实现
//====================
bool SlotOccupancyAlgorithm::detect(const cv::Mat &depth_img,
const cv::Mat &color_img,
const std::string &side,
SlotOccupancyResult &result) {
// 算法启用开关
const bool USE_ALGORITHM = false;
if (USE_ALGORITHM) {
// --- 真实算法逻辑 ---
// 初始化结果
result.success = false;
result.slot_occupied = false;
// 1. 确保算法所需的资源已加载 (懒加载模式)
if (!g_algo_context.ensureInitialized()) {
std::cerr << "[SlotAlgo] Algorithm not initialized, skipping detection."
<< std::endl;
return false;
}
// 2. 输入检查
if (color_img.empty()) {
std::cerr << "[SlotAlgo] Input image is empty." << std::endl;
return false;
}
try {
// 3. 准备当前帧灰度图 (高效转换)
cv::Mat curr_gray;
if (color_img.channels() == 3) {
cv::cvtColor(color_img, curr_gray, cv::COLOR_BGR2GRAY);
} else {
// 如果已经是灰度图，直接引用，避免拷贝
curr_gray = color_img;
}
// 验证输入尺寸 (假设输入应该匹配目标分辨率)
if (curr_gray.size() != Config::TARGET_SIZE) {
// 如果输入尺寸不对，这里选择报错或者 Resize
// 鉴于这是一个工业场景，分辨率突变通常是异常，这里建议打印警告如果必须处理则
// Resize 但为了效率，我们尽量避免每帧 Resize。
// 如果确实不一样，这里做一个临时 Resize 以保证程序不崩，但会影响性能
// std::cout << "[SlotAlgo] Warning: Input size mismatch. Resizing..."
// << std::endl; 暂时不处理 resize，依靠 getSafeROI 防止崩坏，或者在 ROI
// 截取时会出错
}
// 4. 确定 ROI
// 实际应用中根据 side 选择 ROI; 目前使用默认
cv::Rect roi =
getSafeROI(Config::ROI_DEFAULT, curr_gray.cols, curr_gray.rows);
// 5. 截取 ROI
// 直接从 input 和 cached reference 中截取，无需 clone
cv::Mat img_roi = curr_gray(roi);
const cv::Mat &ref_full = g_algo_context.getRefImage();
// 确保 ref_full 够大覆盖 ROI (理论上 init 中已经 resize 到了 TARGET_SIZE)
// 双重保险
cv::Rect ref_roi_rect = getSafeROI(roi, ref_full.cols, ref_full.rows);
if (ref_roi_rect != roi) {
std::cerr << "[SlotAlgo] Error: Reference image size mismatch with ROI."
<< std::endl;
return false;
}
cv::Mat ref_roi = ref_full(ref_roi_rect);
// 6. 图像处理 pipeline
// 只对当前帧 ROI 做高斯模糊 (基准图已经预处理过了)
cv::Mat img_roi_blurred;
cv::GaussianBlur(img_roi, img_roi_blurred,
cv::Size(Config::BLUR_SIZE, Config::BLUR_SIZE), 0);
// 绝对差分
cv::Mat diff;
cv::absdiff(img_roi_blurred, ref_roi, diff);
// 二值化
cv::Mat mask;
cv::threshold(diff, mask, Config::DIFF_THRESHOLD, 255, cv::THRESH_BINARY);
// 形态学滤波 (去除噪点)
cv::morphologyEx(mask, mask, cv::MORPH_OPEN,
g_algo_context.getMorphKernel());
// 7. 统计判定
int non_zero_pixels = cv::countNonZero(mask);
// 可选：仅在状态变化时打印，避免刷屏
// std::cout << "[SlotAlgo] Diff pixels: " << non_zero_pixels <<
// std::endl;
if (non_zero_pixels > Config::AREA_THRESHOLD) {
result.slot_occupied = true;
} else {
result.slot_occupied = false;
}
result.success = true;
// std::cout << "[SlotAlgo] Result: " << (result.slot_occupied ?
// "Occupied" : "Empty") << std::endl;
} catch (const cv::Exception &e) {
std::cerr << "[SlotAlgo] OpenCV Exception: " << e.what() << std::endl;
result.success = false;
return false;
} catch (...) {
std::cerr << "[SlotAlgo] Unknown Exception during detection."
<< std::endl;
result.success = false;
return false;
}
return result.success;
} else {
// --- 模拟数据逻辑 ---
std::cout << "[SlotOccupancyAlgorithm] Using FAKE DATA implementation "
"(Switch OFF)."
<< std::endl;
result.slot_occupied = false; // 模拟无货
result.success = true;
return result.success;
}
}

37
image_capture/src/algorithm/detections/slot_occupancy/slot_occupancy_detection.h Normal file
View File

@@ -0,0 +1,37 @@
#pragma once
#include <string>
namespace cv { class Mat; }
/**
* @brief 货位有无检测算法结果
*/
struct SlotOccupancyResult {
bool slot_occupied; // 货位是否有托盘/货物
bool success; // 算法是否执行成功
SlotOccupancyResult() : slot_occupied(false), success(false) {}
};
/**
* @brief 货位有无检测算法
*
* 分析深度图或彩色图，判断货位是否有托盘/货物
*/
class SlotOccupancyAlgorithm {
public:
/**
* 执行货位有无检测
* @param depth_img 深度图像
* @param color_img 彩色图像
* @param side 货架侧（"left"或"right"）
* @param result [输出] 检测结果
* @return 是否检测成功
*/
static bool detect(const cv::Mat& depth_img,
const cv::Mat& color_img,
const std::string& side,
SlotOccupancyResult& result);
};

129
image_capture/src/algorithm/detections/visual_inventory/visual_inventory_detection.cpp Normal file
View File

@@ -0,0 +1,129 @@
#include "visual_inventory_detection.h"
#include "HalconCpp.h"
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace HalconCpp;
// Helper to convert cv::Mat to Halcon HImage
HImage MatToHImage(const cv::Mat &image) {
HImage hImage;
if (image.empty())
return hImage;
cv::Mat gray;
if (image.channels() == 3) {
cv::cvtColor(image, gray, cv::COLOR_BGR2GRAY);
} else {
gray = image.clone();
}
// Fix: Create a copy of the data to ensure it persists beyond function scope
// GenImage1 with "byte" type expects the data to remain valid
void *data_copy = new unsigned char[gray.total()];
memcpy(data_copy, gray.data, gray.total());
try {
hImage.GenImage1("byte", gray.cols, gray.rows, data_copy);
} catch (...) {
delete[] static_cast<unsigned char *>(data_copy);
throw;
}
// Note: The data_copy will be managed by Halcon's HImage
// We don't delete it here as Halcon takes ownership
return hImage;
}
bool VisualInventoryAlgorithm::detect(const cv::Mat &depth_img,
const cv::Mat &color_img,
const std::string &side,
VisualInventoryResult &result) {
result.success = false;
try {
if (color_img.empty()) {
std::cerr << "[VisualInventoryAlgorithm] Error: Empty image input."
<< std::endl;
return false;
}
// 1. Convert to HImage
HImage hImage = MatToHImage(color_img);
// 2. Setup Halcon QR Code Model
HDataCode2D dataCode2d;
dataCode2d.CreateDataCode2dModel("QR Code", HTuple(), HTuple());
dataCode2d.SetDataCode2dParam("default_parameters", "enhanced_recognition");
HTuple resultHandles, decodedDataStrings;
// 3. Detect
// stop_after_result_num: 100 ensures we get up to 100 codes
HXLDCont symbolXLDs =
dataCode2d.FindDataCode2d(hImage, "stop_after_result_num", 100,
&resultHandles, &decodedDataStrings);
// 4. Transform Results to JSON
// Format: {"A01":["BOX111","BOX112"], "A02":["BOX210"]}
// Since we don't have position information, group all codes under a generic
// key
std::string json_barcodes = "\"" + side + "\":[";
Hlong count = decodedDataStrings.Length();
for (Hlong i = 0; i < count; i++) {
if (i > 0)
json_barcodes += ",";
// Access string from HTuple using S() which returns const char*
HTuple s = decodedDataStrings[i];
std::string code = std::string(s.S());
// Save raw code for deduplication
result.codes.push_back(code);
// Escape special characters in JSON strings
// Replace backslashes first, then quotes
size_t pos = 0;
while ((pos = code.find('\\', pos)) != std::string::npos) {
code.replace(pos, 1, "\\\\");
pos += 2;
}
pos = 0;
while ((pos = code.find('"', pos)) != std::string::npos) {
code.replace(pos, 1, "\\\"");
pos += 2;
}
json_barcodes += "\"" + code + "\"";
}
json_barcodes += "]";
result.result_barcodes = "{" + json_barcodes + "}";
result.success = true;
std::cout << "[VisualInventoryAlgorithm] Side: " << side
<< " | Detected: " << count << " codes." << std::endl;
} catch (HException &except) {
std::cerr << "[VisualInventoryAlgorithm] Halcon Exception: "
<< except.ErrorMessage().Text() << std::endl;
result.result_barcodes = "{\"" + side +
"\":[], \"error\":\"Halcon Exception: " +
std::string(except.ErrorMessage().Text()) + "\"}";
result.success = false;
} catch (std::exception &e) {
std::cerr << "[VisualInventoryAlgorithm] Exception: " << e.what()
<< std::endl;
result.result_barcodes =
"{\"" + side + "\":[], \"error\":\"" + std::string(e.what()) + "\"}";
result.success = false;
} catch (...) {
std::cerr
<< "[VisualInventoryAlgorithm] Unknown Exception during detection."
<< std::endl;
result.result_barcodes =
"{\"" + side + "\":[], \"error\":\"Unknown exception\"}";
result.success = false;
}
return result.success;
}

40
image_capture/src/algorithm/detections/visual_inventory/visual_inventory_detection.h Normal file
View File

@@ -0,0 +1,40 @@
#pragma once
#include <string>
#include <vector>
namespace cv {
class Mat;
}
/**
* @brief 视觉盘点检测算法结果
*/
struct VisualInventoryResult {
std::string
result_barcodes; // 条码扫描结果JSON: {"left":["BOX111","BOX112"]} 或
// 错误时: {"left":[], "error":"error message"}
std::vector<std::string> codes; // 原始条码列表，便于去重
bool success; // 算法是否执行成功
VisualInventoryResult() : success(false) {}
};
/**
* @brief 视觉盘点检测算法
*
* 识别货位位置并扫描条码
*/
class VisualInventoryAlgorithm {
public:
/**
* 执行视觉盘点检测
* @param depth_img 深度图像
* @param color_img 彩色图像
* @param side 货架侧（"left"或"right"）
* @param result [输出] 检测结果
* @return 是否检测成功
*/
static bool detect(const cv::Mat &depth_img, const cv::Mat &color_img,
const std::string &side, VisualInventoryResult &result);
};

150
image_capture/src/algorithm/utils/image_processor.cpp Normal file
View File

@@ -0,0 +1,150 @@
#include "image_processor.h"
#include <opencv2/opencv.hpp>
#include <iomanip>
#include <iostream>
#include <sstream>
#include <chrono>
#include <ctime>
#include <filesystem>
// ========== ImageProcessor 类实现 ==========
/**
* 构造函数
*/
ImageProcessor::ImageProcessor() {}
/**
* 处理深度图像
*
* @param depth_img 输入的深度图像（16位无符号整数，CV_16U类型）
* @return cv::Mat 处理后的深度图（已应用伪彩色映射）
*/
cv::Mat ImageProcessor::processDepthImage(const cv::Mat& depth_img) {
if (depth_img.empty())
return cv::Mat();
// 确保输入是16位深度图
cv::Mat depthMap;
if (depth_img.type() == CV_16U) {
depthMap = depth_img;
} else {
// 如果不是16位，尝试转换或返回空
return cv::Mat();
}
// 创建掩码，标记无效深度值（0值表示无效/无数据）
cv::Mat invalid_mask = (depthMap == 0);
// 性能优化：避免不必要的clone，直接使用depthMap的视图
// 只有在需要修改时才创建副本
cv::Mat depthProcessed;
// 检查是否有无效值需要处理
int invalid_count = cv::countNonZero(invalid_mask);
if (invalid_count > 0) {
// 有无效值，需要创建副本并修改
depthProcessed = depthMap.clone();
// 将无效值设置为一个很大的值，这样在归一化时会被排除
depthProcessed.setTo(65535, invalid_mask);
} else {
// 没有无效值，直接使用原图（避免不必要的复制）
depthProcessed = depthMap;
}
// 计算有效深度值的范围（排除无效值）
double minVal, maxVal;
cv::minMaxLoc(depthProcessed, &minVal, &maxVal, nullptr, nullptr,
~invalid_mask);
// 如果所有值都无效，返回黑色图像
if (maxVal == 0 || minVal == 65535) {
cv::Mat blackImg = cv::Mat::zeros(depthMap.size(), CV_8UC3);
return blackImg;
}
// 如果所有有效深度值都相同（maxVal == minVal），避免除零错误
// 返回一个统一颜色的深度图（中等灰色，对应JET色图的中间值）
if (maxVal == minVal) {
// 创建单通道灰度图，有效区域设置为中等灰度值（128对应JET色图的中间颜色）
cv::Mat grayMat = cv::Mat::zeros(depthMap.size(), CV_8UC1);
grayMat.setTo(128, ~invalid_mask);
// 应用伪彩色映射
cv::Mat uniformImg;
cv::applyColorMap(grayMat, uniformImg, cv::COLORMAP_JET);
// 确保无效区域保持黑色
uniformImg.setTo(cv::Scalar(0, 0, 0), invalid_mask);
return uniformImg;
}
// 归一化有效深度值到0-255范围
cv::Mat depthVis;
depthProcessed.convertTo(depthVis, CV_8U, 255.0 / (maxVal - minVal),
-minVal * 255.0 / (maxVal - minVal));
// 将无效区域设置为0（黑色）
depthVis.setTo(0, invalid_mask);
// 应用伪彩色映射以提高可视性（JET色图：蓝色=近，红色=远）
cv::applyColorMap(depthVis, depthVis, cv::COLORMAP_JET);
// 确保无效区域保持黑色（伪彩色映射后可能改变，需要再次设置）
depthVis.setTo(cv::Scalar(0, 0, 0), invalid_mask);
return depthVis;
}
/**
* 保存图像到文件
*
* @param depth_img 深度图像（可选）
* @param color_img 彩色图像（可选）
* @param frame_num 帧编号，用于文件命名
* @param save_dir 保存目录（可选，默认为当前目录）
*/
void ImageProcessor::saveImages(const cv::Mat& depth_img, const cv::Mat& color_img,
int frame_num, const std::string& save_dir) {
// 创建保存目录（如果指定了且不存在）
std::string actual_dir = save_dir.empty() ? "." : save_dir;
if (!save_dir.empty()) {
try {
std::filesystem::create_directories(actual_dir);
} catch (const std::exception& e) {
std::cerr << "[Save] Failed to create directory: " << e.what() << std::endl;
actual_dir = "."; // 回退到当前目录
}
}
// 获取当前系统时间用于文件命名
auto now = std::chrono::system_clock::now();
auto time_t = std::chrono::system_clock::to_time_t(now);
std::tm* tm = std::localtime(&time_t);
char time_str[64];
std::strftime(time_str, sizeof(time_str), "%Y%m%d_%H%M%S", tm);
// 保存深度图
if (!depth_img.empty()) {
std::stringstream depth_filename;
depth_filename << actual_dir << "/depth_" << time_str << "_frame"
<< std::setfill('0') << std::setw(6) << frame_num << ".png";
if (cv::imwrite(depth_filename.str(), depth_img)) {
std::cout << "[Save] Depth image saved: " << depth_filename.str() << std::endl;
} else {
std::cerr << "[Save] Failed to save depth image: " << depth_filename.str() << std::endl;
}
}
// 保存彩色图
if (!color_img.empty()) {
std::stringstream color_filename;
color_filename << actual_dir << "/color_" << time_str << "_frame"
<< std::setfill('0') << std::setw(6) << frame_num << ".png";
if (cv::imwrite(color_filename.str(), color_img)) {
std::cout << "[Save] Color image saved: " << color_filename.str() << std::endl;
} else {
std::cerr << "[Save] Failed to save color image: " << color_filename.str() << std::endl;
}
}
}

63
image_capture/src/algorithm/utils/image_processor.h Normal file
View File

@@ -0,0 +1,63 @@
#pragma once
// OpenCV图像处理模块头文件
// 负责深度图的处理（伪彩色映射）、显示和保存
// 注意：此模块不依赖SDK，只使用OpenCV标准类型
// 注意：彩色图的颜色空间转换已在图像采集层完成，此处不再处理
#include <string>
namespace cv { class Mat; }
/**
* 图像处理器类
*
* 功能说明：
* - 处理深度图（归一化、伪彩色映射）
* - 保存图像到文件
*
* 注意：彩色图的颜色空间转换已在图像采集层（camera层）完成，
* 统一输出BGR格式，此处不再需要处理
*
* 设计原则：
* - 此模块完全独立于SDK，只使用OpenCV的cv::Mat类型
* - SDK到OpenCV的转换应在图像采集层（camera层）完成
* - 图像显示由GUI层（MainWindow）负责，使用Qt的QLabel分别显示
*/
class ImageProcessor
{
public:
/**
* 构造函数
*/
ImageProcessor();
/**
* 处理深度图像
*
* @param depth_img 输入的深度图像（16位无符号整数，CV_16U类型）
* @return cv::Mat 处理后的深度图（已应用伪彩色映射，BGR格式）
*
* 功能说明：
* - 将深度值归一化到0-255范围
* - 应用JET伪彩色映射（蓝色=近，红色=远）
* - 处理无效深度值（0值）
*/
cv::Mat processDepthImage(const cv::Mat& depth_img);
/**
* 保存图像到文件
*
* @param depth_img 深度图像（可选）
* @param color_img 彩色图像（可选）
* @param frame_num 帧编号，用于文件命名
* @param save_dir 保存目录（可选，默认为当前目录）
*
* 功能说明：
* - 获取当前时间戳用于文件命名
* - 保存深度图和彩色图到文件
*/
void saveImages(const cv::Mat& depth_img, const cv::Mat& color_img,
int frame_num, const std::string& save_dir = "");
};

306
image_capture/src/camera/mvs_multi_camera_capture.cpp Normal file
View File

@@ -0,0 +1,306 @@
/**
* @file mvs_multi_camera_capture.cpp
* @brief 海康 MVS 相机采集实现文件
*
* 此文件包含了 MvsMultiCameraCapture 类的完整实现
* - 封装 MVS SDK (MvCameraControl.h)，管理多相机采集
* - 将 SDK 的原始帧数据转换为 OpenCV 的 cv::Mat 格式
* - 管理采集线程和缓冲区
*
* 设计说明：
* - 每个相机使用独立的采集线程，避免阻塞
* - 使用线程安全的缓冲区存储最新图像
* - 统一输出 BGR 格式的彩色图
*/
#include "mvs_multi_camera_capture.h"
#include "MvCameraControl.h"
#include <iostream>
#include <chrono>
#include <cstring>
/**
* @brief 构造函数
* 初始化运行标志和状态
*/
MvsMultiCameraCapture::MvsMultiCameraCapture() : running_(false), initialized_(false) {}
/**
* @brief 析构函数
* 确保在对象销毁时正确停止所有采集线程和相机，并清理资源
*/
MvsMultiCameraCapture::~MvsMultiCameraCapture() {
stop();
// 清理句柄
for (auto& cam : cameras_) {
if (cam.handle) {
MV_CC_DestroyHandle(cam.handle);
cam.handle = nullptr;
}
}
}
/**
* @brief 初始化相机
*
* 此函数完成以下工作：
* 1. 枚举所有连接的 GenTL GigE 和 USB 设备
* 2. 创建并打开相机句柄
* 3. 配置相机参数（如触发模式、包大小等）
* 4. 初始化图像缓冲区
*
* @return true 初始化成功且至少找到一个设备, false 失败
*/
bool MvsMultiCameraCapture::initialize() {
if (initialized_) return true;
MV_CC_DEVICE_INFO_LIST stDeviceList;
memset(&stDeviceList, 0, sizeof(MV_CC_DEVICE_INFO_LIST));
// 枚举 GenTL GigE 和 USB 设备
int nRet = MV_CC_EnumDevices(MV_GIGE_DEVICE | MV_USB_DEVICE, &stDeviceList);
if (MV_OK != nRet) {
std::cerr << "[MVS] EnumDevices failed: " << std::hex << nRet << std::endl;
return false;
}
if (stDeviceList.nDeviceNum == 0) {
std::cout << "[MVS] No devices found." << std::endl;
return false;
}
std::cout << "[MVS] Found " << stDeviceList.nDeviceNum << " devices." << std::endl;
for (unsigned int i = 0; i < stDeviceList.nDeviceNum; i++) {
MV_CC_DEVICE_INFO* pDeviceInfo = stDeviceList.pDeviceInfo[i];
if (NULL == pDeviceInfo) continue;
CameraInfo camInfo;
camInfo.index = static_cast<int>(cameras_.size());
// 获取序列号
if (pDeviceInfo->nTLayerType == MV_GIGE_DEVICE) {
camInfo.serial_number = std::string((char*)pDeviceInfo->SpecialInfo.stGigEInfo.chSerialNumber);
} else if (pDeviceInfo->nTLayerType == MV_USB_DEVICE) {
camInfo.serial_number = std::string((char*)pDeviceInfo->SpecialInfo.stUsb3VInfo.chSerialNumber);
}
// 创建句柄
nRet = MV_CC_CreateHandle(&camInfo.handle, pDeviceInfo);
if (MV_OK != nRet) {
std::cerr << "[MVS] CreateHandle failed for device " << i << std::endl;
continue;
}
// 打开设备
nRet = MV_CC_OpenDevice(camInfo.handle);
if (MV_OK != nRet) {
std::cerr << "[MVS] OpenDevice failed for device " << i << std::endl;
MV_CC_DestroyHandle(camInfo.handle);
continue;
}
// 确保触发模式为 OFF 以进行连续采集
nRet = MV_CC_SetEnumValue(camInfo.handle, "TriggerMode", MV_TRIGGER_MODE_OFF);
if (MV_OK != nRet) {
std::cerr << "[MVS] Warning: Failed to set TriggerMode to Off. Ret = " << std::hex << nRet << std::endl;
}
// 检查 GigE 的最佳包大小并设置
if (pDeviceInfo->nTLayerType == MV_GIGE_DEVICE) {
int nPacketSize = MV_CC_GetOptimalPacketSize(camInfo.handle);
if (nPacketSize > 0) {
MV_CC_SetIntValue(camInfo.handle, "GevSCPSPacketSize", nPacketSize);
}
}
cameras_.push_back(camInfo);
buffers_.push_back(std::make_shared<ImageBuffer>());
// 记录相机分辨率
MVCC_INTVALUE stWidth = {0};
MVCC_INTVALUE stHeight = {0};
int nRetW = MV_CC_GetIntValue(camInfo.handle, "Width", &stWidth);
int nRetH = MV_CC_GetIntValue(camInfo.handle, "Height", &stHeight);
std::cout << "[MVS] Initialized camera " << camInfo.index << ": " << camInfo.serial_number;
if (MV_OK == nRetW && MV_OK == nRetH) {
std::cout << " (Resolution: " << stWidth.nCurValue << "x" << stHeight.nCurValue << ")";
}
std::cout << std::endl;
}
initialized_ = true;
return !cameras_.empty();
}
/**
* @brief 开始采集
* 启动所有相机的抓图，并为每个相机创建一个采集线程
* @return true 启动成功, false 失败
*/
bool MvsMultiCameraCapture::start() {
if (!initialized_ || running_) return false;
running_ = true;
for (const auto& cam : cameras_) {
// 开始抓取
int nRet = MV_CC_StartGrabbing(cam.handle);
if (MV_OK != nRet) {
std::cerr << "[MVS] StartGrabbing failed for camera " << cam.index << std::endl;
}
threads_.emplace_back(&MvsMultiCameraCapture::captureThreadFunc, this, cam.index);
}
return true;
}
/**
* @brief 停止采集
* 停止所有采集线程和相机抓图
*/
void MvsMultiCameraCapture::stop() {
running_ = false;
for (auto& t : threads_) {
if (t.joinable()) t.join();
}
threads_.clear();
for (const auto& cam : cameras_) {
MV_CC_StopGrabbing(cam.handle);
}
}
/**
* @brief 获取相机 ID (序列号)
* @param camera_index 相机索引
* @return 相机序列号
*/
std::string MvsMultiCameraCapture::getCameraId(int camera_index) const {
if (camera_index >= 0 && camera_index < cameras_.size()) {
return cameras_[camera_index].serial_number;
}
return "";
}
/**
* @brief 获取指定相机的最新图像
* 从线程安全的缓冲区中读取最新图像数据
*
* @param camera_index 相机索引
* @param[out] image 输出图像
* @param[out] fps 当前帧率
* @return true 成功获取, false 索引无效或无新图像
*/
bool MvsMultiCameraCapture::getLatestImage(int camera_index, cv::Mat& image, double& fps) {
if (camera_index < 0 || camera_index >= buffers_.size()) return false;
auto& buffer = buffers_[camera_index];
std::lock_guard<std::mutex> lock(buffer->mtx);
if (buffer->image.empty()) return false;
image = buffer->image.clone();
fps = buffer->fps;
buffer->updated = false;
return true;
}
/**
* @brief 转换为 OpenCV Mat 格式
*
* 将 SDK 返回的帧数据转换为 OpenCV 的 BGR8 Mat。
*
* @param handle 相机句柄
* @param pFrame MVS 帧信息结构体指针 (MV_FRAME_OUT*)
* @param pUser 用户数据 (未使用)
* @return cv::Mat 转换后的 OpenCV 图像
*/
cv::Mat MvsMultiCameraCapture::convertToMat(void* handle, void* pFrame, void* pUser) {
// pFrame 在 captureThreadFunc 中传入的是 MV_FRAME_OUT* 指针
MV_FRAME_OUT* stFrameOut = (MV_FRAME_OUT*)pFrame;
MV_FRAME_OUT_INFO_EX* stUserInfo = stFrameOut->stFrameInfo.enPixelType == 0 ? nullptr : &stFrameOut->stFrameInfo;
if (!handle || !stUserInfo) return cv::Mat();
cv::Mat image;
MV_CC_PIXEL_CONVERT_PARAM stConvertParam = {0};
stConvertParam.nWidth = stUserInfo->nWidth;
stConvertParam.nHeight = stUserInfo->nHeight;
stConvertParam.pSrcData = stFrameOut->pBufAddr;
stConvertParam.nSrcDataLen = stUserInfo->nFrameLen;
stConvertParam.enSrcPixelType = stUserInfo->enPixelType;
stConvertParam.enDstPixelType = PixelType_Gvsp_BGR8_Packed; // 转换为 OpenCV 的 BGR8
stConvertParam.nDstBufferSize = stUserInfo->nWidth * stUserInfo->nHeight * 3;
// 分配目标缓冲区
image.create(stUserInfo->nHeight, stUserInfo->nWidth, CV_8UC3);
stConvertParam.pDstBuffer = image.data;
int nRet = MV_CC_ConvertPixelType(handle, &stConvertParam);
if (MV_OK != nRet) {
std::cerr << "[MVS] ConvertPixelType failed: " << std::hex << nRet << std::endl;
return cv::Mat();
}
return image;
}
/**
* @brief 采集线程函数
*
* 每个相机的独立工作线程：
* 1. 循环调用 MV_CC_GetImageBuffer 获取图像
* 2. 调用 convertToMat 转换为 cv::Mat
* 3. 更新线程安全缓冲区
* 4. 计算并更新 FPS
*
* @param camera_index 相机索引
*/
void MvsMultiCameraCapture::captureThreadFunc(int camera_index) {
auto& cam = cameras_[camera_index];
auto& buffer = buffers_[camera_index];
MV_FRAME_OUT stFrameOut;
memset(&stFrameOut, 0, sizeof(MV_FRAME_OUT));
auto start_time = std::chrono::steady_clock::now();
int frame_count = 0;
while (running_) {
// 获取图像缓冲区，超时 1000ms
int nRet = MV_CC_GetImageBuffer(cam.handle, &stFrameOut, 1000);
if (MV_OK == nRet) {
try {
// 传递 stFrameOut 指针进行转换
cv::Mat image = convertToMat(cam.handle, &stFrameOut, nullptr);
if (!image.empty()) {
std::lock_guard<std::mutex> lock(buffer->mtx);
buffer->image = image;
buffer->updated = true;
frame_count++;
auto now = std::chrono::steady_clock::now();
double elapsed = std::chrono::duration_cast<std::chrono::seconds>(now - start_time).count();
if (elapsed >= 1.0) {
buffer->fps = frame_count / elapsed;
frame_count = 0;
start_time = now;
// std::cout << "[MVS] Cam " << camera_index << " FPS: " << buffer->fps << std::endl;
}
}
} catch (const std::exception& e) {
std::cerr << "[MVS] Exception in conversion: " << e.what() << std::endl;
}
// 释放图像缓冲区
MV_CC_FreeImageBuffer(cam.handle, &stFrameOut);
} else {
// 如果触发器正在等待，超时是预期的，但我们设置的是连续采集。
std::cerr << "[MVS] GetImageBuffer failed: " << std::hex << nRet << std::endl;
}
}
}

122
image_capture/src/camera/mvs_multi_camera_capture.h Normal file
View File

@@ -0,0 +1,122 @@
#pragma once
#include <opencv2/opencv.hpp>
#include <vector>
#include <string>
#include <atomic>
#include <thread>
#include <mutex>
#include <memory>
/**
* @file mvs_multi_camera_capture.h
* @brief 海康 MVS 相机采集类定义
*/
/**
* @brief 图像缓冲区结构体
* 存储从相机采集到的最新图像及相关元数据
*/
struct ImageBuffer {
cv::Mat image; ///< 存储图像数据 (BGR格式)
std::mutex mtx; ///< 互斥锁，保证多线程访问安全
bool updated = false; ///< 标记图像是否已更新
double fps = 0.0; ///< 当前帧率
};
/**
* @brief 相机信息结构体
* 存储相机的句柄和序列号等信息
*/
struct CameraInfo {
void* handle = nullptr; ///< MVS SDK 相机句柄
std::string serial_number; ///< 相机序列号
int index = -1; ///< 相机索引
};
/**
* @brief MVS 多相机采集类
*
* 功能说明：
* - 封装海康 MVS SDK，管理多相机采集
* - 将 SDK 原始图像转换为 OpenCV Mat 格式
* - 管理采集线程和缓冲区
* - 提供最新的图像数据供上层调用
*/
class MvsMultiCameraCapture {
public:
MvsMultiCameraCapture();
~MvsMultiCameraCapture();
/**
* @brief 初始化相机
* 枚举并打开所有连接的 GenTL GigE 和 USB 设备
* @return true 初始化成功且至少找到一个设备, false 失败
*/
bool initialize();
/**
* @brief 开始采集
* 启动所有相机的抓图，并开启采集线程
* @return true 启动成功, false 失败
*/
bool start();
/**
* @brief 停止采集
* 停止抓图并关闭所有线程
*/
void stop();
/**
* @brief 获取相机数量
* @return 已初始化的相机数量
*/
int getCameraCount() const { return static_cast<int>(cameras_.size()); }
/**
* @brief 获取指定相机的最新图像
* @param camera_index 相机索引
* @param[out] image 输出图像 (BGR格式)
* @param[out] fps 当前帧率
* @return true 成功获取, false 索引无效或无新图像
*/
bool getLatestImage(int camera_index, cv::Mat& image, double& fps);
/**
* @brief 获取相机 ID (序列号)
* @param camera_index 相机索引
* @return 相机序列号字符串
*/
std::string getCameraId(int camera_index) const;
/**
* @brief 检查是否正在运行
* @return true 运行中, false 已停止
*/
bool isRunning() const { return running_; }
private:
/**
* @brief 采集线程函数
* 每个相机运行在独立的线程中，持续从 SDK 获取图像
* @param camera_index 相机索引
*/
void captureThreadFunc(int camera_index);
/**
* @brief 转换为 OpenCV Mat 格式
* 使用 SDK 的 MV_CC_ConvertPixelType 函数将原始数据转换为 BGR8 格式
* @param handle 相机句柄
* @param pFrame 帧数据指针 (MV_FRAME_OUT*)
* @param pUser 用户数据 (保留，未使用)
* @return cv::Mat 转换后的图像
*/
static cv::Mat convertToMat(void* handle, void* pFrame, void* pUser);
std::vector<CameraInfo> cameras_; ///< 相机列表
std::vector<std::shared_ptr<ImageBuffer>> buffers_; ///< 缓冲区列表
std::vector<std::thread> threads_; ///< 采集线程列表
std::atomic<bool> running_; ///< 运行状态标志
bool initialized_ = false; ///< 初始化状态标志
};

950
image_capture/src/camera/ty_multi_camera_capture.cpp Normal file
View File

@@ -0,0 +1,950 @@
/**
* @file ty_multi_camera_capture.cpp
* @brief TY相机采集实现文件
*
* 此文件包含了 CameraCapture 类的完整实现
* - 封装TY相机SDK，管理多相机采集
* - 将SDK的TYImage转换为OpenCV的cv::Mat格式
* - 管理采集线程和缓冲区
* - 输出原始cv::Mat格式的图像，供上层使用
*
* 设计说明：
* - 每个相机使用独立的采集线程，避免阻塞
* - 使用线程安全的缓冲区存储最新图像
* - 使用clone()确保数据安全，避免悬空指针
* - 统一输出BGR格式的彩色图，便于上层处理
*/
#include "ty_multi_camera_capture.h"
#include "TYCoordinateMapper.h"
#include <chrono>
#include <iostream>
#ifdef _WIN32
#include <windows.h>
#endif
/**
* @brief 构造函数
*
* 初始化所有成员变量为默认值：初始化列表的方式初始化成员变量，避免在构造函数体中初始化，提高代码可读性。
* - streams_configured_: 流未配置
* - depth_enabled_: 深度流未启用
* - color_enabled_: 彩色流未启用
* - running_: 未运行状态
*/
CameraCapture::CameraCapture()
: streams_configured_(false), depth_enabled_(false), color_enabled_(false),
running_(false) {}
/**
* @brief 析构函数
*
* 确保在对象销毁时正确停止所有采集线程和相机
* 调用stop()来清理资源，避免资源泄漏
*/
CameraCapture::~CameraCapture() { stop(); }
/**
* @brief 初始化并配置相机
*
* 此函数完成以下工作：
* 1. 清理现有资源（如果之前已初始化）
* 2. 查询并打开所有可用的相机设备
* 3. 为每个相机配置深度流和彩色流
* 4. 创建图像处理器和缓冲区
*
* @param enable_depth 是否启用深度流，true表示启用深度图采集
* @param enable_color 是否启用彩色流，true表示启用彩色图采集
* @return true 初始化成功，false 初始化失败（无设备或打开失败）
*
* @note 如果部分相机打开失败，只要至少有一个相机成功打开，函数仍返回true
*/
bool CameraCapture::initialize(bool enable_depth, bool enable_color) {
// 设置控制台代码页为UTF-8，确保中文正确显示
#ifdef _WIN32
SetConsoleOutputCP(65001); // UTF-8代码页
SetConsoleCP(65001); // 设置输入代码页也为UTF-8
#endif
// 清理现有资源，确保重新初始化时状态干净
stop();
cameras_.clear();
depth_processers_.clear();
color_processers_.clear();
camera_running_.clear();
buffers_.clear();
calib_infos_.clear();
has_calib_info_.clear();
// 通过TY SDK的上下文查询设备列表
// TYContext是单例模式，获取全局唯一的上下文实例
auto &context = TYContext::getInstance();
auto device_list = context.queryDeviceList();
// 检查是否找到设备-检查device_list是否为空或者设备数量为0
if (!device_list || device_list->empty()) {
std::cerr << "[CameraCapture] No devices found!" << std::endl;
return false;
}
// 获取所有可用设备数量，使用所有找到的设备
int device_count = device_list->devCount();
std::cout << "[CameraCapture] Found " << device_count
<< " device(s), will use all available devices" << std::endl;
// 遍历设备列表，逐个打开所有可用相机
for (int i = 0; i < device_count; i++) {
// 获取设备信息（包含设备ID等）
auto device_info = device_list->getDeviceInfo(i);
if (!device_info) {
std::cerr << "[CameraCapture] Failed to get camera device info! " << i << std::endl;
continue; // 跳过此设备，继续处理下一个
}
std::cout << "[CameraCapture] Preparing to open camera! " << i << ": "
<< device_info->id() << std::endl;
// 创建FastCamera对象（SDK提供的相机封装类）
auto camera = std::make_shared<FastCamera>();
// 使用设备ID打开相机
TY_STATUS status = camera->open(device_info->id());
// 检查打开是否成功
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Failed to open camera! " << i << ": "
<< device_info->id() << std::endl;
continue; // 打开失败，跳过此设备
} else {
std::cout << "[CameraCapture] Successfully opened camera! " << i << ": "
<< device_info->id() << std::endl;
}
// 成功打开，添加到相机列表
cameras_.push_back(camera);
camera_running_.push_back(false); // 初始状态为未运行
// 图像处理器稍后在配置流时创建，这里先占位
depth_processers_.push_back(nullptr);
color_processers_.push_back(nullptr);
// 获取并保存标定信息
TY_CAMERA_CALIB_INFO calib_info;
TY_STATUS calib_status = TYGetStruct(camera->handle(), TY_COMPONENT_DEPTH_CAM, TY_STRUCT_CAM_CALIB_DATA,
&calib_info, sizeof(calib_info));
if (calib_status == TY_STATUS_OK) {
calib_infos_.push_back(calib_info);
has_calib_info_.push_back(true);
std::cout << "[CameraCapture] Camera " << i << " calibration info fetched." << std::endl;
} else {
calib_infos_.push_back(TY_CAMERA_CALIB_INFO());
has_calib_info_.push_back(false);
std::cerr << "[CameraCapture] Camera " << i << " failed to fetch calibration info: " << calib_status << std::endl;
}
}
// 检查是否至少成功打开一个相机
if (cameras_.empty()) {
std::cerr << "[CameraCapture] No cameras opened successfully!" << std::endl;
return false;
}
// ========== 配置流 ==========
// 保存流配置标志，供后续使用
depth_enabled_ = enable_depth;
color_enabled_ = enable_color;
// 为每个已打开的相机配置流
for (size_t i = 0; i < cameras_.size(); i++) {
auto &camera = cameras_[i];
// 启用深度流
if (enable_depth) {
// 调用SDK接口启用深度流
TY_STATUS status = camera->stream_enable(FastCamera::stream_depth);
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to enable depth stream"
<< std::endl;
} else {
// 创建深度图像处理器
// ImageProcesser用于处理SDK返回的原始图像数据
std::string depth_win_name = "depth_" + std::to_string(i);
depth_processers_[i] =
std::make_shared<ImageProcesser>(depth_win_name.c_str());
std::cout << "[CameraCapture] Camera " << i << " depth stream enabled"
<< std::endl;
}
}
// 启用彩色流
if (enable_color) {
// 调用SDK接口启用彩色流
TY_STATUS status = camera->stream_enable(FastCamera::stream_color);
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to enable color stream"
<< std::endl;
} else {
// 创建彩色图像处理器
std::string color_win_name = "color_" + std::to_string(i);
color_processers_[i] =
std::make_shared<ImageProcesser>(color_win_name.c_str());
std::cout << "[CameraCapture] Camera " << i << " color stream enabled"
<< std::endl;
}
}
}
// ========== 设置分辨率 ==========
// 为每个相机设置深度图和彩色图分辨率为640x480
for (size_t i = 0; i < cameras_.size(); i++) {
auto &camera = cameras_[i];
TY_DEV_HANDLE hDevice = camera->handle();
if (hDevice == 0) {
std::cerr << "[CameraCapture] Camera " << i << " handle is invalid, skip resolution setting"
<< std::endl;
continue;
}
// 设置深度图分辨率为1280x960（使用图像模式）
if (enable_depth) {
// 方法1：尝试使用图像模式（推荐，同时设置分辨率和格式）
TY_IMAGE_MODE depth_mode = TY_IMAGE_MODE_DEPTH16_1280x960;
TY_STATUS status = TYSetEnum(hDevice, TY_COMPONENT_DEPTH_CAM, TY_ENUM_IMAGE_MODE, depth_mode);
if (status != TY_STATUS_OK) {
// 方法2：如果图像模式不支持，回退到单独设置宽高
std::cerr << "[CameraCapture] Camera " << i << " failed to set depth image mode 1280x960, trying width/height: "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
status = TYSetInt(hDevice, TY_COMPONENT_DEPTH_CAM, TY_INT_WIDTH, 1280);
if (status == TY_STATUS_OK) {
status = TYSetInt(hDevice, TY_COMPONENT_DEPTH_CAM, TY_INT_HEIGHT, 960);
}
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to set depth resolution: "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
} else {
std::cout << "[CameraCapture] Camera " << i << " depth resolution set to 1280x960 (via width/height)"
<< std::endl;
}
} else {
std::cout << "[CameraCapture] Camera " << i << " depth resolution set to 1280x960"
<< std::endl;
}
}
// 设置彩色图分辨率为1280x960，使用YUYV格式
if (enable_color) {
// 方法1：尝试使用YUYV格式的1280x960图像模式（推荐）
TY_IMAGE_MODE color_mode = TY_IMAGE_MODE_YUYV_1280x960;
TY_STATUS status = TYSetEnum(hDevice, TY_COMPONENT_RGB_CAM, TY_ENUM_IMAGE_MODE, color_mode);
if (status != TY_STATUS_OK) {
// 方法2：如果YUYV模式不支持，尝试其他格式
std::cerr << "[CameraCapture] Camera " << i << " failed to set YUYV_1280x960 mode, trying alternatives: "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
// 尝试RGB格式
color_mode = TY_IMAGE_MODE_RGB_1280x960;
status = TYSetEnum(hDevice, TY_COMPONENT_RGB_CAM, TY_ENUM_IMAGE_MODE, color_mode);
if (status != TY_STATUS_OK) {
// 方法3：如果图像模式都不支持，回退到单独设置宽高
std::cerr << "[CameraCapture] Camera " << i << " failed to set RGB_1280x960 mode, trying width/height: "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
status = TYSetInt(hDevice, TY_COMPONENT_RGB_CAM, TY_INT_WIDTH, 1280);
if (status == TY_STATUS_OK) {
status = TYSetInt(hDevice, TY_COMPONENT_RGB_CAM, TY_INT_HEIGHT, 960);
}
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to set color resolution: "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
} else {
std::cout << "[CameraCapture] Camera " << i << " color resolution set to 1280x960 (via width/height)"
<< std::endl;
}
} else {
std::cout << "[CameraCapture] Camera " << i << " color resolution set to 1280x960 (RGB format)"
<< std::endl;
}
} else {
std::cout << "[CameraCapture] Camera " << i << " color resolution set to 1280x960 YUYV"
<< std::endl;
}
}
}
// ========== 设置帧率 ==========
// 根据相机技术参数，640x480分辨率下：
// - 深度图：19 fps
// - RGB图（YUYV格式）：25 fps
// 使用连续模式以获得最高帧率
for (size_t i = 0; i < cameras_.size(); i++) {
auto &camera = cameras_[i];
TY_DEV_HANDLE hDevice = camera->handle();
if (hDevice == 0) {
std::cerr << "[CameraCapture] Camera " << i << " handle is invalid, skip frame rate setting"
<< std::endl;
continue;
}
// 方法1：使用连续模式（TY_TRIGGER_MODE_OFF），让相机以最大帧率连续采集
// 640x480分辨率下，连续模式应该能达到：深度19fps，RGB(YUYV)25fps
TY_TRIGGER_PARAM trigger_param;
trigger_param.mode = TY_TRIGGER_MODE_OFF; // 连续模式，不使用触发
trigger_param.fps = 0; // 连续模式下fps参数无效
trigger_param.rsvd = 0;
TY_STATUS status = TYSetStruct(hDevice, TY_COMPONENT_DEVICE, TY_STRUCT_TRIGGER_PARAM,
&trigger_param, sizeof(trigger_param));
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to set trigger mode (continuous): "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
// 方法2：如果连续模式不支持，尝试使用周期性触发模式
// 根据技术参数，640x480下RGB可达25fps，深度可达19fps
// 设置为25fps以匹配RGB的最高帧率
trigger_param.mode = TY_TRIGGER_MODE_M_PER; // 主模式，周期性触发
trigger_param.fps = 25; // 设置帧率为25fps（匹配RGB YUYV格式的最高帧率）
trigger_param.rsvd = 0;
status = TYSetStruct(hDevice, TY_COMPONENT_DEVICE, TY_STRUCT_TRIGGER_PARAM,
&trigger_param, sizeof(trigger_param));
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Camera " << i << " failed to set trigger mode (25fps): "
<< status << "(" << TYErrorString(status) << ")" << std::endl;
} else {
std::cout << "[CameraCapture] Camera " << i << " frame rate set to 25fps (trigger mode)"
<< std::endl;
}
} else {
std::cout << "[CameraCapture] Camera " << i << " set to continuous mode"
<< std::endl;
}
}
// 标记流已配置
streams_configured_ = true;
// ========== 创建图像缓冲区 ==========
// 为每个相机创建一个独立的图像缓冲区
// 缓冲区用于存储采集线程获取的最新图像，供上层读取
for (size_t i = 0; i < cameras_.size(); i++) {
buffers_.push_back(std::make_shared<ImageBuffer>());
}
std::cout << "[CameraCapture] Initialization complete! Total " << cameras_.size() << " camera(s)"
<< std::endl;
return true;
}
/**
* @brief 启动采集
*
* 此函数完成以下工作：
* 1. 检查相机和流配置状态
* 2. 启动所有相机的数据流
* 3. 为每个相机创建独立的采集线程
*
* @return true 启动成功，false 启动失败（无相机或流未配置或启动失败）
*
* @note 如果部分相机启动失败，函数返回false，但已启动的相机需要手动停止
* @note 每个相机使用独立的线程，避免相互阻塞
*/
bool CameraCapture::start() {
// 检查是否已经在运行
if (running_) {
std::cout << "[CameraCapture] System already running" << std::endl;
return true;
}
// 检查是否有相机
if (cameras_.empty()) {
std::cerr << "[CameraCapture] No cameras to start!" << std::endl;
return false;
}
// 检查流是否已配置（必须先调用initialize）
if (!streams_configured_) {
std::cerr << "[CameraCapture] Streams not configured!" << std::endl;
return false;
}
// ========== 启动所有相机 ==========
bool all_started = true;
for (size_t i = 0; i < cameras_.size(); i++) {
auto &camera = cameras_[i];
// 调用SDK接口启动相机数据流
TY_STATUS status = camera->start();
if (status == TY_STATUS_OK) {
camera_running_[i] = true; // 标记相机为运行状态
std::cout << "[CameraCapture] Camera " << i << " started" << std::endl;
} else {
camera_running_[i] = false;
std::cerr << "[CameraCapture] Camera " << i << " failed to start" << std::endl;
all_started = false; // 记录有相机启动失败
}
}
// 如果有相机启动失败，返回false
if (!all_started) {
return false;
}
// ========== 启动采集线程 ==========
// 设置运行标志，采集线程会检查此标志来决定是否开启
running_ = true;
// 为每个相机创建独立的采集线程
// 线程函数：captureThreadFunc
// 参数：this指针指向调用 start() 的 CameraCapture
// 对象和相机索引、static_cast：类型转换操作符
for (size_t i = 0; i < cameras_.size(); i++) {
capture_threads_.emplace_back(&CameraCapture::captureThreadFunc, this,
static_cast<int>(i));
}
return true;
}
/**
* @brief 停止采集
*
* 此函数完成以下工作：
* 1. 设置运行标志为false，通知采集线程退出
* 2. 等待所有采集线程结束（join）
* 3. 停止所有相机的数据流
*
* @note 此函数是线程安全的，可以在任何线程中调用
* @note 析构函数会自动调用此函数，确保资源正确释放
*/
void CameraCapture::stop() {
// 设置运行标志为false，通知所有采集线程退出循环
running_ = false;
// 等待所有采集线程结束
// join()会阻塞直到线程执行完毕，确保线程安全退出
for (auto &t : capture_threads_) {
if (t.joinable()) {
t.join();
}
}
capture_threads_.clear(); // 清空线程列表
// 停止所有相机的数据流
for (size_t i = 0; i < cameras_.size(); i++) {
if (camera_running_[i] && cameras_[i]) {
cameras_[i]->stop(); // 调用SDK接口停止相机
camera_running_[i] = false; // 标记相机为停止状态
}
}
}
/**
* @brief 获取相机数量
*
* @return 当前已初始化的相机数量
*/
int CameraCapture::getCameraCount() const {
return static_cast<int>(cameras_.size());
}
/**
* @brief 获取指定相机的设备ID
*
* @param index 相机索引，从0开始
* @return 相机设备ID字符串，如果索引无效则返回空字符串
*
* @note 此函数会重新查询设备列表，确保返回最新的设备ID
*/
std::string CameraCapture::getCameraId(int index) const {
// 检查索引有效性
if (index < 0 || index >= static_cast<int>(cameras_.size())) {
return "";
}
// 重新查询设备列表获取ID
// 注意：这里使用SDK的设备列表而不是内部存储，确保ID是最新的
auto &context = TYContext::getInstance();
auto device_list = context.queryDeviceList();
if (!device_list || index >= device_list->devCount()) {
return "";
}
auto device_info = device_list->getDeviceInfo(index);
if (!device_info) {
return "";
}
return std::string(device_info->id());
}
/**
* @brief 获取指定相机的最新图像
*
* 从线程安全的缓冲区中读取最新采集的图像数据
*
* @param camera_index 相机索引，从0开始
* @param depth [输出] 深度图，CV_16U格式，包含原始深度值（单位：毫米）
* @param color [输出] 彩色图，BGR格式，CV_8UC3类型
* @param fps [输出] 当前帧率（帧/秒）
* @return true 成功获取图像，false 索引无效或缓冲区为空
*
* @note 此函数是线程安全的，使用互斥锁保护缓冲区访问
* @note 如果某个图像流未启用或尚未采集到数据，对应的Mat将为空
* @note 使用copyTo()复制数据，确保返回的图像数据独立于缓冲区
*/
bool CameraCapture::getLatestImages(int camera_index, cv::Mat &depth,
cv::Mat &color, double &fps) {
// 检查索引有效性
if (camera_index < 0 || camera_index >= static_cast<int>(buffers_.size())) {
return false;
}
auto buffer = buffers_[camera_index];
// 使用互斥锁保护缓冲区访问，确保线程安全
// lock_guard自动管理锁的获取和释放
std::lock_guard<std::mutex> lock(buffer->mtx);
// 复制深度图数据
if (!buffer->depth.empty()) {
buffer->depth.copyTo(depth); // 深拷贝，确保数据独立
} else {
depth = cv::Mat(); // 如果为空，返回空Mat
}
// 复制彩色图数据
if (!buffer->color.empty()) {
buffer->color.copyTo(color); // 深拷贝，确保数据独立
} else {
color = cv::Mat(); // 如果为空，返回空Mat
}
// 复制FPS值
fps = buffer->fps;
return true;
}
/**
* @brief 检查是否正在运行
*
* @return true 正在运行，false 已停止
*/
bool CameraCapture::isRunning() const { return running_; }
/**
* @brief 获取指定相机的深度相机内参
*
* 从图漾相机SDK获取深度相机的内参（fx, fy, cx, cy）
* 内参存储在相机的标定数据中，通过TYGetStruct API获取
*
* @param camera_index 相机索引，从0开始
* @param fx [输出] 焦距x（像素单位）
* @param fy [输出] 焦距y（像素单位）
* @param cx [输出] 主点x坐标（像素单位）
* @param cy [输出] 主点y坐标（像素单位）
* @return true 成功获取内参，false 索引无效或获取失败
*
* @note 内参矩阵格式为3x3：
* | fx 0 cx |
* | 0 fy cy |
* | 0 0 1 |
* @note 此函数需要在相机初始化后调用（initialize之后）
*/
bool CameraCapture::getDepthCameraIntrinsics(int camera_index, float& fx, float& fy, float& cx, float& cy) {
// 检查索引有效性
if (camera_index < 0 || camera_index >= static_cast<int>(cameras_.size())) {
std::cerr << "[CameraCapture] Invalid camera index: " << camera_index << std::endl;
return false;
}
// 检查相机是否已打开
auto camera = cameras_[camera_index];
if (!camera) {
std::cerr << "[CameraCapture] Camera " << camera_index << " is not opened" << std::endl;
return false;
}
// 获取相机设备句柄
TY_DEV_HANDLE hDevice = camera->handle();
if (hDevice == 0) {
std::cerr << "[CameraCapture] Camera " << camera_index << " handle is invalid" << std::endl;
return false;
}
// 获取深度相机的内参
TY_CAMERA_INTRINSIC intrinsic;
TY_STATUS status = TYGetStruct(hDevice, TY_COMPONENT_DEPTH_CAM, TY_STRUCT_CAM_INTRINSIC,
&intrinsic, sizeof(intrinsic));
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] Failed to get depth camera intrinsics for camera "
<< camera_index << ", error: " << status << "(" << TYErrorString(status) << ")" << std::endl;
return false;
}
// 内参矩阵是3x3，按行主序存储：
// data[0] = fx, data[1] = 0, data[2] = cx
// data[3] = 0, data[4] = fy, data[5] = cy
// data[6] = 0, data[7] = 0, data[8] = 1
fx = intrinsic.data[0]; // fx
fy = intrinsic.data[4]; // fy
cx = intrinsic.data[2]; // cx
cy = intrinsic.data[5]; // cy
std::cout << "[CameraCapture] Camera " << camera_index
<< " depth intrinsics: fx=" << fx << ", fy=" << fy
<< ", cx=" << cx << ", cy=" << cy << std::endl;
return true;
}
/**
* @brief 采集线程函数
*
* 这是每个相机独立运行的采集线程的主函数
* 主要工作：
* 1. 从SDK获取原始帧数据（TYFrame）
* 2. 提取深度图和彩色图（TYImage）
* 3. 使用图像处理器处理原始数据（如果需要）
* 4. 转换为OpenCV格式（cv::Mat）
* 5. 进行颜色空间转换（统一为BGR）
* 6. 计算帧率
* 7. 更新线程安全的缓冲区
*
* @param camera_index 相机索引，标识此线程负责哪个相机
*
* @note 此函数运行在独立的线程中，每个相机一个线程
* @note 使用异常处理确保线程异常不会导致程序崩溃
* @note 使用超时机制避免长时间阻塞
* @note 缓冲区更新使用互斥锁保护，确保线程安全
*/
void CameraCapture::captureThreadFunc(int camera_index) {
// 检查相机索引有效性
if (camera_index < 0 || camera_index >= static_cast<int>(cameras_.size())) {
return;
}
// 获取此相机对应的缓冲区
auto buffer = buffers_[camera_index];
if (!buffer) {
std::cerr << "[CameraCapture] Camera " << camera_index << " buffer invalid"
<< std::endl;
return;
}
// 初始化帧计数和FPS计算相关变量
int frame_count = 0;
auto start_time =
std::chrono::steady_clock::now(); // 记录开始时间，用于计算FPS
int consecutive_timeouts = 0; // 连续超时计数，用于检测相机是否异常
// 使用try-catch捕获异常，确保线程异常不会导致程序崩溃
try {
// 主循环：持续采集图像直到停止标志被设置或相机停止运行
while (running_ && camera_running_[camera_index]) {
// 再次检查相机索引有效性（防止在运行过程中相机被移除）
if (camera_index >= static_cast<int>(cameras_.size()) ||
!cameras_[camera_index]) {
break;
}
// 从相机获取帧数据，超时时间500ms
// tryGetFrames是非阻塞的，如果500ms内没有新帧，返回nullptr
// 注意：根据实际测试，单帧处理时间约155ms，加上相机采集时间，500ms是合理的超时值
// 如果相机帧率很低（<2fps），可以适当增加到1000ms
auto frame = cameras_[camera_index]->tryGetFrames(500);
if (!frame) {
// 获取帧失败（超时或错误）
consecutive_timeouts++;
// 如果连续超时超过10次，输出警告并短暂休眠
// 这样可以减少错误日志的噪音，同时避免CPU占用过高
if (consecutive_timeouts == 10) {
std::cerr << "[CameraCapture] Camera " << camera_index
<< " consecutive timeout 10 times, may be low frame rate or connection issue" << std::endl;
}
if (consecutive_timeouts > 10) {
// 连续超时超过10次后，每次超时都休眠100ms，避免CPU空转
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
continue; // 继续下一次循环
}
// 成功获取帧，重置超时计数
consecutive_timeouts = 0;
frame_count++; // 帧计数加1
// 记录帧获取时间，用于性能分析
auto frame_start_time = std::chrono::steady_clock::now();
// ========== 处理深度图 ==========
cv::Mat depthMat;
auto depth_img = frame->depthImage(); // 从帧中提取深度图
if (depth_img) {
// 如果深度流已启用，使用图像处理器处理原始数据
auto depth_processer = depth_processers_[camera_index];
if (depth_processer) {
// parse()处理原始TYImage数据，可能进行格式转换或校正
depth_processer->parse(depth_img);
// image()返回处理后的TYImage
depth_img = depth_processer->image();
}
// 将TYImage转换为OpenCV的Mat格式
// TYImageToMat内部使用clone()，确保数据安全
depthMat = TYImageToMat(depth_img);
}
// ========== 处理彩色图 ==========
cv::Mat colorMat;
auto color_img = frame->colorImage(); // 从帧中提取彩色图
if (color_img) {
// 如果彩色流已启用，使用图像处理器处理原始数据
auto color_processer = color_processers_[camera_index];
if (color_processer) {
// parse()处理原始TYImage数据
color_processer->parse(color_img);
// image()返回处理后的TYImage
color_img = color_processer->image();
}
// 将TYImage转换为OpenCV的Mat格式
cv::Mat rawColorMat = TYImageToMat(color_img);
if (!rawColorMat.empty()) {
// 获取像素格式标识，用于确定颜色空间转换方式
int pixel_format = getPixelFormatId(color_img->pixelFormat());
// 性能优化：根据像素格式进行颜色空间转换，统一输出BGR格式
// 优化：直接转换到目标Mat，避免中间变量
if (pixel_format == 1) {
// RGB格式，转换为BGR
cv::cvtColor(rawColorMat, colorMat, cv::COLOR_RGB2BGR);
} else if (pixel_format == 2 || pixel_format == 3) {
// YUYV或YVYU格式（YUV422），转换为BGR
// 注意：YVYU和YUYV使用相同的转换代码
cv::cvtColor(rawColorMat, colorMat, cv::COLOR_YUV2BGR_YUYV);
} else {
// BGR格式或其他，直接使用（假设已经是BGR）
// 性能优化：使用move语义，避免不必要的复制
colorMat = std::move(rawColorMat);
}
}
}
// ========== 计算FPS ==========
// 使用已采集的帧数和经过的时间计算平均帧率
auto current_time = std::chrono::steady_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
current_time - start_time)
.count();
double fps = 0.0;
if (elapsed > 0) {
// FPS = 帧数 * 1000 / 经过的毫秒数
fps = (frame_count * 1000.0) / elapsed;
}
// ========== 更新缓冲区 ==========
// 使用互斥锁保护缓冲区，确保线程安全
// 注意：这里使用独立的lock_guard作用域，确保锁在更新完成后立即释放
{
std::lock_guard<std::mutex> lock(buffer->mtx);
// 更新深度图缓冲区
// 性能优化：使用swap代替copyTo，避免数据复制，直接交换指针
if (!depthMat.empty()) {
// 使用swap交换数据，这是零拷贝操作，只交换内部指针
std::swap(buffer->depth, depthMat);
} else {
// 如果深度图为空，清空缓冲区
buffer->depth = cv::Mat();
}
// 更新彩色图缓冲区
// 性能优化：使用swap代替copyTo，避免数据复制
if (!colorMat.empty()) {
// 使用swap交换数据，这是零拷贝操作，只交换内部指针
std::swap(buffer->color, colorMat);
} else {
// 如果彩色图为空，清空缓冲区
buffer->color = cv::Mat();
}
// 更新FPS和帧计数
buffer->fps = fps;
buffer->frame_count = frame_count;
buffer->updated = true; // 标记缓冲区已更新
}
// lock_guard在这里自动释放锁
// 性能监控：每100帧输出一次处理时间（可选，用于调试）
if (frame_count % 100 == 0) {
auto frame_end_time = std::chrono::steady_clock::now();
auto frame_process_time =
std::chrono::duration_cast<std::chrono::milliseconds>(
frame_end_time - frame_start_time)
.count();
// if (frame_process_time > 50) { // 如果单帧处理时间超过50ms，输出警告
// std::cout << "[CameraCapture] 相机 " << camera_index
// << " 单帧处理时间: " << frame_process_time << "ms"
// << std::endl;
// }
}
} // while循环结束
} catch (const std::exception &e) {
// 捕获标准异常，记录错误信息
std::cerr << "[CameraCapture] Camera " << camera_index << " capture thread exception: "
<< e.what() << std::endl;
} catch (...) {
// 捕获所有其他异常（非标准异常）
std::cerr << "[CameraCapture] Camera " << camera_index
<< " capture thread unknown exception" << std::endl;
}
// 线程函数结束，线程自动退出
}
/**
* @brief 将TYImage转换为OpenCV的Mat格式
*
* 此函数将SDK的TYImage格式转换为OpenCV的cv::Mat格式
* 关键点：
* 1. 根据像素格式确定OpenCV的Mat类型
* 2. 创建临时Mat包装原始缓冲区（零拷贝视图）
* 3. 使用clone()创建数据副本，确保数据安全
*
* @param img SDK的TYImage智能指针
* @return cv::Mat OpenCV格式的图像矩阵，如果输入无效则返回空Mat
*
* @note 使用clone()创建数据副本是必要的，因为：
* - TYImage的数据可能在frame对象销毁后失效
* - 如果不clone，返回的Mat会引用已释放的内存，导致悬空指针
* - clone()虽然增加内存和CPU开销，但确保了数据安全
*
* @note 支持的像素格式：
* - TY_PIXEL_FORMAT_DEPTH16: 16位深度图 -> CV_16U
* - TY_PIXEL_FORMAT_RGB: RGB彩色图 -> CV_8UC3
* - TY_PIXEL_FORMAT_BGR: BGR彩色图 -> CV_8UC3
* - TY_PIXEL_FORMAT_MONO: 单色图 -> CV_8U
* - TY_PIXEL_FORMAT_YUYV/YVYU: YUV422格式 -> CV_8UC2
*/
cv::Mat CameraCapture::TYImageToMat(const std::shared_ptr<TYImage> &img) {
// 检查输入有效性
if (!img || !img->buffer())
return cv::Mat();
// 根据SDK的像素格式确定OpenCV的Mat数据类型
int type = -1;
switch (img->pixelFormat()) {
case TY_PIXEL_FORMAT_DEPTH16:
type = CV_16U; // 16位无符号整数，用于深度值
break;
case TY_PIXEL_FORMAT_RGB:
type = CV_8UC3; // 8位无符号整数，3通道（RGB）
break;
case TY_PIXEL_FORMAT_MONO:
type = CV_8U; // 8位无符号整数，单通道（灰度）
break;
case TY_PIXEL_FORMAT_YVYU:
case TY_PIXEL_FORMAT_YUYV:
type = CV_8UC2; // 8位无符号整数，2通道（YUV422）
break;
case TY_PIXEL_FORMAT_BGR:
type = CV_8UC3; // 8位无符号整数，3通道（BGR）
break;
default:
type = CV_8U; // 默认单通道
break;
}
// 创建临时Mat对象，直接包装原始缓冲区（零拷贝）
// 注意：这只是创建一个视图，不复制数据
// 参数：高度、宽度、数据类型、原始数据指针
cv::Mat tempMat(img->height(), img->width(), type, img->buffer());
// 使用clone()创建数据副本
// 这是关键步骤：确保返回的Mat拥有独立的数据副本
// 即使原始的TYImage被销毁，返回的Mat仍然有效
// 虽然会增加内存和CPU开销，但这是确保数据安全的必要代价
return tempMat.clone();
}
/**
* @brief 获取像素格式标识
*
* 将SDK的像素格式枚举转换为简单的整数标识
* 用于后续的颜色空间转换判断
*
* @param pixel_format SDK的像素格式枚举值
* @return 像素格式标识：
* - 0: BGR格式（或默认）
* - 1: RGB格式
* - 2: YUYV格式（YUV422）
* - 3: YVYU格式（YUV422）
*
* @note 此函数用于简化颜色空间转换的判断逻辑
*/
int CameraCapture::getPixelFormatId(TY_PIXEL_FORMAT pixel_format) {
switch (pixel_format) {
case TY_PIXEL_FORMAT_RGB:
return 1; // RGB格式
case TY_PIXEL_FORMAT_YUYV:
return 2; // YUYV格式（YUV422）
case TY_PIXEL_FORMAT_YVYU:
return 3; // YVYU格式（YUV422）
case TY_PIXEL_FORMAT_BGR:
default:
return 0; // BGR格式或默认
}
}
/**
* @brief 利用SDK生成点云
* @param camera_index 相机索引
* @param depth_img 深度图
* @param out_points 输出点云
* @return 是否成功
*/
bool CameraCapture::computePointCloud(int camera_index, const cv::Mat& depth_img, std::vector<Point3D>& out_points) {
if (camera_index < 0 || camera_index >= static_cast<int>(cameras_.size())) {
return false;
}
if (!has_calib_info_[camera_index]) {
std::cerr << "[CameraCapture] No calibration info for camera " << camera_index << std::endl;
return false;
}
if (depth_img.empty()) {
return false;
}
// Check for valid intrinsics to prevent division by zero crash
float fx = calib_infos_[camera_index].intrinsic.data[0];
float fy = calib_infos_[camera_index].intrinsic.data[4];
if (std::abs(fx) < 1e-6 || std::abs(fy) < 1e-6) {
std::cerr << "[CameraCapture] Invalid intrinsics for camera " << camera_index
<< " (fx=" << fx << ", fy=" << fy << "). Cannot compute point cloud." << std::endl;
return false;
}
// 调整输出容器大小
out_points.resize(depth_img.cols * depth_img.rows);
// TY_VECT_3F {float x, y, z} 与 Point3D {float x, y, z} 内存布局兼容
// 直接使用 SDK 函数生成点云
TY_VECT_3F* p3d = reinterpret_cast<TY_VECT_3F*>(out_points.data());
TY_STATUS status = TYMapDepthImageToPoint3d(&calib_infos_[camera_index],
depth_img.cols, depth_img.rows,
(const uint16_t*)depth_img.data,
p3d);
if (status != TY_STATUS_OK) {
std::cerr << "[CameraCapture] TYMapDepthImageToPoint3d failed: " << status << std::endl;
return false;
}
return true;
}

159
image_capture/src/camera/ty_multi_camera_capture.h Normal file
View File

@@ -0,0 +1,159 @@
#pragma once
#include "Frame.hpp"
#include "Device.hpp"
#include "TYApi.h"
#include <opencv2/opencv.hpp>
#include <vector>
#include <memory>
#include <thread>
#include <atomic>
#include <mutex>
#include <string>
#include "../common_types.h"
using namespace percipio_layer;
/**
* @brief CameraCapture
* 图像采集层，负责从SDK获取图像并转换为OpenCV格式
*
* 功能说明：
* - 封装TY相机SDK，管理多相机采集
* - 将SDK的TYImage转换为OpenCV的cv::Mat格式
* - 管理采集线程和缓冲区
* - 输出原始cv::Mat格式的图像，供上层使用
*
* 设计原则：
* - 此模块属于图像采集层，可以依赖SDK
* - 输出标准OpenCV格式，实现SDK与算法层的隔离
* - 不进行图像处理（如伪彩色映射等），只负责采集和格式转换
*/
class CameraCapture
{
public:
/**
* @brief 图像缓冲区结构
* 存储原始采集的图像数据（cv::Mat格式）
*/
struct ImageBuffer {
cv::Mat depth; // 原始深度图（CV_16U格式）
cv::Mat color; // 原始彩色图（BGR格式）
double fps = 0.0; // 当前帧率
int frame_count = 0; // 帧计数
std::mutex mtx; // 互斥锁
std::atomic<bool> updated{false}; // 更新标志、std::atomic 确保在多线程环境中对 updated 的操作是原子的，不会发生竞争条件。
};
CameraCapture();
~CameraCapture();
/**
* 初始化并配置相机
* @param enable_depth 是否启用深度流
* @param enable_color 是否启用彩色流
* @return 是否成功
*/
bool initialize(bool enable_depth = true, bool enable_color = true);
/**
* 启动采集
* @return 是否成功
*/
bool start();
/**
* 停止采集
*/
void stop();
/**
* 获取相机数量
* @return 相机数量
*/
int getCameraCount() const;
/**
* 获取相机ID
* @param index 相机索引
* @return 相机ID字符串
*/
std::string getCameraId(int index) const;
/**
* 获取指定相机的最新图像
* @param camera_index 相机索引
* @param depth 输出的深度图（CV_16U格式，原始深度值）
* @param color 输出的彩色图（BGR格式）
* @param fps 输出的帧率
* @return 是否成功获取到图像
*/
bool getLatestImages(int camera_index, cv::Mat& depth, cv::Mat& color, double& fps);
/**
* 检查是否正在运行
* @return 是否运行中
*/
bool isRunning() const;
/**
* 获取指定相机的深度相机内参
* @param camera_index 相机索引
* @param fx [输出] 焦距x
* @param fy [输出] 焦距y
* @param cx [输出] 主点x
* @param cy [输出] 主点y
* @return 是否成功获取内参
*/
// Added method for depth camera intrinsics
bool getDepthCameraIntrinsics(int camera_index, float& fx, float& fy, float& cx, float& cy);
/**
* @brief 利用SDK生成点云
* @param camera_index 相机索引
* @param depth_img 深度图
* @param out_points 输出点云
* @return 是否成功
*/
bool computePointCloud(int camera_index, const cv::Mat& depth_img, std::vector<Point3D>& out_points);
private:
/**
* 采集线程函数
* @param camera_index 相机索引
*/
void captureThreadFunc(int camera_index);
/**
* 将TYImage转换为OpenCV的Mat格式
* @param img 输入的TYImage智能指针
* @return cv::Mat OpenCV格式的图像矩阵
*/
static cv::Mat TYImageToMat(const std::shared_ptr<TYImage> &img);
/**
* 获取像素格式标识，用于颜色空间转换
* @param pixel_format SDK的像素格式枚举
* @return 像素格式标识（0: BGR, 1: RGB, 2: YUYV, 3: YVYU）
*/
static int getPixelFormatId(TY_PIXEL_FORMAT pixel_format);
// SDK相关成员（原MultiCameraCapture的功能）
std::vector<std::shared_ptr<FastCamera>> cameras_; // 相机对象列表
std::vector<std::shared_ptr<ImageProcesser>> depth_processers_; // 深度图像处理器
std::vector<std::shared_ptr<ImageProcesser>> color_processers_; // 彩色图像处理器
std::vector<bool> camera_running_; // 相机运行状态
bool streams_configured_; // 流是否已配置
bool depth_enabled_; // 是否启用深度流
bool color_enabled_; // 是否启用彩色流
// 采集线程和缓冲区
std::vector<std::shared_ptr<ImageBuffer>> buffers_; // 图像缓冲区
std::vector<std::thread> capture_threads_; // 采集线程
std::atomic<bool> running_; // 运行标志
// 标定信息
std::vector<TY_CAMERA_CALIB_INFO> calib_infos_;
std::vector<bool> has_calib_info_;
};

734
image_capture/src/common/config_manager.cpp Normal file
View File

@@ -0,0 +1,734 @@
#include "config_manager.h"
#include <fstream>
#include <iostream>
#include <sstream>
ConfigManager &ConfigManager::getInstance() {
static ConfigManager instance;
return instance;
}
ConfigManager::ConfigManager() {
// 默认配置
config_json_ = json11::Json::object{};
}
bool ConfigManager::loadConfig(const std::string &config_path) {
std::lock_guard<std::mutex> lock(mutex_);
std::ifstream file(config_path);
if (!file.is_open()) {
std::cerr << "ConfigManager: Failed to open config file: " << config_path
<< std::endl;
return false;
}
std::stringstream buffer;
buffer << file.rdbuf();
std::string content = buffer.str();
std::string err;
config_json_ = json11::Json::parse(content, err);
if (!err.empty()) {
std::cerr << "ConfigManager: Failed to parse JSON: " << err << std::endl;
return false;
}
std::cout << "ConfigManager: Successfully loaded config from " << config_path
<< std::endl;
return true;
}
bool ConfigManager::saveConfig(const std::string &config_path) {
std::lock_guard<std::mutex> lock(mutex_);
std::string json_str = config_json_.dump();
std::ofstream file(config_path);
if (!file.is_open()) {
std::cerr << "ConfigManager: Failed to open config file for writing: " << config_path
<< std::endl;
return false;
}
file << json_str;
file.close();
std::cout << "ConfigManager: Successfully saved config to " << config_path << std::endl;
return true;
}
// --- Accessors & Setters ---
std::string ConfigManager::getRedisHost() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["redis"].is_object()) {
return config_json_["redis"]["host"].string_value();
}
return "127.0.0.1"; // Default
}
int ConfigManager::getRedisPort() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["redis"].is_object()) {
int port = config_json_["redis"]["port"].int_value();
return port > 0 ? port : 6379;
}
return 6379;
}
int ConfigManager::getRedisDb() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["redis"].is_object()) {
return config_json_["redis"]["db"].int_value();
}
return 0;
}
bool ConfigManager::isDepthEnabled() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["cameras"].is_object()) {
auto val = config_json_["cameras"]["depth_enabled"];
if (val.is_bool())
return val.bool_value();
}
return true; // Default
}
bool ConfigManager::isColorEnabled() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["cameras"].is_object()) {
auto val = config_json_["cameras"]["color_enabled"];
if (val.is_bool())
return val.bool_value();
}
return true; // Default
}
std::vector<ConfigManager::CameraMapping>
ConfigManager::getCameraMappings() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<CameraMapping> mappings;
if (config_json_["cameras"].is_object() &&
config_json_["cameras"]["mapping"].is_array()) {
for (const auto &item : config_json_["cameras"]["mapping"].array_items()) {
CameraMapping m;
m.id = item["id"].string_value();
m.index = item["index"].int_value();
mappings.push_back(m);
}
}
return mappings;
}
std::string ConfigManager::getSavePath() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["vision"].is_object()) {
return config_json_["vision"]["save_path"].string_value();
}
return "./";
}
int ConfigManager::getLogLevel() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["vision"].is_object()) {
return config_json_["vision"]["log_level"].int_value();
}
return 0;
}
// --- Algorithm Config Accessors ---
// Beam/Rack Deflection - ROI Points
std::vector<cv::Point2i> ConfigManager::getBeamROIPoints() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<cv::Point2i> points;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"]["beam_roi_points"].is_array()) {
const auto& points_array = config_json_["algorithms"]["beam_rack_deflection"]["beam_roi_points"].array_items();
for (const auto& point : points_array) {
if (point.is_object()) {
int x = point["x"].int_value();
int y = point["y"].int_value();
points.push_back(cv::Point2i(x, y));
}
}
}
// Default values if not configured
if (points.empty()) {
points = {
cv::Point2i(100, 50),
cv::Point2i(540, 80),
cv::Point2i(540, 280),
cv::Point2i(100, 280)
};
}
return points;
}
std::vector<cv::Point2i> ConfigManager::getRackROIPoints() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<cv::Point2i> points;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"]["rack_roi_points"].is_array()) {
const auto& points_array = config_json_["algorithms"]["beam_rack_deflection"]["rack_roi_points"].array_items();
for (const auto& point : points_array) {
if (point.is_object()) {
int x = point["x"].int_value();
int y = point["y"].int_value();
points.push_back(cv::Point2i(x, y));
}
}
}
// Default values if not configured
if (points.empty()) {
points = {
cv::Point2i(50, 50),
cv::Point2i(150, 50),
cv::Point2i(150, 430),
cv::Point2i(50, 430)
};
}
return points;
}
// Beam/Rack Deflection - Thresholds
std::vector<float> ConfigManager::getBeamThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"]["beam_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["beam_rack_deflection"]["beam_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
// Default values if not configured
if (thresholds.size() != 4) {
thresholds = {-10.0f, -5.0f, 5.0f, 10.0f};
}
return thresholds;
}
std::vector<float> ConfigManager::getRackThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"].is_object() &&
config_json_["algorithms"]["beam_rack_deflection"]["rack_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["beam_rack_deflection"]["rack_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
// Default values if not configured
if (thresholds.size() != 4) {
thresholds = {-6.0f, -3.0f, 3.0f, 6.0f};
}
return thresholds;
}
// Pallet Offset - Thresholds
std::vector<float> ConfigManager::getPalletOffsetLatThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["pallet_offset"].is_object() &&
config_json_["algorithms"]["pallet_offset"]["offset_lat_mm_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["pallet_offset"]["offset_lat_mm_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
if (thresholds.size() != 4) {
thresholds = {-20.0f, -10.0f, 10.0f, 20.0f};
}
return thresholds;
}
std::vector<float> ConfigManager::getPalletOffsetLonThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["pallet_offset"].is_object() &&
config_json_["algorithms"]["pallet_offset"]["offset_lon_mm_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["pallet_offset"]["offset_lon_mm_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
if (thresholds.size() != 4) {
thresholds = {-20.0f, -10.0f, 10.0f, 20.0f};
}
return thresholds;
}
std::vector<float> ConfigManager::getPalletRotationAngleThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["pallet_offset"].is_object() &&
config_json_["algorithms"]["pallet_offset"]["rotation_angle_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["pallet_offset"]["rotation_angle_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
if (thresholds.size() != 4) {
thresholds = {-5.0f, -2.5f, 2.5f, 5.0f};
}
return thresholds;
}
std::vector<float> ConfigManager::getPalletHoleDefLeftThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["pallet_offset"].is_object() &&
config_json_["algorithms"]["pallet_offset"]["hole_def_mm_left_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["pallet_offset"]["hole_def_mm_left_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
if (thresholds.size() != 4) {
thresholds = {-8.0f, -4.0f, 4.0f, 8.0f};
}
return thresholds;
}
std::vector<float> ConfigManager::getPalletHoleDefRightThresholds() const {
std::lock_guard<std::mutex> lock(mutex_);
std::vector<float> thresholds;
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["pallet_offset"].is_object() &&
config_json_["algorithms"]["pallet_offset"]["hole_def_mm_right_thresholds"].is_object()) {
const auto& thresh = config_json_["algorithms"]["pallet_offset"]["hole_def_mm_right_thresholds"];
thresholds.push_back(static_cast<float>(thresh["A"].number_value()));
thresholds.push_back(static_cast<float>(thresh["B"].number_value()));
thresholds.push_back(static_cast<float>(thresh["C"].number_value()));
thresholds.push_back(static_cast<float>(thresh["D"].number_value()));
}
if (thresholds.size() != 4) {
thresholds = {-8.0f, -4.0f, 4.0f, 8.0f};
}
return thresholds;
}
// Slot Occupancy
float ConfigManager::getSlotOccupancyDepthThreshold() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["slot_occupancy"].is_object()) {
return static_cast<float>(config_json_["algorithms"]["slot_occupancy"]["depth_threshold_mm"].number_value());
}
return 100.0f; // Default
}
float ConfigManager::getSlotOccupancyConfidenceThreshold() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["slot_occupancy"].is_object()) {
return static_cast<float>(config_json_["algorithms"]["slot_occupancy"]["confidence_threshold"].number_value());
}
return 0.8f; // Default
}
// Visual Inventory
float ConfigManager::getVisualInventoryBarcodeConfidence() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["visual_inventory"].is_object()) {
return static_cast<float>(config_json_["algorithms"]["visual_inventory"]["barcode_confidence_threshold"].number_value());
}
return 0.7f; // Default
}
bool ConfigManager::getVisualInventoryROIEnabled() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["visual_inventory"].is_object()) {
return config_json_["algorithms"]["visual_inventory"]["roi_enabled"].bool_value();
}
return true; // Default
}
// General Algorithm Parameters
float ConfigManager::getAlgorithmMinDepth() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["general"].is_object()) {
return static_cast<float>(config_json_["algorithms"]["general"]["min_depth_mm"].number_value());
}
return 800.0f; // Default
}
float ConfigManager::getAlgorithmMaxDepth() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["general"].is_object()) {
return static_cast<float>(config_json_["algorithms"]["general"]["max_depth_mm"].number_value());
}
return 3000.0f; // Default
}
int ConfigManager::getAlgorithmSamplePoints() const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_["algorithms"].is_object() &&
config_json_["algorithms"]["general"].is_object()) {
return config_json_["algorithms"]["general"]["sample_points"].int_value();
}
return 50; // Default
}
// Beam/Rack Deflection - Setters
void ConfigManager::setBeamROIPoints(const std::vector<cv::Point2i>& points) {
std::lock_guard<std::mutex> lock(mutex_);
// Note: json11 is immutable, we need to reconstruct the object or use a mutable json library.
// For this project using json11, we have to rebuild the part of the json tree.
// This is a bit expensive but config saving is rare.
// Actually, to make it easier with json11, we might need to parse, modify and dump if we want to keep comments?
// But json11 parser doesn't keep comments.
// Let's just modify the internal map if possible or rebuild.
// json11::Json is const. We need to cast it away or rebuild the whole structure?
// Rebuilding is safer.
// Implementation Note: Since json11 is immutable, proper way is to create new Json objects.
// For simplicity in this context, we will use a "deep update" strategy helper if we had one.
// But here we need to do it manually.
// Let's cheat a bit and use const_cast for the "value" if it was a simpler lib, but json11 uses shared_ptr...
// Okay, we will use a temporary mutable map approach for the 'algorithms' section.
// Helper to get mutable map from Json object
auto get_mutable_map = [](const json11::Json& j) -> json11::Json::object {
return j.object_items();
};
json11::Json::object root_map = get_mutable_map(config_json_);
json11::Json::object algo_map = get_mutable_map(root_map["algorithms"]);
json11::Json::object beam_rack_map = get_mutable_map(algo_map["beam_rack_deflection"]);
std::vector<json11::Json> points_json;
for(const auto& p : points) {
points_json.push_back(json11::Json::object{{"x", p.x}, {"y", p.y}});
}
beam_rack_map["beam_roi_points"] = points_json;
algo_map["beam_rack_deflection"] = beam_rack_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setRackROIPoints(const std::vector<cv::Point2i>& points) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object beam_rack_map = algo_map["beam_rack_deflection"].object_items();
std::vector<json11::Json> points_json;
for(const auto& p : points) {
points_json.push_back(json11::Json::object{{"x", p.x}, {"y", p.y}});
}
beam_rack_map["rack_roi_points"] = points_json;
algo_map["beam_rack_deflection"] = beam_rack_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setBeamThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object beam_rack_map = algo_map["beam_rack_deflection"].object_items();
beam_rack_map["beam_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["beam_rack_deflection"] = beam_rack_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setRackThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object beam_rack_map = algo_map["beam_rack_deflection"].object_items();
beam_rack_map["rack_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["beam_rack_deflection"] = beam_rack_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
// Pallet Offset Setters
void ConfigManager::setPalletOffsetLatThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object pallet_map = algo_map["pallet_offset"].object_items();
pallet_map["offset_lat_mm_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["pallet_offset"] = pallet_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setPalletOffsetLonThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object pallet_map = algo_map["pallet_offset"].object_items();
pallet_map["offset_lon_mm_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["pallet_offset"] = pallet_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setPalletRotationAngleThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object pallet_map = algo_map["pallet_offset"].object_items();
pallet_map["rotation_angle_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["pallet_offset"] = pallet_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setPalletHoleDefLeftThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object pallet_map = algo_map["pallet_offset"].object_items();
pallet_map["hole_def_mm_left_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["pallet_offset"] = pallet_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setPalletHoleDefRightThresholds(const std::vector<float>& thresholds) {
if(thresholds.size() < 4) return;
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object pallet_map = algo_map["pallet_offset"].object_items();
pallet_map["hole_def_mm_right_thresholds"] = json11::Json::object{
{"A", thresholds[0]}, {"B", thresholds[1]}, {"C", thresholds[2]}, {"D", thresholds[3]}
};
algo_map["pallet_offset"] = pallet_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
// Slot Occupancy Setters
void ConfigManager::setSlotOccupancyDepthThreshold(float value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object slot_map = algo_map["slot_occupancy"].object_items();
slot_map["depth_threshold_mm"] = value;
algo_map["slot_occupancy"] = slot_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setSlotOccupancyConfidenceThreshold(float value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object slot_map = algo_map["slot_occupancy"].object_items();
slot_map["confidence_threshold"] = value;
algo_map["slot_occupancy"] = slot_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
// Visual Inventory Setters
void ConfigManager::setVisualInventoryBarcodeConfidence(float value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object vis_map = algo_map["visual_inventory"].object_items();
vis_map["barcode_confidence_threshold"] = value;
algo_map["visual_inventory"] = vis_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setVisualInventoryROIEnabled(bool value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object vis_map = algo_map["visual_inventory"].object_items();
vis_map["roi_enabled"] = value;
algo_map["visual_inventory"] = vis_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
// General Setters
void ConfigManager::setAlgorithmMinDepth(float value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object gen_map = algo_map["general"].object_items();
gen_map["min_depth_mm"] = value;
algo_map["general"] = gen_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setAlgorithmMaxDepth(float value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object gen_map = algo_map["general"].object_items();
gen_map["max_depth_mm"] = value;
algo_map["general"] = gen_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
void ConfigManager::setAlgorithmSamplePoints(int value) {
std::lock_guard<std::mutex> lock(mutex_);
json11::Json::object root_map = config_json_.object_items();
json11::Json::object algo_map = root_map["algorithms"].object_items();
json11::Json::object gen_map = algo_map["general"].object_items();
gen_map["sample_points"] = value;
algo_map["general"] = gen_map;
root_map["algorithms"] = algo_map;
config_json_ = root_map;
}
// Generic access
std::string ConfigManager::getString(const std::string &key,
const std::string &default_value) const {
std::lock_guard<std::mutex> lock(mutex_);
// Simple top-level access, or implement dot notation parsing if needed.
// For now assuming top level.
if (config_json_[key].is_string()) {
return config_json_[key].string_value();
}
return default_value;
}
int ConfigManager::getInt(const std::string &key, int default_value) const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_[key].is_number()) {
return config_json_[key].int_value();
}
return default_value;
}
bool ConfigManager::getBool(const std::string &key, bool default_value) const {
std::lock_guard<std::mutex> lock(mutex_);
if (config_json_[key].is_bool()) {
return config_json_[key].bool_value();
}
return default_value;
}

124
image_capture/src/common/config_manager.h Normal file
View File

@@ -0,0 +1,124 @@
#pragma once
#include "json11.hpp"
#include <map>
#include <mutex>
#include <string>
#include <vector>
#include <opencv2/core.hpp>
/**
* @brief ConfigManager
* 全局配置管理器，单例模式
* 负责加载和提供系统配置参数
*/
class ConfigManager {
public:
static ConfigManager &getInstance();
// 禁止拷贝
ConfigManager(const ConfigManager &) = delete;
ConfigManager &operator=(const ConfigManager &) = delete;
/**
* 加载配置文件
* @param config_path 配置文件路径，默认在当前目录查找 config.json
* @return 是否成功加载
*/
bool loadConfig(const std::string &config_path = "config.json");
/**
* 保存配置文件
* @param config_path 配置文件路径，默认在当前目录查找 config.json
* @return 是否成功保存
*/
bool saveConfig(const std::string &config_path = "config.json");
// --- Accessors & Setters ---
// Redis Config
std::string getRedisHost() const;
int getRedisPort() const;
int getRedisDb() const;
// Camera Config
bool isDepthEnabled() const;
bool isColorEnabled() const;
struct CameraMapping {
std::string id;
int index;
};
std::vector<CameraMapping> getCameraMappings() const;
// Vision/Global Config
std::string getSavePath() const;
int getLogLevel() const;
// Algorithm Config - Beam/Rack Deflection
std::vector<cv::Point2i> getBeamROIPoints() const;
void setBeamROIPoints(const std::vector<cv::Point2i>& points);
std::vector<cv::Point2i> getRackROIPoints() const;
void setRackROIPoints(const std::vector<cv::Point2i>& points);
std::vector<float> getBeamThresholds() const; // Returns [A, B, C, D]
void setBeamThresholds(const std::vector<float>& thresholds);
std::vector<float> getRackThresholds() const; // Returns [A, B, C, D]
void setRackThresholds(const std::vector<float>& thresholds);
// Algorithm Config - Pallet Offset
std::vector<float> getPalletOffsetLatThresholds() const;
void setPalletOffsetLatThresholds(const std::vector<float>& thresholds);
std::vector<float> getPalletOffsetLonThresholds() const;
void setPalletOffsetLonThresholds(const std::vector<float>& thresholds);
std::vector<float> getPalletRotationAngleThresholds() const;
void setPalletRotationAngleThresholds(const std::vector<float>& thresholds);
std::vector<float> getPalletHoleDefLeftThresholds() const;
void setPalletHoleDefLeftThresholds(const std::vector<float>& thresholds);
std::vector<float> getPalletHoleDefRightThresholds() const;
void setPalletHoleDefRightThresholds(const std::vector<float>& thresholds);
// Algorithm Config - Slot Occupancy
float getSlotOccupancyDepthThreshold() const;
void setSlotOccupancyDepthThreshold(float value);
float getSlotOccupancyConfidenceThreshold() const;
void setSlotOccupancyConfidenceThreshold(float value);
// Algorithm Config - Visual Inventory
float getVisualInventoryBarcodeConfidence() const;
void setVisualInventoryBarcodeConfidence(float value);
bool getVisualInventoryROIEnabled() const;
void setVisualInventoryROIEnabled(bool value);
// Algorithm Config - General
float getAlgorithmMinDepth() const;
void setAlgorithmMinDepth(float value);
float getAlgorithmMaxDepth() const;
void setAlgorithmMaxDepth(float value);
int getAlgorithmSamplePoints() const;
void setAlgorithmSamplePoints(int value);
// Generic access (for dynamic access)
std::string getString(const std::string &key,
const std::string &default_value = "") const;
int getInt(const std::string &key, int default_value = 0) const;
bool getBool(const std::string &key, bool default_value = false) const;
private:
ConfigManager();
~ConfigManager() = default;
json11::Json config_json_;
mutable std::mutex mutex_;
};

104
image_capture/src/common/log_manager.cpp Normal file
View File

@@ -0,0 +1,104 @@
#include "log_manager.h"
#include <iostream>
LogManager &LogManager::getInstance() {
static LogManager instance;
return instance;
}
void LogManager::setCallback(LogCallback callback) {
std::lock_guard<std::mutex> lock(mutex_);
callback_ = callback;
}
#include "config_manager.h"
#include <cstdarg>
#include <cstdio>
#include <ctime>
#include <iomanip>
#include <sstream>
#include <vector>
void LogManager::logFormat(LogLevel level, const char *fmt, ...) {
// 1. Level Filter
if (static_cast<int>(level) < ConfigManager::getInstance().getLogLevel()) {
return;
}
// 2. Format Message
va_list args;
va_start(args, fmt);
// Determine required size
va_list args_copy;
va_copy(args_copy, args);
int size = std::vsnprintf(nullptr, 0, fmt, args_copy);
va_end(args_copy);
if (size < 0) {
va_end(args);
return; // Encoding error
}
std::vector<char> buffer(size + 1);
std::vsnprintf(buffer.data(), buffer.size(), fmt, args);
va_end(args);
std::string message(buffer.data(), size);
// 3. Delegate
logInternal(level, message);
}
void LogManager::logInternal(LogLevel level, const std::string &message) {
// 1. Add Timestamp and Level Prefix
const char *levelStr = "[INFO] ";
bool isError = false;
switch (level) {
case LogLevel::DEBUG:
levelStr = "[DEBUG] ";
break;
case LogLevel::INFO:
levelStr = "[INFO] ";
break;
case LogLevel::WARNING:
levelStr = "[WARN] ";
break;
case LogLevel::ERROR:
levelStr = "[ERROR] ";
isError = true;
break;
}
auto t = std::time(nullptr);
auto tm = *std::localtime(&t);
std::ostringstream oss;
oss << std::put_time(&tm, "[%Y-%m-%d %H:%M:%S] ") << levelStr << message
<< "\n";
std::string formattedMsg = oss.str();
// 2. Output
std::lock_guard<std::mutex> lock(mutex_);
if (callback_) {
callback_(formattedMsg, isError);
} else {
if (isError) {
std::cerr << formattedMsg;
} else {
std::cout << formattedMsg;
}
}
}
// 兼容旧接口
void LogManager::log(const std::string &message, bool isError) {
logInternal(isError ? LogLevel::ERROR : LogLevel::INFO, message);
}
void LogManager::clearCallback() {
std::lock_guard<std::mutex> lock(mutex_);
callback_ = nullptr;
}

80
image_capture/src/common/log_manager.h Normal file
View File

@@ -0,0 +1,80 @@
#pragma once
#include <functional>
#include <memory>
#include <mutex>
#include <string>
/**
* @brief 全局日志管理器（单例模式）
*
* 用于将标准输出（cout/cerr）重定向到Qt日志系统
* 任何模块都可以通过LogManager输出日志，这些日志会被重定向到MainWindow的日志文本框
*/
// 日志级别枚举
enum class LogLevel { DEBUG = 0, INFO = 1, WARNING = 2, ERROR = 3 };
/**
* @brief 全局日志管理器（单例模式）
*
* 用于将标准输出（cout/cerr）重定向到Qt日志系统
* 支持格式化输出和日志级别过滤
*/
class LogManager {
public:
// 日志回调函数类型
using LogCallback =
std::function<void(const std::string &message, bool isError)>;
/**
* @brief 获取单例实例
*/
static LogManager &getInstance();
/**
* @brief 设置日志回调函数
* @param callback 回调函数，接收日志消息和错误标志
*/
void setCallback(LogCallback callback);
/**
* @brief 格式化并输出日志消息
* @param level 日志级别
* @param fmt 格式化字符串 (printf style)
* @param ... 可变参数
*/
void logFormat(LogLevel level, const char *fmt, ...);
/**
* @brief 兼容旧接口的日志输出
* @param message 日志消息
* @param isError 是否为错误消息
*/
void log(const std::string &message, bool isError = false);
/**
* @brief 清除回调函数
*/
void clearCallback();
private:
LogManager() = default;
~LogManager() = default;
// 内部实际执行日志输出的方法
void logInternal(LogLevel level, const std::string &message);
LogCallback callback_;
std::mutex mutex_; // 保护回调函数的线程安全
};
// 宏定义以便于使用
#define LOG_DEBUG(fmt, ...) \
LogManager::getInstance().logFormat(LogLevel::DEBUG, fmt, ##__VA_ARGS__)
#define LOG_INFO(fmt, ...) \
LogManager::getInstance().logFormat(LogLevel::INFO, fmt, ##__VA_ARGS__)
#define LOG_WARN(fmt, ...) \
LogManager::getInstance().logFormat(LogLevel::WARNING, fmt, ##__VA_ARGS__)
#define LOG_ERROR(fmt, ...) \
LogManager::getInstance().logFormat(LogLevel::ERROR, fmt, ##__VA_ARGS__)

39
image_capture/src/common/log_streambuf.h Normal file
View File

@@ -0,0 +1,39 @@
#pragma once
#include <streambuf>
#include <string>
#include "log_manager.h"
/**
* @brief 自定义streambuf，用于重定向cout/cerr到LogManager
*/
class LogStreamBuf : public std::streambuf {
public:
LogStreamBuf(bool isError) : isError_(isError), buffer_() {}
protected:
virtual int_type overflow(int_type c) override {
if (c != EOF) {
buffer_ += static_cast<char>(c);
if (c == '\n') {
// 遇到换行符，输出完整行
LogManager::getInstance().log(buffer_, isError_);
buffer_.clear();
}
}
return c;
}
virtual int sync() override {
if (!buffer_.empty()) {
LogManager::getInstance().log(buffer_, isError_);
buffer_.clear();
}
return 0;
}
private:
bool isError_;
std::string buffer_;
};

27
image_capture/src/common_types.h Normal file
View File

@@ -0,0 +1,27 @@
#pragma once
#include <vector>
/**
* @brief 点云数据结构
* 表示一个三维点
*/
struct Point3D {
float x, y, z;
Point3D() : x(0), y(0), z(0) {}
Point3D(float x, float y, float z) : x(x), y(y), z(z) {}
};
/**
* @brief 相机内参结构
*/
struct CameraIntrinsics {
float fx; // 焦距x
float fy; // 焦距y
float cx; // 主点x
float cy; // 主点y
CameraIntrinsics() : fx(0), fy(0), cx(0), cy(0) {}
CameraIntrinsics(float fx, float fy, float cx, float cy)
: fx(fx), fy(fy), cx(cx), cy(cy) {}
};

300
image_capture/src/device/device_manager.cpp Normal file
View File

@@ -0,0 +1,300 @@
/**
* @file device_manager.cpp
* @brief 设备管理器实现文件
*
* 此文件实现了DeviceManager类的完整功能：
* - 设备初始化（扫描和配置相机）
* - 设备启动和停止
* - 图像获取接口
* - 设备信息查询
*
* 设计说明：
* - DeviceManager是对CameraCapture的封装，提供统一的设备管理接口
* - 不涉及业务逻辑，只负责设备层的管理
* - 使用智能指针管理CameraCapture，自动释放资源
*/
#include "device_manager.h"
#include "../camera/ty_multi_camera_capture.h"
#include "../camera/mvs_multi_camera_capture.h"
#include <iostream>
/**
* @brief 获取单例实例
*
* @return DeviceManager单例引用-DeviceManager&返回的是实例的引用
*/
DeviceManager& DeviceManager::getInstance() {
static DeviceManager instance; // C++11保证线程安全的单例
return instance;
}
/**
* @brief 构造函数（私有）
*
* 初始化设备管理器，设置初始状态为未初始化
*/
DeviceManager::DeviceManager() : initialized_(false) {}
/**
* @brief 析构函数
*
* 确保在对象销毁时正确停止所有设备
* 调用stopAll()清理资源
*/
DeviceManager::~DeviceManager() {
stopAll();
}
/**
* @brief 初始化并扫描设备
*
* 初始化相机采集模块，扫描并配置所有可用的相机设备
*
* @param enable_depth 是否启用深度流，true表示启用深度图采集
* @param enable_color 是否启用彩色流，true表示启用彩色图采集
* @return 发现的设备数量，0表示初始化失败或未找到设备
*
* @note 如果已经初始化，直接返回当前设备数量（避免重复初始化）
* @note 初始化失败时，capture_会被重置为nullptr
*/
int DeviceManager::initialize(bool enable_depth, bool enable_color) {
// 如果已经初始化，直接返回当前设备数量
if (initialized_) {
return getDeviceCount();
}
int total_count = 0;
// 创建深度相机采集对象
capture_ = std::make_shared<CameraCapture>();
// 初始化深度相机采集（扫描设备、配置流）
if (capture_->initialize(enable_depth, enable_color)) {
total_count += capture_->getCameraCount();
std::cout << "[DeviceManager] Initialized " << capture_->getCameraCount() << " depth camera(s)" << std::endl;
} else {
std::cerr << "[DeviceManager] Failed to initialize depth cameras" << std::endl;
capture_.reset(); // 重置智能指针，释放资源
}
// 初始化MVS 2D相机
mvs_cameras_ = std::make_unique<MvsMultiCameraCapture>();
if (mvs_cameras_->initialize()) {
total_count += mvs_cameras_->getCameraCount();
std::cout << "[DeviceManager] Initialized " << mvs_cameras_->getCameraCount() << " 2D camera(s)" << std::endl;
} else {
std::cout << "[DeviceManager] No 2D cameras found or initialization failed" << std::endl;
mvs_cameras_.reset();
}
// 获取设备数量并标记为已初始化
initialized_ = true;
std::cout << "[DeviceManager] Total devices initialized: " << total_count << std::endl;
return total_count;
}
/**
* @brief 启动所有设备
*
* 启动所有相机的数据采集
*
* @return true 启动成功，false 启动失败（capture_为空或启动失败）
*
* @note 必须先调用initialize()初始化设备
*/
bool DeviceManager::startAll() {
bool success = true;
// 启动深度相机
if (capture_) {
if (!capture_->start()) {
success = false;
std::cerr << "[DeviceManager] Failed to start depth cameras" << std::endl;
}
}
// 启动2D相机
if (mvs_cameras_) {
if (!mvs_cameras_->start()) {
success = false;
std::cerr << "[DeviceManager] Failed to start 2D cameras" << std::endl;
}
}
return success;
}
/**
* @brief 停止所有设备
*
* 停止所有相机的数据采集
*
* @note 此函数是幂等的，可以安全地多次调用
*/
void DeviceManager::stopAll() {
if (capture_) {
capture_->stop();
}
}
/**
* @brief 获取设备数量
*
* @return 当前已初始化的设备数量，0表示未初始化或无设备
*/
int DeviceManager::getDeviceCount() const {
int count = 0;
if (capture_) {
count += capture_->getCameraCount();
}
if (mvs_cameras_) {
count += mvs_cameras_->getCameraCount();
}
return count;
}
int DeviceManager::getDepthCameraCount() const {
return capture_ ? capture_->getCameraCount() : 0;
}
/**
* @brief 获取设备ID
*
* @param index 设备索引，从0开始
* @return 设备ID字符串，如果索引无效或未初始化则返回空字符串
*/
std::string DeviceManager::getDeviceId(int index) const {
int percipio_count = capture_ ? capture_->getCameraCount() : 0;
if (index < percipio_count) {
return capture_->getCameraId(index);
}
int mvs_index = index - percipio_count;
if (mvs_cameras_ && mvs_index >= 0 && mvs_index < mvs_cameras_->getCameraCount()) {
return "2D-" + mvs_cameras_->getCameraId(mvs_index);
}
return "";
}
/**
* @brief 获取指定设备的最新图像
*
* 从设备缓冲区中获取最新采集的图像数据
*
* @param device_index 设备索引，从0开始
* @param depth [输出] 深度图，CV_16U格式，包含原始深度值（单位：毫米）
* @param color [输出] 彩色图，BGR格式，CV_8UC3类型
* @param fps [输出] 当前帧率（帧/秒）
* @return true 成功获取图像，false 获取失败（设备未初始化、索引无效、缓冲区为空）
*
* @note 此函数是线程安全的，使用互斥锁保护缓冲区访问
* @note 如果某个图像流未启用或尚未采集到数据，对应的Mat将为空
*/
bool DeviceManager::getLatestImages(int device_index, cv::Mat& depth, cv::Mat& color, double& fps) {
int percipio_count = capture_ ? capture_->getCameraCount() : 0;
// 深度相机
if (device_index < percipio_count) {
if (!capture_) return false;
return capture_->getLatestImages(device_index, depth, color, fps);
}
// 2D相机
int mvs_index = device_index - percipio_count;
if (mvs_cameras_ && mvs_index >= 0 && mvs_index < mvs_cameras_->getCameraCount()) {
depth = cv::Mat(); // 2D相机没有深度图
return mvs_cameras_->getLatestImage(mvs_index, color, fps);
}
return false;
}
/**
* @brief 检查是否正在运行
*
* @return true 正在运行，false 已停止或未初始化
*/
bool DeviceManager::isRunning() const {
bool anyScaleRunning = capture_ && capture_->isRunning();
bool anyMVSRunning = mvs_cameras_ && mvs_cameras_->isRunning();
return anyScaleRunning || anyMVSRunning;
}
/**
* @brief 获取指定设备的深度相机内参
*
* 从相机SDK获取深度相机的内参（fx, fy, cx, cy）
* 内参存储在相机的标定数据中
*
* @param device_index 设备索引，从0开始
* @param cy [输出] 主点y坐标（像素单位）
* @return 是否成功获取内参
*/
bool DeviceManager::getDepthCameraIntrinsics(int device_index, float& fx, float& fy, float& cx, float& cy) {
if (!capture_) return false;
// 只有深度相机有内参
int percipio_count = capture_->getCameraCount();
if (device_index < percipio_count) {
return capture_->getDepthCameraIntrinsics(device_index, fx, fy, cx, cy);
}
return false;
}
/**
* @brief 利用SDK生成点云
* @param device_index 设备索引
* @param depth_img 深度图
* @param out_points 输出点云
* @return 是否成功
*/
bool DeviceManager::computePointCloud(int device_index, const cv::Mat& depth_img, std::vector<Point3D>& out_points) {
if (!capture_) return false;
// 只有深度相机可以生成点云
int percipio_count = capture_->getCameraCount();
if (device_index < percipio_count) {
return capture_->computePointCloud(device_index, depth_img, out_points);
}
return false;
}
int DeviceManager::get2DCameraCount() const {
return mvs_cameras_ ? mvs_cameras_->getCameraCount() : 0;
}
bool DeviceManager::get2DCameraImage(int camera_index, cv::Mat& image, double& fps) {
if (!mvs_cameras_) {
return false;
}
return mvs_cameras_->getLatestImage(camera_index, image, fps);
}
std::string DeviceManager::get2DCameraId(int camera_index) const {
if (!mvs_cameras_) {
return "";
}
return mvs_cameras_->getCameraId(camera_index);
}

154
image_capture/src/device/device_manager.h Normal file
View File

@@ -0,0 +1,154 @@
#pragma once
// #include "../camera/ty_multi_camera_capture.h" -> Moved to cpp
// #include "../camera/mvs_multi_camera_capture.h" -> Moved to cpp
#include <opencv2/opencv.hpp>
#include <vector>
class CameraCapture;
class MvsMultiCameraCapture;
#include <memory>
#include <string>
#include "../common_types.h"
/**
* @brief DeviceManager
* 设备管理器（Device Manager），负责管理硬件设备（相机、读码器等）
*
* 采用单例模式，确保全局只有一个设备管理器实例
* 任何模块都可以通过getInstance()访问设备
*
* 功能说明：
* - 管理相机采集设备的初始化、启动、停止
* - 管理读码器设备的初始化、启动、停止
* - 提供设备访问接口（获取图像、设备信息等）
* - 支持未来扩展其他设备类型
*
* 职责范围：
* - 设备生命周期管理（初始化、启动、停止）
* - 设备数据获取（图像、设备信息）
* - 不涉及业务逻辑（任务管理、结果处理等）
*/
class DeviceManager {
public:
/**
* 获取单例实例
* @return DeviceManager单例引用
*/
static DeviceManager& getInstance();
// 禁止拷贝和赋值
DeviceManager(const DeviceManager&) = delete;
DeviceManager& operator=(const DeviceManager&) = delete;
~DeviceManager();
/**
* 初始化并扫描设备
* @param enable_depth 是否启用深度流
* @param enable_color 是否启用彩色流
* @return 发现的设备数量
*/
int initialize(bool enable_depth = true, bool enable_color = true);
/**
* 启动所有设备
* @return 是否成功
*/
bool startAll();
/**
* 停止所有设备
*/
void stopAll();
/**
* 获取设备数量
* @return 设备数量
*/
int getDeviceCount() const;
/**
* 获取设备ID
* @param index 设备索引
* @return 设备ID字符串
*/
std::string getDeviceId(int index) const;
/**
* 获取指定设备的最新图像
* @param device_index 设备索引
* @param depth 输出的深度图
* @param color 输出的彩色图
* @param fps 输出的帧率
* @return 是否成功获取到图像
*/
bool getLatestImages(int device_index, cv::Mat& depth, cv::Mat& color, double& fps);
/**
* 检查是否正在运行
* @return 是否运行中
*/
bool isRunning() const;
/**
* 获取指定设备的深度相机内参
* @param device_index 设备索引
* @param fx [输出] 焦距x
* @param fy [输出] 焦距y
* @param cx [输出] 主点x
* @param cy [输出] 主点y
* @return 是否成功获取内参
*/
bool getDepthCameraIntrinsics(int device_index, float& fx, float& fy, float& cx, float& cy);
/**
* @brief 利用SDK生成点云
* @param device_index 设备索引
* @param depth_img 深度图
* @param out_points 输出点云
* @return 是否成功
*/
bool computePointCloud(int device_index, const cv::Mat& depth_img, std::vector<Point3D>& out_points);
/**
* 获取深度相机数量
* @return 深度相机数量
*/
int getDepthCameraCount() const;
/**
* 获取2D (MVS)相机数量
* @return 2D相机数量
*/
int get2DCameraCount() const;
/**
* 获取2D相机图像
* @param camera_index 2D相机索引（从0开始）
* @param image 输出的彩色图
* @param fps 输出的帧率
* @return 是否成功
*/
bool get2DCameraImage(int camera_index, cv::Mat& image, double& fps);
/**
* 获取2D相机ID
* @param camera_index 2D相机索引
* @return 相机ID字符串
*/
std::string get2DCameraId(int camera_index) const;
private:
DeviceManager(); // 私有构造函数，确保单例
std::shared_ptr<CameraCapture> capture_; // Percipio深度相机采集对象
std::unique_ptr<MvsMultiCameraCapture> mvs_cameras_; // MVS 2D相机采集对象
bool initialized_; // 是否已初始化
};

1014
image_capture/src/gui/mainwindow.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

106
image_capture/src/gui/mainwindow.h Normal file
View File

@@ -0,0 +1,106 @@
#pragma once
#include <QMainWindow>
#include <QLabel>
#include <QPushButton>
#include <QVBoxLayout>
#include <QHBoxLayout>
#include <QTimer>
#include <QPlainTextEdit>
#include <QTextStream>
#include <QTabWidget>
#include <QDoubleSpinBox>
#include <QSpinBox>
#include <QFormLayout>
#include <QGroupBox>
#include <QScrollArea>
#include <QScrollArea>
#include <memory>
#include <vector>
#include <streambuf>
#include <fstream>
#include <opencv2/core/mat.hpp>
// Forward declarations
class SettingsWidget;
class ImageProcessor;
class VisionController;
class LogStreamBuf;
// 必须在QT_BEGIN_NAMESPACE之前包含，确保MOC能看到完整类型定义
#include "../common/log_streambuf.h"
QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE
class MainWindow : public QMainWindow
{
Q_OBJECT
public:
MainWindow(QWidget *parent = nullptr);
~MainWindow();
private slots:
void onStartCapture();
void onStopCapture();
void onSaveImage();
void onSavePointCloud();
void updateImage();
private:
// Helper to reduce redundancy in save functions
bool prepareCapturedData(cv::Mat& depth, cv::Mat& color, QString& timestamp);
Ui::MainWindow *ui;
std::shared_ptr<VisionController> visionController_; // Vision系统控制器（Redis监控和算法触发）
std::vector<std::shared_ptr<ImageProcessor>> processors_;
// Supports max 4 depth cameras
static const int MAX_DEPTH_CAMERAS = 4;
static const int MAX_2D_CAMERAS = 5;
QLabel* depthImageLabels_[MAX_DEPTH_CAMERAS];
QLabel* depthInfoLabels_[MAX_DEPTH_CAMERAS];
// 2D camera display control array
QLabel* twoDImageLabels_[MAX_2D_CAMERAS];
QLabel* twoDInfoLabels_[MAX_2D_CAMERAS];
void update2DDisplay();
void update2DCameraDisplay(int camera_index, const cv::Mat& image, double fps);
// 深度图信息标签（显示相机编号、FPS等）
QPushButton* startButton_;
QPushButton* stopButton_;
QPushButton* saveButton_;
QPushButton* savePointCloudButton_;
QPlainTextEdit* logTextEdit_;
QTimer* updateTimer_;
bool isCapturing_;
int currentDeviceIndex_;
// 日志重定向相关
std::unique_ptr<LogStreamBuf> coutBuf_;
std::unique_ptr<LogStreamBuf> cerrBuf_;
std::streambuf* originalCout_;
std::streambuf* originalCerr_;
QImage cvMatToQImage(const cv::Mat& mat);
void updateDepthDisplay();
void updateCameraDisplay(int cameraIndex);
void appendLog(const QString& message);
// 辅助函数：简化代码
void startImageDisplay(); // 启动图像显示（更新按钮状态和定时器）
void stopImageDisplay(); // 停止图像显示（更新按钮状态、停止定时器、清空显示）
// Settings tab
// Settings Widget
SettingsWidget* settingsWidget_;
};

1205
image_capture/src/gui/mainwindow.ui Normal file
View File

File diff suppressed because it is too large Load Diff

305
image_capture/src/gui/settings_widget.cpp Normal file
View File

@@ -0,0 +1,305 @@
#include "settings_widget.h"
#include "../common/config_manager.h"
#include <QSpinBox>
#include <QDoubleSpinBox>
#include <QVBoxLayout>
#include <QHBoxLayout>
#include <QTabWidget>
#include <QPushButton>
#include <QGroupBox>
#include <QFormLayout>
#include <QLabel>
#include <QScrollArea>
#include <QMessageBox>
SettingsWidget::SettingsWidget(QWidget *parent) : QWidget(parent) {
setupUi();
loadSettings();
}
void SettingsWidget::setupUi() {
auto mainLayout = new QVBoxLayout(this);
auto tabWidget = new QTabWidget(this);
tabWidget->addTab(createBeamRackTab(), "Beam/Rack Deflection");
tabWidget->addTab(createPalletOffsetTab(), "Pallet Offset");
tabWidget->addTab(createOtherAlgorithmsTab(), "Other Algorithms");
tabWidget->addTab(createGeneralTab(), "General");
mainLayout->addWidget(tabWidget);
auto buttonLayout = new QHBoxLayout();
buttonLayout->addStretch();
auto saveButton = new QPushButton("Save Settings", this);
saveButton->setMinimumSize(120, 35);
QFont font;
font.setPointSize(12);
saveButton->setFont(font);
connect(saveButton, &QPushButton::clicked, this, &SettingsWidget::saveSettings);
buttonLayout->addWidget(saveButton);
mainLayout->addLayout(buttonLayout);
}
QWidget* SettingsWidget::createBeamRackTab() {
auto widget = new QWidget(this);
auto layout = new QVBoxLayout(widget);
auto roiGroup = new QGroupBox("感兴趣区域点", this);
auto roiLayout = new QGridLayout(roiGroup);
roiLayout->addWidget(new QLabel("点", this), 0, 0);
roiLayout->addWidget(new QLabel("横梁 X", this), 0, 1);
roiLayout->addWidget(new QLabel("横梁 Y", this), 0, 2);
roiLayout->addWidget(new QLabel("立柱 X", this), 0, 3);
roiLayout->addWidget(new QLabel("立柱 Y", this), 0, 4);
const QStringList pointNames = {"左上", "右上", "右下", "左下"};
for (int i = 0; i < 4; ++i) {
beamRoiX_[i] = new QSpinBox(this); beamRoiX_[i]->setRange(0, 5000);
beamRoiY_[i] = new QSpinBox(this); beamRoiY_[i]->setRange(0, 5000);
rackRoiX_[i] = new QSpinBox(this); rackRoiX_[i]->setRange(0, 5000);
rackRoiY_[i] = new QSpinBox(this); rackRoiY_[i]->setRange(0, 5000);
roiLayout->addWidget(new QLabel(pointNames[i], this), i+1, 0);
roiLayout->addWidget(beamRoiX_[i], i+1, 1);
roiLayout->addWidget(beamRoiY_[i], i+1, 2);
roiLayout->addWidget(rackRoiX_[i], i+1, 3);
roiLayout->addWidget(rackRoiY_[i], i+1, 4);
}
layout->addWidget(roiGroup);
auto threshGroup = new QGroupBox("阈值 (mm)", this);
auto threshLayout = new QFormLayout(threshGroup);
auto createDoubleSpin = [this](QDoubleSpinBox*& box) {
box = new QDoubleSpinBox(this);
box->setRange(-1000.0, 1000.0);
box->setSingleStep(0.1);
box->setDecimals(1);
};
createDoubleSpin(beamThresholdA_);
createDoubleSpin(beamThresholdB_);
createDoubleSpin(beamThresholdC_);
createDoubleSpin(beamThresholdD_);
createDoubleSpin(rackThresholdA_);
createDoubleSpin(rackThresholdB_);
createDoubleSpin(rackThresholdC_);
createDoubleSpin(rackThresholdD_);
threshLayout->addRow("横梁负向报警 (A):", beamThresholdA_);
threshLayout->addRow("横梁负向预警 (B):", beamThresholdB_);
threshLayout->addRow("横梁正向预警 (C):", beamThresholdC_);
threshLayout->addRow("横梁正向报警 (D):", beamThresholdD_);
threshLayout->addRow("立柱负向报警 (A):", rackThresholdA_);
threshLayout->addRow("立柱负向预警 (B):", rackThresholdB_);
threshLayout->addRow("立柱正向预警 (C):", rackThresholdC_);
threshLayout->addRow("立柱正向报警 (D):", rackThresholdD_);
layout->addWidget(threshGroup);
layout->addStretch();
return widget;
}
QWidget* SettingsWidget::createPalletOffsetTab() {
auto widget = new QWidget(this);
auto scroll = new QScrollArea(widget);
auto contentWidget = new QWidget();
auto layout = new QVBoxLayout(contentWidget);
auto createThreshGroup = [this](const QString& title, QDoubleSpinBox*& A, QDoubleSpinBox*& B, QDoubleSpinBox*& C, QDoubleSpinBox*& D) {
auto group = new QGroupBox(title, this);
auto flo = new QFormLayout(group);
auto createDS = [this](QDoubleSpinBox*& box) {
box = new QDoubleSpinBox(this);
box->setRange(-1000.0, 1000.0);
box->setSingleStep(0.1);
box->setDecimals(1);
};
createDS(A); createDS(B); createDS(C); createDS(D);
flo->addRow("低位报警 (A):", A);
flo->addRow("低位预警 (B):", B);
flo->addRow("高位预警 (C):", C);
flo->addRow("高位报警 (D):", D);
return group;
};
layout->addWidget(createThreshGroup("横向偏移 (mm)", palletLatA_, palletLatB_, palletLatC_, palletLatD_));
layout->addWidget(createThreshGroup("纵向偏移 (mm)", palletLonA_, palletLonB_, palletLonC_, palletLonD_));
layout->addWidget(createThreshGroup("旋转角度 (deg)", palletRotA_, palletRotB_, palletRotC_, palletRotD_));
layout->addWidget(createThreshGroup("左孔变形 (mm)", palletHoleLeftA_, palletHoleLeftB_, palletHoleLeftC_, palletHoleLeftD_));
layout->addWidget(createThreshGroup("右孔变形 (mm)", palletHoleRightA_, palletHoleRightB_, palletHoleRightC_, palletHoleRightD_));
layout->addStretch();
auto mainLayout = new QVBoxLayout(widget);
scroll->setWidget(contentWidget);
scroll->setWidgetResizable(true);
mainLayout->addWidget(scroll);
return widget;
}
QWidget* SettingsWidget::createOtherAlgorithmsTab() {
auto widget = new QWidget(this);
auto layout = new QVBoxLayout(widget);
auto slotGroup = new QGroupBox("库位占用", this);
auto slotLayout = new QFormLayout(slotGroup);
slotDepthThreshold_ = new QDoubleSpinBox(this);
slotDepthThreshold_->setRange(0.0, 10000.0);
slotConfidenceThreshold_ = new QDoubleSpinBox(this);
slotConfidenceThreshold_->setRange(0.0, 1.0);
slotConfidenceThreshold_->setSingleStep(0.05);
slotLayout->addRow("深度阈值 (mm):", slotDepthThreshold_);
slotLayout->addRow("置信度阈值:", slotConfidenceThreshold_);
layout->addWidget(slotGroup);
auto visGroup = new QGroupBox("视觉盘点", this);
auto visLayout = new QFormLayout(visGroup);
visualBarcodeConfidence_ = new QDoubleSpinBox(this);
visualBarcodeConfidence_->setRange(0.0, 1.0);
visualBarcodeConfidence_->setSingleStep(0.05);
visLayout->addRow("条码置信度:", visualBarcodeConfidence_);
layout->addWidget(visGroup);
layout->addStretch();
return widget;
}
QWidget* SettingsWidget::createGeneralTab() {
auto widget = new QWidget(this);
auto layout = new QFormLayout(widget);
minDepth_ = new QDoubleSpinBox(this);
minDepth_->setRange(0.0, 10000.0);
maxDepth_ = new QDoubleSpinBox(this);
maxDepth_->setRange(0.0, 10000.0);
samplePoints_ = new QSpinBox(this);
samplePoints_->setRange(1, 1000);
layout->addRow("最小深度 (mm):", minDepth_);
layout->addRow("最大深度 (mm):", maxDepth_);
layout->addRow("采样点数:", samplePoints_);
return widget;
}
void SettingsWidget::loadSettings() {
auto& config = ConfigManager::getInstance();
auto beamPoints = config.getBeamROIPoints();
for(int i=0; i<4 && i<beamPoints.size(); ++i) {
beamRoiX_[i]->setValue(beamPoints[i].x);
beamRoiY_[i]->setValue(beamPoints[i].y);
}
auto rackPoints = config.getRackROIPoints();
for(int i=0; i<4 && i<rackPoints.size(); ++i) {
rackRoiX_[i]->setValue(rackPoints[i].x);
rackRoiY_[i]->setValue(rackPoints[i].y);
}
auto beamT = config.getBeamThresholds();
if(beamT.size() >= 4) {
beamThresholdA_->setValue(beamT[0]);
beamThresholdB_->setValue(beamT[1]);
beamThresholdC_->setValue(beamT[2]);
beamThresholdD_->setValue(beamT[3]);
}
auto rackT = config.getRackThresholds();
if(rackT.size() >= 4) {
rackThresholdA_->setValue(rackT[0]);
rackThresholdB_->setValue(rackT[1]);
rackThresholdC_->setValue(rackT[2]);
rackThresholdD_->setValue(rackT[3]);
}
auto setThresh = [](std::vector<float> v, QDoubleSpinBox* a, QDoubleSpinBox* b, QDoubleSpinBox* c, QDoubleSpinBox* d) {
if(v.size() >= 4) {
a->setValue(v[0]); b->setValue(v[1]); c->setValue(v[2]); d->setValue(v[3]);
}
};
setThresh(config.getPalletOffsetLatThresholds(), palletLatA_, palletLatB_, palletLatC_, palletLatD_);
setThresh(config.getPalletOffsetLonThresholds(), palletLonA_, palletLonB_, palletLonC_, palletLonD_);
setThresh(config.getPalletRotationAngleThresholds(), palletRotA_, palletRotB_, palletRotC_, palletRotD_);
setThresh(config.getPalletHoleDefLeftThresholds(), palletHoleLeftA_, palletHoleLeftB_, palletHoleLeftC_, palletHoleLeftD_);
setThresh(config.getPalletHoleDefRightThresholds(), palletHoleRightA_, palletHoleRightB_, palletHoleRightC_, palletHoleRightD_);
slotDepthThreshold_->setValue(config.getSlotOccupancyDepthThreshold());
slotConfidenceThreshold_->setValue(config.getSlotOccupancyConfidenceThreshold());
visualBarcodeConfidence_->setValue(config.getVisualInventoryBarcodeConfidence());
minDepth_->setValue(config.getAlgorithmMinDepth());
maxDepth_->setValue(config.getAlgorithmMaxDepth());
samplePoints_->setValue(config.getAlgorithmSamplePoints());
}
void SettingsWidget::saveSettings() {
auto& config = ConfigManager::getInstance();
std::vector<cv::Point2i> beamPts, rackPts;
for(int i=0; i<4; ++i) {
beamPts.push_back(cv::Point2i(beamRoiX_[i]->value(), beamRoiY_[i]->value()));
rackPts.push_back(cv::Point2i(rackRoiX_[i]->value(), rackRoiY_[i]->value()));
}
config.setBeamROIPoints(beamPts);
config.setRackROIPoints(rackPts);
config.setBeamThresholds({
(float)beamThresholdA_->value(), (float)beamThresholdB_->value(),
(float)beamThresholdC_->value(), (float)beamThresholdD_->value()
});
config.setRackThresholds({
(float)rackThresholdA_->value(), (float)rackThresholdB_->value(),
(float)rackThresholdC_->value(), (float)rackThresholdD_->value()
});
config.setPalletOffsetLatThresholds({
(float)palletLatA_->value(), (float)palletLatB_->value(),
(float)palletLatC_->value(), (float)palletLatD_->value()
});
config.setPalletOffsetLonThresholds({
(float)palletLonA_->value(), (float)palletLonB_->value(),
(float)palletLonC_->value(), (float)palletLonD_->value()
});
config.setPalletRotationAngleThresholds({
(float)palletRotA_->value(), (float)palletRotB_->value(),
(float)palletRotC_->value(), (float)palletRotD_->value()
});
config.setPalletHoleDefLeftThresholds({
(float)palletHoleLeftA_->value(), (float)palletHoleLeftB_->value(),
(float)palletHoleLeftC_->value(), (float)palletHoleLeftD_->value()
});
config.setPalletHoleDefRightThresholds({
(float)palletHoleRightA_->value(), (float)palletHoleRightB_->value(),
(float)palletHoleRightC_->value(), (float)palletHoleRightD_->value()
});
config.setSlotOccupancyDepthThreshold((float)slotDepthThreshold_->value());
config.setSlotOccupancyConfidenceThreshold((float)slotConfidenceThreshold_->value());
config.setVisualInventoryBarcodeConfidence((float)visualBarcodeConfidence_->value());
config.setAlgorithmMinDepth((float)minDepth_->value());
config.setAlgorithmMaxDepth((float)maxDepth_->value());
config.setAlgorithmSamplePoints(samplePoints_->value());
if (config.saveConfig()) {
QMessageBox::information(this, "Success", "Configuration saved successfully.");
emit settingsSaved();
} else {
QMessageBox::critical(this, "Error", "Failed to save configuration.");
}
}

75
image_capture/src/gui/settings_widget.h Normal file
View File

@@ -0,0 +1,75 @@
#pragma once
#include <QWidget>
#include <vector>
class QSpinBox;
class QDoubleSpinBox;
class SettingsWidget : public QWidget {
Q_OBJECT
public:
explicit SettingsWidget(QWidget *parent = nullptr);
~SettingsWidget() = default;
public slots:
void loadSettings();
void saveSettings();
signals:
void settingsSaved();
private:
void setupUi();
QWidget* createBeamRackTab();
QWidget* createPalletOffsetTab();
QWidget* createOtherAlgorithmsTab();
QWidget* createGeneralTab();
// Beam/Rack Deflection
QSpinBox* beamRoiX_[4];
QSpinBox* beamRoiY_[4];
QSpinBox* rackRoiX_[4];
QSpinBox* rackRoiY_[4];
QDoubleSpinBox* beamThresholdA_;
QDoubleSpinBox* beamThresholdB_;
QDoubleSpinBox* beamThresholdC_;
QDoubleSpinBox* beamThresholdD_;
QDoubleSpinBox* rackThresholdA_;
QDoubleSpinBox* rackThresholdB_;
QDoubleSpinBox* rackThresholdC_;
QDoubleSpinBox* rackThresholdD_;
// Pallet Offset
QDoubleSpinBox* palletLatA_;
QDoubleSpinBox* palletLatB_;
QDoubleSpinBox* palletLatC_;
QDoubleSpinBox* palletLatD_;
QDoubleSpinBox* palletLonA_;
QDoubleSpinBox* palletLonB_;
QDoubleSpinBox* palletLonC_;
QDoubleSpinBox* palletLonD_;