LeRobot-SO100入门教程-配置摄像头

说明：

介绍如何配置两个摄像头
获取摄像头的帧数，分辨率，更改配置文件

步骤：

两个摄像头摆放到合理的位置，尽可能拍到从臂附近的工作区即可
查找两个摄像机索引，将保存每个检测到的摄像机的几帧：

python3 ~/lerobot/lerobot/common/robot_devices/cameras/opencv.py \
    --images-dir outputs/images_from_opencv_cameras

结果如下：

相机1，/dev/video0，索引camera_index索引为0，fps帧数为25，分辨率 width为640，height为480
相机2，/dev/video2，索引camera_index索引为2，fps帧数为25，分辨率 width为640，height为480

Linux detected. Finding available camera indices through scanning '/dev/video*' ports
Camera found at index /dev/video0
Camera found at index /dev/video2
Connecting cameras
OpenCVCamera(0, fps=25, width=640, height=480, color_mode=rgb)
OpenCVCamera(2, fps=25, width=640, height=480, color_mode=rgb)
Saving images to outputs/images_from_opencv_cameras
Frame: 0000	Latency (ms): 1011.64
Frame: 0001	Latency (ms): 31.14
Frame: 0002	Latency (ms): 44.36
Frame: 0003	Latency (ms): 35.54
Frame: 0004	Latency (ms): 44.39
Frame: 0005	Latency (ms): 35.47
Frame: 0006	Latency (ms): 44.44
Frame: 0007	Latency (ms): 39.86
Frame: 0008	Latency (ms): 40.01
Frame: 0009	Latency (ms): 40.18
Frame: 0010	Latency (ms): 35.13
Frame: 0011	Latency (ms): 44.25
Frame: 0012	Latency (ms): 35.58
Frame: 0013	Latency (ms): 44.06
Frame: 0014	Latency (ms): 40.26
Frame: 0015	Latency (ms): 39.65
Frame: 0016	Latency (ms): 40.38
Frame: 0017	Latency (ms): 35.12
Frame: 0018	Latency (ms): 43.70
Frame: 0019	Latency (ms): 35.68
Frame: 0020	Latency (ms): 44.38
Frame: 0021	Latency (ms): 35.59
Frame: 0022	Latency (ms): 48.05
Frame: 0023	Latency (ms): 40.52
Frame: 0024	Latency (ms): 35.23
Frame: 0025	Latency (ms): 43.76
Frame: 0026	Latency (ms): 36.08
Frame: 0027	Latency (ms): 43.83
Frame: 0028	Latency (ms): 35.83
Frame: 0029	Latency (ms): 45.41
Frame: 0030	Latency (ms): 34.63
Frame: 0031	Latency (ms): 44.24
Frame: 0032	Latency (ms): 39.66
Frame: 0033	Latency (ms): 35.82
Frame: 0034	Latency (ms): 44.21
Frame: 0035	Latency (ms): 35.52
Frame: 0036	Latency (ms): 43.96
Frame: 0037	Latency (ms): 40.45
Frame: 0038	Latency (ms): 39.29
Frame: 0039	Latency (ms): 40.12
Frame: 0040	Latency (ms): 33.09
Frame: 0041	Latency (ms): 44.61
Frame: 0042	Latency (ms): 40.80
Frame: 0043	Latency (ms): 38.72
Frame: 0044	Latency (ms): 40.10
Frame: 0045	Latency (ms): 35.17
Frame: 0046	Latency (ms): 44.40
Frame: 0047	Latency (ms): 40.18
Frame: 0048	Latency (ms): 35.15
Frame: 0049	Latency (ms): 43.80
Frame: 0050	Latency (ms): 35.69
Frame: 0051	Latency (ms): 44.16
Frame: 0052	Latency (ms): 40.66
Frame: 0053	Latency (ms): 35.12
Frame: 0054	Latency (ms): 43.73
Frame: 0055	Latency (ms): 36.00
Frame: 0056	Latency (ms): 44.50
Frame: 0057	Latency (ms): 39.45
Frame: 0058	Latency (ms): 35.71
Frame: 0059	Latency (ms): 44.01
Frame: 0060	Latency (ms): 35.29
Frame: 0061	Latency (ms): 44.40
Frame: 0062	Latency (ms): 39.37
Frame: 0063	Latency (ms): 35.89
Frame: 0064	Latency (ms): 44.25
Frame: 0065	Latency (ms): 35.80
Frame: 0066	Latency (ms): 44.38
Frame: 0067	Latency (ms): 39.98
Frame: 0068	Latency (ms): 35.25
Frame: 0069	Latency (ms): 43.99
Frame: 0070	Latency (ms): 39.89
Frame: 0071	Latency (ms): 39.91
Frame: 0072	Latency (ms): 39.66
Frame: 0073	Latency (ms): 35.63
Frame: 0074	Latency (ms): 44.44
Frame: 0075	Latency (ms): 38.93

保存的图像在~/lerobot/outputs/images_from_opencv_cameras，确定哪个相机索引对应于哪个相机，0 对应 camera_00， 1 对应camera_01

camera_00_frame_000000.png
    [...]
    camera_00_frame_000047.png
    camera_01_frame_000000.png
    [...]
    camera_01_frame_000047.png

修改两个相机的索引 camera_index

vim ~/lerobot/lerobot/common/robot_devices/robots/configs.py

修改的内容如下，camera_index相机索引，fps帧数

@RobotConfig.register_subclass("so100")
    ...

cameras: dict[str, CameraConfig] = field( default_factory=lambda: { "laptop": OpenCVCameraConfig( camera_index=0, fps=25, width=640, height=480, ), "phone": OpenCVCameraConfig( camera_index=2, fps=25, width=640, height=480, ), } )

加载lerobot环境

conda activate lerobot

cd ~/lerobot

运行程序，操控机械臂，显示两个相机的画面

python ~/lerobot/lerobot/scripts/control_robot.py \
      --robot.type=so100 \
      --control.type=teleoperate

请输入图片描述

演示视频

LeRobot-SO100入门教程-配置摄像头

说明：

相关设备

步骤：

演示视频