LeRobot-SO100双臂入门教程-配置摄像头

说明：

介绍如何配置摄像头
获取摄像头的帧数，分辨率，更改配置文件
介绍如何增加摄像头

步骤：

摄像头摆放到合理的位置，尽可能拍到从臂附近的工作区即可
查找三个摄像机索引，将保存每个检测到的摄像机的几帧：

python3 ~/lerobot/lerobot/common/robot_devices/cameras/opencv.py \
    --images-dir outputs/images_from_opencv_cameras

结果如下：

相机1，/dev/video0，索引camera_index索引为0，fps帧数为30，分辨率 width为640，height为480
相机2，/dev/video2，索引camera_index索引为2，fps帧数为30，分辨率 width为640，height为480
相机3，/dev/video4，索引camera_index索引为4，fps帧数为30，分辨率 width为640，height为480

Linux detected. Finding available camera indices through scanning '/dev/video*' ports
Camera found at index /dev/video0
Camera found at index /dev/video2
Camera found at index /dev/video4
Connecting cameras
OpenCVCamera(0, fps=30, width=640, height=480, color_mode=rgb)
OpenCVCamera(2, fps=30, width=640, height=480, color_mode=rgb)
OpenCVCamera(4, fps=30, width=640, height=480, color_mode=rgb)
Saving images to outputs/images_from_opencv_cameras
Frame: 0000	Latency (ms): 1226.09
Frame: 0001	Latency (ms): 2.71
Frame: 0002	Latency (ms): 8.48
Frame: 0003	Latency (ms): 3.20
Frame: 0004	Latency (ms): 38.33
Frame: 0005	Latency (ms): 31.96
Frame: 0006	Latency (ms): 35.62
Frame: 0007	Latency (ms): 32.50
Frame: 0008	Latency (ms): 35.13
Frame: 0009	Latency (ms): 39.02
Frame: 0010	Latency (ms): 43.77
Frame: 0011	Latency (ms): 41.04
Frame: 0012	Latency (ms): 39.12
Frame: 0013	Latency (ms): 43.88
Frame: 0014	Latency (ms): 39.50
Frame: 0015	Latency (ms): 43.85
Frame: 0016	Latency (ms): 40.17
Frame: 0017	Latency (ms): 44.16
Frame: 0018	Latency (ms): 40.01
Frame: 0019	Latency (ms): 39.66
Frame: 0020	Latency (ms): 43.80
Frame: 0021	Latency (ms): 38.67
Frame: 0022	Latency (ms): 44.26
Frame: 0023	Latency (ms): 41.18
Frame: 0024	Latency (ms): 38.64
Frame: 0025	Latency (ms): 44.60
Frame: 0026	Latency (ms): 39.21
Frame: 0027	Latency (ms): 43.78
Frame: 0028	Latency (ms): 40.08
Frame: 0029	Latency (ms): 44.22
Frame: 0030	Latency (ms): 40.46
Frame: 0031	Latency (ms): 38.54
Frame: 0032	Latency (ms): 44.09
Frame: 0033	Latency (ms): 40.54
Frame: 0034	Latency (ms): 43.36
Frame: 0035	Latency (ms): 39.68
Frame: 0036	Latency (ms): 40.78
Frame: 0037	Latency (ms): 43.99
Frame: 0038	Latency (ms): 39.63
Frame: 0039	Latency (ms): 43.80
Frame: 0040	Latency (ms): 39.93
Frame: 0041	Latency (ms): 43.58
Frame: 0042	Latency (ms): 40.27
Frame: 0043	Latency (ms): 40.40
Frame: 0044	Latency (ms): 43.87
Frame: 0045	Latency (ms): 39.60
Frame: 0046	Latency (ms): 44.33
Frame: 0047	Latency (ms): 39.19
Frame: 0048	Latency (ms): 40.56
Frame: 0049	Latency (ms): 43.97
Frame: 0050	Latency (ms): 40.57
Frame: 0051	Latency (ms): 43.21
Frame: 0052	Latency (ms): 39.59
Frame: 0053	Latency (ms): 44.51
Frame: 0054	Latency (ms): 39.70
Frame: 0055	Latency (ms): 41.77
Frame: 0056	Latency (ms): 41.66
Frame: 0057	Latency (ms): 40.38
Frame: 0058	Latency (ms): 44.05
Frame: 0059	Latency (ms): 39.31
Frame: 0060	Latency (ms): 40.69
Frame: 0061	Latency (ms): 45.08
Frame: 0062	Latency (ms): 38.67
Frame: 0063	Latency (ms): 43.07
Frame: 0064	Latency (ms): 45.87
Frame: 0065	Latency (ms): 38.71
Frame: 0066	Latency (ms): 39.75
Frame: 0067	Latency (ms): 43.59
Frame: 0068	Latency (ms): 40.40
Frame: 0069	Latency (ms): 116.08

保存的图像在~/lerobot/outputs/images_from_opencv_cameras，确定哪个相机索引对应于哪个相机，0 对应 camera_00， 1 对应camera_01……

camera_00_frame_000000.png
    [...]
    camera_00_frame_000069.png
    camera_02_frame_000000.png
    [...]
    camera_02_frame_000069.png
    camera_04_frame_000000.png
    [...]
    camera_0_frame_000069.png

修改三个相机的索引 camera_index

vim ~/lerobot/lerobot/common/robot_devices/robots/configs.py

修改的内容如下，camera_index相机索引，fps帧数

@RobotConfig.register_subclass("so100")
    ...

cameras: dict[str, CameraConfig] = field( default_factory=lambda: { "laptop": OpenCVCameraConfig( camera_index=0, fps=30, width=640, height=480, ), "phone1": OpenCVCameraConfig( camera_index=2, fps=30, width=640, height=480, ), "phone2": OpenCVCameraConfig( camera_index=4, fps=30, width=640, height=480, ), } )

加载lerobot环境

conda activate lerobot

cd ~/lerobot

运行程序，操控机械臂，显示两个相机的画面

python ~/lerobot/lerobot/scripts/control_robot.py \
      --robot.type=so100 \
      --control.type=teleoperate

请输入图片描述

标签: none