LeRobot-SO100双臂入门教程-配置摄像头

说明：

• 介绍如何配置摄像头
• 获取摄像头的帧数，分辨率，更改配置文件
• 介绍如何增加摄像头

步骤：

• 摄像头摆放到合理的位置，尽可能拍到从臂附近的工作区即可

• 查找三个摄像机索引，将保存每个检测到的摄像机的几帧：

    python3 ~/lerobot/lerobot/common/robot_devices/cameras/opencv.py \
    --images-dir outputs/images_from_opencv_cameras

• 结果如下：
  相机1，/dev/video0，索引camera_index索引为0，fps帧数为30，分辨率 width为640，height为480
  相机2，/dev/video2，索引camera_index索引为2，fps帧数为30，分辨率 width为640，height为480
  相机3，/dev/video4，索引camera_index索引为4，fps帧数为30，分辨率 width为640，height为480

Linux detected. Finding available camera indices through scanning '/dev/video*' ports
Camera found at index /dev/video0
Camera found at index /dev/video2
Camera found at index /dev/video4
Connecting cameras
OpenCVCamera(0, fps=30, width=640, height=480, color_mode=rgb)
OpenCVCamera(2, fps=30, width=640, height=480, color_mode=rgb)
OpenCVCamera(4, fps=30, width=640, height=480, color_mode=rgb)
Saving images to outputs/images_from_opencv_cameras
Frame: 0000 Latency (ms): 1226.09
Frame: 0001 Latency (ms): 2.71
Frame: 0002 Latency (ms): 8.48
Frame: 0003 Latency (ms): 3.20
Frame: 0004 Latency (ms): 38.33
Frame: 0005 Latency (ms): 31.96
Frame: 0006 Latency (ms): 35.62
Frame: 0007 Latency (ms): 32.50
Frame: 0008 Latency (ms): 35.13
Frame: 0009 Latency (ms): 39.02
Frame: 0010 Latency (ms): 43.77
Frame: 0011 Latency (ms): 41.04
Frame: 0012 Latency (ms): 39.12
Frame: 0013 Latency (ms): 43.88
Frame: 0014 Latency (ms): 39.50
Frame: 0015 Latency (ms): 43.85
Frame: 0016 Latency (ms): 40.17
Frame: 0017 Latency (ms): 44.16
Frame: 0018 Latency (ms): 40.01
Frame: 0019 Latency (ms): 39.66
Frame: 0020 Latency (ms): 43.80
Frame: 0021 Latency (ms): 38.67
Frame: 0022 Latency (ms): 44.26
Frame: 0023 Latency (ms): 41.18
Frame: 0024 Latency (ms): 38.64
Frame: 0025 Latency (ms): 44.60
Frame: 0026 Latency (ms): 39.21
Frame: 0027 Latency (ms): 43.78
Frame: 0028 Latency (ms): 40.08
Frame: 0029 Latency (ms): 44.22
Frame: 0030 Latency (ms): 40.46
Frame: 0031 Latency (ms): 38.54
Frame: 0032 Latency (ms): 44.09
Frame: 0033 Latency (ms): 40.54
Frame: 0034 Latency (ms): 43.36
Frame: 0035 Latency (ms): 39.68
Frame: 0036 Latency (ms): 40.78
Frame: 0037 Latency (ms): 43.99
Frame: 0038 Latency (ms): 39.63
Frame: 0039 Latency (ms): 43.80
Frame: 0040 Latency (ms): 39.93
Frame: 0041 Latency (ms): 43.58
Frame: 0042 Latency (ms): 40.27
Frame: 0043 Latency (ms): 40.40
Frame: 0044 Latency (ms): 43.87
Frame: 0045 Latency (ms): 39.60
Frame: 0046 Latency (ms): 44.33
Frame: 0047 Latency (ms): 39.19
Frame: 0048 Latency (ms): 40.56
Frame: 0049 Latency (ms): 43.97
Frame: 0050 Latency (ms): 40.57
Frame: 0051 Latency (ms): 43.21
Frame: 0052 Latency (ms): 39.59
Frame: 0053 Latency (ms): 44.51
Frame: 0054 Latency (ms): 39.70
Frame: 0055 Latency (ms): 41.77
Frame: 0056 Latency (ms): 41.66
Frame: 0057 Latency (ms): 40.38
Frame: 0058 Latency (ms): 44.05
Frame: 0059 Latency (ms): 39.31
Frame: 0060 Latency (ms): 40.69
Frame: 0061 Latency (ms): 45.08
Frame: 0062 Latency (ms): 38.67
Frame: 0063 Latency (ms): 43.07
Frame: 0064 Latency (ms): 45.87
Frame: 0065 Latency (ms): 38.71
Frame: 0066 Latency (ms): 39.75
Frame: 0067 Latency (ms): 43.59
Frame: 0068 Latency (ms): 40.40
Frame: 0069 Latency (ms): 116.08

• 保存的图像在~/lerobot/outputs/images_from_opencv_cameras，确定哪个相机索引对应于哪个相机，0 对应 camera_00， 1 对应camera_01……

    camera_00_frame_000000.png
    [...]
    camera_00_frame_000069.png
    camera_02_frame_000000.png
    [...]
    camera_02_frame_000069.png
    camera_04_frame_000000.png
    [...]
    camera_0_frame_000069.png

• 修改三个相机的索引 camera_index

    vim ~/lerobot/lerobot/common/robot_devices/robots/configs.py

• 修改的内容如下，camera_index相机索引，fps帧数

    @RobotConfig.register_subclass("so100")
    ...

    cameras: dict[str, CameraConfig] = field(
        default_factory=lambda: {
            "laptop": OpenCVCameraConfig(
                camera_index=0,
                fps=30,
                width=640,
                height=480,
            ),
            "phone1": OpenCVCameraConfig(
                camera_index=2,
                fps=30,
                width=640,
                height=480,
            ),
            "phone2": OpenCVCameraConfig(
                camera_index=4,
                fps=30,
                width=640,
                height=480,
            ),
        }
    )

• 加载lerobot环境

    conda activate lerobot

• 进入lerobot目录

    cd ~/lerobot

• 运行程序，操控机械臂，显示两个相机的画面

    python ~/lerobot/lerobot/scripts/control_robot.py \
      --robot.type=so100 \
      --control.type=teleoperate