LeRobot-SO100双臂入门教程-配置摄像头

说明：

介绍如何配置摄像头
获取摄像头的帧数，分辨率，更改配置文件
介绍如何增加摄像头

步骤：

摄像头摆放到合理的位置，尽可能拍到从臂附近的工作区即可
查找三个摄像机索引，将保存每个检测到的摄像机的几帧：

python3 ~/lerobot/lerobot/common/robot_devices/cameras/opencv.py \
    --images-dir outputs/images_from_opencv_cameras

结果如下：

相机1，/dev/video0，索引camera_index索引为0，fps帧数为30，分辨率 width为640，height为480
相机2，/dev/video2，索引camera_index索引为2，fps帧数为30，分辨率 width为640，height为480
相机3，/dev/video4，索引camera_index索引为4，fps帧数为30，分辨率 width为640，height为480

Linux detected. Finding available camera indices through scanning '/dev/video*' ports Camera found at index /dev/video0 Camera found at index /dev/video2 Camera found at index /dev/video4 Connecting cameras OpenCVCamera(0, fps=30, width=640, height=480, color_mode=rgb) OpenCVCamera(2, fps=30, width=640, height=480, color_mode=rgb) OpenCVCamera(4, fps=30, width=640, height=480, color_mode=rgb) Saving images to outputs/images_from_opencv_cameras Frame: 0000 Latency (ms): 1226.09 Frame: 0001 Latency (ms): 2.71 Frame: 0002 Latency (ms): 8.48 Frame: 0003 Latency (ms): 3.20 Frame: 0004 Latency (ms): 38.33 Frame: 0005 Latency (ms): 31.96 Frame: 0006 Latency (ms): 35.62 Frame: 0007 Latency (ms): 32.50 Frame: 0008 Latency (ms): 35.13 Frame: 0009 Latency (ms): 39.02 Frame: 0010 Latency (ms): 43.77 Frame: 0011 Latency (ms): 41.04 Frame: 0012 Latency (ms): 39.12 Frame: 0013 Latency (ms): 43.88 Frame: 0014 Latency (ms): 39.50 Frame: 0015 Latency (ms): 43.85 Frame: 0016 Latency (ms): 40.17 Frame: 0017 Latency (ms): 44.16 Frame: 0018 Latency (ms): 40.01 Frame: 0019 Latency (ms): 39.66 Frame: 0020 Latency (ms): 43.80 Frame: 0021 Latency (ms): 38.67 Frame: 0022 Latency (ms): 44.26 Frame: 0023 Latency (ms): 41.18 Frame: 0024 Latency (ms): 38.64 Frame: 0025 Latency (ms): 44.60 Frame: 0026 Latency (ms): 39.21 Frame: 0027 Latency (ms): 43.78 Frame: 0028 Latency (ms): 40.08 Frame: 0029 Latency (ms): 44.22 Frame: 0030 Latency (ms): 40.46 Frame: 0031 Latency (ms): 38.54 Frame: 0032 Latency (ms): 44.09 Frame: 0033 Latency (ms): 40.54 Frame: 0034 Latency (ms): 43.36 Frame: 0035 Latency (ms): 39.68 Frame: 0036 Latency (ms): 40.78 Frame: 0037 Latency (ms): 43.99 Frame: 0038 Latency (ms): 39.63 Frame: 0039 Latency (ms): 43.80 Frame: 0040 Latency (ms): 39.93 Frame: 0041 Latency (ms): 43.58 Frame: 0042 Latency (ms): 40.27 Frame: 0043 Latency (ms): 40.40 Frame: 0044 Latency (ms): 43.87 Frame: 0045 Latency (ms): 39.60 Frame: 0046 Latency (ms): 44.33 Frame: 0047 Latency (ms): 39.19 Frame: 0048 Latency (ms): 40.56 Frame: 0049 Latency (ms): 43.97 Frame: 0050 Latency (ms): 40.57 Frame: 0051 Latency (ms): 43.21 Frame: 0052 Latency (ms): 39.59 Frame: 0053 Latency (ms): 44.51 Frame: 0054 Latency (ms): 39.70 Frame: 0055 Latency (ms): 41.77 Frame: 0056 Latency (ms): 41.66 Frame: 0057 Latency (ms): 40.38 Frame: 0058 Latency (ms): 44.05 Frame: 0059 Latency (ms): 39.31 Frame: 0060 Latency (ms): 40.69 Frame: 0061 Latency (ms): 45.08 Frame: 0062 Latency (ms): 38.67 Frame: 0063 Latency (ms): 43.07 Frame: 0064 Latency (ms): 45.87 Frame: 0065 Latency (ms): 38.71 Frame: 0066 Latency (ms): 39.75 Frame: 0067 Latency (ms): 43.59 Frame: 0068 Latency (ms): 40.40 Frame: 0069 Latency (ms): 116.08

保存的图像在~/lerobot/outputs/images_from_opencv_cameras，确定哪个相机索引对应于哪个相机，0 对应 camera_00， 1 对应camera_01……

camera_00_frame_000000.png
    [...]
    camera_00_frame_000069.png
    camera_02_frame_000000.png
    [...]
    camera_02_frame_000069.png
    camera_04_frame_000000.png
    [...]
    camera_0_frame_000069.png

修改三个相机的索引 camera_index

vim ~/lerobot/lerobot/common/robot_devices/robots/configs.py

修改的内容如下，camera_index相机索引，fps帧数

@RobotConfig.register_subclass("so100")
    ...

    cameras: dict[str, CameraConfig] = field(
        default_factory=lambda: {
            "laptop": OpenCVCameraConfig(
                camera_index=0,
                fps=30,
                width=640,
                height=480,
            ),
            "phone1": OpenCVCameraConfig(
                camera_index=2,
                fps=30,
                width=640,
                height=480,
            ),
            "phone2": OpenCVCameraConfig(
                camera_index=4,
                fps=30,
                width=640,
                height=480,
            ),
        }
    )

加载lerobot环境

conda activate lerobot

cd ~/lerobot

运行程序，操控机械臂，显示两个相机的画面

python ~/lerobot/lerobot/scripts/control_robot.py \
      --robot.type=so100 \
      --control.type=teleoperate

请输入图片描述

标签: none