Skip to content

Add integration tests #422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
joaoantoniocardoso wants to merge 20 commits into
base: master
Choose a base branch
from
Draft

Add integration tests #422

joaoantoniocardoso wants to merge 20 commits into from
+2,014 −1,018

Conversation

@joaoantoniocardoso
Copy link
Collaborator

@joaoantoniocardoso joaoantoniocardoso commented Jul 1, 2024 •
edited
Loading

First step towards #378

How to test locally

1. Install v4l2loopback:

  • Ubuntu: sudo apt-get install -y v4l2loopback-dkms
  • Arch: sudo pacman -S v4l2loopback-dkms v4l2loopback-utils

2. Add a v4l2loopback device:

sudo modprobe v4l2loopback video_nr=42 && \
sudo v4l2loopback-ctl delete /dev/video42 && \
sudo v4l2loopback-ctl add --exclusive-caps 0 --max-openers 2 --name potato --verbose /dev/video42

3. Get qrtimestamp plugins:

wget -q https://github.com/patrickelectric/qrtimestamp-gst/releases/download/0.1.0/libgstqrtimestamp-x86_64.so.tar -O - | tar -x
export GST_PLUGIN_PATH="$PWD:$GST_PLUGIN_PATH"

4. Run the test

cargo test --release --locked --test v4l2_latency_and_jitter

5. Clean up

sudo modprobe -r v4l2loopback

Test output

Results for ~100 frames at ~30 FPS, in milliseconds:

Baseline Run Added latency
UDP H264 25.99 34.80 8.81
UDP MJPEG 26.71 34.94 8.23
UDP YUYV 26.99 36.61 9.62
RTSP H264 27.27 35.00 7.73
RTSP MJPEG 26.66 35.18 8.53
RTSP YUYV 26.90 37.52 10.62
Details 
2024-12-12T21:34:42.214270Z  INFO src/lib/logger/manager.rs:193: mavlink-camera-manager, version: 0.2.4-fd5634e, build date: 2024-12-12
2024-12-12T21:34:42.215378Z  INFO src/lib/logger/manager.rs:200: Starting at 2024-12-12T21:34:42
2024-12-12T21:34:42.215586Z  INFO src/lib/logger/manager.rs:204: Server running at 0.0.0.0:6020
2024-12-12T21:34:42.231049Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "udp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    5600,
                ),
                path: "",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: H264,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:34:42.231486Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:34:42.329457Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:34:43.239030Z  INFO src/lib/stream/manager.rs:553: Stream 208c433c-fb32-4dab-a23d-f4e85c5c0610 successfully added!
2024-12-12T21:34:43.239163Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
2024-12-12T21:34:46.566817Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 5.0,
    latency_max: 30.0,
    latency_mean: 25.989898989898983,
    latency_std_dev: 3.3911497944603344,
    latency_variance: 11.499896928468367,
    jitter_min: -4.0,
    jitter_max: 11.0,
    jitter_mean: 0.21428571428571463,
    jitter_std_dev: 1.9117110595635871,
    jitter_variance: 3.654639175257733,
}
2024-12-12T21:34:46.567015Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:34:46.574273Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:34:48.138566Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "set state timed-out (5 seconds)"
2024-12-12T21:34:48.140233Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "set state timed-out (5 seconds)"
2024-12-12T21:35:05.627754Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:05.629443Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:05.630675Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:05.736154Z  INFO src/lib/stream/manager.rs:572: Stream 208c433c-fb32-4dab-a23d-f4e85c5c0610 successfully removed!
2024-12-12T21:35:05.736869Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:05.737021Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:05.737094Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 34.0,
    latency_max: 36.0,
    latency_mean: 34.7979797979798,
    latency_std_dev: 0.4512974036336674,
    latency_variance: 0.2036693465264893,
    jitter_min: -1.0,
    jitter_max: 2.0,
    jitter_mean: 2.0816681711721685e-17,
    jitter_std_dev: 0.717958158617738,
    jitter_variance: 0.5154639175257731,
}
2024-12-12T21:35:05.737197Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 8.808080808080817 <= 21...
2024-12-12T21:35:05.737321Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting 0.000000000000000020816681711721685 <= 11...
2024-12-12T21:35:05.750789Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "udp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    5600,
                ),
                path: "",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: Mjpg,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:35:05.751364Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:35:07.995063Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:35:08.204992Z  INFO src/lib/stream/manager.rs:553: Stream 59ed3d69-663c-4fdb-9f13-ed94db0b8504 successfully added!
2024-12-12T21:35:08.205221Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
2024-12-12T21:35:11.522232Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 24.0,
    latency_max: 29.0,
    latency_mean: 26.70707070707071,
    latency_std_dev: 0.610328202983983,
    latency_variance: 0.37250051535765805,
    jitter_min: -5.0,
    jitter_max: 3.0,
    jitter_mean: 0.010204081632653078,
    jitter_std_dev: 0.8909036066106302,
    jitter_variance: 0.7937092362718285,
}
2024-12-12T21:35:11.522365Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:35:11.524738Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:35:14.835531Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:14.836886Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:14.838058Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:14.939475Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:14.940660Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:14.977206Z  INFO src/lib/stream/manager.rs:572: Stream 59ed3d69-663c-4fdb-9f13-ed94db0b8504 successfully removed!
2024-12-12T21:35:14.977357Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:14.977423Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:14.977477Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 34.0,
    latency_max: 37.0,
    latency_mean: 34.93939393939396,
    latency_std_dev: 0.5499198794737696,
    latency_variance: 0.30241187384044527,
    jitter_min: -1.0,
    jitter_max: 2.0,
    jitter_mean: 3.469446951953614e-18,
    jitter_std_dev: 0.7321765321469982,
    jitter_variance: 0.5360824742268042,
}
2024-12-12T21:35:14.977588Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 8.23232323232325 <= 13...
2024-12-12T21:35:14.977643Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting 0.000000000000000003469446951953614 <= 3...
2024-12-12T21:35:14.992097Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "udp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    5600,
                ),
                path: "",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: Yuyv,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:35:14.992450Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:35:15.004949Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:35:15.219595Z  INFO src/lib/stream/manager.rs:553: Stream 450619fe-a712-4199-951f-d2dc657fa13a successfully added!
2024-12-12T21:35:15.219750Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
2024-12-12T21:35:18.535712Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 25.0,
    latency_max: 28.0,
    latency_mean: 26.98989898989899,
    latency_std_dev: 0.5049742263568345,
    latency_variance: 0.2549989692846836,
    jitter_min: -2.0,
    jitter_max: 2.0,
    jitter_mean: 0.020408163265306124,
    jitter_std_dev: 0.6731919938258556,
    jitter_variance: 0.4531874605512308,
}
2024-12-12T21:35:18.535832Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:35:18.537830Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:35:25.447452Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:25.448738Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:25.449915Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:25.550753Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:25.552025Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:25.586767Z  INFO src/lib/stream/manager.rs:572: Stream 450619fe-a712-4199-951f-d2dc657fa13a successfully removed!
2024-12-12T21:35:25.586869Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:25.586919Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:25.586962Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 35.0,
    latency_max: 68.0,
    latency_mean: 36.6060606060606,
    latency_std_dev: 5.499198794737694,
    latency_variance: 30.24118738404451,
    jitter_min: -29.0,
    jitter_max: 30.0,
    jitter_mean: 7.554897750478585e-17,
    jitter_std_dev: 4.333994134126069,
    jitter_variance: 18.783505154639172,
}
2024-12-12T21:35:25.587033Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 9.616161616161612 <= 32...
2024-12-12T21:35:25.587075Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting 0.00000000000000007554897750478585 <= 2...
2024-12-12T21:35:25.597295Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "rtsp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    8554,
                ),
                path: "/test",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: H264,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:35:25.597595Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:35:25.611000Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:35:25.826640Z  INFO src/lib/stream/manager.rs:553: Stream c0768f78-2e40-4350-ab3b-46ec8a9d2950 successfully added!
2024-12-12T21:35:25.826761Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
2024-12-12T21:35:29.153783Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 25.0,
    latency_max: 30.0,
    latency_mean: 27.272727272727273,
    latency_std_dev: 1.3983823147945773,
    latency_variance: 1.9554730983302404,
    jitter_min: -4.0,
    jitter_max: 3.0,
    jitter_mean: -2.5773034500226846e-17,
    jitter_std_dev: 1.5860214114335829,
    jitter_variance: 2.5154639175257745,
}
2024-12-12T21:35:29.153956Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:35:29.172989Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:35:38.921449Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:38.923241Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:38.924244Z  INFO src/lib/stream/manager.rs:572: Stream c0768f78-2e40-4350-ab3b-46ec8a9d2950 successfully removed!
2024-12-12T21:35:38.924372Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:38.924427Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:38.924418Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:38.924475Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 34.0,
    latency_max: 36.0,
    latency_mean: 35.0,
    latency_std_dev: 0.4285714285714286,
    latency_variance: 0.1836734693877551,
    jitter_min: -2.0,
    jitter_max: 1.0,
    jitter_mean: -0.010204081632653062,
    jitter_std_dev: 0.6175257907579325,
    jitter_variance: 0.38133810225120984,
}
2024-12-12T21:35:38.924556Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 7.727272727272727 <= 13...
2024-12-12T21:35:38.924606Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting -0.010204081632653062 <= 3...
2024-12-12T21:35:38.937615Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "rtsp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    8554,
                ),
                path: "/test",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: Mjpg,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:35:38.937950Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:35:38.946408Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:35:39.026659Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:39.154905Z  INFO src/lib/stream/manager.rs:553: Stream a35494e6-4586-4dbe-97c1-ccabf9a1dd4f successfully added!
2024-12-12T21:35:39.155047Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
test main has been running for over 60 seconds
2024-12-12T21:35:42.472290Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 25.0,
    latency_max: 28.0,
    latency_mean: 26.65656565656566,
    latency_std_dev: 0.5182705967557434,
    latency_variance: 0.2686044114615544,
    jitter_min: -1.0,
    jitter_max: 2.0,
    jitter_mean: 0.010204081632653078,
    jitter_std_dev: 0.739112004948815,
    jitter_variance: 0.5462865558594572,
}
2024-12-12T21:35:42.472413Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:35:42.474585Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:35:45.822244Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:45.824481Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:45.825657Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:45.927880Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:45.930938Z  INFO src/lib/stream/manager.rs:572: Stream a35494e6-4586-4dbe-97c1-ccabf9a1dd4f successfully removed!
2024-12-12T21:35:45.931147Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:45.931241Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:45.931318Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 35.0,
    latency_max: 36.0,
    latency_mean: 35.18181818181818,
    latency_std_dev: 0.38765744302855326,
    latency_variance: 0.15027829313543603,
    jitter_min: -1.0,
    jitter_max: 1.0,
    jitter_mean: -8.673617379884035e-18,
    jitter_std_dev: 0.5920434645509787,
    jitter_variance: 0.3505154639175259,
}
2024-12-12T21:35:45.931434Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 8.525252525252519 <= 12...
2024-12-12T21:35:45.931513Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting -0.000000000000000008673617379884035 <= 2...
2024-12-12T21:35:45.945611Z  INFO tests/v4l2_latency_and_jitter.rs:632: Testing for: VideoAndStreamInformation {
    name: "QRTimeStamp - QR",
    stream_information: StreamInformation {
        endpoints: [
            Url {
                scheme: "rtsp",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: Some(
                    Domain(
                        "127.0.0.1",
                    ),
                ),
                port: Some(
                    8554,
                ),
                path: "/test",
                query: None,
                fragment: None,
            },
        ],
        configuration: Video(
            VideoCaptureConfiguration {
                encode: Yuyv,
                height: 320,
                width: 320,
                frame_interval: FrameInterval {
                    numerator: 1,
                    denominator: 30,
                },
            },
        ),
        extended_configuration: Some(
            ExtendedConfiguration {
                thermal: false,
                disable_mavlink: true,
            },
        ),
    },
    video_source: Local(
        VideoSourceLocal {
            name: "potato",
            device_path: "/dev/video42",
            typ: V4L2Loopback(
                "platform:v4l2loopback-042",
            ),
        },
    ),
}
2024-12-12T21:35:45.946026Z  INFO tests/v4l2_latency_and_jitter.rs:634: Building v4lloopback pipeline (video generation with qrtimestampsrc)...
2024-12-12T21:35:45.954693Z  INFO tests/v4l2_latency_and_jitter.rs:638: Building MCM stream...
2024-12-12T21:35:46.168292Z  INFO src/lib/stream/manager.rs:553: Stream 4b7fd412-017c-4ac9-86de-8c9ebcd12860 successfully added!
2024-12-12T21:35:46.168431Z  INFO tests/v4l2_latency_and_jitter.rs:645: Getting baseline latency...
2024-12-12T21:35:49.484139Z  INFO tests/v4l2_latency_and_jitter.rs:650: Baseline latency: LatenciesStats {
    latency_min: 26.0,
    latency_max: 28.0,
    latency_mean: 26.898989898989896,
    latency_std_dev: 0.41637280917881647,
    latency_variance: 0.1733663162234591,
    jitter_min: -1.0,
    jitter_max: 2.0,
    jitter_mean: -6.372453585220925e-18,
    jitter_std_dev: 0.6092076990801715,
    jitter_variance: 0.3711340206185567,
}
2024-12-12T21:35:49.484274Z  INFO tests/v4l2_latency_and_jitter.rs:652: Building qrtimestamp pipeline (video receiver with qrtimestampsink)...
2024-12-12T21:35:49.486078Z  INFO tests/v4l2_latency_and_jitter.rs:660: Waiting for QrTimestampSink Pipeline to finish...
2024-12-12T21:35:52.833175Z  INFO tests/v4l2_latency_and_jitter.rs:663: Finishing loopback pipeline...
2024-12-12T21:35:52.834516Z  INFO tests/v4l2_latency_and_jitter.rs:666: Finishing MCM stream...
2024-12-12T21:35:52.835569Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:52.937186Z  WARN src/lib/stream/pipeline/runner.rs:59: task ended with error: "Some(\"Received EndOfStream: Eos { structure: None, source: None }\")"
2024-12-12T21:35:53.015688Z  INFO src/lib/stream/manager.rs:572: Stream 4b7fd412-017c-4ac9-86de-8c9ebcd12860 successfully removed!
2024-12-12T21:35:53.015834Z  INFO tests/v4l2_latency_and_jitter.rs:671: Results appended...
2024-12-12T21:35:53.015901Z  INFO tests/v4l2_latency_and_jitter.rs:673: Evaluating results...
2024-12-12T21:35:53.015954Z  INFO tests/v4l2_latency_and_jitter.rs:676: Pipeline latency: LatenciesStats {
    latency_min: 37.0,
    latency_max: 39.0,
    latency_mean: 37.51515151515151,
    latency_std_dev: 0.5952813837073163,
    latency_variance: 0.3543599257884971,
    jitter_min: -2.0,
    jitter_max: 1.0,
    jitter_mean: -0.020408163265306117,
    jitter_std_dev: 0.7992211651730134,
    jitter_variance: 0.6387544708605092,
}
2024-12-12T21:35:53.016040Z  INFO tests/v4l2_latency_and_jitter.rs:681: Asserting 10.616161616161612 <= 32...
2024-12-12T21:35:53.016090Z  INFO tests/v4l2_latency_and_jitter.rs:685: Asserting -0.020408163265306117 <= 2...
test main ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 71.32s

@joaoantoniocardoso joaoantoniocardoso force-pushed the add_integration_tests branch 5 times, most recently from 76d8a58 to 1677dd8 Compare July 18, 2024 20:36
@joaoantoniocardoso joaoantoniocardoso force-pushed the add_integration_tests branch 2 times, most recently from 21006ab to 4eda092 Compare July 22, 2024 19:20
@joaoantoniocardoso joaoantoniocardoso marked this pull request as ready for review July 23, 2024 11:46
@joaoantoniocardoso joaoantoniocardoso force-pushed the add_integration_tests branch 20 times, most recently from 07c8b16 to 8b7b654 Compare October 1, 2024 19:22
@joaoantoniocardoso joaoantoniocardoso force-pushed the add_integration_tests branch 19 times, most recently from be2ddb4 to e95b166 Compare December 14, 2024 02:58
@joaoantoniocardoso joaoantoniocardoso marked this pull request as draft March 29, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@patrickelectric patrickelectric Awaiting requested review from patrickelectric

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@joaoantoniocardoso