{"id":3609,"date":"2021-03-09T06:35:00","date_gmt":"2021-03-09T14:35:00","guid":{"rendered":"http:\/\/192.168.3.4\/?p=3609"},"modified":"2021-09-10T06:32:48","modified_gmt":"2021-09-10T13:32:48","slug":"software-gimbal","status":"publish","type":"post","link":"https:\/\/www.cloudacm.com\/?p=3609","title":{"rendered":"Software Gimbal"},"content":{"rendered":"<p>In this post I&#8217;ll demonstrate how to stabilize and enhance low resolution video using FFMpeg. I will also discuss the pros and cons of hardware options available. From there the discussion will show examples of software options that have supplanted hardware. Lastly, I will introduce some examples of AI that others have used.<\/p>\n<p><strong>The Z51 660mm<\/strong><\/p>\n<p>Below is a side by side comparison of video taken from one of my fixed wing remote control (RC) planes.<\/p>\n<p><iframe loading=\"lazy\" title=\"Z51 Footage with 808-16, Original and Processed\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/RYswci3fKVQ?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>Here is the script I used to process that video.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\">#!\/bin\/bash\r\n\r\n# Take a 40 second clip out of the original - this small sample will be uploaded to Youtube\r\nffmpeg -ss 00:00:00 -i MINI0005.MOV -t 00:00:40 -c copy MINI0005_Segment.MOV\r\n\r\n# Increase the FPS and interpolate missing frames so the video appearance flows better \r\nffmpeg -i MINI0005_Segment.MOV -vf minterpolate=fps=120 MINI0005_Int.MOV\r\n\r\n# Sharpen and lens correct the video based on the 808-16 720p camera\r\nffmpeg -i MINI0005_Int.MOV -vf \"unsharp=3:3:5, lenscorrection=cx=0.5:cy=0.5:k1=-0.222:k2=-0.0\" Combine.MOV\r\n\r\n# Create a stabilization profile of the video\r\nffmpeg -i Combine.MOV -vf vidstabdetect=shakiness=10:accuracy=15 -f null -\r\n\r\n# Apply video stabilization from the profile\r\nffmpeg -i Combine.MOV -vf vidstabtransform=smoothing=240:input=\"transforms.trf\" Combine_Stab_Extreme.MOV\r\n\r\n# Merge the source and processed videos together for a side by side view\r\n# Left is the original while right is the processed\r\nffmpeg -i \"MINI0005_Segment.MOV\" -vf \"[in] scale=iw:ih, pad=2*iw:ih [left];movie=Combine_Stab_Extreme.MOV, scale=iw:ih [right]; [left][right] overlay=main_w\/2:0 [out]\" \"MINI0005_Youtube.MOV\"\r\n<\/pre>\n<p>The video was shot with the 808-16 720p camera mounted inside the fuselage of a Z51 rc plane, featured here<\/p>\n<p><iframe loading=\"lazy\" title=\"Z51 Predator Drone (MQ-9 Reaper) Flight Test Review\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/6PjXSmILRZA?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><iframe loading=\"lazy\" title=\"Z51 Predator Drone (MQ-9 Reaper) FPV Camera Conversion Flight Test Review\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/9UWWScnLBxw?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>The 808-16 was selected becuase of its size and weight (19 grams), as well as due to the limits of what the Z51 could handle. I had tested heaver devices, but the phugoid oscillations became more pronounced. Attempts to fly with both the camera and a barometric altimeter (total weight of 31 grams) repeatedly ended in a crash at take off.<\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3612\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3610\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/IMG_1522-768x1024.jpg\" alt=\"\" width=\"360\" height=\"480\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-768x1024.jpg 768w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-225x300.jpg 225w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-1152x1536.jpg 1152w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-1536x2048.jpg 1536w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-203x270.jpg 203w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1522-scaled.jpg 1920w\" sizes=\"auto, (max-width: 360px) 100vw, 360px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3611\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3611\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/IMG_1524-1024x768.jpg\" alt=\"\" width=\"480\" height=\"360\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-1024x768.jpg 1024w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-300x225.jpg 300w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-768x576.jpg 768w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-1536x1152.jpg 1536w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-2048x1536.jpg 2048w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1524-360x270.jpg 360w\" sizes=\"auto, (max-width: 480px) 100vw, 480px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3612\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3612\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/IMG_1525-1024x768.jpg\" alt=\"\" width=\"480\" height=\"360\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-1024x768.jpg 1024w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-300x225.jpg 300w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-768x576.jpg 768w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-1536x1152.jpg 1536w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-2048x1536.jpg 2048w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/IMG_1525-360x270.jpg 360w\" sizes=\"auto, (max-width: 480px) 100vw, 480px\" \/><\/a><\/p>\n<p><strong>The Volantex Ranger 600mm<\/strong><\/p>\n<p>Here is another side by side video. The script used to process the video was almost identical to the one listed above, since it used the same camera.<\/p>\n<p><iframe loading=\"lazy\" title=\"Volantex Ranger 600 808-16 Camera Footage, Original and Processed\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/fqrau6ipc-Y?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>This time video was shot onboard a different plane, the Volantex Ranger 600 featured here.<\/p>\n<p><iframe loading=\"lazy\" title=\"Volantex Ranger 600 - PERFECT for the beginner in you\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/xliePn17qXI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><iframe loading=\"lazy\" title=\"V761-2 Ranger 600 RTF Three Channel Stabilized RC Glider Flight Test Review\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/GQXp0-QKQUo?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>I want to draw attention to the comment about FPV. Both of the fliers were prone to rolls and pitches in moderate wind. I can attest to this based on all of the video I shot. Since FPV is a live stream, the video stabilzation I have demonstrated here would not apply. Here is a list of pros and cons of software versus hardware gimbal stabilzation.<\/p>\n<p><strong>Hardware Limits<\/strong><\/p>\n<p>Weight is the biggest problem facing hardware. This is especially true for the planes I used. The Z51 was limited to 20 grams, where the Ranger 600 could handle more than double of that. Adding weight limits the operation of the plane by draining power for the batteries more quickly.<\/p>\n<p>This leads us to power being another problem for hardware. Either that engery is coming from the onboard battery or a differnt power source is needed, which adds more weight.<\/p>\n<p>Hardware gimbals aren&#8217;t free. Costs can range with the quality of the gimbal.<\/p>\n<p>Hardware gimbals take up space and most are fitted on the exterior of the plane. Here is an example of a gimble that doesn&#8217;t include servos or the controller board and is fitted for the Mobius 1080p camera, <a href=\"https:\/\/www.amazon.com\/dp\/B07TGK8276\/ref=cm_sw_em_r_mt_dp_QQ7HWVFMZJX8EG9EQST\">https:\/\/www.amazon.com\/dp\/B07TGK8276\/ref=cm_sw_em_r_mt_dp_QQ7HWVFMZJX8EG9EQST<\/a>. If we were to mount this gimbal underneath the aircraft, it would stick out around 90mm. That surface area would be a lot of drag.<\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3613\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3613\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/Mobius-Gimbal-Under.jpg\" alt=\"\" width=\"370\" height=\"366\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Mobius-Gimbal-Under.jpg 500w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Mobius-Gimbal-Under-300x296.jpg 300w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Mobius-Gimbal-Under-273x270.jpg 273w\" sizes=\"auto, (max-width: 370px) 100vw, 370px\" \/><\/a><\/p>\n<p><strong>Software Limits<\/strong><\/p>\n<p>The software can not be used for FPV flight, as I pointed out earlier. There is no realtime stablizaion available to live streams, at least non feature here. My focus is purely recorded flight, not FPV.<\/p>\n<p>Software processing is lossy, that means video from the original is either missing or distorted. Altough the distortion is mostly artifacts introduced from extreme stabilzation, the loss of the outer edge of video is noticable.<\/p>\n<p><strong>Software Replacing Hardware<\/strong><\/p>\n<p>Interestingly, software is being increasingly utilized as a means to replace tasks traditionly handled by hardware. My use of software was to attempted to overcome the inherint limitation of the 808-16 camera. Other examples of this can be found in music, photography, and radio. The SDR has replaced a significant amount of hardware radios had used in the past.<\/p>\n<p><iframe loading=\"lazy\" title=\"SDR School Part One The Basics\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/ncxyycmSeWU?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><iframe loading=\"lazy\" title=\"SDR School Part Two   the Hardware\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/1JSyaZ_VS6k?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><iframe loading=\"lazy\" title=\"SDR School Part 3   Software Puts it all Together\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/D2yyDGbw8H8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><strong>Advanced Processing<\/strong><\/p>\n<p>I had mentioned that FPV processing was a limit of software, however this is likely to change over time. Tasks done using cloud based processing have shown some remarkable results. Here is an example of footage with both limitaions but also age.<\/p>\n<p><iframe loading=\"lazy\" title=\"[4k, 60 fps] San Francisco, a Trip down Market Street, April 14, 1906\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/VO_1AdYRGW8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><strong>Side Note<\/strong><\/p>\n<p>I just wanted to finish this post with a return to an earlier subject. The Ranger 600 footage was calculated to be at an elevation of 550 feet at this location, 47.660393, -122.112908. Again, it was critical that I used un-process source footage to determine position.<\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3614\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3614\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/Arial-1024x576.png\" alt=\"\" width=\"720\" height=\"405\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Arial-1024x576.png 1024w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Arial-300x169.png 300w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Arial-768x432.png 768w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Arial-480x270.png 480w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Arial.png 1280w\" sizes=\"auto, (max-width: 720px) 100vw, 720px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/www.cloudacm.com\/?attachment_id=3615\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-3615\" src=\"http:\/\/192.168.3.4\/wp-content\/uploads\/2021\/03\/Map-1024x729.png\" alt=\"\" width=\"720\" height=\"513\" srcset=\"https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Map-1024x729.png 1024w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Map-300x214.png 300w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Map-768x547.png 768w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Map-379x270.png 379w, https:\/\/www.cloudacm.com\/wp-content\/uploads\/2021\/03\/Map.png 1259w\" sizes=\"auto, (max-width: 720px) 100vw, 720px\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post I&#8217;ll demonstrate how to stabilize and enhance low resolution video using FFMpeg. I will also discuss the pros and cons of hardware options available. From there the discussion will show examples of software options that have supplanted hardware. Lastly, I will introduce some examples of AI that others have used. The Z51 660mm Below is a side by side comparison of video taken from one of my fixed wing remote control (RC) planes. Here is the script&#8230;<\/p>\n<p class=\"read-more\"><a class=\"btn btn-default\" href=\"https:\/\/www.cloudacm.com\/?p=3609\"> Read More<span class=\"screen-reader-text\">  Read More<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,3],"tags":[],"class_list":["post-3609","post","type-post","status-publish","format-standard","hentry","category-computer-vision","category-rd"],"_links":{"self":[{"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/posts\/3609","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3609"}],"version-history":[{"count":5,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/posts\/3609\/revisions"}],"predecessor-version":[{"id":3622,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=\/wp\/v2\/posts\/3609\/revisions\/3622"}],"wp:attachment":[{"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3609"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3609"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cloudacm.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3609"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}