Some data from waveshare IMX219 based Stero CSI camera exploration on jetson nano.
Some informationabout tis stereo CSI camera can be found in the following link:
https://www.waveshare.com/wiki/IMX219-83_Stereo_Camera
ls /dev/video*
Testing by show video image from both cameras:
camera 0
gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=3280, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=2 ! 'video/x-raw, width=816, height=616' ! nvvidconv ! nvegltransform ! nveglglessink -e
camera 1
gst-launch-1.0 nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM),width=3280, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=2 ! 'video/x-raw, width=816, height=616' ! nvvidconv ! nvegltransform ! nveglglessink -e
Test the ROS Package for Jetson CSI Stereo Camera and GPU Accelerated Depth Processing, available on the following repo:
- https://github.com/borongyuan/jetson_csi_stereo_ros
This packages is tested on JetPack 4.4 (OpenCV 4.1, VisionWorks 1.6), but i have JetPack 4.5.11, with opencv 4.1.1.
The installation describe a special ros install process where the opencv parts are installed by compilation from git repositories.
The package on this repository supply some launch files, and is depend from the following one:
https://github.com/WHILL/gpu_stereo_image_proc
error at caktkin_make:
Project ‘cv_bridge’ specifies ‘/usr/include/opencv’ as an include dir,
which is not found.
Possible solution:
sudo ln -s /usr/include/opencv4/opencv2/ /usr/include/opencv
O que nos leva ao próximo erro, na qual é requerida uma versão especifica do opencv.
CMake Error at gpu_stereo_image_proc/CMakeLists.txt:24 (find_package):
Could not find a configuration file for package “OpenCV” that is compatible
with requested version “3.2”.
The following configuration files were considered but not accepted:
/usr/lib/aarch64-linux-gnu/cmake/opencv4/OpenCVConfig.cmake, version: 4.1.1
On CMakeLists.txt, replace
find_package(OpenCV 3.2 REQUIRED)
with
find_package(OpenCV REQUIRED)
As is, its not working, because it is a standard melodic install. So the special procedure about opencv ros instalation was not done.
I was unable to make the package jetson_csi_stereo_ros working, because i cant build the gpu_stereo_image_proc package.
The problem is a dependency to opencv 3.2.0 that i did not manage to solve.
This package use the gpu to process the disparity data, by using libSGM.
I check building libSGM outside the package and it works, so the problem is inside gpu_stereo_image_proc package.
So… frustrated check the code demos on the product wiki,
- https://www.waveshare.com/wiki/IMX219-83_Stereo_Camera
And start another aproach described HERE demos_on_wiki_TODO
But. Later after a check the possibility of have multiple opencv versions by build alternatives, I also look what are the apt opencv install possibilities.
the issue about opencv include was solved by a symbolic link:
# stero camera, build deep package
sudo ln -s /usr/include/opencv4/opencv2 /usr/include/opencv
The next problem is the building fail with the following messages.
[ 19%] Building CXX object gpu_stereo_image_proc/CMakeFiles/vx_stereo_image_proc.dir/src/nodelets/vx_disparity.cpp.o
make[2]: *** No rule to make target ‘/usr/lib/aarch64-linux-gnu/libopencv_objdetect.so.3.2.0’, needed by ‘/home/inaciose/catkin_ws/devel/lib/libvx_stereo_image_proc.so’. Stop.
Solving by install the following package
sudo apt install libopencv-contrib3.2
The following additional packages will be installed:
liblept5 libopencv-objdetect3.2 libopencv-photo3.2 libopencv-shape3.2 libopencv-stitching3.2 libopencv-superres3.2 libopencv-video3.2 libopencv-videostab3.2 libopencv-viz3.2 libtesseract4
The following NEW packages will be installed:
liblept5 libopencv-contrib3.2 libopencv-objdetect3.2 libopencv-photo3.2 libopencv-shape3.2 libopencv-stitching3.2 libopencv-superres3.2 libopencv-video3.2 libopencv-videostab3.2 libopencv-viz3.2 libtesseract4
0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
After install the 3.2 contribs it works. But there is this waring message:
[ 95%] Linking CXX executable /home/inaciose/catkin_ws/devel/lib/gpu_stereo_image_proc/vx_gpu_stereo_image_proc
/usr/bin/ld: warning: libopencv_imgproc.so.3.2, needed by /opt/ros/melodic/lib/libimage_proc.so, may conflict with libopencv_imgproc.so.4.1
Remark: this is not the better option. The better is build the opencv 3.2 to an alternate directory and link to it. For now its ok.
Check the built package
rosrun nodelet nodelet manager __name:=manager roslaunch gpu_stereo_image_proc comparison.launch manager:=/manager __ns:=/camera_topicname rosrun rviz rviz
the name manager is our choice and will be the name of the node
It shows the topics to select in rviz but i cant any see image from cameras
To try the package jetson_csi_stereo_ros, we need to install the package:
sudo apt install ros-melodic-gscam
(no dependencies on this one)
it also builds, but this package have only launchers and configs.
I have to do the calibration of cameras later.
- http://wiki.ros.org/camera_calibration/Tutorials/StereoCalibration
Test
roslaunch jetson_csi_stereo_ros jetson_csi_stereo.launch
Stereo Image Processing using VisionWorks
roslaunch jetson_csi_stereo_ros jetson_csi_stereo_pipeline.launch
it worked, slow and strange maybe because the cams are not calibrated.
https://github.com/borongyuan/jetson_csi_stereo_ros
https://github.com/WHILL/gpu_stereo_image_proc
https://github.com/fixstars/libSGM