6. ROS API
The ROS API for the MultiSense S7 sensor, provided by the ros_driver executable, is split into distinct subsystems for each piece of hardware. Each of these subsystems is documented below.
Streams from the sensor are initiated on an "on-demand" basis. That is, on subscription of a given data stream (such as /multisense/left/image_rect), the driver will initialize the data stream on the sensor. Thus, the driver uses very little bandwidth and CPU when there are no subscriptions to any of the ROS topics.
6.1. Camera Topics
Camera topics outlined below are supported on the MultiSense S7.
All The focal lengths and image center in K and P are scaled based on the current operating resolution. Both the plumb_bob and rational polynomial distortion models are supported. |
6.1.1. Depth Camera
Topic |
/multisense/camera_info |
Notes |
Camera projection matrix and metadata. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/depth |
Notes |
Depth image in canonical representation (values are depth in meters and are float32). |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/openni_depth |
Notes |
Depth image in OpenNI representation (values are depth in millimeters and are uint16). |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/image_points2 |
Notes |
Greyscale stereo point cloud. Each point contains 4 fields (x, y, z, lumanince). |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/image_points2_color |
Notes |
Color stereo point cloud. Each point contains 4 fields (x, y, z, rgb). Color images are rectified on the host machine before combining with range data. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/organized_image_points2 |
Notes |
Stereo point cloud. Each point contains 4 fields (x,y,z, lumanince). |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/organized_image_points2_color |
Notes |
Color stereo point cloud. Each point contains 4 fields (x,y,z, rgb). The number of points is equal to the size of the left rectified camera image, allowing for direct indexing based off camera pixel coordinates. Invalid points are populated with std::numeric_limits::quiet_NaN values for the X,Y, and Z point fields. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/disparity |
Notes |
Left disparity image in "mono16" format (units are 1/16th of a disparity.) |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/disparity_image |
Notes |
Left disparity image in DisparityImage format. Note that this type has been deprecated by ROS, see REP-118. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/disparity/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the left disparity topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/disparity |
Notes |
Right disparity image in "mono8" format. This stream is provided for debugging purposes ONLY. Please use the left/disparity topic for ALL purposes. Note that this topic will only be published if the left/disparity topic is also subscribed to. |
Message Type |
|
Minimum Firmware |
3.0 |
Topic |
/multisense/right/disparity_image |
Notes |
Right disparity image in DisparityImage format. This stream is provided for debugging purposes ONLY. Please use the left/disparity topic for ALL purposes. Note that this type has been deprecated by ROS, see REP-118. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/disparity/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the right disparity topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/cost |
Notes |
Left stereo cost in "mono8" format. |
Message Type |
|
Minimum Firmware |
3.0 |
Topic |
/multisense/left/cost/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the cost image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
6.1.2. Left Camera
Topic |
/multisense/left/camera_info |
Notes |
Camera projection matrix and metadata. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_rect |
Notes |
Rectified greyscale images from left camera in mono8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_rect/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the left rectified image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_rect_color |
Notes |
Rectified color images from left camera in RGB8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_rect_color/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the left rectified color image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_mono |
Notes |
Unrectified greyscale images from left camera in mono8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_mono/camera_info |
Notes |
Camera projection matrix and metadata corresponding to the left mono image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_color |
Notes |
Unrectified color images from left camera in RGB8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/left/image_color/camera_info |
Notes |
Camera projection matrix and metadata corresponding to the left color image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
6.1.3. Right Camera
Topic |
/multisense/right/camera_info |
Notes |
Camera projection matrix and metadata. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/image_rect |
Notes |
Rectified greyscale images from right camera in mono8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/image_rect/camera_info |
Notes |
Rectified camera projection matrix and metadata corresponding to the right rectified mono image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/image_mono |
Notes |
Unrectified greyscale images from right camera in mono8 format. |
Message Type |
|
Minimum Firmware |
2.0 |
Topic |
/multisense/right/image_mono/camera_info |
Notes |
Camera projection matrix and metadata corresponding to the right mono image topic. See the Camera Info Note at the top of this section for matrix scale information. |
Message Type |
|
Minimum Firmware |
2.0 |
In the current release, color images are not supported for the right camera. |
6.1.4. Calibration
Topic |
/multisense/calibration/device_info |
Notes |
Hardware and software versioning information. |
Message Type |
multisense_ros/DeviceInfo |
Minimum Firmware |
2.0 |
Topic |
/multisense/calibration/raw_cam_cal |
Notes |
Rectification and stereo projection matrices:
These matrices are for the maximum operating resolution of the MultiSense unit. |
Message Type |
multisense_ros/RawCamCal |
Minimum Firmware |
2.0 |
Topic |
/multisense/calibration/raw_cam_config |
Notes |
Current operating resolution and camera parameters: fx, fy, cx, cy, tx, ty, tz, roll, pitch, and yaw. |
Message Type |
multisense_ros/RawCamCal |
Minimum Firmware |
2.0 |
Topic |
/multisense/calibration/raw_cam_data |
Notes |
Synchronized left-rectified (greyscale) and left-disparity images. |
Message Type |
multisense_ros/RawCamData |
Minimum Firmware |
2.0 |
6.1.5. Histogram
Topic |
/multisense/histogram |
Notes |
Image histograms for each of the 4 Bayer channels in the left camera image. Note that there must be a subscription to a MultiSense image topic for the histogram topic to be published. Note that histograms from the MultiSense can only record 2^18 pixels in each intensity bin (this is limited by hardware buffers on the FPGA). Above that they saturate and the sum of all the histogram bins does not equal the number of pixels in the image. |
Message Type |
multisense_ros/Histogram |
Minimum Firmware |
3.1 |
6.2. PPS Topics
PPS topics are supported for all MultiSense products including: SL, S7, S7S, and S21.
Topic |
/multisense/pps |
Notes |
Contains the time of the last PPS in the sensor’s clock frame. Published once per second, immediately after the pulse. |
Message Type |
std_msgs/Time |
Minimum Firmware |
2.2 |
Topic |
/multisense/stamped_pps |
Notes |
Contains the time of the last PPS in the sensor’s clock frame. Published once per second, immediately after the pulse. Also includes the host time corresponding to this PPS event. |
Message Type |
multisense_ros/StampedPps |
Minimum Firmware |
2.2 |
Firmware version 2.1 contained a bug that could duplicate PPS events. |
Firmware version 3.2 adds a directed streams implementation, allowing data streams to be sent to IP addresses that differ from control device. This changes the PPS implementation slightly and requires that a data stream (of any type) be created for PPS events to be emitted by the MultiSense {S7}. Prior to firmware 3.2, PPS events were emitted upon system boot. |
6.3. IMU Topics
IMU topics are supported for MultiSense S7.
The ROS driver includes a command line tool for Querying and Changing the IMU Configuration.
Accelerometer and gyroscope messages may arrive out of order with each other. The accelerometer and gyroscope are on two different chips with two different hardware clocks. The MultiSense attempts to estimate the skew between these clocks and the main clock, but this estimation may not be perfectly correct. Additionally, each chip has a hardware FIFO where sensor measurements are buffered before being dumped to an internal FIFO on the MultiSense. This can cause older gyro measurements to occur after newer accelerometer measurements. To minimize latency these measurements are sent directly to the client without any sorting based on time. The best way to minimize these errors is to run the accelerometer and gyroscope at approximately the same frequency. Note that clock skew is taken into account in the sensor head. You can treat all the measurements as though they were in the same timebase. |
Topic |
/multisense/imu/accelerometer |
Notes |
Raw accelerometer data. Each message contains 4 fields: time and x,y,z accelerations in g. |
Message Type |
multisense_ros/RawImuData |
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/accelerometer_vector |
Notes |
Accelerometer vector. |
Message Type |
|
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/gyroscope |
Notes |
Raw gyroscope data. Each message contains 4 fields: time and x,y,z rates in degrees-per-second. |
Message Type |
multisense_ros/RawImuData |
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/gyroscope_vector |
Notes |
Gyroscope vector. |
Message Type |
|
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/magnetometer |
Notes |
Raw magnetometer data. Each message contains 4 fields: time and x,y,z measurements in gauss. |
Message Type |
multisense_ros/RawImuData |
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/magnetometer_vector |
Notes |
Magnetometer vector. |
Message Type |
|
Minimum Firmware |
2.3 |
Topic |
/multisense/imu/imu_data |
Notes |
Standard ROS IMU message. Note that the orientation quaternion is not set in the current implementation, only linear_acceleration and angular_velocity fields are populated. |
Message Type |
|
Minimum Firmware |
2.3 |
6.4. Status Message
The status topic is supported for all MultiSense products including: SL, S7, S7S, and S21. Note that the units without laser will always return "false" for laserOk and laserMotorOk.
Topic |
/multisense/status |
Notes |
This is a custom message, please see the table below for details. |
Message Type |
multisense_ros/DeviceStatus |
Minimum Firmware |
3.4 |
This custom message contains the following fields:
Field |
Type |
Notes |
time |
time |
|
uptime |
time |
|
systemOk |
bool |
|
laserOk |
bool |
Will turn false if the laser reports an error or becomes non-responsive. |
laserMotorOk |
bool |
Will turn false if the spindle motor stalls. |
camerasOk |
bool |
|
imuOk |
bool |
|
externalLedsOk |
bool |
|
processingPipelineOk |
bool |
|
powerSupplyTemp |
float32 |
Returned value is in Celsius and is obtained by a thermistor near the device. |
fpgaTemp |
float32 |
Returned value is in Celsius and is obtained by a thermistor near the device. |
leftImagerTemp |
float32 |
Returned value is in Celsius and is obtained by a thermistor near the device. |
rightImagerTemp |
float32 |
Returned value is in Celsius and is obtained by a thermistor near the device. |
inputVoltage |
float32 |
This field is not supported on the S7S. |
inputCurrent |
float32 |
Returned value is in amps. This field is not supported on the S7S. |
fpgaPower |
float32 |
Returned value is in watts. |
logicPower |
float32 |
Returned value is in watts. |
imagerPower |
float32 |
Returned value is in watts. |
6.5. Parameters
These parameters employ dynamic_reconfigure, and can be modified at runtime by executing the following commands:
Noetic, Melodic, Kinetic |
|
Selecting "/multisense" from the drop-down list will bring up the following configurable parameters.
Parameter | Data Type | Description |
---|---|---|
resolution |
string |
Resolution of the camera images streamed from the sensor, in the format of "width x height x number_of_disparities". In the 3.0_beta and newer releases, each resolution can be configured for 64, 128, or 256 disparities. Since the hardware stereo core has a finite throughput, lower disparities will have the effect of increasing frame-rate at the cost of near-field coverage. Note that changing this value will pause image streams while rectification lookup table is recomputed. |
fps |
double |
Frames per second. |
gain |
double |
Camera gain. |
auto_exposure |
bool |
Enable or disable auto exposure. |
auto_exposure_max_time |
int |
Maximum time (in microseconds) that the AE algorithm will allow for frame capture. |
auto_exposure_decay |
int |
Adjust the auto exposure decay. Increasing this parameter makes the auto exposure algorithm respond more slowly to changes in lighting. |
auto_exposure_thresh |
double |
Adjust the auto exposure threshold. This parameter changes the overall scene brightness target for the AE algorithm. |
exposure_time |
double |
Time (in seconds) for camera exposure. Ignored if auto exposure is enabled. |
auto_white_balance |
bool |
Enable or disable auto white balance. |
auto_white_balance_decay |
int |
Adjust auto white balance decay. Increasing this parameter makes the auto white balance algorithm respond more slowly to changes. |
auto_white_balance_thresh |
double |
Adjust the auto white balance threshold. |
white_balance_red |
double |
Adjust the red white balance. Ignored if auto white balance enabled. |
white_balance_blue |
double |
Adjust the blue white balance. Ignored if auto white balance enabled. |
stereo_post_filtering |
double |
Adjust the strength of the hardware stereo post-filter. This parameter only exists in firmware 3.0 or later. |
lighting |
bool |
Enable or disable LEDs. |
flash |
bool |
Enable or disable flashing of LEDs. |
duty_cycle |
double |
Change brightness of LEDs (0 = Off, 1 = Full Brightness). |
network_time_sync |
bool |
Enable network-based time sync between the host computer and sensor. If disabled, all datum timestamps will be in the frame of the MultiSense’s internal clock, which is free-running from zero on power up. This parameter only exists in firmware 2.1 or later. |
border_clip_type |
int |
Can either be 0:rectangular or 1:circular. When rectangular it operates like the legacy MULITSENSE_POINTCLOUD_BORDER_CLIP environment variable, excluding a When circular it generates a circle, centered around the center of the image, whose radius is equal to the radius of the circle which circumscribes the image minus the Disparity pixels which lie within these regions are included in the point cloud and pixels which lie outside are excluded. This removes points on the edge of the image where the lens model is less accurate and results in poor stereo matches. |
border_clip_value |
double |
Number of pixels trimmed from the edges of disparity images. See |
crop_mode |
bool |
When true, crops a 4MP imager to 2MP resolution, FOV, and frame rate limits. The |
crop_offset |
int |
Offset from the bottom of the imager’s FOV for the cropped FOV. A value of 480 will center the ROI in the 4MP imager. Note that changing this value will pause image streams while rectification lookup table is recomputed. |
desired_transmit_delay |
int |
Desired transmit delay in milliseconds. If longer than interval between exposures, will be shortened so that the current frame is transmitted before the next frame capture. |
max_pointcloud_range |
double |
The euclidean distance, in meters, for a point to be included in published pointclouds. Distances are computed from the focal point of the |
|
Parameter | Data Type | Description |
---|---|---|
imu_samples_per_message |
int |
Adjust the number of IMU readings (aggregate from accel/gyro/mag sensors) that the device will accumulate before shipping over the network. This can be used to make tradeoffs between sample rates, processor load, and latency. |
accelerometer_enabled |
bool |
Enable or disable the accelerometer sensor. |
accelerometer_rate |
int |
Selects the sample rate index of the accelerometer. |
accelerometer_range |
int |
Selects the sample range index of the accelerometer. |
gyroscope_enabled |
bool |
Enable or disable the gyroscope sensor. |
gyroscope_rate |
int |
Selects the sample rate index of the gyroscope. |
gyroscope_range |
int |
Selects the sample range index of the gyroscope. |
magnetometer_enabled |
bool |
Enable or disable the magnetometer sensor. |
magnetometer_rate |
int |
Selects the sample rate index of the magnetometer. |
magnetometer_range |
int |
Selects the sample range index of the magnetometer. |