Mech-Vision Interface
This section introduces Adapter interfaces related to Mech-Vision, as listed below.
Get Vision Target(s)
The vision result from Mech-Vision can be obtained by calling the “find_vision_pose()” function in adapter.py.
def find_vision_pose(self, project_name=None, timeout=default_vision_timeout):
vision_result = self.call_vision("findPoses", project_name=project_name, timeout=timeout)
logging.info("Find vision result: {}".format(vision_result))
return vision_resultThe vision points in the result output by Mech-Vision are usually object poses (obj_pose) in quaternions. If you need the vision points in TCP, please make the transformation in the Mech-Vision project.
Example
A function needs to be created in Adapter to convert object poses into TCP (tcp_pose), and convert the quaternions into Euler angles and radians into degrees (if required by the robot side), and the converted data will be packed and send to the robot. The data format and unit should be consistent with those used on the robot side. Besides, Adapter can check the vision points output by Mech-Vision, as shown below:
def check_vision_result(self, vision_result):
if vision_result["noCloudInRoi"]: # Determine whether it is an empty bin
logging.info("Layer has no objects")
self.send(pack('>2B6i', CODE_NO_CLOUD, vision_num, *EMPTY_PLACEHOLDER))
return
poses = vision_result.get("poses", []) # Get pose
if len(poses) = 0: # Determine whether there is a vision point
logging.warning("No pose from vision")
self.send(pack('>2B6i', CODE_NO_POSE, vision_num, *EMPTY_PLACEHOLDER))
return
self.send(pack_pose(poses[0], vision_num)) # Send after format conversionIn the above example, “vision_result” is obtained from “find_vision_pose()”, and the statement is shown below.
self.check_vision_result(json.loads(self.find_vision_pose().decode()))After “vision_result” is passed into the “check_vision_result” function, for projects with an ROI set, the function will determine if it is an empty bin first, and then obtain the vision points. If there is nothing wrong with the vision points, they will be passed into the “pack_pose()” function and sent out.
Set Step Parameters
To set Step parameters in Mech-Vision dynamically, usually you need to call only the “set_step_property()” function in the adapter.py.
def set_step_property(self, msg, project_name=None, timeout=None):
return self.call_vision("setStepProperties", msg, project_name, timeout)In the above statement, “msg” determines the specific Step name and the parameter which needs to be set.
Example
You can create a function as shown below to set “msg” when the point cloud matching model file in the Mech-Vision project needs to be set dynamically according to different types of workpieces.
def _step_matching_model_cell(step_name, model_type):
msg = {"name": step_name,
"values":
{"modelFile": model_type["ply"],
"pickPointFilePath": model_type["json"]}}
return msgIn the above example, “step_name” is the name of the Mech-Vision Step to be set; “model_type” indicates the file directory of the point cloud matching model file and pick point file for corresponding workpiece. By setting “ply” and “json” in “model_type”, the directory of the point cloud matching model file in .ply format and the directory of the pick point file in .json can be obtained and will be assigned to the parameters in corresponding Mech-Vision Step.
If “step_name” is “Local Matching”, then the statement is shown below.
msg = _step_matching_model_cell("Local Matching", model_type)
self.set_step_property(msg)Read Step Parameters
You can obtain the parameter settings of a certain Mech-Vision Step by calling “read_step_property()” function in adapter.py.
def read_step_property(self, msg):
result = self.call_vision("readStepProperties", msg)
logging.info("Property result: {}".format(result))
return resultIn the above function, “msg” determines the Step name and the parameters to be obtained. You can create a function to rewrite the “msg”.
Example
An example to obtain the camera IP address is shown below.
def read_camera_property(self):
msg = {"type": "Camera",
"properties": ["MechEye"]}
property_results = json.loads(self.read_step_property(msg).decode())
camera_ip = property_results["MechEye"]["NetCamIp"]If only one camera is used in the Mech-Vision project, the Step can be found according to the “type” field. If there are multiple Steps of the same type in the Mech-Vision project, and you would like to obtain or configure parameters of a certain Step, the “name” field can be used to find the Step. All parameter settings of the camera can be obtained and converted into the JSON format by calling the “read_step_property()” function, as shown below:
Property result:
{
"MechEye": {
"NetCamIp": "127.0.0.1",
"TimeOut": "10",
"configGroup": "",
}
}In this example, the camera IP address (127.0.0.1) is obtained according to the specific parameter fields “MechEye” and “NetCamIp”.
Switch the Parameter Recipe
When using Mech-Vision, you can switch the parameter recipe for a project by calling the select_parameter_group() function in adapter.py in scenarios where multiple recipes are used to distinguish the parameter settings for the same workflow of different workpieces.
def select_parameter_group(self, project_name, group_index, timeout=None):
msg = {"parameter_group_idx": group_index}
result = self.call_vision("selectParameterGroup", msg, project_name, timeout)
logging.info("selectParameterGroup result: {}".format(result))
return resultIn the above function, “project_name” is the Mech-Vision project name, and “group_index” is the ID of the parameter recipe.
Example
When the parameter recipe needs to be switched in a project, you can call the “select_parameter_group()” function as shown below and use it for error handling.
try:
result = self.select_parameter_group(self.vision_project_name, model_code-1)
if result:
result = result.decode()
if result.startswith("CV-E0401"):
return -1
elif result.startswith("CV-E0403"):
return -1
raise RuntimeError(result)
except Exception as e:
logging.exception('Exception when switch model: {}'.format(e))
return -1
return 0In this example, “self.vision_project_name” passes the Mech-Vision project name, and “model_code minus 1” passes the ID of the recipe.