Mech-Vision Interface

This section introduces Adapter interfaces related to Mech-Vision, as listed below.

Get Vision Target(s)

The vision result from Mech-Vision can be obtained by calling the find_vision_pose() function in adapter.py.

def find_vision_pose(self, project_name=None, timeout=default_vision_timeout):
    vision_result = self.call_vision("findPoses", project_name=project_name, timeout=timeout)
    logging.info("Find vision result: {}".format(vision_result))
    return vision_result

The vision results output by Mech-Vision are usually object poses (obj_pose) in quaternions. If you need the vision results in TCP, please adjust the settings in Mech-Vision.

Example

A function needs to be created in Adapter to convert object poses into TCP (tcp_pose), and convert the quaternions into Euler angles and radians into degrees, and the converted data will be packed and send to the robot. The data format and unit should be consistent with those used on the robot side. Besides, Adapter can check the vision points output by Mech-Vision, as shown below.

def check_vision_result(self, vision_result):
    if vision_result["noCloudInRoi"]:                      # Determine whether it is an empty bin
    logging.info("Layer has no objects")
    self.send(pack('>2B6i', CODE_NO_CLOUD, vision_num, *EMPTY_PLACEHOLDER))
    return
    poses = vision_result.get("poses", [])                 # Get pose
    if len(poses) == 0:                                   # Determine whether there is a vision point
        logging.warning("No pose from vision")
        self.send(pack('>2B6i', CODE_NO_POSE, vision_num, *EMPTY_PLACEHOLDER))
        return
    self.send(pack_pose(poses[0], vision_num))       # Send after format conversion

In the above example, vision_result is obtained from find_vision_pose(), and the statement is shown below.

self.check_vision_result(json.loads(self.find_vision_pose().decode()))

After vision_result is passed into the check_vision_result function, for projects with an ROI, the function will determine if it is an empty bin first, and then obtain the vision point. If there is nothing wrong with the vision point, it will be passed into the pack_pose() function and send out.

Set Step Parameters

To set Step parameters in Mech-Vision dynamically, usually only the set_step_property() function in the adapter.py need to be called.

def set_step_property(self, msg, project_name=None, timeout=None):
    return self.call_vision("setStepProperties", msg, project_name, timeout)

In the above statement, msg determines the specific Step name and the parameter which need to be configured.

Example

You can create a function as shown below to set msg when the point cloud model file need to be set dynamically according to different types of workpieces.

def _step_matching_model_cell(step_name, model_type):
    msg = {"name": step_name,
            "values":
           {"modelFile": model_type["ply"],
         "pickPointFilePath": model_type["json"]}}
return msg

step_name is the name of the Step to be configured; model_type is the file directory of the corresponding workpiece. The directory of point cloud model file in PLY and JSON format can be obtained via ply and json, and they will be assigned to the parameters in corresponding Step.

If the step_name is “Local Matching”, then the statement is shown below.

msg = _step_matching_model_cell("Local Matching", model_type)
self.set_step_property(msg)

The Step Parameters panel in Mech-Vision is shown in the figure below.

Read Step Parameters

You can obtain the parameters of a certain Step by calling read_step_property() in adapter.py.

def read_step_property(self, msg):
     result = self.call_vision("readStepProperties", msg)
     logging.info("Property result: {}".format(result))
     return result

In the above function, msg determines the Step name and the parameters to be obtained. You can create a function to rewrite the msg.

Example

An example to obtain the camera IP address is shown below.

def read_camera_property(self):
     msg = {"type": "Camera",
            "properties": ["MechEye"]}
     property_results = json.loads(self.read_step_property(msg).decode())
     camera_ip = property_results["MechEye"]["NetCamIp"]

If only one camera is used in the Mech-Vision project, the Step can be found according to type. If there are multiple Steps of the same type in the project, and you would like to obtain or configure parameters of a certain Step, the name can be used to find the Step. All parameters of the camera can be obtained by calling the read_step_property() function and converting the data into JSON format.

Property result:
{
  "MechEye": {
    "NetCamIp": "127.0.0.1",
    "TimeOut": "10",
    "configGroup": "",
  }
}

In this example, the camera IP address (127.0.0.1) is obtained according to the specific parameter field “MechEye” and “NetCamIp”.

Switch Parameter Recipe

When using Mech-Vision, you can switch parameter recipes between different projects by calling the select_parameter_group() function in adapter.py if the workflow of the projects are the same, but the parameters are configured differently.

def select_parameter_group(self, project_name, group_index, timeout=None):
    msg = {"parameter_group_idx": group_index}
    result = self.call_vision("selectParameterGroup", msg, project_name, timeout)
    logging.info("selectParameterGroup result: {}".format(result))
    return result

In the above function, project_name is the Mech-Vision project name, and group_index is the number of recipe.

Example

When the parameter recipe need to be switched in a project, you can call the select_parameter_group() function as shown below and use it for error handling.

try:
   result = self.select_parameter_group(self.vision_project_name, model_code-1)
   if result:
      result = result.decode()
      if result.startswith("CV-E0401"):
            return -1
      elif result.startswith("CV-E0403"):
            return -1
      raise RuntimeError(result)
   except Exception as e:
      logging.exception('Exception when switch model: {}'.format(e))
      return -1
   return 0

In this example, self.vision_project_name passes the Mech-Vision project name, and model_code-1 passes the number of the recipe.