Task 2: Rearrange Dice

TriFinger robot with multiple dice in the arena

Task: Rearrange Dice

The goal of this task is to arrange multiple dice into a given pattern.

The pattern is given as a list of N target positions where N is the number of dice:

goal = [
    (0.10, 0.04, 0.01),
    (0.04, -0.08, 0.01),
    (0.0, 0.15, 0.01),

Since the single dice are indistinguishable the target positions are not linked to a specific die, there should just be one die at each position in the end.

The duration of a run is 120000 steps (~2 minutes). This value is also given by EPISODE_LENGTH.

The cost of each step is computed using the camera images. Based on the colour, it is determined how many “die pixels” are outside of the target regions (see evaluate_state()).

trifinger_simulation.tasks.rearrange_dice.EPISODE_LENGTH = 120000

Duration of the episode in time steps (corresponds to ~2 minutes).

trifinger_simulation.tasks.rearrange_dice.NUM_DICE = 25

Number of dice in the arena


Sample a random list of die goal positions.

trifinger_simulation.tasks.rearrange_dice.evaluate_state(goal_masks, actual_masks)[source]

Compute cost of a given state. Less is better.

The cost is computed as the number of “die pixels” in the actual masks that do not overlap with the goal mask:

cost = count(actual_masks AND (NOT goal_masks))
  • goal_masks (Sequence[numpy.ndarray]) – Masks of the desired die positions in the camera images, one mask per camera. Use generate_goal_mask() to generate the goal mask for a given goal.

  • actual_masks (Sequence[numpy.ndarray]) – Masks of the actual die positions in the camera images, one mask per camera using the same order as goal_masks.


The cost of the given state.

Return type



Verify that the goal has the proper shape and all positions are valid.

  • OutOfArenaError – If a die position is outside the valid range.

  • InvalidGoalError – If the goal does not have the expected shape.


Load or sample a goal based on the given goal config file.


filename (str) – Path to the goal config JSON file. If it contains an entry “goal”, its value is used as goal. Otherwise a random goal is sampled.


The goal as JSON-encoded string.

Return type



Set random seed for this module.


seed (int) –


Visualise the target positions in 2d.

Shows a top-down view of the arena with the goal positions marked by squares.


target_positions (Sequence[Sequence[float]]) – The goal that is visualised.

Example on how the visualisation look like:

trifinger_simulation.tasks.rearrange_dice.generate_goal_mask(camera_parameters, goal)[source]

Generate goal masks that can be used with evaluate_state().

A goal mask is a single-channel image where the areas at which dice are supposed to be placed are white and everything else is black. So it corresponds more or less to a segmentation mask where all dice are at the goal positions.

For rendering the mask, TARGET_WIDTH is used for the die width to add some tolerance.

  • camera_parameters (Sequence[trifinger_simulation.camera.CameraParameters]) – List of camera parameters, one per camera.

  • goal (Sequence[Sequence[float]]) – The goal die positions.


List of masks. The number and order of masks corresponds to the input camera_parameters.

Return type


Example on how the generated goal masks look like: