Mastering 3D Annotation Tools:- Empowering your Data Pre-Processing Workflow

Data preprocessing is a crucial step in machine learning, involving the transformation of raw data into a suitable format for algorithmic interpretation. It encompasses data acquisition, cleaning, integration, transformation, and feature selection, all of which are essential for optimizing the performance of machine learning algorithms.

Among the various steps in data preprocessing, data cleaning holds significant importance. It focuses on removing noise, biases, and outliers from the acquired dataset, ensuring that the data is reliable and representative of the underlying patterns. By eliminating unwanted elements, data cleaning lays the foundation for accurate and effective machine learning models.

Data labeling is another critical aspect in the machine learning pipeline. It involves assigning labels or annotations to objects within the collected data, enabling algorithms to recognize and classify patterns accurately. In supervised machine learning, data labeling is instrumental in training algorithms to make predictions on unseen data.

While 2D data labeling tools have been widely used in computer vision applications, annotating 3D data poses unique challenges. 3D data, often stored in formats like .pcd, .ply, .bag, and .csv, requires specialized tools for accurate annotation. In particular, the .ply format is prevalent for storing 3D point cloud data.

In this analysis, we will focus on four annotation tools: 3D BAT, LabelCloud, CVAT, and Photo.annotate, specifically designed to annotate both 3D and 2D data. Understanding the strengths and weaknesses of these tools will empower researchers, developers, and data scientists to choose the most suitable annotation tool for their specific needs. By comparing their features and capabilities, we aim to provide valuable insights that facilitate the efficient annotation of data and the development of robust computer vision models.

Now, let us delve into the detailed analysis of these annotation tools, considering their unique features, pros, and cons, to gain a comprehensive understanding of their capabilities and potential applications.

1) 3D BAT

3D BAT, which stands for 3D Bounding Box Annotation Tool, is a web-based tool that allows direct annotation of point cloud images or point cloud data. 

However, during our evaluation, we encountered challenges in getting 3D BAT to recognize the .ply file format. Additionally, we found the interface to be relatively complex compared to other available annotation tools. As a result, we decided to exclude 3D BAT from our consideration and explored alternative tools for annotating 3D point cloud data.

2) Label Cloud

LabelCloud is an alternative tool for annotating 3D data. It offers a lightweight and straightforward setup process on your device. Here are the instructions for setting up LabelCloud:

 

1.     Clone the LabelCloud repository from GitHub.

2.     Navigate to the directory of the cloned repository using the Windows Terminal or your Conda environment.

3.     Use pip to install all the necessary libraries listed in the requirements.txt file.

4.     Run the labelcloud.py file from your Conda environment or terminal.

 

To effectively use LabelCloud, follow this guide:

1.      Launch the application to access a simple graphical user interface.

2.  Select the desired labeling mode; in our case, we will focus on object detection.

3. Customize the class labels by assigning a class number, class name, and color for the 3D bounding box.

4. If you have multiple labels, use the "add new label" button located at the bottom middle to add new classes.

5.     Configure your default class and label export format. The available annotation export formats include vertices, centroid_rel, centroid_abs, kitti, and kitti_untransformed.

6.     Once you have adjusted all the necessary variables, click on the save button.

The labeling interface will then appear, providing the following features:

 

-        On the left-hand side, you will find the file menu and bounding box controls.                  

-        The center panel displays a view of your 3D data, enabling navigation through it.

-        The right-hand panel offers access to menus and controls related to the labels.

 

To create labels on your point cloud data, follow these steps:

 

1.     Set the point cloud folder from the file menu at the top left corner to input your point cloud data. Make sure all your .ply files are stored in this folder.

2.     Define the label folder from the file menu to specify the directory where your labels will be stored.

3.     Click on the "pick a bounding box" button in the left panel.

4.     Position the bounding box over the object in your scene that you wish to label.

5.     Use your mouse to explore different viewing angles and ensure that the object is entirely enclosed within the bounding box.

6.     The bounding box controls on the left panel allow you to adjust the position of the bounding box as needed.

7.     Hovering over any side of the cuboid enables you to resize it using the mouse scroll wheel or pinch and zoom on a laptop trackpad.

8.     Once you are satisfied with the adjusted bounding box, proceed to save the label.

9.     The "save label" button, located in the left side panel, will store the label in the assigned format within the designated folder.

 

LabelCloud presents several advantages as a 3D point cloud annotation tool:

 

-        Easy setup process.

-        Lightweight and efficient.

-        Smooth navigation within the 3D space.

 

However, it is worth noting a potential drawback:

 

-        Drawing the bounding box without a mouse may prove challenging.

 

LabelCloud offers a convenient solution for labeling 3D point cloud data, and by following the provided instructions, users can effectively utilize its features.

 

3) CVAT

CVAT, which stands for Computer Vision Annotation Tool, is a versatile tool for annotating 3D data. It allows users to annotate both 2D and 3D images and videos with ease.

Source: CVAT

Here's a step-by-step guide on how to annotate using CVAT:

 

1.     Visit the website www.cvat.ai.

2.     Click on "Start using CVAT."

3.     Create an account and sign in to CVAT. Alternatively, you can sign in using your Google or GitHub account.

4.     Navigate to "Tasks" from the menu.

5.     Click on the "+" button at the top right corner to create a new task.

6.     Select "Create new task."

7.     Provide a valid name for your task and optionally specify a project name.

8.     Click on "Add label."

9.     Enter a label name and choose a bounding box shape for your annotation. The default shape is set to "any," and you can also select a color for your bounding box.

10.  Select the files you want to annotate. You can upload files from your computer, remote sources, or cloud storage as needed.

11.  You can explore advanced configurations and make any necessary changes. If not, proceed to submit and open the task.

12.  Now, click on "Jobs" from the menu.

13.  Select the task you have created.

14.  Choose "Draw a new rectangle" from the left panel.

15.  Use your mouse to select the area of interest and draw the bounding box around it.

16.  CVAT offers various annotation shapes for different areas of interest.

17.  Once you are done, save your annotation. Go to the menu in the top left corner and click on "Export job dataset."

18.  Select the desired export format. CVAT provides more than 20+ formats for exporting your data.

19.  Optionally, give a custom name to your export.

20.  Use the default settings to save the files to your local device. If you prefer saving to cloud storage, uncheck "Use default settings" and select the cloud storage option, following the provided steps.

CVAT offers several advantages as a 3D data annotation tool:

 

-        It supports both 2D and 3D data annotation.

-        There are numerous advanced configurations available.

-        Task and project management is made easier.

-        It can be accessed directly from your browser, whether locally or online.

-        Multiple export formats are supported.

 

However, it is important to be aware of the potential drawbacks:

 

-        Not all data types are supported.

-        The setup and initial setup process can be somewhat cumbersome.

-        Some users have encountered difficulties when uploading .ply and .bag files.

-        Issues may arise when importing 3D data and annotations.

 

CVAT's versatility and advanced features make it a powerful tool for annotating 3D data, but it is advisable to be mindful of these limitations during usage.

 

4) Annotate.photo

Annotate.photo is a versatile tool for 3D and 2D data annotations. It allows users to create 3D bounding boxes on both 3D and 2D data, as well as 2D bounding boxes on images. The greatest advantage of annotate.photo is that it operates entirely within your web browser.

 

Here's a step-by-step guide on how to create 3D bounding boxes on point cloud data using annotate.photo:

 

1.     Click on "3D bounding box for point cloud data" in the interface.

2.     Click on "New" in the appearing menu.

3.     Click on the "+" button at the bottom right-hand corner.

4.     Browse and upload your .ply file from your local device.

5.     To begin annotation, it is important to create a label class.

6.     Click on the "+" button in the left panel. In the "New class" menu that appears, enter a name for your class.

7.     You have the option to select a color for your bounding box or leave it as black. However, if you are using a dark theme in your browser, ensure that you change the color from black for better visibility.

8.     The perspective panel is visible in the middle. Select the class you want to label from the classes panel.

9.     Hover your mouse over the perspective panel, then press "B" on your keyboard and simultaneously left-click using your mouse. This action will create a bounding box.

10.  Adjust the bounding box to cover your area of interest within the scene.

11.  Use your mouse to click and drag the bounding box to the desired location.

12.  At the bottom, you can find the top view, side view, and front view. Utilize these different views to adjust the boundaries of the bounding box for a precise fit.

13.  Switch between various annotations in your scene using the annotations panel on the right-hand side.

14.  You can also explore different annotation settings and make any necessary changes if required.

15.  Once everything is set, navigate to the "File" option in the menu. Select "Export" and choose the desired format from the available list.

16.  Provide a file name and export the annotation.

 

Photo.annotate offers an intuitive way to create accurate 3D bounding box annotations on point cloud data. By following these steps, you can effectively utilize the tool for your annotation needs.

 

To annotate a 3D bounding box on a 2D image using Photo.annotate, follow these steps:

 

1.     Click on "3D bounding box from 2D images" in the interface.

2.     Click on "New" in the appearing menu.

Source: annotate.photo

3.     Click on the "+" button at the bottom right-hand corner.

4.     Import the 2D image that you wish to annotate.

5.     Provide the camera intrinsic values in the input fields that follow. This includes the focal length (X and Y) and principal point (X and Y). By default, the intrinsic values of the Intel RealSense D435i camera are provided.

6.     Once you have set everything, click on "Save."

7.     To begin annotation, it is important to create a label class.

8.     Click on the "+" button in the left panel. In the "New class" window that pops up, enter a name for your class.

9.     You can select a color for your bounding box or leave it as black. However, if you are using a dark theme in your browser, ensure that you change the color from black for better visibility.

10.  You will also need to upload the 3D model of your image in .ply format.

11.  Pro tip: If you encounter issues while uploading the .ply file, try compressing it using Meshlab, a reliable tool that minimizes data loss. Follow these steps to compress your 3D data:

-        Install Meshlab from the provided link.

-        Open Meshlab.

-        Go to "File" > "Import Mesh" and select your .ply file.

-        Once your .ply file is loaded, navigate to "Filters" > "Remeshing, Simplification, Reconstruction" > "Simplification: Quadratic Edge Collapse Decimation."

-        Adjust the percentage reduction to a value between 0 and 1, then apply the changes.

-        Go to "File" > "Export Mesh" and save your compressed .ply file.

 

12.  Once you have created a label name and imported the compressed 3D model, click on "Save."

13.  In the image view at the center, use your mouse to pin 'n' points to create the bounding box. Here, 'n' represents the number of key points in your 3D data.

14.  The annotation panel on the right-hand side can be used to align your bounding box.

15.  Select your class name and adjust the rotation and translation values as needed.

16.  Once you have completed the annotation, go to "File" > "Export," and your 3D on 2D annotation will be saved in .json format.

 

By following these steps, you can effectively annotate a 3D bounding box on a 2D image using annotate.photo.

  

To annotate a 2D bounding box on a 2D image using  annotate.photo, follow these steps:

 

1.     Select "2D on 2D annotation" from the interface.

2.     Click on "New" in the appearing menu.

Source: annotate.photo

3.     Click on the "+" button at the bottom right-hand corner.

4.     Import the 2D image that you wish to annotate.

5.     Create a class for the annotation.

6.     Drag and create your bounding box on the image.

 

By following these steps, you can effectively annotate a 2D bounding box on a 2D image using Photo.annotate.

 

The pros of using annotate.photo for annotation are:

 

-        Provides options for 3D on 2D, 3D on 3D, and 2D on 2D annotations.

-        Relatively simple to create a 3D on 3D bounding box.

-        Supports exporting annotations in Votenet format.

 

The cons of using annotate.photo for annotation are:

 

-        Issues with uploading .ply files for 3D on 2D annotations.

-        Challenges when annotating multiple scenes within the same session.

-        Controls for adjusting the bounding boxes are not user-friendly.

 

In conclusion, we have explored four annotation tools for 3D and 2D data: 3D BAT, LabelCloud, CVAT, and annotate.photo. Each tool has its own strengths and weaknesses, and the choice of the tool depends on specific requirements and preferences.

3D BAT offers the flexibility to annotate point cloud data directly from a web browser. However, it may have limitations in recognizing certain file formats and its interface complexity compared to other tools. LabelCloud, on the other hand, is lightweight and easy to set up, making it convenient for quick annotation tasks. CVAT stands out with its support for both 2D and 3D annotation, advanced configurations, and project/task management capabilities. Lastly, Photo.annotate provides a browser-based solution with options for various annotation modes.

When choosing an annotation tool, it is essential to consider factors such as data type support, ease of setup and use, available annotation features, export formats, and compatibility with different platforms. It is also crucial to assess the specific requirements of the annotation task, such as the need for 3D annotations or the ability to handle multiple scenes simultaneously.

 

Here is a side-by-side comparison of the key features and limitations of the four annotation tools:

 

It's important to note that the comparison table provides a simplified overview of the tools, and individual preferences and specific use cases may influence the final choice of annotation tool. Therefore, it is recommended to explore and evaluate each tool based on the unique requirements of the annotation task at hand.

Overall, the availability of multiple annotation tools allows users to select the most suitable option based on their specific needs, whether it's simplicity, advanced features, or compatibility with different data types and platforms.