Auto annotate with yolov3 pretrained model for training new model

https://www.reddit.com/r/deeplearning/comments/cye6y6/an_opensource_data_annotation_tool_for_yolo/

In this article, I will explain how to do annotation automatically by utilizing the existing pre trained model. but it needs to be agreed that the object or class to be annotated is a general object such as cars, motorbikes, people, dogs, cats etc. I do this way to reduce the number of classes in the new model and also to increase the accuracy of the desired object.

this time I use yolo to detect objects using the pre trained model yolov3. The initial goal of this project is to reduce the number of classes in the model which results in faster detection times and better accuracy. In general, Yolov3 uses the Coco dataset where vehicles come from various countries so that if this pre trained model is used in certain countries with different vehicle types and models, the accuracy will decrease. at this time I will process the annotation automatically and manually.

In the automatic annotation process, the pre trained model will be used to detect vehicles in the dataset then create a bounding box for each object and produce a text file with the yolo style format, namely (class, x, y, w, h). This conversion is used to fit the Yolo format if you want to train again with a new dataset. If the algorithm cannot detect objects in an image, the image will be moved to another folder to be checked again and annotated manually.

In the manual annotation process, it is done using a tool, namely Yolo_mark. How to use it is quite simple, we only need to move the image to be annotated in the Yolo_mark /x64 /Release/data/img folder, then write down each class or object name in the obj.names file.

Before we annotate automatically, we need to do initial filtering to clean out the dataset that does not meet criteria such as image formats (jpeg and JPG) and sizes that are too small. we need to create a service to separate these files as follows:

import os
import shutil
import difflib

# folder_parent = "/home/dsc-01/dsc_erwin/google_image_downloader/google-images-download/mamam/"
# path = "/home/dsc-01/dsc_bariqi/yolov4/darknet-master/data/obj" 
# destination = "/home/dsc-01/dsc_erwin/google_image_downloader/google-images-download/images_check_MP4_JPG"

folder_parent = "/home/iwin/Documents/tools/Yolo_mark/x64/Release/data/img/"

jpg = []
txt = []

for folder in os.listdir(folder_parent):
 	for image in os.listdir(folder_parent+folder):
 		print(folder_parent+folder+image)
 		if image.endswith('.JPG') or image.endswith('.mp4'):
 			image_first = os.path.splitext(image)[0]
			shutil.move(folder_parent+folder+'/'+image, destination)
			shutil.move(path+'/'+image_first+'.txt',destination)
print(image,image_first+'.txt')
			

for image in os.listdir(folder_parent):
	# for image in os.listdir(folder_parent+folder):
	if image.endswith('.png') or image.endswith('.jpg'):
		image_first = os.path.splitext(image)[0]
		jpg.append(image_first)
	if image.endswith('.txt'):
		txt_first = os.path.splitext(image)[0]
		txt.append(txt_first)

data = list(set(jpg) - set(txt)) 

for img in data:
	image_moved = folder_parent+img+'.png'
	print(image_moved)

After the dataset has been separated from damaged or incorrectly formatted images, the next step is to perform auto annotation using the existing pre trained model. Prepare the model files for yolov3.weight, yolov3.cfg, and coco.names. In this article we only annotate 6 objects, namely car, bus, truck, motorbike, bicycle, person. other than that class it will not be denoted or ignored. We need to remember that the yolo coordinate format and style are different from other algorithms, therefore we need to prepare a special function to change the bounding box format (x1, y1) and (x2, y2) to (x, y, w, h).

def convert(size, box):
    dw = 1./size[0]
    dh = 1./size[1]
    x = (box[0] + box[1])/2.0
    y = (box[2] + box[3])/2.0
    w = box[1] - box[0]
    h = box[3] - box[2]
    x = x*dw
    w = w*dw
    y = y*dh
    h = h*dh
    return (x,y,w,h)

after that we create a script to create a folder. That folder will be used to move problem files such as damaged files or images that are not detected by objects in it.

try:
    if not os.path.exists("output_txt"):
        os.makedirs("output_txt")
        print("directory is ready!")
    if not os.path.exists("image_check"):
        os.makedirs("image_check")
        print("directory is ready!")
except OSError:
    print ('Error: Creating directory. ')

To be able to describe a train data that has been obtained, it is necessary to carry out the analysis with the processed data. The data is the value or classification result of YOLO which is stored in the CSV file. The data stored includes path, width, height, class, total_class, xmin, xmax, ymin, ymax, x, y, w, h, confidence_level.

data = pd.DataFrame({'path': path_df, 'width': W_df, 'height': H_df ,'total_class': class_total_df,'class_object' : class_df, 'xmin': xmin_df, 'xmax': xmax_df, 'ymin': ymin_df, 'ymax': ymax_df,'x': x_df, 'y':y_df, 'w': w_df, 'h': h_df, 'confidence': confidence_df})
                    listToStr = ' '.join(map(str, data_txt))

run auto_annotation.py, if there is no error, the output_txt folder will be filled with the image file name with a text format extension which contains the class values, x, y, w, h. the data have been converted from the coordinates of the detected object.

if you are going to do training using Yolo, the folder containing the txt must be combined with the image in one folder. make sure each image is paired with the txt file.

you can see full source code in my github here.