Custom object detection_gcp

Custom object detection_gcp

• 1.
  Training Custom Object Detection Model in GCP CloudML Engine
• 3.
  Prerequisite • Install Git •Install Tensorflow • Requires Python 3.4, 3.5, or 3.6 • pip install tensorflow # CPU Version • https://www.tensorflow.org/install • Download Tensorflow models git • git clone https://github.com/tensorflow/models • Some other python libraries • Pillow, lxml, contextlib2 (use pip install) • Install Google cloud SDK • https://cloud.google.com/sdk/docs/quickstart-windows
• 4.
  Setup Google CloudPlatform • Create Google Cloud Platform (GCP) Account • Create a project • Enable Billing • On Command prompt (Terminal) • gcloud init • gcloud config auth list • gcloud config list • gcloud config set project
• 5.
  Protobuf • The TensorFlowObject Detection API uses .proto files which need to be compiled into .py files. Google provides a program called Protobuf that will batch compile these. • cd models/research • For windows • Download version 3.4.0 for Windows from https://github.com/protocolbuffers/protobuf/releases/tag/v3.4.0 • “E:Object_Detectiondemoprotoc-3.4.0-win32binprotoc.exe" object_detectionprotos*.proto --python_out=. • For Linux • sudo apt-get install protobuf-compiler • protoc object_detection/protos/*.proto --python_out=. • You will see multiple python files now in “object_detectionprotos” folder
• 6.
  Environment Setup • Linux •export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim • Windows • Add to Environment Variable • modelsresearch • modelsresearchslim • Test Installation • python object_detection/builders/model_builder_test.py • If no errors installation is perfect
• 7.
  Create Dataset • Createtwo folders: • images • Image Name Format: categoryName_imageID.jpg (only one "_") • annotations • trainval.txt • categoryName_imageID category_ID (80% images for train/val) • test.txt • categoryName_imageID category_ID (20% images for test) • list.txt • categoryName_imageID category_ID (all images) • xmls • categoryName_imageID.xml • Label_map.pbtxt • Note: • Make sure no image is corrupt, bounding boxes are well defined. • Bounding box co-ordinates do not fall outside of image height and width. • Image Height and width are correct. • If one image has many object of same kind, only one copy of image is needed with respective object category name and all bounding box locations in xml. • If image has many object of different category, multiple copies of same image will be stored with one object category in image name in one copy
• 8.
  Samples images annotations xmls Sample xml Label_map.pbtxt
• 9.
  Create TFRecord • TheTensorflow Object Detection API uses the TFRecord format for training and validation datasets • python object_detection/dataset_tools/create_pet_tf_record.py --label_map_path=E:Object_Detectiondemolabel_map.pbtxt --data_dir=E:Object_Detectiondemo --output_dir=E:Object_Detectiondemo • You will see two new files at output_dir location • pet_train.record • pet_val.record
• 10.
  Move data toGCS (Google Cloud Storage) • Create a new bucket • gsutil mb gs://train_coke • Set bucket name for future use • set YOUR_GCS_BUCKET=gs://train_coke • Move file to new GCS Bucket • gsutil cp pet_train.record %YOUR_GCS_BUCKET%/data/pet_train.record • gsutil cp pet_val.record %YOUR_GCS_BUCKET%/data/pet_val.record • gsutil cp label_map.pbtxt %YOUR_GCS_BUCKET%/data/pet_label_map.pbtxt
• 11.
  Transfer Learning • Uploadtrained model to GCS • wget https://storage.googleapis.com/download.tensorflow.org/models/object_detection/fast er_rcnn_resnet101_coco_11_06_2017.tar.gz • tar -xvf faster_rcnn_resnet101_coco_11_06_2017.tar.gz • gsutil cp faster_rcnn_resnet101_coco_11_06_2017/model.ckpt.* %YOUR_GCS_BUCKET%/data/ • Edit Config File • sed -i "s|PATH_TO_BE_CONFIGURED|"${YOUR_GCS_BUCKET}" data|g" object_detection/samples/configs/faster_rcnn_resnet101_pets.config #Linux • gsutil cp object_detection/samples/configs/faster_rcnn_resnet101_pets.config %YOUR_GCS_BUCKET%/data/faster_rcnn_resnet101_pets.config • Tensorflow detection model zoo https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/det ection_model_zoo.md
• 12.
  Train Model • Packagethe TensorFlow Object Detection API and TF Slim • python setup.py sdist • cd slim • python setup.py sdist • Enable Cloud Machine Learning Engine API • bash object_detection/dataset_tools/create_pycocotools_package.sh /tmp/pycocotools (#Linux) • gcloud ml-engine jobs submit training object_detection_coke --runtime-version 1.9 --job-dir=%YOUR_GCS_BUCKET%/model_dir --packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,pycocotools/pycocotools-2.0.tar.gz --module-name object_detection.model_main --region us-east1 --config object_detection/samples/cloud/cloud.yml -- --model_dir=%YOUR_GCS_BUCKET%/model_dir --pipeline_config_path=%YOUR_GCS_BUCKET%/data/faster_rcnn_resnet101_pets.config • Get Logs • gcloud ml-engine jobs stream-logs object_detection_pets
• 13.
  Tensorboard • pip installtensorboard • From cloudshell • tensorboard --logdir=${YOUR_GCS_BUCKET}/model_dir --port=5000
• 14.
  Export Trained Model •Get model checkpoint number • Set CHECKPOINT_NUMBER=2975 • set YOUR_LOCAL_CHK_DIR=exported_model • Copy model checkpoint files to local from GCS • gsutil cp %YOUR_GCS_BUCKET%/model_dir/model.ckpt- %CHECKPOINT_NUMBER%.* % YOUR_LOCAL_CHK_DIR% • Export Inference Graph • python object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path object_detection/samples/configs/faster_rcnn_resnet101_pets.config --trained_checkpoint_prefix %YOUR_LOCAL_CHK_DIR%/model.ckpt- %CHECKPOINT_NUMBER% --output_directory exported_graphs
• 15.
  Test Model
• 16.
  References • https://cloud.google.com/blog/products/gcp/training-an-object- detector-using-cloud-machine-learning-engine • https://towardsdatascience.com/google-object-detection-api-to- detect-brand-logos-fd9e113725d8 •https://towardsdatascience.com/google-object-detection-api-to- detect-brand-logos-part-2-384055e4e109 • Git Repo • https://github.com/sahuvaibhav/Object_Detection