opencv_zoo / tools /eval /README.md
Wanli
Add script to evaluate face recognition by LFW (#72)
f2e3176
|
raw
history blame
4.28 kB

Accuracy evaluation of models in OpenCV Zoo

Make sure you have the following packages installed:

pip install tqdm
pip install scikit-learn
pip install scipy

Generally speaking, evaluation can be done with the following command:

python eval.py -m model_name -d dataset_name -dr dataset_root_dir

Supported datasets:

ImageNet

Prepare data

Please visit https://image-net.org/ to download the ImageNet dataset and the labels from caffe. Organize files as follow:

$ tree -L 2 /path/to/imagenet
.
β”œβ”€β”€ caffe_ilsvrc12
β”‚   β”œβ”€β”€ det_synset_words.txt
β”‚   β”œβ”€β”€ imagenet.bet.pickle
β”‚   β”œβ”€β”€ imagenet_mean.binaryproto
β”‚   β”œβ”€β”€ synsets.txt
β”‚   β”œβ”€β”€ synset_words.txt
β”‚   β”œβ”€β”€ test.txt
β”‚   β”œβ”€β”€ train.txt
β”‚   └── val.txt
β”œβ”€β”€ caffe_ilsvrc12.tar.gz
β”œβ”€β”€ ILSVRC
β”‚   β”œβ”€β”€ Annotations
β”‚   β”œβ”€β”€ Data
β”‚   └── ImageSets
β”œβ”€β”€ imagenet_object_localization_patched2019.tar.gz
β”œβ”€β”€ LOC_sample_submission.csv
β”œβ”€β”€ LOC_synset_mapping.txt
β”œβ”€β”€ LOC_train_solution.csv
└── LOC_val_solution.csv

Evaluation

Run evaluation with the following command:

python eval.py -m mobilenet -d imagenet -dr /path/to/imagenet

WIDERFace

The script is modified based on WiderFace-Evaluation.

Prepare data

Please visit http://shuoyang1213.me/WIDERFACE to download the WIDERFace dataset Validation Images, Face annotations and eval_tools. Organize files as follow:

$ tree -L 2 /path/to/widerface
.
β”œβ”€β”€ eval_tools
β”‚   β”œβ”€β”€ boxoverlap.m
β”‚   β”œβ”€β”€ evaluation.m
β”‚   β”œβ”€β”€ ground_truth
β”‚   β”œβ”€β”€ nms.m
β”‚   β”œβ”€β”€ norm_score.m
β”‚   β”œβ”€β”€ plot
β”‚   β”œβ”€β”€ read_pred.m
β”‚   └── wider_eval.m
β”œβ”€β”€ wider_face_split
β”‚   β”œβ”€β”€ readme.txt
β”‚   β”œβ”€β”€ wider_face_test_filelist.txt
β”‚   β”œβ”€β”€ wider_face_test.mat
β”‚   β”œβ”€β”€ wider_face_train_bbx_gt.txt
β”‚   β”œβ”€β”€ wider_face_train.mat
β”‚   β”œβ”€β”€ wider_face_val_bbx_gt.txt
β”‚   └── wider_face_val.mat
└── WIDER_val
    └── images

Evaluation

Run evaluation with the following command:

python eval.py -m yunet -d widerface -dr /path/to/widerface

LFW

The script is modified based on evaluation of InsightFace.

This evaluation uses YuNet as face detector. The structure of the face bounding boxes saved in lfw_face_bboxes.npy is shown below. Each row represents the bounding box of the main face that will be used in each image.

[
  [x, y, w, h, x_re, y_re, x_le, y_le, x_nt, y_nt, x_rcm, y_rcm, x_lcm, y_lcm],
  ...
  [x, y, w, h, x_re, y_re, x_le, y_le, x_nt, y_nt, x_rcm, y_rcm, x_lcm, y_lcm]
]

x1, y1, w, h are the top-left coordinates, width and height of the face bounding box, {x, y}_{re, le, nt, rcm, lcm} stands for the coordinates of right eye, left eye, nose tip, the right corner and left corner of the mouth respectively. Data type of this numpy array is np.float32.

Prepare data

Please visit http://vis-www.cs.umass.edu/lfw to download the LFW all images(needs to be decompressed) and pairs.txt(needs to be placed in the view2 folder). Organize files as follow:

$ tree -L 2 /path/to/lfw
.
β”œβ”€β”€ lfw
β”‚   β”œβ”€β”€ Aaron_Eckhart
β”‚   β”œβ”€β”€ ...
β”‚   └── Zydrunas_Ilgauskas
└── view2
    └── pairs.txt

Evaluation

Run evaluation with the following command:

python eval.py -m sface -d lfw -dr /path/to/lfw