update

README.md

@@ -1,3 +1,53 @@
 # PyTorch implementation of the [OpenPose](https://arxiv.org/abs/1611.08050)
 
-readme comming soon...
+The OpenPose is one of the most popular keypoint estimator, which uses two branches of feature map (is trained and enhanced via multiple stages) to estimate (via a [postprocess procedure](https://github.com/ruiminshen/pyopenpose)) the position of keypoints (via Gaussian heatmap) and the relationship between keypoints (called part affinity fields), respectively.
+This project adopts [PyTorch](http://pytorch.org/) as the developing framework to increase productivity, and utilize [ONNX](https://github.com/onnx/onnx) to convert models into [Caffe 2](https://caffe2.ai/) to benefit engineering deployment.
+If you are benefited from this project, a donation will be appreciated (via [PayPal](https://www.paypal.me/minimumshen), [微信支付](donate_mm.jpg) or [支付宝](donate_alipay.jpg)).
+
+## Designs
+
+- Flexible configuration design.
+Program settings are configurable and can be modified (via **configure file overlaping** (-c/--config option) or **command editing** (-m/--modify option)) using command line argument.
+
+- Monitoring via [TensorBoard](https://github.com/tensorflow/tensorboard).
+Such as the loss values and the debugging images (such as IoU heatmap, ground truth and predict bounding boxes).
+
+- Parallel model training design.
+Different models are saved into different directories so that can be trained simultaneously.
+
+- Time-based output design.
+Running information (such as the model, the summaries (produced by TensorBoard), and the evaluation results) are saved periodically via a predefined time.
+
+- Checkpoint management.
+Several latest checkpoint files (.pth) are preserved in the model directory and the older ones are deleted.
+
+- NaN debug.
+When a NaN loss is detected, the running environment (data batch) and the model will be exported to analyze the reason.
+
+- Unified data cache design.
+Various dataset are converted into a unified data cache via a programmable (a series of Python lambda expressions, which means some points can be flexibly generated) configuration.
+Some plugins are already implemented. Such as [MS COCO](http://cocodataset.org/).
+
+- Arbitrarily replaceable model plugin design.
+The deep neural network (both the feature extraction network and the stage networks) can be easily replaced via configuration settings.
+Multiple models are already provided. Such as the oringal VGG like network, [Inception v4](https://arxiv.org/abs/1602.07261), [MobileNet v2](https://arxiv.org/abs/1801.04381) and [U-Net](https://arxiv.org/abs/1505.04597).
+
+- Extendable data preprocess plugin design.
+The original images (in different sizes) and labels are processed via a sequence of operations to form a training batch (images with the same size, and bounding boxes list are padded).
+Multiple preprocess plugins are already implemented. Such as
+augmentation operators to process images and labels (such as random rotate and random flip) simultaneously,
+operators to resize both images and labels into a fixed size in a batch (such as random crop),
+and operators to augment images without labels (such as random blur, random saturation and random brightness).
+
+## Quick Start
+
+This project uses [Python 3](https://www.python.org/). To install the dependent libraries, make sure the [pyopenpose](https://github.com/ruiminshen/pyopenpose) is installed, and type the following command in a terminal.
+
+```
+sudo pip3 install -r requirements.txt
+```
+
+`quick_start.sh` contains the examples to perform detection and evaluation. Run this script.
+The COCO dataset is downloaded ([aria2](https://aria2.github.io/) is required) and cached, and the original pose model (18 parts and 19 limbs) is converted into PyTorch's format.
+If a webcam is present, the keypoint estimation demo will be shown.
+Finally, the training program is started.

convert_caffe_torch.py

@@ -78,7 +78,7 @@ def main():
     inference.eval()
     state_dict = inference.state_dict()
     # Caffe
-    net = caffe.Net(model_dir + '.prototxt', model_dir + '.caffemodel', caffe.TEST)
+    net = caffe.Net(os.path.expanduser(os.path.expandvars(args.prototxt)), os.path.expanduser(os.path.expandvars(args.caffemodel)), caffe.TEST)
     if args.debug:
         logging.info('Caffe variables')
         for name, blobs in net.params.items():
@@ -136,6 +136,8 @@ def main():
 def make_args():
     parser = argparse.ArgumentParser()
     parser.add_argument('mapper')
+    parser.add_argument('prototxt')
+    parser.add_argument('caffemodel')
     parser.add_argument('-c', '--config', nargs='+', default=['config.ini'], help='config file')
     parser.add_argument('-m', '--modify', nargs='+', default=[], help='modify config')
     parser.add_argument('--logging', default='logging.yml', help='logging config')

quick_start.sh

@@ -0,0 +1,14 @@
+echo cache data
+python3 cache.py -c config.ini config/original_person18_19.ini -m cache/name=cache_original
+
+echo download and cache the original model
+ROOT=~/model/openpose/pose/coco
+aria2c --auto-file-renaming=false -d $ROOT https://raw.githubusercontent.com/CMU-Perceptual-Computing-Lab/openpose/master/models/pose/coco/pose_deploy_linevec.prototxt
+aria2c --auto-file-renaming=false -d $ROOT http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel
+python3 convert_caffe_torch.py config/convert_caffe_torch/original_person18_19.tsv $ROOT/pose_deploy_linevec.prototxt $ROOT/pose_iter_440000.caffemodel -c config.ini config/original_person18_19.ini -m model/name=model_original -d
+
+echo demo keypoint estimation via a webcam
+python3 estimate.py -c config.ini config/original_person18_19.ini -m model/name=model_original
+
+echo training
+python3 train.py -c config.ini config/original_person18_19.ini -m cache/name=cache_original model/name=model_original

requirements.txt

@@ -0,0 +1,23 @@
+tqdm
+pybenchmark
+graphviz
+torch<=0.3.1
+pandas
+onnx
+onnx_caffe2
+pretrainedmodels
+torchvision
+matplotlib
+filelock
+scikit_image
+inflection
+numpy
+humanize
+Pillow
+PyQt5
+scipy
+skimage
+tensorboardX
+tensorflow
+PyYAML
+pycocotools