2016年11月6日星期日

Fine-tuning pre-trained ResNet model with different learning rate

Fine-tuning Facebook Torch implementation of ResNet model with different learning rate.

  • with learning rate 0.1
  • with learning rate 0.01


  • with learning rate 0.001




2016年3月5日星期六

One note for building Caffe RC3 on Ubuntu 14.04

When building Caffe RC3 on Ubuntu 14.04, you may run into this error:
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/test_net.bin
//usr/lib/x86_64-linux-gnu/libunwind.so.8:对‘lzma_index_buffer_decode@XZ_5.0’未定义的引用
//usr/lib/x86_64-linux-gnu/libunwind.so.8:对‘lzma_index_size@XZ_5.0’未定义的引用
//usr/lib/x86_64-linux-gnu/libunwind.so.8:对‘lzma_index_uncompressed_size@XZ_5.0’未定义的引用
//usr/lib/x86_64-linux-gnu/libunwind.so.8:对‘lzma_stream_footer_decode@XZ_5.0’未定义的引用
解决方法是在~/.bashrc文件最后添加路径
export LD_LIBRARY_PATH=/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH

2016年3月1日星期二

Some notes about building Caffe RC3 with Mac OS X 10.11.3, Anaconda, CUDA 7.5, cuDNN 4, Intel MKL and MATLAB R2015b

Environment: Mac OS X 10.11.3, Xcode 7.2, Anaconda Python 2.7.11, CUDA 7.5, cuDNN 4, Intel parallel_studio_xe_2016.1.043 mkl, homebrew boost 1.6.0, homebrew OpenCV 2.4.12, MATLAB R2015b.

Note 1: If your run into this problem afterwards "ld: warning: directory not found for option '-L/opt/intel/mkl/lib/intel64'", you can solve this problem as:
cd /opt/intel/mkl/lib/
sudo ln -s . /opt/intel/mkl/lib/intel64
Suppose the environment is setup as above, first follow the official Caffe OS X Installation guide step by step.

Then modify your .bash_profile file as:
export PATH=/usr/local/bin:/Applications/MATLAB_R2015b.app/bin:/Developer/NVIDIA/CUDA-7.5/bin:$PATH
export PATH="/Users/ylzhao/anaconda/bin:$PATH"
export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH
export DYLD_FALLBACK_LIBRARY_PATH=/usr/local/cuda/lib::$HOME/anaconda/lib:/usr/local/lib:/usr/lib:$DYLD_FALLBACK_LIBRARY_PATH
export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
Note 2: If you run into this problem when invoking "make runtest"afterwards:
dyld: Library not loaded: @rpath/libcudart.7.0.dylib
Referenced from: .../caffe/.build_release/tools/caffe
Reason: image not found
make: *** [runtest] Trace/BPT trap: 5
That is because on Mac OS X El Capitan, DYLD_FALLBACK_LIBRARY_PATH is cleared by the new System Integrity Protection feature of El Capitan. See the discussions 2320 and 3628. One possible way to solve this problem is disabled System Integrity Protection as:
    1. Boot to Recovery OS by restarting your machine and holding down the Command and R keys at startup.
    2. Launch Terminal from the Utilities menu.
    3. Enter the following command: csrutil disable
    4. restart your computer
     Then follow the official Caffe compilation with Make guide, my 'Makefile.config' file is:
    ## Refer to http://caffe.berkeleyvision.org/installation.html
    # Contributions simplifying and improving our build system are welcome!
    # cuDNN acceleration switch (uncomment to build with cuDNN).
    USE_CUDNN := 1
    # CUDA directory contains bin/ and lib/ directories that we need.
    CUDA_DIR := /usr/local/cuda
    # BLAS choice:
    # atlas for ATLAS (default)
    # mkl for MKL
    # open for OpenBlas
    BLAS := mkl

    # This is required only if you will compile the matlab interface.
    # MATLAB directory should contain the mex binary in /bin.
    # MATLAB_DIR := /usr/local
    MATLAB_DIR := /Applications/MATLAB_R2015b.app

    # NOTE: this is required only if you will compile the python interface.
    # We need to be able to find Python.h and numpy/arrayobject.h.
    # PYTHON_INCLUDE := /usr/include/python2.7 \
    # /usr/lib/python2.7/dist-packages/numpy/core/include
    # Anaconda Python distribution is quite popular. Include path:
    # Verify anaconda location, sometimes it's in root.
    ANACONDA_HOME := $(HOME)/anaconda
    PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
    $(ANACONDA_HOME)/include/python2.7 \
    $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

    # We need to be able to find libpythonX.X.so or .dylib.
    # PYTHON_LIB := /usr/lib
    PYTHON_LIB := $(ANACONDA_HOME)/lib

    # Uncomment to support layers written in Python (will link against Python libs)
    # WITH_PYTHON_LAYER := 1

    # Whatever else you find you need goes here.
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
    My modification parts are labelled as bold.

    Then you can start the compilation in Terminal:
    make all
    make test
    make runtest
    Note 3: If your "LIBRARY_DIRS" like these:
    LIBRARY_DIRS := $(PYTHON_LIB) $(MATLAB_DIR)/bin/maci64 usr/local/lib  /usr/lib
    you may run into this protobuf problem afterwards:
    Undefined symbols for architecture x86_64:
     "google::protobuf::io::CodedOutputStream::WriteStringWithSizeToArray(std::__1::basic_string, std::__1::allocator > const&, unsigned char*)", referenced from:
    caffe::Datum::SerializeWithCachedSizesToArray(unsigned char*) const in caffe.pb.o
    This is because Caffe tries to link with MATLAB's protobuf instead of homebrew protobuf, and this will also happen with the boost library and CUDA. (More detail: Caffe was seeing Matlab's internal libraries earlier in the library path than the local homebrew libraries in /usr/local/lib. Caffe might be trying to link Matlab's version of the boost library, which, needless to say, isn't compatible with Caffe.) See discussion 915.

    So before compile the Caffe Matlab interface, your "LIBRARY_DIRS" should like:
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
    After the commands "make all", "make test", "make runtest" successfully, you should modify the "LIBRARY_DIRS" like:
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib $(MATLAB_DIR)/bin/maci64 /usr/lib
    Thus we can avoid the conflict between homebrew's protobuf/boost/cuda libraries and MATLAB's protobuf/boost/cuda libraries.

    Then you can build the Caffe Matlab interface as:
    make matcaffe
    make mattest 
    Note 4: If you forget to add "(MATLAB_DIR)/bin/maci64" in "LIBRARY_DIRS" line, you may run into this problem:
    Undefined symbols for architecture x86_64:
      "_mxArrayToString", referenced from:
          get_solver(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          solver_restore(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          get_net(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          net_copy_from(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          net_save(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          read_mean(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          write_mean(int, mxArray_tag**, int, mxArray_tag const**) in caffe_.o
          ...
    That why we need to change the LIBRARY_DIRS := ... line in your Makefile.config so that the /usr/local/lib directory is before your Matlab library directory as above described.

    Note 5: when you compile Caffe Python interface like:
    make pycaffe
    make pytest 
    you may run into this problem:
    ======================================================================
    ERROR: test_forward_backward (test_net.TestNet)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/ylzhao/Project/caffe/python/caffe/test/test_net.py", line 45, in setUp
        size=self.net.blobs['label'].data.shape)
      File "caffe/pycaffe.py", line 28, in _Net_blobs
        return OrderedDict(zip(self._blob_names, self._blobs))
    TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr >
    It seems that this problem is related to Boost 1.60, not 1.59. You can find the discussion at 3494 and 3575, and the fix is at 3575. The commit you want to apply is at 3575 commit.

    Unfortunately, this commit has not been merged into Caffe master branch, so you should do the merge by hand.

    That's all, have fun with Caffe building!

    2015年4月21日星期二

    Mac OS X 10.10安装深度学习框架Caffe教程

    Mac OS X 10.10安装深度学习框架Caffe教程

    Author: Hoondy.com
    翻译和修订:赵毅力(panovr at gmail dot com)

    这份教程是关于在Mac OS X 10.10上安装深度学习框架Caffe的详细指南(成功测试的操作系统是Mac OS X10.10.3,2.3 GHz的英特尔酷睿i7 CPU,NVIDIA的GeForce GT650M显卡)

    在过去的几天,我一直尝试着在我的MacBook Pro上面安装Caffe框架。Caffe是由Berkeley视觉小组开发的深度学习框架,你可以从Caffe的主页上http://caffe.berkekeyvision.org/获取更多关于它的介绍。如果你正在阅读这篇教程,你可能已经注意到了要在Mac OS X上面安装Caffe框架,你必须安装正确的依赖,并从不同的地方下载几个第三方库和工具包。Caffe网站上关于Mac OS X的安装文档有点过时,并不十分完整,有时还有点令人费解。所以,我决定分享我的成功故事,并为简单起见,我把它做成一个一步一步的安装教程,用于帮助和指导其他人如何在Mac OS X上安装Caffe框架。为了获得最大的计算性能,我将使用我的MacBook Pro上面的NVIDIA GPU并链接NVIDA的cuDNN GPU库进行加速。

    2015年4月19日星期日

    从特征描述符到深度学习:计算机视觉发展20年


    从特征描述符到深度学习:计算机视觉研究20年

    Author: Tomasz Malisiewicz
    翻译:赵毅力(panovr at gmail dot com)


    我们都知道在过去两年(2012-2014年)深度卷积神经网络在目标检测与识别的基准测试中有过辉煌的成绩,所以你可能会问:在此之前的物体识别技术是什么样子?早期识别系统的设计和现代以多层卷积为基础的框架之间的关系是什么?

    让我们先来回顾一下过去20年里计算机视觉研究中的一些重要理论和方法。

    2013年10月10日星期四

    BLAS & LAPACK for Windows

    BLAS and LAPACK from Netlib are the de facto libraries for linear algebra. For those interested in using BLAS and LAPACK on Windows platform, I have compiled them in 32bit and 64bit libraries by Intel® Fortran Compiler XE 13.1 and Visual Studio 2010.