Deep Learning

Machine Learning – Configuring our Linux (Debian) environment

Posted on by David Mata in Deep Learning

In this tutorial, I am going to show you how to configure your Debian environment to start developing machine learning.

We are going to install miniconda which is a package manager who helps us create virtual environments and download the package necessary.

debina.PNGIn my VirtualBox machine, I installed Debian Jessie amd64.

First, install miniconda.

To install miniconda you will need to go to https://conda.io/miniconda.html and download the 64 bits bash installer for Linux.

It´s important and at least it was for me, do the steps with your user and not as a root user. If your user can´t use the sudo command add it to the sudoest file.

# usermod -aG sudo username

In order to install miniconda type the next command in your terminal and follow the instructions to install it. (Watch where you do the installation, in my case I did the installation in /home/myuser/miniconda3)

bash Miniconda3-latest-Linux-x86_64.sh

After the installation, finish type the command

conda -V

If it shows you the version everything is correct, but if an error shows saying that the command conda couldn´t be found you need to add the path to the environmental variable. To do this type the next command.

export PATH=$PATH:/home/myuser/miniconda3/bin

Now you should be able to use conda, if not try to close your terminal and open it again or try to see if the path is correct typing echo $PATH.

Now what we have to do is to create our virtual environment called tensorflow35 with python 3.5

conda create -n tensorflow35 python=3.5

Get into the environment, your prompt should change adding the (tensorflow35).

source activate tensorflow35

Now we can install any library that we need, we are going to install the basic library to start developing deep nets.

Tensorflow

Tensorflow is a framework for machine learning developed by google. In order to install the framework, we can follow the instructions on their web page.

 (tensorflow35)$ pip install --ignore-installed --upgrade TF_PYTHON_URL

Where is TF_PYTHON_URL .the URL of the TensorFlow Python package? For example, the following command installs the CPU-only version of TensorFlow for Python 3.5:

(tensorflow35)$ pip install --ignore-installed --upgrade 
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.1-cp35-cp35m-linux_x86_64.whl

If you wanna try the installation go into python and type import Tensorflow as tf.

Pandas, Sklearns, Matlib

The easiest way for the majority of users to install pandas is to install it as part of the Anaconda distribution, a cross-platform distribution for data analysis and scientific computing. This is the recommended installation method for most users.

Installing pandas and the rest of the NumPy and SciPy stack can be a little difficult for inexperienced users.

The simplest way to install not only pandas but Python and the most popular packages that make up the SciPystack (IPython, NumPy, Matplotlib, …) is with Anaconda, a cross-platform (Linux, Mac OS X, Windows) Python distribution for data analytics and scientific computing.

After running a simple installer, the user will have access to pandas and the rest of the SciPy stack without needing to install anything else, and without needing to wait for any software to be compiled.

conda install pandas

conda install scikit-learn

conda install -c conda-forge matplotlib=2.0.0

conda install jupyter

OpenCV

OpenCV is a library for image processing that is going to help us to build deep learning application that required image processing.

If you are not going to need load video you can use the updated version to install conda, with this version you can not load video because the FFMPEG encoders are not configured to work with OpenCV. If you want to install a version of the FFMPEG enabled, follow the step below the updated step.

Updated : 

To avoid the “manual installation of OpenCV” you can install OpenCV 3.1 using conda.

conda install -c menpo opencv3

How to install OpenCV with FFMPEG enabled.

Outside of your environment you are going to need to install the conda-build
$ conda install conda-build
$ git clone https://github.com/menpo/conda-opencv3
$ cd conda-opencv3
#To enabling FFMPEG functionality, edit the build.sh file in order to enable (1) or disable (0) the required settings
#Enter into you conda environment.
$ conda config --add channels menpo
$ conda build conda/

 

 

If you installed OpenCV using conda, you don`t need to follow the next steps.

In order to install you need to follow the script that can be found on the GitHub page. But just in case I am going to left a copy of the script here.

# KEEP UBUNTU OR DEBIAN UP TO DATE

sudo apt-get -y update
sudo apt-get -y upgrade
sudo apt-get -y dist-upgrade
sudo apt-get -y autoremove


# INSTALL THE DEPENDENCIES

# Build tools:
sudo apt-get install -y build-essential cmake

# GUI (if you want to use GTK instead of Qt, replace 'qt5-default' with 'libgtkglext1-dev' and remove '-DWITH_QT=ON' option in CMake):
sudo apt-get install -y qt5-default libvtk6-dev

# Media I/O:
sudo apt-get install -y zlib1g-dev libjpeg-dev libwebp-dev libpng-dev libtiff5-dev libjasper-dev libopenexr-dev libgdal-dev

# Video I/O:
sudo apt-get install -y libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev

# Parallelism and linear algebra libraries:
sudo apt-get install -y libtbb-dev libeigen3-dev

# Python:
sudo apt-get install -y python-dev python-tk python-numpy python3-dev python3-tk python3-numpy

# Java:
sudo apt-get install -y ant default-jdk

# Documentation:
sudo apt-get install -y doxygen


# INSTALL THE LIBRARY (YOU CAN CHANGE '3.2.0' FOR THE LAST STABLE VERSION)

sudo apt-get install -y unzip wget
wget https://github.com/opencv/opencv/archive/3.2.0.zip
unzip 3.2.0.zip
rm 3.2.0.zip
mv opencv-3.2.0 OpenCV
cd OpenCV
mkdir build
cd build
cmake -DWITH_QT=ON -DWITH_OPENGL=ON -DFORCE_VTK=ON -DWITH_TBB=ON -DWITH_GDAL=ON -DWITH_XINE=ON -DBUILD_EXAMPLES=ON ..
make -j4
sudo make install
sudo ldconfig


# EXECUTE SOME OPENCV EXAMPLES AND COMPILE A DEMONSTRATION

# To complete this step, please visit 'http://milq.github.io/install-opencv-ubuntu-debian'.

After the manual installation  I got the next error

OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script) in cvShowImage

To fix this error you need to follow the next steps:

  1. Install the packages : libgtk2.0-dev and pkg-config
  2. cd to your opencv directory
  3. mkdir Release
  4. cd Release
  5. Run the command : cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_GTK=ON -D WITH_OPENGL=ON ..
  6. make
  7. sudo make install

 

Stastmodels

statsmodels is a Python module that provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests, and statistical data exploration.

conda install statsmodels

Xlrd

Library for developers to extract data from Microsoft Excel ™ spreadsheet files

conda install -c anaconda xlrd=1.0.0

 

Pydotplus

PyDotPlus is an improved version of the old pydot project that provides a Python Interface to Graphviz’s Dot language.

conda install -c conda-forge pydotplus=2.0.2

 

Leave a comment