Deep Learning For Coders - How to Setup AWS P2 instance

This blog elaborates steps to setup AWS P2 instance for fast.ai course on deep learning.

Create AWS Account

Create aws account and select us-west oregon as region by default its ohio (us-west-2).

Go to create case and select “ change limit” option.Give limit as 1 and select p2xlarge give desc as “fast.ai MOOC”And submit request. Got immediate approval.

Install Anaconda python

Get link to latest anaconda 2.x version from continuum.io. I have used below version:

wget https://repo.continuum.io/archive/Anaconda2-5.0.0.1-Linux-x86_64.sh

bash Anaconda2-5.0.0.1-Linux-x86_64.sh

Verify installation:

source ~/.bashrc

echo $PATH

python

exit()

 Install awscli

$pip install awscli

$aws

$ aws configure

provide keys

clone git hub material

go to sources/setup

bash setup_p2.sh

Got AWS console  to get the publicDNS

ssh to the aws instance using the command displayed in the output of
setup_p2.sh

and start jupyter

access notebook bia hostname:8888

Give password as  dl_course

Create a new notebook from top right new > Python [conda root]

test by entering 1 + 1
print ('hello world')

How to terminate the stack

 Use fastai-remove.sh to terminate the stack.

Refer below links for more details:

 

https://www.youtube.com/watch?v=8rjRfW4JM2I

http://wiki.fast.ai/index.php/Lesson_1_Notes

http://wiki.fast.ai/index.php/AWS_install

http://forums.fast.ai/t/reduce-size-of-volume/149/50

How to download files from Google Drive to Google Colaboratory

How to download files from Google Drive to Google Colaboratory

Colaboratory is a research tool for machine learning education and research. It’s a Jupyter notebook environment that requires no setup to use. More info here.


 Navigate to https://colab.research.google.com and open a notebook.
Open File > New Python2 Notebook

How to download a file from google drive to Colaboratory 

from google.colab import auth
auth.authenticate_user()

from googleapiclient.discovery import build
drive_service = build('drive', 'v3')


import io
from googleapiclient.http import MediaIoBaseDownload

file_id = '1uIafI8_GgJSB-XgVa9_lHhJ3JunhT6GH'
request = drive_service.files().get_media(fileId=file_id)
downloaded = io.BytesIO()
downloader = MediaIoBaseDownload(downloaded, request)
done = False
while done is False:
  # _ is a placeholder for a progress object that we ignore.
  # (Our file is small, so we skip reporting progress.)
  _, done = downloader.next_chunk()
 
downloaded.seek(0)
print 'Downloaded file contents are:', downloaded.read()
downloaded.

How to download folder 

 this will throw error "HttpError: "

One option is to zip the folder and get file id of the archive.

file_id = '1VcwmEBcIEEFgYJVM1IT9ZmPI-SqSQEWe'
request = drive_service.files().get_media(fileId=file_id)
downloaded = io.BytesIO()
downloader = MediaIoBaseDownload(downloaded, request)
done = False
while done is False:
  # _ is a placeholder for a progress object that we ignore.
  # (Our file is small, so we skip reporting progress.)
  _, done = downloader.next_chunk()

print 'Download complete' 


downloaded

How to install PyCharm on Oracle Linux

 How to install PyCharm on Oracle Linux

Set proxy if you are behind a firewall.

bash-4.1$ export http_proxy=HTTP_PROXY_HOST_PORT_HERE
bash-4.1$ export https_proxy=HTTPS_PROXY_HOST_PORT_HERE

Download pycharm archive:

bash-4.1$ wget https://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz

--2017-11-05 23:02:33--  https://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz
Resolving ... ...
Connecting to |...|:80... connected.
ERROR: certificate common name â??www.jetbrains.comâ? doesnâ??t match requested host name â??download.jetbrains.comâ?.
To connect to download.jetbrains.com insecurely, use â??--no-check-certificateâ??.

So tried again with " --no-check-certificate" option.

bash-4.1$ wget https://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz --no-check-certificate

--2017-11-05 23:02:47--  https://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz
Resolving ... ...
Connecting to |...|:80... connected.
WARNING: certificate common name â??www.jetbrains.comâ? doesnâ??t match requested host name â??download.jetbrains.comâ?.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: https://download-cf.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz [following]
--2017-11-05 23:02:48--  https://download-cf.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz
Connecting to |...|:80... connected.
OpenSSL: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
Unable to establish SSL connection.

" --no-check-certificate" didn't help. So tried with http url, though its not mentioned in the docs and it worked.


Try using HTTP URL

$ wget http://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz

--2017-11-05 23:03:10--  http://download.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz
Connecting to |:80... connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: http://download-cf.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz [following]
--2017-11-05 23:03:10--  http://download-cf.jetbrains.com/python/pycharm-community-2017.2.4.tar.gz
..
Proxy request sent, awaiting response... 200 OK
Length: 200392450 (191M) [binary/octet-stream]
Saving to: â??pycharm-community-2017.2.4.tar.gzâ?

100%[======================================================================================================================>] 200,392,450 3.16M/s   in 64s

2017-11-05 23:04:14 (2.99 MB/s) - â??pycharm-community-2017.2.4.tar.gzâ? saved [200392450/200392450]

Extract archive:

$ sudo tar xf pycharm-community-2017.2.4.tar.gz -C /opt/
Password:

cd /opt

ln -s  /opt/pycharm-community-2017.2.4 pycharm

Start pycharm:

sh pycharm/bin/pycharm.sh