Deploy (always running) Jupyter (ipython) Notebook server (remote access) in Linux auto start when you start your machine (using supervisor in Linux)

I use Ubuntu and it works. red colored words can be run in shell $ command

 

first, you need the ipython, you can pip, like $pip install jupyter notebook, if you don’t have pip, install it using $apt-get install python-pip

But I use Anaconda, I guess its the basic for a data scientist using Python, even an amateur, just go to their website https://www.continuum.io/downloads

in Linux shell, choose a directory you’d like to save your download files, wget the installer.

e.g. $wget https://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh

https://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh

then install, but you have to make your downloaded file executable (don’t forget the ‘.’ before ‘/’) $ ./chmod 777 Anaconda2-4.4.0-Linux-x86_64.sh

run the installer $ ./Anaconda2-4.4.0-Linux-x86_64.sh

 

Next, set up notebook

  • generate config file for the notebook : in shell $ jupyter notebook – – generate-config

 /home/iri/.jupyter/jupyter_notebook_config.py

remember the path of yours machine returns, will be used later

  • to get a encrypted password, open ipython $ ipython 
    In [1]: from notebook.auth import passwd
    In [2]: passwd()
    Enter password: 
    Verify password: 
    Out[2]: 'sha1:a1a7e6611365:4db3c012ed2dc11a6348c7a79d9e881e6992fc07'

    save the ‘sha1:a…’ in somewhere, like in a txt to be used later, exit()

  • edit the config generated in the earlier step (use vi, vim, nano …), e.g $ vim /home/iri/.jupyter/jupyter_notebook_config.py
    c = get_config()
    c.IPKernelApp.pylab = 'inline'
    c.NotebookApp.ip='*' 
    c.NotebookApp.password = 'sha1:ce23d945972f:34769685a7ccd3d08c84a18c63968a41f1140274'
    c.NotebookApp.open_browser = False 
    c.NotebookApp.port = 1314 #any port you like
    c.NotebookApp.notebook_dir = '/home/notebook' #the directory you set, where jupyter notebook starts 
    

save and exit

  • find your machine’s ip using $ ifconfig
  • to verify, execute $jupyter notebook

keep $jupyter notebook running, open a browser on another machine, put url, http://(the ip of your server machine):1314 e.g.  in internal network (machines using the same wifi), http://192.168.1.112:1314,  enter password (the password you input earlier in ipython to get ‘sha1:…’, the you will land to your jupyter notebook.

if you are deploying a server, e.g. in AWS, you can access your jupyter notebook from anywhere of our planet, by inputting your server’s public IP and the port you set for the notebook, e.g. mine is http://shichaoji.com:520

Here why mine starts from https:// , because I use ssh certificate, it is optional but using certificate will be safer, especially deploying to public

note: if encounter error like Permission denied: ‘/run/user/1000/jupyter’

solution: $ unset XDG_RUNTIME_DIR

optional, add a certificate (from http to https)

  • in shell, do $ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

you can press ‘enter’ through the end, and it will generate a certificate file mycert.pem

  • copy the generated certificate file to a directory  $ cp mycert.pem /(some_ directory), e.g. $ cp mycert.pem /home/iri/.jupyter
  • add another line to the config file, e.g. , $ nano/home/iri/.jupyter/jupyter_notebook_config.py
     c.NotebookApp.certfile=u'/home/iri/.jupyter/mycert.pem'
  • run notebook $ jupyter notebook 
  • go from another computer (proceed any security warning), https://192.168.1.112:1314

 

Deployment, auto start when you start your machine, better do below as root, if will be running forever in servers like in AWS

  • install supervisor, as root or sudoer,  $ apt-get install supervisor
  • go to the config directory of supervisor (where it installed), $ cd /etc/supervisor/conf.d/
  • create and edit a config file : $ vim deploy.conf (name whatever but with .conf)
[program:notebook]
command = /home/iri/anaconda2/bin/jupyter notebook 
user = iri # the user you install jupyter notebook
directory = /home/notebook
autostart = true
autorestart = true
logfile= /home/notebook/book.log # log

command —- the directory you install python/command

user —- the user you install python in the earlier step

directory —- the same directory in the jupyter notebook config file, where it starts

autostart —- run jupyter notebook when the machine starts

autorestart —- restart jupyter notebook if the program fails

  • final step, in shell, as root, run $ service supervisor restart
  • you can check status as root , run $ supervisorctl, then command supervisor> status, supervisor> help to get commands

 

Add Anaconda new Python environment & add to jupyter notebook kernel

Create a virtual environment for your project

conda create -n yourenvname python=x.x anaconda

Activate your virtual environment

source activate yourenvname

list conda environment

conda info --envs

 

Install additional Python packages to a virtual environment

conda install -n yourenvname [package]

Add the new environment to jupyter notebook

pip install ipykernel

python -m ipykernel install --user --name=yourenvname

list jupyter notebook’s kernels info (config) , list kernel environment

jupyter kernelspec list

conda info --envs

Change name/ properties of the environment:

go to the path under “jupyter kernelspec list”, edit the config file directly

delete environment

conda remove --name myenv --all

 

 

 

bokeh 5th: project interactive data visualization web app

 

How to make useful and fun interactive data visualization web apps and how to deploy them online for public access? 

Click App runs on Here, will be interactive and animated.

bokeh 4th: server

how to write simple bokeh program that runs on a server?

bokeh 3rd: high-level charts

Where to get Bokeh high-level charts that can be simply created through pandas DataFrame?