Category Archives: Deploy on Linux
Face Similarity searching ~ landmark detecting
Thanks to this post of facial landmarks and the openface project!
11/11 updated the image pool to 710000
11/3 updated the image pool to 540000
10/14 add face similarity searching! from a 4000-photo pool
Deep Learning model find 128 features of each face –Then Cosine distance ~ simple but powerful.
face similarity searching from celebrities (which superstar looks like you the most)?
I’ve implemented the idea to a web app that can batch processing face chopping from pics & facial landmarks pinpointing!
The next step will be training beauty score rating model & a face similarity comparing to superstars (which star looks like you the most)?
Sentiment Analysis model deployed!
I’ve trained a sentiment analysis on simple data set:
Amazon Reviews: Unlocked Mobile Phones
based on the amazon phone purchase reviews. Simple linear SVM classifier using scikit-learn. The code is down below, please scroll down
Yet I’ve successful deployed the model on an AWS server! original deployment page
Model building
Deploy (always running) Jupyter (ipython) Notebook server (remote access) in Linux auto start when you start your machine (using supervisor in Linux)
I use Ubuntu and it works. red colored words can be run in shell $ command
first, you need the ipython, you can pip, like $pip install jupyter notebook, if you don’t have pip, install it using $apt-get install python-pip
But I use Anaconda, I guess its the basic for a data scientist using Python, even an amateur, just go to their website https://www.continuum.io/downloads
in Linux shell, choose a directory you’d like to save your download files, wget the installer.
e.g. $wget https://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
https://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh
then install, but you have to make your downloaded file executable (don’t forget the ‘.’ before ‘/’) $ ./chmod 777 Anaconda2-4.4.0-Linux-x86_64.sh
run the installer $ ./Anaconda2-4.4.0-Linux-x86_64.sh
Next, set up notebook
- generate config file for the notebook : in shell $ jupyter notebook – – generate-config
/home/iri/.jupyter/jupyter_notebook_config.py
remember the path of yours machine returns, will be used later
- to get a encrypted password, open ipython $ ipython
In [1]: from notebook.auth import passwd In [2]: passwd() Enter password: Verify password: Out[2]: 'sha1:a1a7e6611365:4db3c012ed2dc11a6348c7a79d9e881e6992fc07'
save the ‘sha1:a…’ in somewhere, like in a txt to be used later, exit()
- edit the config generated in the earlier step (use vi, vim, nano …), e.g $ vim /home/iri/.jupyter/jupyter_notebook_config.py
c = get_config() c.IPKernelApp.pylab = 'inline' c.NotebookApp.ip='*' c.NotebookApp.password = 'sha1:ce23d945972f:34769685a7ccd3d08c84a18c63968a41f1140274' c.NotebookApp.open_browser = False c.NotebookApp.port = 1314 #any port you like c.NotebookApp.notebook_dir = '/home/notebook' #the directory you set, where jupyter notebook starts
save and exit
- find your machine’s ip using $ ifconfig
- to verify, execute $jupyter notebook
keep $jupyter notebook running, open a browser on another machine, put url, http://(the ip of your server machine):1314 e.g. in internal network (machines using the same wifi), http://192.168.1.112:1314, enter password (the password you input earlier in ipython to get ‘sha1:…’, the you will land to your jupyter notebook.
if you are deploying a server, e.g. in AWS, you can access your jupyter notebook from anywhere of our planet, by inputting your server’s public IP and the port you set for the notebook, e.g. mine is http://shichaoji.com:520
Here why mine starts from https:// , because I use ssh certificate, it is optional but using certificate will be safer, especially deploying to public
note: if encounter error like Permission denied: ‘/run/user/1000/jupyter’
solution: $ unset XDG_RUNTIME_DIR
optional, add a certificate (from http to https)
- in shell, do $ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem
you can press ‘enter’ through the end, and it will generate a certificate file mycert.pem
- copy the generated certificate file to a directory $ cp mycert.pem /(some_ directory), e.g. $ cp mycert.pem /home/iri/.jupyter
- add another line to the config file, e.g. , $ nano/home/iri/.jupyter/jupyter_notebook_config.py
c.NotebookApp.certfile=u'/home/iri/.jupyter/mycert.pem'
- run notebook $ jupyter notebook
- go from another computer (proceed any security warning), https://192.168.1.112:1314
Deployment, auto start when you start your machine, better do below as root, if will be running forever in servers like in AWS
- install supervisor, as root or sudoer, $ apt-get install supervisor
- go to the config directory of supervisor (where it installed), $ cd /etc/supervisor/conf.d/
- create and edit a config file : $ vim deploy.conf (name whatever but with .conf)
[program:notebook] command = /home/iri/anaconda2/bin/jupyter notebook user = iri # the user you install jupyter notebook directory = /home/notebook autostart = true autorestart = true logfile= /home/notebook/book.log # log
command —- the directory you install python/command
user —- the user you install python in the earlier step
directory —- the same directory in the jupyter notebook config file, where it starts
autostart —- run jupyter notebook when the machine starts
autorestart —- restart jupyter notebook if the program fails
- final step, in shell, as root, run $ service supervisor restart
- you can check status as root , run $ supervisorctl, then command supervisor> status, supervisor> help to get commands
Add Anaconda new Python environment & add to jupyter notebook kernel
Create a virtual environment for your project
conda create -n yourenvname python=x.x anaconda
Activate your virtual environment
source activate yourenvname
list conda environment
conda info --envs
Install additional Python packages to a virtual environment
conda install -n yourenvname [package]
Add the new environment to jupyter notebook
pip install ipykernel python -m ipykernel install --user --name=yourenvname
list jupyter notebook’s kernels info (config) , list kernel environment
jupyter kernelspec list conda info --envs
Change name/ properties of the environment:
go to the path under “jupyter kernelspec list”, edit the config file directly
delete environment
conda remove --name myenv --all
bokeh 5th: project interactive data visualization web app
How to make useful and fun interactive data visualization web apps and how to deploy them online for public access?
set up email service for wordpress
install plugin Postman SMTP
go to settings -> Postman SMTP
follow the instructions,
input the email you want to use as admin account of the site
it will automatically detect which email service you are using, e.g. I use google gmail,
it will suggest the SMTP port, and OAuth 2.0 for authentication
redirecting to https://www.google.com/accounts/Logout?continue=https://console.developers.google.com/start/api?id=gmail
register for google API,
register for Authorized JavaScript origins, and Authorized redirect URI in google’s API page
get back
Final step, you should grant permission with google
http://jishichao.com/wp-admin/admin-post.php?action=postman/requestOauthGrant
Then you have everything.