Category Archives: project
Sentiment Analysis model deployed!
I’ve trained a sentiment analysis on simple data set:
Amazon Reviews: Unlocked Mobile Phones
based on the amazon phone purchase reviews. Simple linear SVM classifier using scikit-learn. The code is down below, please scroll down
Yet I’ve successful deployed the model on an AWS server! original deployment page
Model building
Add Anaconda new Python environment & add to jupyter notebook kernel
Create a virtual environment for your project
conda create -n yourenvname python=x.x anaconda
Activate your virtual environment
source activate yourenvname
list conda environment
conda info --envs
Install additional Python packages to a virtual environment
conda install -n yourenvname [package]
Add the new environment to jupyter notebook
pip install ipykernel python -m ipykernel install --user --name=yourenvname
list jupyter notebook’s kernels info (config) , list kernel environment
jupyter kernelspec list conda info --envs
Change name/ properties of the environment:
go to the path under “jupyter kernelspec list”, edit the config file directly
delete environment
conda remove --name myenv --all
bokeh 5th: project interactive data visualization web app
How to make useful and fun interactive data visualization web apps and how to deploy them online for public access?
Package a Python program to a single Executable file with Pyinstaller
The advantage of packaging a program is that the user do NOT need any programming environment to use the packaged software.
Packaging a program is very common.For example, when you install most of the software, the ‘setup’ installer will install a packaged software into your computer as a folder and add a shortcut of the executable to your desktop.
However, packaging the program into a single file is much more difficult, I’ll explain why later, let us see how to do it for a Python project.
First things first, make sure you have pyinstaller. I’ve used other libraries like py2exe, but pyinstaller is the best in terms of its compatibility, stability, and user-friendly, the list goes on and on.
in the command window
-> pyinstaller file_tobe_pack.py -o image.ico -F –upx-dir upxfolder
-c (default) open a console window
-w will NOT open a console window
The biggest problem when packaging, especially to a single file:
can not find some specific library dependencies
these are tons of solutions online, but much are so complex and not direct
these is a very simple solution:
Locate the library you import in your program (the one failed packaging) then move the source code to your current directory with the program you wrote, the CHANGE the name of the source code file and change the library name you import.
The packaging software will then search dependency files from the files you made.
Here’s an example:
A Image Crawler I wrote Added GUI (graphical user interface) & packaged into a single .exe file
in 2014, I’ ve written a Python image crawler for a specific website Cure WorldCosplay, a website that attracts cosplayers all over the world post their own pictures . Which has about 10k active members and up to 10 million pictures posted.
The pros is the program is packaged into a single executable file, no programming environment needed. But some virus detection software could report unsafe file.
Here the program is! |
Click names below for download |
With interface |
Without interface |
Theoretically, if you have enough disk space, you can download all the pics of that website (about 9800 Gigabyte), the only limit is your bandwidth. I have deployed 36 crawlers on a Linux server at the same time, they download pics 24/7 at the maximum internet bandwidth.
How to use it: