$ ssh-copy-id user@host (from client to host, the host is the server you want to log in)
enter password, done! next time you don’t need password to log in.
Port forwarding and X11 forwarding
When login, simple add -L (port forwarding), and optional -X (X11 forwarding)
$ ssh -X -L port: localhost: port user@host
first port is the client machine’s localhost’s port
localhost is the server machine’s url , can be diff of localhost, e.g. google.com , where server machine’s internet access google.com, that is the idea of proxy, e.g. access google from China (client machine) login to server machine, forwarding server machine’s access of google to client machine’s localhost
second port is the port of the middle url (localhost in this case), if this port set to 80, it is the default webpage, where we do not need to specify port
Cumulative Distribution Function from an image histogram¶
A histogram of a continuous random variable is sometimes called a Probability Distribution Function (or PDF).
The area under a PDF (a definite integral) is called a Cumulative Distribution Function (or CDF). The CDF quantifies the probability of observing certain pixel intensities.
The histogram option cumulative=True permits viewing the CDF instead of the PDF.
The command plt.twinx() allows two plots to be overlayed sharing the x-axis but with different scales on the y-axis.
In [8]:
# Display a histogram of the pixelsplt.hist(pixels,bins=64,range=(0,256),normed=False,color='red',alpha=0.3)# Use plt.twinx() to overlay the CDF plt.twinx()# Display a cumulative histogram of the pixelsplt.hist(pixels,bins=64,range=(0,256),normed=True,cumulative=True,color='blue',alpha=0.3)plt.title('PDF & CDF (original image)')plt.show()
Histogram equalization is an image processing procedure that reassigns image pixel intensities. The basic idea is to use interpolation to map the original CDF of pixel intensities to a CDF that is almost a straight line. In essence, the pixel intensities are spread out and this has the practical effect of making a sharper, contrast-enhanced image. This is particularly useful in astronomy and medical imaging to help us see more features.
# Load the image into an array: imageimage=plt.imread('cat.jpg')# Flatten the image into 1 dimension: pixelspixels=image.flatten()# Generate a cumulative histogramcdf,bins,patches=plt.hist(pixels,bins=256,range=(0,256),normed=True,cumulative=True)new_pixels=np.interp(pixels,bins[:-1],cdf*255)# Reshape new_pixels as a 2-D array: new_imagenew_image=new_pixels.reshape(image.shape)# Display the new image with 'gray' color mapplt.subplot(2,1,1)plt.title('Equalized image')plt.axis('off')plt.imshow(new_image,cmap='gray')# Generate a histogram of the new pixelsplt.subplot(2,1,2)pdf=plt.hist(new_pixels,bins=64,range=(0,256),normed=False,color='red',alpha=0.4)plt.grid('off')# Use plt.twinx() to overlay the CDF in the bottom subplotplt.twinx()plt.xlim((0,256))plt.grid('off')# Add titleplt.title('PDF & CDF (equalized image)')# Generate a cumulative histogram of the new pixelscdf=plt.hist(new_pixels,bins=64,range=(0,256),cumulative=True,normed=True,color='blue',alpha=0.4)plt.show()
The separate RGB (red-green-blue) channels will be extracted for you as two-dimensional arrays red, green, and blue respectively. You will plot three overlaid color histograms on common axes (one for each channel) in a subplot as well as the original image in a separate subplot.
In [10]:
# Load the image into an array: imageimage=plt.imread('cat.jpg')# Display image in top subplotplt.subplot(2,1,1)plt.title('Original image')plt.axis('off')plt.imshow(image)# Extract 2-D arrays of the RGB channels: red, blue, greenred,blue,green=image[:,:,0],image[:,:,1],image[:,:,2]# Flatten the 2-D arrays of the RGB channels into 1-Dred_pixels=red.flatten()blue_pixels=blue.flatten()green_pixels=green.flatten()# Overlay histograms of the pixels of each color in the bottom subplotplt.subplot(2,1,2)plt.title('Histograms from color image')plt.xlim((0,256))plt.hist(red_pixels,bins=64,normed=True,color='red',alpha=0.2)plt.hist(blue_pixels,bins=64,normed=True,color='blue',alpha=0.2)plt.hist(green_pixels,bins=64,normed=True,color='green',alpha=0.2)# Display the plotplt.show()
Extracting bivariate histograms from a color image¶
Rather than overlaying univariate histograms of intensities in distinct channels, it is also possible to view the joint variation of pixel intensity in two different channels.
The separate RGB (red-green-blue) channels will be extracted for you as one-dimensional arrays red_pixels, green_pixels, & blue_pixels respectively.
In [11]:
# Load the image into an array: imageimage=plt.imread('star.jpg')# Extract RGB channels and flatten into 1-D arrayred,blue,green=image[:,:,0],image[:,:,1],image[:,:,2]red_pixels=red.flatten()blue_pixels=blue.flatten()green_pixels=green.flatten()# Generate a 2-D histogram of the red and green pixelsplt.subplot(2,2,1)plt.grid('off')plt.xticks(rotation=60)plt.xlabel('red')plt.ylabel('green')plt.hist2d(red_pixels,green_pixels,bins=(32,32))# Generate a 2-D histogram of the green and blue pixelsplt.subplot(2,2,2)plt.grid('off')plt.xticks(rotation=60)plt.yticks(size=6)plt.xlabel('green')plt.ylabel('blue')plt.hist2d(green_pixels,blue_pixels,bins=(32,32))# Generate a 2-D histogram of the blue and red pixelsplt.subplot(2,2,3)plt.grid('off')plt.xticks(rotation=60)plt.xlabel('blue',size=5,color='r')plt.ylabel('red')plt.hist2d(blue_pixels,red_pixels,bins=(32,32))plt.subplot(2,2,4)plt.grid('off')plt.xticks(rotation=60)plt.title('orig')plt.imshow(image)# Display the plotplt.show()
When there are more complex relationships between two variables, a simple first order regression is often not sufficient to accurately capture the relationship between the variables. Seaborn makes it simple to compute and visualize regressions of varying orders.
sns.regplot()
the function sns.lmplot() is a higher-level interface to sns.regplot().
A principal difference between sns.lmplot() and sns.regplot() is the way in which matplotlib options are passed (sns.regplot() is more permissive).
For both sns.lmplot() and sns.regplot(), the keyword order is used to control the order of polynomial regression.
The function sns.regplot() uses the argument scatter=None to prevent plotting the scatter plot points again.
In [10]:
tip.head(1)
Out[10]:
total_bill
tip
sex
smoker
day
time
size
0
16.99
1.01
Female
No
Sun
Dinner
2
In [11]:
# Generate a scatter plot of 'weight' and 'mpg' using red circlesplt.scatter(tip['total_bill'],tip['tip'],label='data',color='red',marker='o',alpha=.5)# Plot in blue a linear regression of order 1 between 'weight' and 'mpg'sns.regplot(x='total_bill',y='tip',data=tip,scatter=None,color='blue',label='order 1')# Plot in green a linear regression of order 2 between 'weight' and 'mpg'sns.regplot(x='total_bill',y='tip',data=tip,scatter=None,order=2,color='green',label='order 2')sns.regplot(x='total_bill',y='tip',data=tip,scatter=None,order=3,color='purple',label='order 2')# Add a legend and display the plotplt.legend(loc='upper right')plt.show()
plt.figure(figsize=(5,3))# Generate two 1-D arrays: u, vu=np.linspace(-2,2,41)v=np.linspace(-1,1,21)# Generate 2-D arrays from u and v: X, YX,Y=np.meshgrid(u,v)# Compute Z based on X and YZ=np.sin(3*np.sqrt(X**2+Y**2))# Display the resulting image with pcolor()plt.pcolor(Z,cmap='Blues')plt.colorbar()plt.axis('tight')# Save the figure to 'sine_mesh.png'plt.savefig('sine_mesh.jpg')plt.show()
<matplotlib.contour.QuadContourSet at 0x7f15340dca10>
Improve the spacing between the subplots with plt.tight_layout() and display the figure.¶
In [15]:
plt.set_cmap('winter')# Generate a default contour map of the array Zplt.subplot(2,2,1)plt.contour(X,Y,Z)# Generate a contour map with 20 contoursplt.subplot(2,2,2)plt.contour(X,Y,Z,20)# Generate a default filled contour map of the array Zplt.subplot(2,2,3)plt.contourf(X,Y,Z)# Generate a contour map with 20 contoursplt.subplot(2,2,4)plt.contourf(X,Y,Z,20)# Improve the spacing between subplotsplt.tight_layout()# Display the figureplt.show()
You specify the coordinates of the points using plt.hist2d(x,y) assuming x and y are two vectors of the same length.
You can specify the number of bins with the argument bins=(nx, ny) where nx is the number of bins to use in the horizontal direction and ny is the number of bins to use in the vertical direction.
You can specify the rectangular region in which the samples are counted in constructing the 2D histogram. The optional parameter required is range=((xmin, xmax), (ymin, ymax))
xmin and xmax are the respective lower and upper limits for the variables on the x-axis and
ymin and ymax are the respective lower and upper limits for the variables on the y-axis. Notice that the optional range argument can use nested tuples or lists.
In [16]:
# Generate a 2-D histogramplt.hist2d(np.random.randint(0,10,1000),np.random.randint(0,30,1000),bins=(20,20),range=((0,13),(0,35)),cmap='Blues')# Add a color bar to the histogramplt.colorbar()# Add labels, title, and display the plotplt.xlabel('Horse power [hp]')plt.ylabel('Miles per gallon [mpg]')plt.title('hist2d() plot')plt.show()
The color image can be plotted as usual using plt.imshow()
The resulting image loaded is a NumPy array of three dimensions. The array typically has dimensions M×N×3, where M×N is the dimensions of the image. The third dimensions are referred to as color channels (typically red, green, and blue).
In [18]:
# Load the image into an array: imgimg=plt.imread('cat.jpg')# Print the shape of the imageprint(img.shape)# Display the imageplt.imshow(img)# Hide the axesplt.axis('off')plt.show()
(194, 259, 3)
In [19]:
# Load the image into an array: imgimg=plt.imread('cat.jpg')# Print the shape of the imageprint(img.shape)# Compute the sum of the red, green and blue channels: intensityintensity=img.sum(axis=2)# Print the shape of the intensityprint(intensity.shape)# Display the intensity with a colormap of 'gray'plt.imshow(intensity,cmap='gray')# Add a colorbarplt.colorbar()# Hide the axes and show the figureplt.axis('off')plt.show()
When using plt.imshow() to display an array, the default behavior is to keep pixels square so that the height to width ratio of the output matches the ratio determined by the shape of the array. In addition, by default, the x- and y-axes are labeled by the number of samples in each direction.
The ratio of the displayed width to height is known as the image aspect
the range used to label the x- and y-axes is known as the image extent.
The default aspect value of 'auto' keeps the pixels square and the extents are automatically computed from the shape of the array if not specified otherwise.
In [21]:
# Load the image into an array: imgimg=plt.imread('cat.jpg')# Specify the extent and aspect ratio of the top left subplotplt.subplot(2,2,1)plt.title('extent=(-1,1,-1,1),\naspect=0.5')plt.xticks([-1,0,1])plt.yticks([-1,0,1])plt.imshow(img,extent=(-1,1,-1,1),aspect=0.5)# Specify the extent and aspect ratio of the top right subplotplt.subplot(2,2,2)plt.title('extent=(-1,1,-1,1),\naspect=1')plt.xticks([-1,0,1])plt.yticks([-1,0,1])plt.imshow(img,extent=(-1,1,-1,1),aspect=1)# Specify the extent and aspect ratio of the bottom left subplotplt.subplot(2,2,3)plt.title('extent=(-1,1,-1,1),\naspect=2')plt.xticks([-1,0,1])plt.yticks([-1,0,1])plt.imshow(img,extent=(-1,1,-1,1),aspect=2)# Specify the extent and aspect ratio of the bottom right subplotplt.subplot(2,2,4)plt.title('extent=(-2,2,-1,1),\naspect=2')plt.xticks([-2,-1,0,1,2])plt.yticks([-1,0,1])plt.imshow(img,extent=(-2,2,-1,1),aspect=2)# Improve spacing and display the figureplt.tight_layout()plt.show()
Sometimes, low contrast images can be improved by rescaling their intensities.
In [23]:
# Load the image into an array: imageimage=plt.imread('cat.jpg')# Extract minimum and maximum values from the image: pmin, pmaxpmin,pmax=image.min(),image.max()print("The smallest & largest pixel intensities are %d & %d."%(pmin,pmax))# Rescale the pixels: rescaled_imagerescaled_image=256*(image-pmin)/(pmax-pmin)print("The rescaled smallest & largest pixel intensities are %.1f & %.1f."%(rescaled_image.min(),rescaled_image.max()))# Display the original image in the top subplotplt.subplot(2,1,1)plt.title('original image')plt.axis('off')plt.imshow(image)# Display the rescaled image in the bottom subplotplt.subplot(2,1,2)plt.title('rescaled image')plt.axis('off')plt.imshow(rescaled_image)plt.show()
The smallest & largest pixel intensities are 0 & 255.
The rescaled smallest & largest pixel intensities are 0.0 & 256.0.
plt.figure(figsize=[6,3])# Plot in blue the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='blue')# Plot in red the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='red')# Display the plotplt.show()
In calling plt.axes([xlo, ylo, width, height]), a set of axes is created and made active with lower corner at coordinates (xlo, ylo) of the specified width and height. Note that these coordinates are passed to plt.axes() in the form of a list.
The coordinates and lengths are values between 0 and 1 representing lengths relative to the dimensions of the figure. After issuing a plt.axes() command, plots generated are put in that set of axes.
In [6]:
plt.figure(figsize=[9,3])# Create plot axes for the first line plotplt.axes([0.05,0.05,0.425,0.9])# Plot in blue the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='blue')# Create plot axes for the second line plotplt.axes([.525,0.05,0.425,0.9])# Plot in red the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='red')# Display the plotplt.show()
The command plt.axes() requires a lot of effort to use well because the coordinates of the axes need to be set manually. A better alternative is to use plt.subplot() to determine the layout automatically.
plt.subplot(m, n, k) to make the subplot grid of dimensions m by n and to make the kth subplot active (subplots are numbered starting from 1 row-wise from the top left corner of the subplot grid).
In [7]:
plt.figure(figsize=[9,3])# Create a figure with 1x2 subplot and make the left subplot activeplt.subplot(1,2,1)# Plot in blue the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='blue')plt.title('Physical Sciences')# Make the right subplot active in the current 1x2 subplot gridplt.subplot(1,2,2)# Plot in red the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='red')plt.title('Computer Science')# Use plt.tight_layout() to improve the spacing between subplotsplt.tight_layout()plt.show()
# Create a figure with 2x2 subplot layout and make the top left subplot activeplt.subplot(2,2,1)# Plot in blue the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='blue')plt.title('Physical Sciences')# Make the top right subplot active in the current 2x2 subplot grid plt.subplot(2,2,2)# Plot in red the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='red')plt.title('Computer Science')# Make the bottom left subplot active in the current 2x2 subplot gridplt.subplot(2,2,3)# Plot in green the % of degrees awarded to women in Health Professionsplt.plot(year,health,color='green')plt.title('Health Professions')# Make the bottom right subplot active in the current 2x2 subplot gridplt.subplot(2,2,4)# Plot in yellow the % of degrees awarded to women in Educationplt.plot(year,education,color='yellow')plt.title('Education')# Improve the spacing between subplots and display themplt.tight_layout()plt.show()
set x- and y-limits of plots, e.g. plt.xlim() to set the x-axis range
In [10]:
plt.figure(figsize=[9,3])# Plot the % of degrees awarded to women in Computer Science and the Physical Sciencesplt.plot(year,computer_science,color='red')plt.plot(year,physical_sciences,color='blue')# Add the axis labelsplt.xlabel('Year')plt.ylabel('Degrees awarded to women (%)')# Set the x-axis rangeplt.xlim([1990,2010])# Set the y-axis rangeplt.ylim([0,50])# Add a title and display the plotplt.title('Degrees awarded to women (1990-2010)\nComputer Science (red)\nPhysical Sciences (blue)')plt.show()# Save the image as 'xlim_and_ylim.png'plt.savefig('xlim_and_ylim.png')
alternatively, you can pass a 4-tuple to plt.axis() to set limits for both axes at once.
save plot using savefig()
In [11]:
plt.figure(figsize=[7,3])# Plot in blue the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='blue')# Plot in red the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='red')# Set the x-axis and y-axis limitsplt.axis((1990,2010,0,50))# Show the figureplt.show()# Save the figure as 'axis_limits.png'plt.savefig('axis_limits.png')
plt.figure(figsize=[20,3])plt.subplot(1,2,1)# Plot in blue the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='blue')# Plot in red the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='red')# Set the x-axis and y-axis limitsplt.axis('equal')plt.subplot(1,2,2)# Plot in blue the % of degrees awarded to women in Computer Scienceplt.plot(year,computer_science,color='blue')# Plot in red the % of degrees awarded to women in the Physical Sciencesplt.plot(year,physical_sciences,color='red')# Set the x-axis and y-axis limitsplt.axis('tight')# Show the figureplt.show()
plt.figure(figsize=[7,2.5])# Specify the label 'Computer Science'plt.plot(year,computer_science,color='red',label='Computer Science')# Specify the label 'Physical Sciences' plt.plot(year,physical_sciences,color='blue',label='Physical Sciences')# Add a legend at the lower centerplt.legend(loc='upper right')# Add axis labels and titleplt.xlabel('Year')plt.ylabel('Enrollment (%)')plt.title('Undergraduate enrollment of women')plt.show()
To enable an arrow, set arrowprops=dict(facecolor='black'). The arrow will point to the location given by xy and the text will appear at the location given by xytext
In [14]:
plt.figure(figsize=[7,2.5])# Plot with legend as beforeplt.plot(year,computer_science,color='red',label='Computer Science')plt.plot(year,physical_sciences,color='blue',label='Physical Sciences')plt.legend(loc='lower right')# Compute the maximum enrollment of women in Computer Science: cs_maxcs_max=computer_science.max()# Calculate the year in which there was maximum enrollment of women in Computer Science: yr_maxyr_max=year[computer_science.argmax()]# Add a black arrow annotationplt.annotate(s='Maximum',xy=(yr_max,cs_max),xytext=(yr_max-30,cs_max+8),arrowprops={'facecolor':'cyan'})# Add axis labels and titleplt.xlabel('Year')plt.ylabel('Enrollment (%)')plt.title('Undergraduate enrollment of women')plt.show()
Matplotlib comes with a number of different stylesheets to customize the overall look of different plots. To activate a particular stylesheet you can simply call plt.style.use() with the name of the style sheet you want.
To list all the available style sheets you can execute: print(plt.style.available)
plt.figure(figsize=[7,2.5])# Set the style to 'ggplot'plt.style.use('ggplot')# Plot the enrollment % of women in Computer Scienceplt.plot(year,computer_science,'ro-',alpha=.2,linewidth=2,markersize=12)plt.title('Computer Science',fontsize=11,alpha=.8,color='orange')plt.xlabel('test x lable',fontsize=8,color='g')plt.ylabel('test y lable',fontsize=9,color='purple',alpha=.8)plt.tick_params(labelsize=7)# Add annotationcs_max=computer_science.max()yr_max=year[computer_science.argmax()]plt.annotate('Maximum',xy=(yr_max,cs_max),xytext=(yr_max-1,cs_max-15),arrowprops=dict(facecolor='green'))# Improve spacing between subplots and display themplt.tight_layout()plt.show()