1. show all the rows or columns from a DataFrame in Jupyter QTConcole
if the df has a lot of rows or columns, then when you try to show the df, pandas will auto detect the size of the displaying area and automatically hide some part of the data by replacing with .... To show the full data without any hiding, you can use pd.set_option('display.max_rows', 500) and pd.set_option('display.max_rows', 500) to change the max number of rows or max number of columns to display.
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.random(200).reshape(2, 100))
df
# set up display area to show dataframe in jupyter qtconsole
pd.set_option('display.height', 1000)
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
df
2. display all text in a cell without truncation
pandas will automatically truncate the long string to display by default. Taking the example below, the string_x is long so by default it will not display the full string. However the full text is wanted. pd.set_option('display.max_colwidth', -1) will help to show all the text strings in the column.
string_x = "if the df has a lot of rows or columns, then when you try to show the df, pandas will auto detect
the size of the displaying area and automatically hide some part of the data by replacing with"
pd.DataFrame({'string_x': string_x}, index = [0])
| string_x | |
|---|---|
| 0 | if the df has a lot of rows or columns, then w... |
pd.options.display.max_rows
pd.set_option('display.max_colwidth', -1)
pd.DataFrame({'string_x': string_x}, index = [0])
| string_x | |
|---|---|
| 0 | if the df has a lot of rows or columns, then when you try to show the df, pandas will auto detect the size of the displaying area and automatically hide some part of the data by replacing with |
3. display html in Jupyter
If the dataframe is shown in html, then it is easier to copy the splited data to excel. Otherwise data copied to excel will be in one row. To display in html in jupyter, use from IPython.display import display, HTML
from IPython.display import display, HTML
display(HTML(' <span style="color:red">the title is: <h1>Hello, world!</h1> </span> '))
the title is:
Hello, world!
4. jupyter run magic command
Jupyter has some magic command making your job easier. Some useful commands like %lsmagic %env %ls %pwd %cd
%lsmagic
Available line magics:
%alias %alias_magic %autocall %automagic %autosave %bookmark %cat %cd %clear %colors %config %connect_info %cp %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history %killbgscripts %ldir %less %lf %lk %ll %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %lx %macro %magic %man %matplotlib %mkdir %more %mv %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %popd %pprint %precision %profile %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %rm %rmdir %run %save %sc %set_env %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode
Available cell magics:
%%! %%HTML %%SVG %%bash %%capture %%debug %%file %%html %%javascript %%js %%latex %%perl %%prun %%pypy %%python %%python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile
Automagic is ON, % prefix IS NOT needed for line magics.
5. run shell command
jupyter also provides the facility to run linux shell command in the way !shall_command.
! ls
20160625_deploy_flask_on_heroku.txt
20170318_pandas_multiple_agg_summary.ipynb
20170402_dl_notes.ipynb
20170402_jupyter_pandas_display.ipynb
build_by_pelican.md
figures
6. list function definiton
?? gives the facility to check the difinicion of a function.
??pd.merge
7. download the server files
FileLink will return a link to the file on your remote server in your browser. You can then click on it to download the file.
from IPython.display import FileLink
FileLink('build_by_pelican.md')