Skip to content

Commit

Permalink
Added IPython Notebook Spark integration files through the pyspark pr…
Browse files Browse the repository at this point in the history
…ofile.
  • Loading branch information
donnemartin committed Jul 23, 2015
1 parent b9bda35 commit eef444b
Show file tree
Hide file tree
Showing 9 changed files with 2,787 additions and 0 deletions.
Binary file added init/profile_pyspark/history.sqlite
Binary file not shown.
532 changes: 532 additions & 0 deletions init/profile_pyspark/ipython_config.py

Large diffs are not rendered by default.

844 changes: 844 additions & 0 deletions init/profile_pyspark/ipython_nbconvert_config.py

Large diffs are not rendered by default.

666 changes: 666 additions & 0 deletions init/profile_pyspark/ipython_notebook_config.py

Large diffs are not rendered by default.

663 changes: 663 additions & 0 deletions init/profile_pyspark/ipython_qtconsole_config.py

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions init/profile_pyspark/startup/00-pyspark-setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Configure the necessary Spark environment
import os
import sys


# Note: Some Spark installations do not need the extra libexec path
spark_home = os.environ.get('SPARK_HOME', None)
sys.path.insert(0, spark_home + "/libexec/python")

# Add the py4j to the path.
# You may need to change the version number to match your install
sys.path.insert(0, os.path.join(spark_home, 'libexec/python/lib/py4j-0.8.2.1-src.zip'))

# Initialize PySpark to predefine the SparkContext variable 'sc'
execfile(os.path.join(spark_home, 'libexec/python/pyspark/shell.py'))
11 changes: 11 additions & 0 deletions init/profile_pyspark/startup/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
This is the IPython startup directory

.py and .ipy files in this directory will be run *prior* to any code or files specified
via the exec_lines or exec_files configurables whenever you load this profile.

Files will be run in lexicographical order, so you can control the execution order of files
with a prefix, e.g.::

00-first.py
50-middle.py
99-last.ipy
7 changes: 7 additions & 0 deletions init/profile_pyspark/static/custom/custom.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
Placeholder for custom user CSS
mainly to be overridden in profile/static/custom/custom.css
This will always be an empty file in IPython
*/
49 changes: 49 additions & 0 deletions init/profile_pyspark/static/custom/custom.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// leave at least 2 line with only a star on it below, or doc generation fails
/**
*
*
* Placeholder for custom user javascript
* mainly to be overridden in profile/static/custom/custom.js
* This will always be an empty file in IPython
*
* User could add any javascript in the `profile/static/custom/custom.js` file
* (and should create it if it does not exist).
* It will be executed by the ipython notebook at load time.
*
* Same thing with `profile/static/custom/custom.css` to inject custom css into the notebook.
*
* Example :
*
* Create a custom button in toolbar that execute `%qtconsole` in kernel
* and hence open a qtconsole attached to the same kernel as the current notebook
*
* $([IPython.events]).on('app_initialized.NotebookApp', function(){
* IPython.toolbar.add_buttons_group([
* {
* 'label' : 'run qtconsole',
* 'icon' : 'icon-terminal', // select your icon from http://fortawesome.github.io/Font-Awesome/icons
* 'callback': function () {
* IPython.notebook.kernel.execute('%qtconsole')
* }
* }
* // add more button here if needed.
* ]);
* });
*
* Example :
*
* Use `jQuery.getScript(url [, success(script, textStatus, jqXHR)] );`
* to load custom script into the notebook.
*
* // to load the metadata ui extension example.
* $.getScript('/static/notebook/js/celltoolbarpresets/example.js');
* // or
* // to load the metadata ui extension to control slideshow mode / reveal js for nbconvert
* $.getScript('/static/notebook/js/celltoolbarpresets/slideshow.js');
*
*
* @module IPython
* @namespace IPython
* @class customjs
* @static
*/

0 comments on commit eef444b

Please sign in to comment.