forked from donnemartin/dev-setup
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added IPython Notebook Spark integration files through the pyspark pr…
…ofile.
- Loading branch information
1 parent
b9bda35
commit eef444b
Showing
9 changed files
with
2,787 additions
and
0 deletions.
There are no files selected for viewing
Binary file not shown.
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# Configure the necessary Spark environment | ||
import os | ||
import sys | ||
|
||
|
||
# Note: Some Spark installations do not need the extra libexec path | ||
spark_home = os.environ.get('SPARK_HOME', None) | ||
sys.path.insert(0, spark_home + "/libexec/python") | ||
|
||
# Add the py4j to the path. | ||
# You may need to change the version number to match your install | ||
sys.path.insert(0, os.path.join(spark_home, 'libexec/python/lib/py4j-0.8.2.1-src.zip')) | ||
|
||
# Initialize PySpark to predefine the SparkContext variable 'sc' | ||
execfile(os.path.join(spark_home, 'libexec/python/pyspark/shell.py')) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
This is the IPython startup directory | ||
|
||
.py and .ipy files in this directory will be run *prior* to any code or files specified | ||
via the exec_lines or exec_files configurables whenever you load this profile. | ||
|
||
Files will be run in lexicographical order, so you can control the execution order of files | ||
with a prefix, e.g.:: | ||
|
||
00-first.py | ||
50-middle.py | ||
99-last.ipy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
/* | ||
Placeholder for custom user CSS | ||
mainly to be overridden in profile/static/custom/custom.css | ||
This will always be an empty file in IPython | ||
*/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
// leave at least 2 line with only a star on it below, or doc generation fails | ||
/** | ||
* | ||
* | ||
* Placeholder for custom user javascript | ||
* mainly to be overridden in profile/static/custom/custom.js | ||
* This will always be an empty file in IPython | ||
* | ||
* User could add any javascript in the `profile/static/custom/custom.js` file | ||
* (and should create it if it does not exist). | ||
* It will be executed by the ipython notebook at load time. | ||
* | ||
* Same thing with `profile/static/custom/custom.css` to inject custom css into the notebook. | ||
* | ||
* Example : | ||
* | ||
* Create a custom button in toolbar that execute `%qtconsole` in kernel | ||
* and hence open a qtconsole attached to the same kernel as the current notebook | ||
* | ||
* $([IPython.events]).on('app_initialized.NotebookApp', function(){ | ||
* IPython.toolbar.add_buttons_group([ | ||
* { | ||
* 'label' : 'run qtconsole', | ||
* 'icon' : 'icon-terminal', // select your icon from http://fortawesome.github.io/Font-Awesome/icons | ||
* 'callback': function () { | ||
* IPython.notebook.kernel.execute('%qtconsole') | ||
* } | ||
* } | ||
* // add more button here if needed. | ||
* ]); | ||
* }); | ||
* | ||
* Example : | ||
* | ||
* Use `jQuery.getScript(url [, success(script, textStatus, jqXHR)] );` | ||
* to load custom script into the notebook. | ||
* | ||
* // to load the metadata ui extension example. | ||
* $.getScript('/static/notebook/js/celltoolbarpresets/example.js'); | ||
* // or | ||
* // to load the metadata ui extension to control slideshow mode / reveal js for nbconvert | ||
* $.getScript('/static/notebook/js/celltoolbarpresets/slideshow.js'); | ||
* | ||
* | ||
* @module IPython | ||
* @namespace IPython | ||
* @class customjs | ||
* @static | ||
*/ |