Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: qubole/qds-sdk-py
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: V1.9.6
Choose a base ref
...
head repository: qubole/qds-sdk-py
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Loading
19 changes: 19 additions & 0 deletions .deepsource.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version = 1

test_patterns = [
'tests/**'
]

exclude_patterns = [
'example/**'
]

[[analyzers]]
name = 'python'
enabled = true
runtime_version = '2.x.x'

# Test coverage analyzer
[[analyzers]]
name = "test-coverage"
enabled = true
71 changes: 71 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
name: "CodeQL"

on:
push:
branches: [master, unreleased]
pull_request:
# The branches below must be a subset of the branches above
branches: [master]
schedule:
- cron: '0 2 * * 4'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
# Override automatic language detection by changing the below list
# Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript', 'python']
language: ['python']
# Learn more...
# https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection

steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2

# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -47,3 +47,6 @@ tags

# Idea IDE
.idea

#scratch directory
./scratch/*
13 changes: 9 additions & 4 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
language: python
python:
- "2.6"
- "2.7"
- "3.3"
- "3.4"
- "3.5"
# command to install dependencies
install:
- pip install urllib3==1.22 requests
- "python setup.py install"
- pip install mock
- pip install pytest
- pip install pytest==3.2.0 pytest-cov==2.6.0
- if [[ $TRAVIS_PYTHON_VERSION == 2.6 ]]; then pip install unittest2; fi
- curl https://deepsource.io/cli | sh
# command to run tests
script: py.test
env: BOTO_CONFIG=/tmp/nowhere
script:
- py.test --cov=./ --cov-report xml
- ./bin/deepsource report --analyzer test-coverage --key python --value-file ./coverage.xml
env:
- BOTO_CONFIG=/tmp/nowhere DEEPSOURCE_DSN=https://c9d4fb28ce6f41798861936c25b0361e@deepsource.io
dist: trusty
28 changes: 25 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
@@ -21,9 +21,9 @@ The SDK is available on `PyPI <https://pypi.python.org/pypi/qds_sdk>`_.

From source
~~~~~~~~~~~
* Download the source code:
* Get the source code:

- Either clone the project: ``git clone git@github.com:qubole/qds-sdk-py.git``
- Either clone the project: ``git clone git@github.com:qubole/qds-sdk-py.git`` and checkout latest release tag from `Releases <https://github.com/qubole/qds-sdk-py/releases>`_.

- Or download one of the releases from https://github.com/qubole/qds-sdk-py/releases

@@ -91,19 +91,34 @@ Examples:
$ qds.py hivecmd check 12345678
{"status": "done", ... }

5. If you are hitting api\_url other than api.qubole.com, then you can pass it in command line as ``--url`` or set in as env variable

::

$ qds.py --token 'xxyyzz' --url https://<env>.qubole.com/api hivecmd ...

or

$ export QDS_API_URL=https://<env>.qubole.com/api


SDK API
-------

An example Python application needs to do the following:

1. Set the api\_token:
1. Set the api\_token and api\_url (if api\_url other than api.qubole.com):

::

from qds_sdk.qubole import Qubole

Qubole.configure(api_token='ksbdvcwdkjn123423')

# or

Qubole.configure(api_token='ksbdvcwdkjn123423', api_url='https://<env>.qubole.com/api')

2. Use the Command classes defined in commands.py to execute commands.
To run Hive Command:

@@ -115,3 +130,10 @@ An example Python application needs to do the following:
print "Id: %s, Status: %s" % (str(hc.id), hc.status)

``example/mr_1.py`` contains a Hadoop Streaming example


Reporting Bugs and Contributing Code
------------------------------------

* Want to report a bug or request a feature? Please open `an issue <https://github.com/qubole/qds-sdk-py/issues/new>`_.
* Want to contribute? Fork the project and create a pull request with your changes against ``unreleased`` branch.
Loading