Skip to content

Latest commit

 

History

History
274 lines (181 loc) · 7.24 KB

GUIDE.md

File metadata and controls

274 lines (181 loc) · 7.24 KB

Guide

How to do Readme

How to get readme

Visual Programming

ASGI server

Use Uvicorn or more better Hypercorn

Making things fast

ToDo:

  • Especially caching with marshalling.HashableClass
  • Note avoid using __dict__ to keep things fast (listen to talk)

https://www.youtube.com/watch?v=xKk7IXm0XO0

  • Adaptive specializing interpreter (PEP 659)
  • Consecutively allocated execution frames
  • Zero cost try-except
  • More regular object layout
  • Lazily created object dictionaries.

https://www.youtube.com/watch?v=6P68IBou_cg

  • The fastest way of removing duplicates from a list
  • How much faster your code is when you reuse the built-in functions instead of trying to reinvent the wheel
  • What is faster than the “for loop”
  • If the lookup is faster in a list or a set
  • When it’s better to beg for forgiveness than to ask for permission

https://www.youtube.com/watch?v=9OOJcTp8dqE

  • Multi-threaded python without GIL

conda env

create or delete environment

conda remove --name toolcraft --all
conda create --name toolcraft python=3.10

Poetry installation

Note that poetry needs to be installed in isolated environment. So refrain from pip install poetry

https://python-poetry.org/docs/master/#installation https://python-poetry.org/docs/1.2/

activate base
conda update --all
conda install python=3.10

curl -sSL https://install.python-poetry.org | python - --uninstall
curl -sSL https://install.python-poetry.org | python - --version 1.2.0
poetry --version

Go to your repo i.e. toolcraft and update packages with poetry

activate toolcraft
poetry update

Also do some more things docs and pytest to work Rest all dependencies will be taken care on cloud

poetry install --with dev --with test --with docs --with pre-commit --with build --no-root

Pre-commit (use in future if needed) -- IGNORE

We will use this in future ...

This works only locally to do automated tests before commit ... should be taken care by individuals

pre-commit install --install-hooks

Poetry publish

With tasks.py

Note that there will be draft release on Github ... Once you release there things will be uploaded to pypi No need to do below stuffs as Github Actions ill handle it :)

git commit -m "update to next alpha"
git push
poetry run invoke bump --alpha

In case you want to publish from local machine -- IGNORE

Note that you need to store pypi creds in poetry config and then you can publish

poetry config --list
poetry config repositories.<repo_name> https://upload.pypi.org/legacy/
poetry config --list
poetry config http-basic.<repo_name> __token__ <pypi-token>
poetry publish -r <repo_name>
poetry cache clear -all .
poetry update

EXTRAS

Getting started with ppw

  • use ppw to start with
  • use invoke instead of fire
  • use docausaurus instead of mkdocs

Need to stick with something as there are lot of things to try

We use ppw to make this project

We will replace tox commands from tox.ini to use tasks.py which uses invoke

Then ...

  • Then replace mkdocs to docausaurus
    • As this has website and blog
    • support for API documentation
    • MDX components
    • multi-platform doc i.e. for mobile, desktop etc ...

Unittests

hypothesis

Maybe look hypothesis .... it is modern ... The author has added feature add_node in python 3.11 for exceptions Also look it has specialized numpy pandas and pyarrow api https://hypothesis.readthedocs.io/en/latest/numpy.html#array-api

Documentations

Docusauraus

Docusauraus will be great Discussion to get sphinx inside it are going on https://github.com/${organizationName}/${projectName}/issues/1059

mkdocs

https://www.mkdocs.org Not sure but thsi soes not have stuff for Docausaurus

pydoc=markdown

pydoc-markdown has docausaurus renderer

but I assume this one is completely different as it does not use sphinx instead has its own markdowm

Badges

LOGO Icons Images

Gen inspirations for different files from

pyproject.toml

.pre-commit-config.yaml

workflows

CLI for toolcraft using cleo

Understand tox and tox-gh-actions

Refer

Security (check dependencies and check python code)

Three options for dependencies:

For code analysis

  • code-ql (from github)

Github has a lab for it ... use tools from them maybe (https://securitylab.github.com/)

Currently pyup it expects requirements file ... need to see how and when they will support poetry based files

Snyk seems to support poetry but the badge link does not work ... need to see that

Add workflow for security

Make sure that before release the vulnerabilities are addressed in workflow Either use

  • github dependabot
  • snyk
  • or something else

Add to conda-forge

Example repo https://github.com/pytest-dev/pytest

Pre-commit

Currently, we let pre-commit serves handle it But if it fails our workflows will not know that So we can make our own pre-commit workflow ... this also make it dependent on release tags

Coverage

There are two options

  • codecov
  • coveralls

We are using codecov But looks like both do not have release-tag specific badges We need to just have badge on main branch and disable it for release

Note that codecov report can be specialized for branch but same is not known for tags

You can see branches here https://app.codecov.io/gh/SpikingNeurons/toolcraft/branches?page=1&order=-updatestamp

Release tag specific readme badge

We might need to have badges specific to release tage And the badges that cannot be release-tag specific must be removed from readme...

Having workflows as dependency

https://github.com/KarmaComputing/Github-Trigger-workflow-from-another-workflow/tree/main/.github/workflows

Netlify

We will use this as facebook uses it

Cloud functions

Netlify allows deploying cloud functions We have set that to website/functions dir Netlify Functions