Support for building NEURON on windows under Azure CI with installer #572

pramodk · 2020-05-31T23:46:55Z

Later edit:

We will open separate tickets for testing installer and adding a developer README. At this stage, the pipeline can be used as Windows CI for every PR.

pramodk · 2020-05-31T23:58:23Z

By the way, the scripts under ci folder are put together so that one can use those same scripts to setup development/build environment and install NEURON from source. Only missing part is Visual Studio but I will cover that tomorrow.

ci/azure-win-installer-upload.yml

pramodk · 2020-06-01T09:24:31Z

CI fails with unrelated error on OS X:

+ /Users/runner/runners/2.169.1/work/1/s/nrn_test_venv_37/bin/python -c 'import neuron; from neuron import h; s = h.Section(); s.insert('\''cacum'\''); quit()'
Warning: no DISPLAY environment variable.
--No graphics will be displayed.
dlopen failed - 
dlopen(x86_64/.libs/libnrnmech.so, 2): no suitable image found.  Did find:
	file system relative paths not allowed in hardened programs
Traceback (most recent call last):
  File "<string>", line 1, in <module>

This is because of new restrictive Python distributions on OSX. I created #574.

pramodk · 2020-06-02T00:02:30Z

@nrnhines : slide 27 of this pdf says,

you might first need to copy all the dlls cp c:/nrn/bin/*.dll c:/nrn/lib/python/neuron

Is that still applicable? If so, that should be addressed?

By the way, this PR is ready for first review except README.

nrnhines · 2020-06-02T00:25:05Z

slide 27 of this pdf says,

This was also mentioned in #319 with the comment

Here is a big hint relevant to the 'dll dependency' quote above. When I copy all c:/nrn/bin/*.dll into the neuron module, 'import neuron' works.

This is resolved by the use of os.add_dll_directory in pull request #565

ferdonline · 2020-06-02T00:39:46Z

setup.py

@@ -272,7 +272,7 @@ def setup_package():
        rxd_params = extension_common_params.copy()
        rxd_params['libraries'].append("rxdmath")
        rxd_params.update(dict(
-            extra_compile_args=["-O0"],  # cython files take too long to compile with O3
+            extra_compile_args=["-O2"],


We should simply remove this flag so that other configurations may change this, like env vars

to be tackled here : #518

Ok, I am reverting.

nrnhines

I like the organization where the builder can do this on their own machine. This is filled with idioms that work around surprising problems. I wonder if comments about these would be helpful.

nrnhines · 2020-06-02T11:29:36Z

mingw_files/nrnmingwenv.sh

@@ -85,6 +85,8 @@ copy() {
 copy mingw64/bin '
 as.exe
 ld.exe
+zlib1.dll
+libzstd.dll


We should make sure that the changes from pull request #563 where we now automatically determinge needed dlls take precedence over this.

Simply removed those, since cp_dlls is doing the job.

- setup azure CI pipelines with windows server 2019 - build installer with python 2.7, 3.5, 3.6, 3.7, 3.8 - use Microsoft MPI for parallel support - installer is also tested with system provided 2.7 - todo - todo - todo Installer is (temporarily) being uploaded to : https://github.com/neuronsimulator/installers

src/mswin/nrnsetupmingw.nsi.in

packaging/python/test_wheels.sh

alexsavulescu · 2020-10-22T11:59:57Z

This PR is ready for review. Testing the installer doesn't work (see ci/azure-win-installer-upload.yml) and will be resolved in a separate ticket. Also a developer README will be added at a later step.

pramodk · 2020-10-22T17:29:29Z

I will review this tonight.

pramodk

I added minor comments / questions. Apart from those, it's ready for merge.

NOTE: While merging, we should squash commit messages into meaningful summary.

src/nrnpython/nrnpy_hoc.cpp

src/mswin/nrnsetupmingw.nsi.in

share/lib/python/neuron/rxd/rxd.py

pramodk · 2020-10-22T21:15:31Z

setup.py

@@ -272,7 +272,7 @@ def setup_package():
        rxd_params = extension_common_params.copy()
        rxd_params['libraries'].append("rxdmath")
        rxd_params.update(dict(
-            extra_compile_args=["-O0"],  # cython files take too long to compile with O3
+            extra_compile_args=["-O2"],


Ok, I am reverting.

packaging/python/test_wheels.sh

pramodk · 2020-10-22T22:13:21Z

@nrnhines : when you will merge this, here is commit message you can use after squashing all commits:

Support for building windows under Azure CI with installer
  - setup azure CI pipelines with windows server 2019
  - build installer with python 2.7, 3.5, 3.6, 3.7, 3.8
  - use Microsoft MPI for parallel support
  - installer is (temporarily) being uploaded to : https://github.com/neuronsimulator/installers
  - Add test for nrnivmodl on windows
  - Reorder tests
  - Specify timeoutInMinutes for all platforms
  - Run ctest after building on Windows
  - dl_info and dladdr not defined on windows
  - installer testing is currently disabled on windows due to silent instalaltion bug

pramodk · 2020-10-23T14:56:42Z

@nrnhines @alexsavulescu : should we merge this then? (I put commit message here).

alexsavulescu · 2020-10-23T15:06:07Z

OK for me, @adamjhn clarified rxd commit. So green light to merge on my part

One of the feature of github ci is the capability to split jobs. This way job are smaller and more readable. Split build and test from build documentation and publish. Fix live-debug following neuron simulator

pramodk added wip CI Continuous Integration windows labels May 31, 2020

pramodk requested a review from nrnhines May 31, 2020 23:46

pramodk commented Jun 1, 2020

View reviewed changes

ci/azure-win-installer-upload.yml Outdated Show resolved Hide resolved

pramodk requested review from adamjhn and ramcdougal June 1, 2020 00:02

pramodk force-pushed the pramodk/windows-ci branch 4 times, most recently from beeb589 to 43c09ee Compare June 1, 2020 18:43

ferdonline reviewed Jun 2, 2020

View reviewed changes

nrnhines approved these changes Jun 2, 2020

View reviewed changes

This was referenced Jun 26, 2020

Issue with binary installer on OS X : ModuleNotFoundError: No module named 'neuron.hoc38' #611

Closed

Can I compile NEURON on Windows with Cmake and Visual Studio 2019? #623

Closed

alexsavulescu closed this Oct 14, 2020

alexsavulescu reopened this Oct 14, 2020

alexsavulescu closed this Oct 15, 2020

alexsavulescu reopened this Oct 15, 2020

alexsavulescu force-pushed the pramodk/windows-ci branch 2 times, most recently from 13e8f9b to 125c490 Compare October 16, 2020 20:53

pramodk added 5 commits October 19, 2020 09:45

Avoid insert('cacum') test on Darwin with Python37

a716935

Add test for nrnivmodl on windows

2882cf0

Use call nrnivmodl instead of nrnivmodl

68c63b0

Reorder tests

5487119

alexsavulescu force-pushed the pramodk/windows-ci branch 2 times, most recently from 1a5cd2a to ebd4377 Compare October 22, 2020 08:54

alexsavulescu added 3 commits October 22, 2020 11:12

dl_info and dladdr not defined on windows

ca2464b

fix install and setup_exe for MinGW

1f0e3d1

make test installer not blocking for now

7be7ef1

alexsavulescu force-pushed the pramodk/windows-ci branch from ebd4377 to 7be7ef1 Compare October 22, 2020 09:46

dlls now copied by cp_dlls

a182498

alexsavulescu marked this pull request as ready for review October 22, 2020 11:48

alexsavulescu requested a review from nrnhines October 22, 2020 11:49

alexsavulescu reviewed Oct 22, 2020

View reviewed changes

src/mswin/nrnsetupmingw.nsi.in Show resolved Hide resolved

packaging/python/test_wheels.sh Outdated Show resolved Hide resolved

alexsavulescu requested a review from ferdonline October 22, 2020 11:57

nrnhines approved these changes Oct 22, 2020

View reviewed changes

Address review comments with minor notes

73bc37c

pramodk mentioned this pull request Oct 22, 2020

Add README about how to use CI scripts for local development setup on windows #788

Open

1 task

pramodk commented Oct 22, 2020

View reviewed changes

Merge branch 'master' into pramodk/windows-ci

a4505ca

alexsavulescu added 2 commits October 23, 2020 00:18

use MINGW macro

8666c48

this is reverted in master (#587)

4fc2e1e

nrnhines merged commit 4963240 into master Oct 23, 2020

nrnhines deleted the pramodk/windows-ci branch October 23, 2020 18:35

alexsavulescu mentioned this pull request Oct 30, 2020

Make windows silent installer work on Azure CI #802

Closed

alexsavulescu linked an issue Oct 30, 2020 that may be closed by this pull request

Make windows silent installer work on Azure CI #802

Closed

alexsavulescu mentioned this pull request Dec 8, 2020

build fixes in master that should get cherry-picked into release/7.8.1 #859

Closed

alexsavulescu mentioned this pull request Apr 26, 2021

Create Release Notes for 8.0 releases #1211

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for building NEURON on windows under Azure CI with installer #572

Support for building NEURON on windows under Azure CI with installer #572

pramodk commented May 31, 2020 •

edited

Loading

pramodk commented May 31, 2020

pramodk commented Jun 1, 2020

pramodk commented Jun 2, 2020

nrnhines commented Jun 2, 2020

ferdonline Jun 2, 2020

alexsavulescu Oct 19, 2020

pramodk Oct 22, 2020

nrnhines left a comment

nrnhines Jun 2, 2020

alexsavulescu Oct 22, 2020

alexsavulescu commented Oct 22, 2020

pramodk commented Oct 22, 2020

pramodk left a comment

pramodk Oct 22, 2020

pramodk commented Oct 22, 2020

pramodk commented Oct 23, 2020

alexsavulescu commented Oct 23, 2020 •

edited

Loading

Support for building NEURON on windows under Azure CI with installer #572

Support for building NEURON on windows under Azure CI with installer #572

Conversation

pramodk commented May 31, 2020 • edited Loading

Later edit:

pramodk commented May 31, 2020

pramodk commented Jun 1, 2020

pramodk commented Jun 2, 2020

nrnhines commented Jun 2, 2020

ferdonline Jun 2, 2020

Choose a reason for hiding this comment

alexsavulescu Oct 19, 2020

Choose a reason for hiding this comment

pramodk Oct 22, 2020

Choose a reason for hiding this comment

nrnhines left a comment

Choose a reason for hiding this comment

nrnhines Jun 2, 2020

Choose a reason for hiding this comment

alexsavulescu Oct 22, 2020

Choose a reason for hiding this comment

alexsavulescu commented Oct 22, 2020

pramodk commented Oct 22, 2020

pramodk left a comment

Choose a reason for hiding this comment

pramodk Oct 22, 2020

Choose a reason for hiding this comment

pramodk commented Oct 22, 2020

pramodk commented Oct 23, 2020

alexsavulescu commented Oct 23, 2020 • edited Loading

pramodk commented May 31, 2020 •

edited

Loading

alexsavulescu commented Oct 23, 2020 •

edited

Loading