diff --git a/.gitignore b/.gitignore new file mode 100644 index 000000000..9a5e3a800 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +openjdk diff --git a/LICENSE b/LICENSE new file mode 100644 index 000000000..8f71f43fe --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/NOTICE b/NOTICE new file mode 100644 index 000000000..8cab3205f --- /dev/null +++ b/NOTICE @@ -0,0 +1,2 @@ +Portions of this software are +(C) Copyright IBM Corporation 2017 diff --git a/README.md b/README.md new file mode 100644 index 000000000..f9dfb50ed --- /dev/null +++ b/README.md @@ -0,0 +1,53 @@ +# Repository for useful files to build and develop OpenJDK + +## This repository contains three folders and one script you should be calling to build OpenJDK + +1. The `ansible` folder contains Ansible playbooks that can be used to quickly configure potentially multiple target machines +2. The `docker` folder contains a Docker file which can be used to create a Docker container for building OpenJDK +3. The `security` folder contains a script and cacerts file that is bundled with the JDK and used when building OpenJDK: the cacerts file is an important file that's used to enable SSL connections + +### Got Docker? + +``` +Usage: ./makejdk.sh [options] + +Options: + + -s, --source specify the location for the source and dependencies to be cloned + -d, --destination specify the location for the tarball (eg. /path/ or /path/here.tar.gz) + -r, --repository specify a custom repository (eg. username/openjdk-jdk8u) + -b, --branch specify a custom branch (eg. dev) + -k, --keep reuse docker container (prevents deleting) + -j, --jtreg run jtreg after building + -S, --ssh use ssh when cloning git +``` + + +The simplest way to build OpenJDK using our scripts is to run `makejdk.sh` and have your user be in the Docker group on the machine (or prefix all of your Docker commands with `sudo`. This script can be used to create a Docker container that will be configured with all of the required dependencies and a base operating system in order to build OpenJDK + +By default the docker container is removed each time and your build will be copied from the container to the host + +To override this behaviour, specify the `-k` or `--keep` options. + +By providing the -d option to `makejdk.sh`, the resulting zipped tarball will be copied to the value for -d, for example: +`makejdk.sh /target/directory` will result in the JDK being built inside of your Docker container and then copied to /target/directory on the host + +For help with getting docker follow the instructions [here](https://docs.docker.com/engine/installation/) + +### Prefer to use Ansible? + +Remember that when using Ansible the changes wil be persistent on your local filesystem: the build process includes downloading and configuring a number of dependencies including gcc and various development libraries: see the Ansible playbook itself to see the full listing + +### None of the above? + +You can use the `makejdk.sh` script by providing two parameters: the "working directory" (which is where files will be downloaded to: this includes a number of libraries used with OpenJDK itself such as FreeType and ALSA) and the "target directory" which will be used to store the final .tar.gz file containing the j2sdk-image + +e.g `./makejdk.sh -s /path/to/workspace -d /target/directory` + +#### Configuring Docker + +To use the Docker commands without using the sudo prefix, you will need to be in the Docker group which can be achieved with the following three commands (performed as root) + +1. `sudo groupadd docker`: creates the Docker group if it doesn't already exist +2. `sudo gpasswd -a yourusernamehere docker`: adds a user to the Docker group +3. `sudo service docker restart`: restarts the Docker service so the above changes can take effect diff --git a/ansible/README.md b/ansible/README.md new file mode 100644 index 000000000..ee82a0fe1 --- /dev/null +++ b/ansible/README.md @@ -0,0 +1,40 @@ +# Ansible playbooks to download and install dependencies for OpenJDK on various platforms + +## Do I need to be a superuser to run the playbooks? + +Yes, in order to access the package repositories (we will perform either `yum install` or `apt-get` commands) + +## How do I run the playbooks? + +1) Install Ansible + +On Ubuntu 16.x +`apt install ansible` + +On RHEL 7.x +`yum install epel-release` then `yum install ansible` + +For Ubuntu 14.x +`sudo apt-add-repository ppa:ansible/ansible` +`sudo apt update` +`sudo install ansible` + +2) Run a playbook to install dependencies, for Ubuntu 14.x on x86: +`ansible-playbook -s ubuntu_14_x86.yml` + +3) The Ansible playbook will download and install any dependencies needed to build OpenJDK + +## Which playbook do I run? + +Our playbooks are named according to the operating system they are supported for, keep in mind that package availability may differ between operating system releases + +## Where can I run the playbooks? + +On any machine you have SSH access to: in the playbooks here we are using `hosts: local`, +our playbook will run on the hosts defined in the Ansible install directory's `hosts` file. To run on the local machine, +we will have the following text in our `/etc/ansible/hosts` file: +``` +[local] +127.0.01 +``` +Running `ansible --version` will display your Ansible configuration folder that contains the `hosts` file you can modify diff --git a/ansible/playbooks/rhel_7_x86.yml b/ansible/playbooks/rhel_7_x86.yml new file mode 100644 index 000000000..0c07539c0 --- /dev/null +++ b/ansible/playbooks/rhel_7_x86.yml @@ -0,0 +1,26 @@ +--- + +# Usage: ansible-playbook -s --ask-sudo-pass rhel_7_x86.yml + +- hosts: local + + tasks: + + - name: Install tools + yum: name={{item}} state=installed + with_items: + - make + - gcc + - gcc-c++ + - java-1.7.0-openjdk-devel # Our base JDK + - libXtst-devel + - libXt-devel + - libXrender-devel + - libXext-devel + - libXrender-devel + - libXtst-devel + - libXt-devel + - cups-devel + - freetype-devel + - ccache + - alsa-lib-devel diff --git a/ansible/playbooks/ubuntu_14_x86.yml b/ansible/playbooks/ubuntu_14_x86.yml new file mode 100644 index 000000000..4e241bb0d --- /dev/null +++ b/ansible/playbooks/ubuntu_14_x86.yml @@ -0,0 +1,24 @@ +--- + +# Usage: ansible-playbook -s --ask-sudo-pass ubuntu_14_x86.yml + +- hosts: local + + tasks: + + - name: Install tools + apt: name={{item}} state=installed + with_items: + - make + - gcc + - g++ + - openjdk-7-jdk + - libx11-dev + - libxext-dev + - libxrender-dev + - libxtst-dev + - libxt-dev + - libcups2-dev + - libasound2-dev + - ccache + - zip diff --git a/docker/.gitignore b/docker/.gitignore new file mode 100644 index 000000000..b25c15b81 --- /dev/null +++ b/docker/.gitignore @@ -0,0 +1 @@ +*~ diff --git a/docker/README.md b/docker/README.md new file mode 100644 index 000000000..50f10e300 --- /dev/null +++ b/docker/README.md @@ -0,0 +1,23 @@ +# OpenJDK and Docker +Dockerfiles and build scripts for generating various Docker Images related to IBM Runtimes + +# License +The Dockerfiles and associated scripts found in this project are licensed under the [Apache License 2.0.](http://www.apache.org/licenses/LICENSE-2.0.html). + +# Steps to build + +1. Checkout the source code into +2. Run the docker image (as below) +3. Take a look at the results in the build directory + +# Hints & Tips + + - Build (from build/docker dir): + `docker build -t dockeropenjdk .` + + - Run: + `docker run -it -v :/openjdk/jdk8u dockeropenjdk` + + - Debug (to shell): + `docker run -it -v :/openjdk/jdk8u --entrypoint /bin/bash dockeropenjdk` + diff --git a/docker/jdk8u/x86_64/ubuntu/.gitignore b/docker/jdk8u/x86_64/ubuntu/.gitignore new file mode 100644 index 000000000..c97f963b3 --- /dev/null +++ b/docker/jdk8u/x86_64/ubuntu/.gitignore @@ -0,0 +1 @@ +*.sh diff --git a/docker/jdk8u/x86_64/ubuntu/Dockerfile b/docker/jdk8u/x86_64/ubuntu/Dockerfile new file mode 100644 index 000000000..c3865e7c6 --- /dev/null +++ b/docker/jdk8u/x86_64/ubuntu/Dockerfile @@ -0,0 +1,55 @@ +# (C) Copyright IBM Corporation 2017. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM ubuntu:14.04 + +MAINTAINER Tim Ellison + +# Install required OS tools +RUN apt-get update \ + && apt-get install -qq -y --no-install-recommends \ + cpio \ + make \ + gcc \ + g++ \ + libx11-dev \ + libxext-dev \ + libxrender-dev \ + libxtst-dev \ + libxt-dev \ + libcups2-dev \ + libfreetype6-dev \ + libasound2-dev \ + openjdk-7-jdk \ + ccache \ + zip \ + wget \ + git \ + unzip \ +&& rm -rf /var/lib/apt/lists/* + +# Pick up build instructions +RUN mkdir -p /openjdk/jdk8u + +COPY build.sh /bin/ +COPY jtreg.sh /bin/ + +WORKDIR /openjdk/jdk8u/ + +ENV JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-amd64 + +# Default actions +ENTRYPOINT ["build.sh"] + +CMD ["images"] diff --git a/makejdk.1 b/makejdk.1 new file mode 100755 index 000000000..951e00eae --- /dev/null +++ b/makejdk.1 @@ -0,0 +1,31 @@ +.\" Manpage for makejdk.sh +.TH "0.1.0" "Apache 2" +.SH NAME +makejdk.sh +.SH SYNOPSIS +"./makejdk [options]" +.SH DESCRIPTION +The simplest way to build OpenJDK using our scripts is to run makejdk.sh and have your user be in the Docker group on the machine (or prefix all of your Docker commands with sudo. This script can be used to create a Docker container that will be configured with all of the required dependencies and a base operating system in order to build OpenJDK. +.SH OPTIONS +.TP +.BR \-s ", " \-\-source " " \fI\fR +specify the location for the source and dependencies to be cloned +.TP +.BR \-d ", " \-\-destination " " \fI\fR +specify the location for the tarball (eg. /path/ or /path/here.tar.gz) +.TP +.BR \-r ", " \-\-repository " " \fI\fR +specify a custom repository (eg. username/openjdk-jdk8u) +.TP +.BR \-b ", " \-\-branch " " \fI\fR +specify a custom branch (eg. dev) +.TP +.BR \-k ", " \-\-keep +reuse docker container (prevents deleting) +.TP +.BR \-j ", " \-\-jtreg +run jtreg after building +.TP +.BR \-S ", " \-\-ssh +use ssh when cloning git +.TP diff --git a/makejdk.sh b/makejdk.sh new file mode 100755 index 000000000..f8d85c544 --- /dev/null +++ b/makejdk.sh @@ -0,0 +1,205 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# Script to clone the OpenJDK source then build it + +# Optionally uses Docker, otherwise you can provide two arguments: +# the area to build the JDK e.g. $HOME/mybuilddir as -s or --source +# and the target destination for the tar.gz e.g. -d or --destination $HOME/mytargetdir +# Both must be absolute paths! You can use $PWD/mytargetdir + +# A simple way to install dependencies persistently is to use our Ansible playbooks + +# You can set the JDK boot directory with the JDK_BOOT_DIR environment variable + +REPOSITORY=breadbin/openjdk-jdk8u + +# Escape code +esc=`echo -en "\033"` + +# Set colors +error="${esc}[0;31m" +good="${esc}[0;32m" +info="${esc}[0;33m" +git="${esc}[0;34m" +normal=`echo -en "${esc}[m\017"` + +source sbin/signalhandler.sh + +while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do + opt="$1"; + shift; + case "$opt" in + "--" ) break 2;; + + "--source" | "-s" ) + WORKING_DIR="$1"; shift;; + + "--ssh" | "-S" ) + USE_SSH=true; shift;; + + "--destination" | "-d" ) + TARGET_DIR="$1"; shift;; + + "--repository" | "-r" ) + REPOSITORY="$1"; shift;; + + "--branch" | "-b" ) + BRANCH="$1"; shift;; + + "--keep" | "-k" ) + KEEP=true; shift;; + + "--jtreg" | "-j" ) + JTREG=true; shift;; + + *) echo >&2 "${error}Invalid option: ${opt}${normal}"; man ./makejdk.1; exit 1;; + esac +done + +# Both a working directory and a target directory provided +if [ ! -z "${WORKING_DIR}" ] && [ ! -z "${TARGET_DIR}" ] ; then + # This uses sbin/build.sh directly + echo "${info} Not using Docker, working area will be ${WORKING_DIR}, target for the JDK will be ${TARGET_DIR} ${normal}" +fi + +# No working directory and no target directory provided +if [ -z "${WORKING_DIR}" ] && [ -z "${TARGET_DIR}" ] ; then + echo "${info}No parameters provided, using Docker ${normal}" + USE_DOCKER=true +elif [ ! -z "${TARGET_DIR}" ] && [ -z "${WORKING_DIR}" ] ; then + # Target dir is defined but the working dir isn't + # Calls sbin/build.sh inside of Docker followed by a docker cp command + echo "${info}Using Docker, target directory for the tgz on the host: ${TARGET_DIR}" + USE_DOCKER=true +fi + +echo ${info} +if [ "${KEEP}" == true ] ; then + echo "We'll keep the built Docker container if you're using Docker" +else + echo "We'll remove the built Docker container if you're using Docker" +fi +echo $normal + +if [ -z "${BRANCH}" ] ; then + echo "${info}BRANCH is undefined so checking out dev${normal}" + BRANCH="dev" +fi + +if [ -z "${WORKING_DIR}" ] ; then + echo "${info}WORKING_DIR is undefined so setting to $PWD${normal}" + WORKING_DIR=$PWD +else + echo "${info}Working dir is $WORKING_DIR${normal}" +fi + +echo $info +if [ -z "${TARGET_DIR}" ] ; then + echo "${info}TARGET_DIR is undefined so setting to $PWD" + TARGET_DIR=$PWD + # Only makes a difference if we're in Docker + echo "If you're using Docker The build artifact will not be copied to the host" +else + echo "${info}Target directory is $TARGET_DIR${normal}" + COPY_TO_HOST=true + echo "If you're using Docker we'll copy the build artifact to the host" +fi + +echo $git +if [ -d "$WORKING_DIR"/openjdk/.git ] && [ $REPOSITORY == "breadbin/openjdk-jdk8u" ] ; then + # It does exist and it's a repo other than the breadbin one + cd $WORKING_DIR/openjdk + echo "${info}Will reset the repository at $PWD in 10 seconds...${git}" + sleep 10 + echo "${git}Pulling latest changes from git repo" + git fetch --all + git reset --hard origin/$BRANCH + echo $normal + cd $WORKING_DIR +elif [ ! -d "${WORKING_DIR}"/openjdk/.git ] ; then + # If it doesn't exixt, clone it + echo "${info}Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" + if [[ "${USE_SSH}" == true ]] ; then + echo "git clone -b ${BRANCH} git@github.com:${REPOSITORY}.git $WORKING_DIR/openjdk" + git clone -b ${BRANCH} git@github.com:${REPOSITORY}.git $WORKING_DIR/openjdk + else + echo "git clone -b ${BRANCH} https://github.com/${REPOSITORY}.git $WORKING_DIR/openjdk" + git clone -b ${BRANCH} https://github.com/${REPOSITORY}.git $WORKING_DIR/openjdk + fi +fi +echo $normal + +if [ "${USE_DOCKER}" ] ; then + PS_DOCKER=`ps -ef | grep "docker" | wc -l` + + if [ -z `which docker` ] || [ ${PS_DOCKER} -lt 2 ]; then + echo "${error}Error, please install docker and ensure that it is in your path and running!${normal}" + exit + fi + + echo "${info}Using Docker to build the JDK${normal}" + + CONTAINER=openjdk_container + + # Copy our script for usage inside of the container + rm docker/jdk8u/x86_64/ubuntu/build.sh + cp sbin/build.sh docker/jdk8u/x86_64/ubuntu 2>/dev/null + + rm docker/jdk8u/x86_64/ubuntu/jtreg.sh + cp sbin/jtreg.sh docker/jdk8u/x86_64/ubuntu 2>/dev/null + # Keep is undefined so we'll kill the docker image + + if [[ $KEEP == true ]] ; then + if [ `docker ps -a | grep openjdk_container | wc -l` == 0 ]; then + echo "${info}No docker container found so creating one${normal}" + docker build -t $CONTAINER docker/jdk8u/x86_64/ubuntu + fi + else + echo "${info}Building as you've not specified -k or --keep" + echo $good + docker ps -a | awk '{ print $1,$2 }' | grep $CONTAINER | awk '{print $1 }' | xargs -I {} docker rm -f {} + docker build -t $CONTAINER docker/jdk8u/x86_64/ubuntu + echo $normal + fi + + docker run --privileged -t -v $WORKING_DIR/openjdk:/openjdk/jdk8u/openjdk --entrypoint build.sh $CONTAINER + + if [[ ! -z $JTREG ]]; then + docker run --privileged -t -v $WORKING_DIR/openjdk:/openjdk/jdk8u/openjdk --entrypoint jtreg.sh $CONTAINER + fi + + CONTAINER_ID=$(docker ps -a | awk '{ print $1,$2 }' | grep openjdk_container | awk '{print $1 }'| head -1) + + if [[ ${COPY_TO_HOST} == true ]] ; then + echo "Copying to the host with docker cp $id:/openjdk/jdk8u/OpenJDK.tar.gz $TARGET_DIR" + docker cp $CONTAINER_ID:/openjdk/jdk8u/OpenJDK.tar.gz $TARGET_DIR + fi + + if [[ ${JTREG} == true ]] ; then + echo "Copying jtreg reports from docker" + docker cp $CONTAINER_ID:/openjdk/jdk8u/jtreport.zip $TARGET_DIR + docker cp $CONTAINER_ID:/openjdk/jdk8u/jtwork.zip $TARGET_DIR + fi + + # Didn't specify to keep + if [[ -z ${KEEP} ]] ; then + docker ps -a | awk '{ print $1,$2 }' | grep $CONTAINER | awk '{print $1 }' | xargs -I {} docker rm {} + fi + +else + echo "Calling sbin/build.sh $WORKING_DIR $TARGET_DIR" + ./sbin/build.sh $WORKING_DIR $TARGET_DIR +fi diff --git a/sbin/build.sh b/sbin/build.sh new file mode 100755 index 000000000..2f4686ce4 --- /dev/null +++ b/sbin/build.sh @@ -0,0 +1,217 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# Script to download any additional packages for building OpenJDK +# before calling ./configure (using JDK 7 as the base) + +WORKING_DIR=$1 +TARGET_DIR=$2 + +# Escape code +esc=`echo -en "\033"` + +# Set colors +error="${esc}[0;31m" +good="${esc}[0;32m" +info="${esc}[0;33m" +git="${esc}[0;34m" +normal=`echo -en "${esc}[m\017"` + + +# If on docker + +if [[ -f /.dockerenv ]] ; then + echo "Detected we're in docker" + WORKING_DIR=/openjdk/jdk8u + TARGET_DIR=$WORKING_DIR +fi + +# E.g. /openjdk/jdk8u if you're building in a Docker container +# otherwise ensure it's a writable area e.g. /home/youruser/myopenjdkarea + +if [ -z "$WORKING_DIR" ] || [ -z "$TARGET_DIR" ] ; then + echo "build.sh is called by makejdk.sh and requires two parameters" + echo "Are you sure you want to call it directly?" + echo "Usage: bash ./build.sh " + echo "Note that you must have the OpenJDK source before using this script!" + echo "This script will try to move ./openjdk to the source directory for you, " + echo "and this will be your working area where all required files will be downloaded to." + echo "You can override the JDK boot directory by setting the environment variable JDK_BOOT_DIR" + exit; +fi + +echo "Making the working directory to store source files and extensions: $WORKING_DIR" + +mkdir -p $WORKING_DIR + +cd $WORKING_DIR + +############################################ + +echo "Downloading required dependencies..." + +# ALSA first for sound + +echo "Checking for ALSA" + +FOUND_ALSA=$(find "$WORKING_DIR" -name "alsa-lib-1.0.27.2") + +if [[ ! -z $FOUND_ALSA ]] ; then + echo "Skipping ALSA download" +else + wget -nc ftp://ftp.alsa-project.org/pub/lib/alsa-lib-1.0.27.2.tar.bz2 + tar xvf alsa-lib-1.0.27.2.tar.bz2 + rm alsa-lib-1.0.27.2.tar.bz2 +fi + +echo "Checking for freetype" + +FOUND_FREETYPE=$(find "$WORKING_DIR/installedfreetype/lib" -name "libfreetype.so.6.5.0") + +if [[ ! -z $FOUND_FREETYPE ]] ; then + echo "Skipping FreeType download" +else + # Then FreeType for fonts: make it and use + wget -nc http://download.savannah.gnu.org/releases/freetype/freetype-2.4.0.tar.gz + tar xvf freetype-2.4.0.tar.gz + rm freetype-2.4.0.tar.gz + + cd freetype-2.4.0 + + # We get the files we need at $WORKING_DIR/installedfreetype + + ./configure --prefix=$WORKING_DIR/installedfreetype && make all && make install + + if [ $? -ne 0 ]; then + echo "${error}Failed to configure and build libfreetype, exiting" + exit; + else + echo "${good}Built the JDK!" + fi + echo $normal +fi + +########################################### + +cd $WORKING_DIR + +echo "Retrieving cacerts file" + +# Ensure it's the latest we pull in +rm -rf $WORKING_DIR/cacerts_area + +git clone https://github.com/breadbin/build.git cacerts_area +echo "cacerts should be here..." +file $WORKING_DIR/cacerts_area/security/cacerts + +if [ $? -ne 0 ]; then + echo "Failed to retrieve the cacerts file, exiting..." + exit; +fi + +########################################### + +echo "Building up the configure command..." + +if [ -z "$JDK_BOOT_DIR" ] ; then + echo "JDK_BOOT_DIR is $JDK_BOOT_DIR" + JDK_BOOT_DIR=/usr/lib/java-1.7.0 +else + echo "Overriding JDK_BOOT_DIR, set to $JDK_BOOT_DIR" +fi + +echo "Boot dir set to $JDK_BOOT_DIR" + +CONFIGURE_CMD=" --with-boot-jdk=$JDK_BOOT_DIR" + +CONFIGURE_CMD="$CONFIGURE_CMD --with-cacerts-file=$WORKING_DIR/cacerts_area/security/cacerts" +CONFIGURE_CMD="$CONFIGURE_CMD --with-alsa=$WORKING_DIR/alsa-lib-1.0.27.2" +CONFIGURE_CMD="$CONFIGURE_CMD --with-freetype=$WORKING_DIR/installedfreetype" + +# These will have been installed by the package manager (see our Dockerfile) +CONFIGURE_CMD="$CONFIGURE_CMD --with-x=/usr/include/X11" + +# We don't want any extra debug symbols - ensure it's set to release, +# other options include fastdebug and slowdebug +CONFIGURE_CMD="$CONFIGURE_CMD --with-debug-level=release" + +#CONFIGURE_CMD="$CONFIGURE_CMD --with-adds-and-overrides=$WORKING_DIR/openjdk/addsandoverrides" + +########################################### + +# Make sure we're in the source directory for OpenJDK now + +cd $WORKING_DIR/openjdk + +echo "Should have the source, I'm at $PWD" + +CONFIGURED_JDK_ALREADY=$(find -name "config.status") + +if [[ ! -z $CONFIGURED_JDK_ALREADY ]] ; then + echo "Not reconfiguring due to the presence of config.status in $WORKING_DIR" +else + echo "Running ./configure with $CONFIGURE_CMD" + bash ./configure $CONFIGURE_CMD + if [ $? -ne 0 ]; then + echo ${fail} + echo "Failed to configure the JDK, exiting" + echo "Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR" + echo "For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64" + echo "Current JDK_BOOT_DIR value: $JDK_BOOT_DIR" + exit; + else + echo "${good}Configured the JDK" + fi + echo ${normal} +fi + +########################################### + +echo "Building the JDK: calling make images" + +make images + +if [ $? -ne 0 ]; then + echo "${fail}Failed to make the JDK, exiting" + exit; +else + echo "${good}Built the JDK!" +fi +echo ${normal} + +########################################### + +echo "Removing unneccessary files now..." + +rm -rf cacerts_area + +cd build/linux*/images + +# Remove files we don't need +rm -rf j2sdk-image/demo/applets +rm -rf j2sdk-image/demo/jfc/Font2DTest +rm -rf j2sdk-image/demo/jfc/SwingApplet +find . -name "*.diz" -type f -delete + +GZIP=-9 tar -cvzf OpenJDK.tar.gz ./j2sdk-image + +mkdir -p $TARGET_DIR + +mv OpenJDK.tar.gz $TARGET_DIR + +echo "${good}Your final tar.gz is here at $PWD${normal}" +cd $TARGET_DIR +ls +echo "All done!" diff --git a/sbin/jtreg.sh b/sbin/jtreg.sh new file mode 100755 index 000000000..628d323ac --- /dev/null +++ b/sbin/jtreg.sh @@ -0,0 +1,70 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +WORKING_DIR=$1 + +if [[ -f /.dockerenv ]] ; then + echo "Detected we're in docker" + WORKING_DIR=/openjdk/jdk8u/openjdk + # Keep as a variable for potential use later + # if we wish to copy the results to the host + IN_DOCKER=true +fi + +echo "Running jtreg" + +cd $WORKING_DIR/jdk/test + +# This is the JDK we'll test +export PRODUCT_HOME=$WORKING_DIR/build/linux-x86_64-normal-server-release/images/j2sdk-image + +echo $PRODUCT_HOME +ls $PRODUCT_HOME + +# Download then add jtreg to our path +wget https://adopt-openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/jtreg-4.2.0-tip.tar.gz + +if [ $? -ne 0 ]; then + echo "Failed to retrieve the jtreg binary, exiting" + exit +fi + +tar xvf *.tar.gz + +mv jtreg* $WORKING_DIR +ls $WORKING_DIR/jtreg* + +export PATH=$WORKING_DIR/jtreg/bin:$PATH + +export JT_HOME=$WORKING_DIR/jtreg + +# Clean up after ourselves by removing jtreg tgz +rm -f jtreg*.tar.gz + +echo "Running jtreg with: jtreg -conc:2 -a -verbose:fail -jdk:$PRODUCT_HOME ./ || true" + +jtreg -agentvm -conc:2 -a -verbose:fail -jdk:$PRODUCT_HOME ./ || true + +if [ $? -ne 0 ]; then + echo "Failed to run jtreg, exiting" + exit +fi + +echo "Archiving your jtreg results" + +zip -r jtreport.zip ./JTreport +zip -r jtwork.zip ./JTwork + +mv *.zip $WORKING_DIR/ diff --git a/sbin/signalhandler.sh b/sbin/signalhandler.sh new file mode 100755 index 000000000..1ce76143e --- /dev/null +++ b/sbin/signalhandler.sh @@ -0,0 +1,10 @@ +exit_script() { + if [[ -z ${KEEP} ]] ; then + docker ps -a | awk '{ print $1,$2 }' | grep $CONTAINER | awk '{print $1 }' | xargs -I {} docker rm -f {} + fi + echo "${error}Process exited${normal}" + trap - SIGINT SIGTERM # clear the trap + kill -- -$$ # Sends SIGTERM to child/sub processes +} + +trap exit_script SIGINT SIGTERM diff --git a/security/GenerateCertsFile.sh b/security/GenerateCertsFile.sh new file mode 100644 index 000000000..40611070c --- /dev/null +++ b/security/GenerateCertsFile.sh @@ -0,0 +1,26 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# Delete any existing file +rm -f certdata.txt* + +# Grab our certificate information to use from the Mozilla site +wget https://hg.mozilla.org/mozilla-central/raw-file/tip/security/nss/lib/ckfw/builtins/certdata.txt . + +# Call the script to generate our crt file +perl mk-ca-bundle.pl -n > ca-bundle.crt + +# Finally use the keyutil application +java -jar keyutil-0.4.0.jar --import --new-keystore trustStore.jks --password changeit --force-new-overwrite --import-pem-file ca-bundle.crt diff --git a/security/README.md b/security/README.md new file mode 100644 index 000000000..ae50633d3 --- /dev/null +++ b/security/README.md @@ -0,0 +1,13 @@ +# Security files for OpenJDK + +### This repository contains the cacerts file used with OpenJDK + +#### Steps we use to create the cacerts file + +1. Download the following Perl script: https://raw.githubusercontent.com/curl/curl/master/lib/mk-ca-bundle.pl + +2. Download the following Java application: https://github.com/use-sparingly/keyutil/releases/download/0.4.0/keyutil-0.4.0.jar (source available at https://github.com/use-sparingly/keyutil) + +3. Run the provided `GenerateCertsFile.sh` script with: `bash ./GenerateCertsFile.sh` - this will use the above files assuming they're located in the same directory as the script + +4. Use the cacerts provided: it must be in the `jdk/jre/lib/security` folder diff --git a/security/cacerts b/security/cacerts new file mode 100644 index 000000000..b2171b53e Binary files /dev/null and b/security/cacerts differ