-
Notifications
You must be signed in to change notification settings - Fork 9
/
Copy pathtesting-process.html
142 lines (129 loc) · 11.8 KB
/
testing-process.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="user-scalable=no, initial-scale=1, maximum-scale=1, minimum-scale=1, width=device-width">
<title>rtc.io</title>
<link rel="stylesheet" type="text/css" href="css/reset.css">
<link rel="stylesheet" type="text/css" href="css/main.css">
<!-- responsive -->
<link rel="stylesheet" media="screen and (max-width: 960px)" href="css/tablet.css">
<link rel="stylesheet" media="screen and (max-width: 710px)" href="css/phone.css">
<link rel="stylesheet" type="text/css" href="fonts/source-sans/stylesheet.css">
<link rel="stylesheet" type="text/css" href="css/code.css">
</head>
<body>
<a class="scroll-point pt-top" name="top"></a>
<header>
<a href="https://github.com/rtc-io"><img class="fork" src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png" alt="Fork me on GitHub"></a>
<a class="scroll-point pt-top" name="top"></a>
<div class="site">
<div class="mascot">
<img src="images/artsio.png">
</div>
<div class="logo" data-subtext="OpenSource WebRTC">
<a href="index.html">rtc.io</a>
</div>
<nav>
<ul>
<li><a href="index.html">About</a></li>
<li><a href="tutorials.html">Tutorials</a></li>
<li><a href="demos.html">Demos</a></li>
<li><a href="modules.html">Modules</a></li>
</ul>
</nav>
</div>
<div class="shadow"></div>
</header>
<div class="main" role="content"><h1 id="testing-webrtc">Testing WebRTC</h1>
<p>This is a set of scripts designed to help you get up and running testing WebRTC applications on <a href="http://travis-ci.org">travis</a>. The scripts themselves are designed to be fetched during a travis <code>before_install</code> process and used to prepare your environment.</p>
<p><a href="https://travis-ci.org/rtc-io/webrtc-testing-on-travis"><img src="https://travis-ci.org/rtc-io/webrtc-testing-on-travis.svg?branch=master" alt="Build Status"></a></p>
<h2 id="usage">Usage</h2>
<p>First, create a <code>.travis.yml</code> folder in your project that looks similar to the followng:</p>
<div class="highlight"><pre><span class="l-Scalar-Plain">language</span><span class="p-Indicator">:</span> <span class="l-Scalar-Plain">node_js</span>
<span class="l-Scalar-Plain">node_js</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">0.10</span>
<span class="l-Scalar-Plain">env</span><span class="p-Indicator">:</span>
<span class="l-Scalar-Plain">matrix</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=chrome BVER=stable</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=chrome BVER=beta</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=chrome BVER=unstable</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=firefox BVER=stable</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=firefox BVER=beta</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">BROWSER=firefox BVER=nightly</span>
<span class="l-Scalar-Plain">matrix</span><span class="p-Indicator">:</span>
<span class="l-Scalar-Plain">fast_finish</span><span class="p-Indicator">:</span> <span class="l-Scalar-Plain">true</span>
<span class="l-Scalar-Plain">allowed_failures</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">env</span><span class="p-Indicator">:</span> <span class="l-Scalar-Plain">BROWSER=chrome BVER=unstable</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">env</span><span class="p-Indicator">:</span> <span class="l-Scalar-Plain">BROWSER=firefox BVER=nightly</span>
<span class="l-Scalar-Plain">before_install</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">./setup.sh</span>
<span class="l-Scalar-Plain">before_script</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">export DISPLAY=:99.0</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">sh -e /etc/init.d/xvfb start</span>
<span class="l-Scalar-Plain">after_failure</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="l-Scalar-Plain">for file in *.log; do echo $file; echo "======================"; cat $file; done || true</span>
</pre></div>
<p>The most interesting part of the configuration file above is definitely the <a href="http://docs.travis-ci.com/user/build-configuration/#before_install"><code>before_install</code></a> section which defines a number of commands that will be executed in the TRAVIS environment prior to running the language appropriate <code>install</code> command.</p>
<p>The commands above are used to pull down the contents of this repository into a <code>.travis</code> within the travis build environment. Once this is completed, the <code>./travis/setup.sh</code> command is run which will knows how to provision a browser based on a <code>BROWSER</code> (specifying the browser name) and <code>BVER</code> which defines the version to test. This script is an only very slightly modified version of the <a href="https://github.com/web-animations/web-animations-js/blob/master/.travis-setup.sh">.travis-setup.sh</a> script written by <a href="https://github.com/mithro">@mithro</a>.</p>
<p>This is where the <a href="http://docs.travis-ci.com/user/build-configuration/#The-Build-Matrix"><code>matrix</code></a> section comes into play. In the <code>.travis.yml</code> file above, I've used the matrix to tell travis that I want to a few variants of both chrome and firefox. It should be noted that the original script does this is a slightly different way, but I wanted to use the <code>BROWSER</code> environment variable later when running my <code>npm test</code> command.</p>
<h3 id="testing-rtc-io-https-github-com-rtc-io-">Testing <a href="https://github.com/rtc-io">rtc.io</a></h3>
<p>I'll now talk about how the scripts in this repo are used to test particular modules within the <a href="https://github.com/rtc-io">rtc.io suite</a>. If you'd prefer to just dig around in a repo, then feel free to take a look at <a href="https://github.com/rtc-io/rtc-quickconnect">rtc-quickconnect</a>.</p>
<p>Up until now, everything here is useful to anyone wanting to do headless browser testing using travis. We are about to take a more opinionated turn into the world of node, and specifically <a href="https://github.com/substack/node-browserify">browserify</a>, <a href="https://github.com/substack/testling">testling</a> and <a href="https://github.com/substack/tape">tape</a>. I won't cover <em>why</em> I've chosed to use this toolchain here, but simply cover how to use these tools to achieve a level of testing zen seldom discovered on the Internet.</p>
<p>Let's begin our journey by taking a look at the relevant bits of a <code>package.json</code> file from the <code>rtc-quickconnect</code> repo:</p>
<div class="highlight"><pre><span class="p">{</span>
<span class="nt">"scripts"</span><span class="p">:</span> <span class="p">{</span>
<span class="nt">"test"</span><span class="p">:</span> <span class="s2">"testling -x ./.travis/start-$BROWSER.sh"</span><span class="p">,</span>
<span class="p">},</span>
<span class="nt">"devDependencies"</span><span class="p">:</span> <span class="p">{</span>
<span class="nt">"tape"</span><span class="p">:</span> <span class="s2">"^2"</span><span class="p">,</span>
<span class="nt">"testling"</span><span class="p">:</span> <span class="s2">"^1"</span>
<span class="p">},</span>
<span class="nt">"testling"</span><span class="p">:</span> <span class="p">{</span>
<span class="nt">"files"</span><span class="p">:</span> <span class="s2">"test/all.js"</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>I'm obvsiouly leaving out a whole pile of stuff here, but what you see above are the bits that are required to "make it go". You can see in the <code>scripts.test</code> section we are defining our test command as:</p>
<pre><code>testling -x ./.travis/start-$BROWSER.sh</code></pre>
<p>Executing this command in combination with our previously defined matrix, will execute either <code>testling -x ./.travis/start-chrome.sh</code> or <code>testling -x ./.travis/start-firefox.sh</code> depending on the environment. At this stage, the <a href="https://github.com/DamonOehlman/webrtc-testing-on-travis/blob/master/start-firefox.sh"><code>start-firefox.sh</code></a> file doesn't do anything clever at all, but the <a href="https://github.com/DamonOehlman/webrtc-testing-on-travis/blob/master/start-chrome.sh"><code>start-chrome.sh</code></a> starts chrome with a number of <a href="http://peter.sh/experiments/chromium-command-line-switches/">command-line switches</a> enabled that make testing WebRTC a whole lot easier.</p>
<p>And that's it (more or less). Testling does all of the hard work by creating a bridge between the running browser instance and itself to monitor test success, reporting failure through an error code that travis will pick up as build failure.</p>
<h2 id="testing-locally">Testing Locally</h2>
<p>For testing locally, I have a cloned version of this repo checked out on my local machine and I then create a symlink from it to a <code>.travis</code> directory in each project that uses this approach. It's then a simple matter of running a command similar to the following to have magic happen locally:</p>
<pre><code>BROWSER=chrome npm start</code></pre>
<h2 id="understanding-fake-media-devices">Understanding Fake Media Devices</h2>
<p>Both chrome and firefox provide approaches for simulating media capture with "fake" media devices. It is important to note, however, that the approach outlined here will only provide fake media streams on chrome. When testing rtc.io modules we typically make use of <a href="https://github.com/rtc-io/rtc-media">rtc-media</a> as we have enabled use of firefox's "fake" capture constraint when we detect that we are running in a testling run testing environment:</p>
<p><a href="https://github.com/rtc-io/rtc-media/commit/7b6cbce0efb2010e5218dfef0ea53b514385a395">https://github.com/rtc-io/rtc-media/commit/7b6cbce0efb2010e5218dfef0ea53b514385a395</a></p>
<h2 id="prior-art">Prior Art</h2>
<p>None of this would have been possible without the docs and code listed below:</p>
<ul>
<li><p><a href="http://lynckia.com/licode/travis-webrtc.html">Automated testing in Travis CI for WebRTC</a></p>
</li>
<li><p><code>.travis-setup.sh</code> from the <a href="https://github.com/web-animations/web-animations-js">web-animations-js</a> project</p>
</li>
<li><p>Travis docs on <a href="http://docs.travis-ci.com/user/gui-and-headless-browsers/">GUI and Headless Browsers</a></p>
</li>
</ul>
</div>
<footer>
<p>
<a href="http://nicta.com.au">
<img src="images/nicta-logo.gif" alt="NICTA logo">
</a>© NICTA 2013 - 2014
</p>
<p class="license">Project source code is licensed under the <a href="https://github.com/rtc-io/rtc/blob/master/LICENSE">Apache 2.0</a>.</p>
<a class="closing" href="#top"></a>
</footer>
</body>
<script src="js/app.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-26567546-2', 'rtc.io');
ga('send', 'pageview');
</script>
</html>