rss.xml

<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Engineering at MindLink]]></title><description><![CDATA[Insights and musings from the MindLink software engineers]]></description><link>https://engineering.mindlinksoft.com</link><generator>RSS for Node</generator><lastBuildDate>Tue, 25 Jun 2019 13:08:50 GMT</lastBuildDate><item><title><![CDATA[Rebuilding a performant emoji picker in React with hooks]]></title><description><![CDATA[Before I start, we didn’t really want to have to do this. There are a couple of really good projects out there that offer an emoji picker…]]></description><link>https://engineering.mindlinksoft.comrebuilding-a-performant-emoji-picker-in-react-with-hooks</link><guid isPermaLink="false">https://engineering.mindlinksoft.comrebuilding-a-performant-emoji-picker-in-react-with-hooks</guid><pubDate>Thu, 23 May 2019 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Before I start, we didn’t really want to have to do this. There are a couple of really good projects out there that offer an emoji picker, they just didn’t &lt;em&gt;quite&lt;/em&gt; work how we needed them to.&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;It is surprisingly easy to build out a complex component with React and hooks. However, you have to be careful about accessibility and profile the performance routinely!&lt;/p&gt;
&lt;p&gt;In a couple of days we managed to put together a performant and feature-rich emoji picker UX that supports down to IE 10 using &lt;a href=&quot;https://reactjs.org/&quot;&gt;React&lt;/a&gt;, &lt;a href=&quot;https://reactjs.org/docs/hooks-intro.html&quot;&gt;hooks&lt;/a&gt;, &lt;a href=&quot;https://github.com/bvaughn/react-window&quot;&gt;react-window&lt;/a&gt; and an emoji data source!&lt;/p&gt;
&lt;h2&gt;Background&lt;/h2&gt;
&lt;p&gt;As you perhaps already know one of MindLink’s core products is a single page application for accessing chat systems.&lt;/p&gt;
&lt;p&gt;We support legacy chat systems from a time where emojis were not a mainstream expectation from users. We bring up to date emoji support to those chat systems via our server-side abstraction layer, which allows us to build modern features on top of existing Enterprise chat systems.&lt;/p&gt;
&lt;p&gt;To bring emoji support to our clients we originally we went with &lt;a href=&quot;https://github.com/missive/emoji-mart&quot;&gt;emoji-mart&lt;/a&gt; by missive. It’s sleek and stylish and has a lot of customisation options. However, we encountered some issues with performance:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;On IE10/11 the showing of the picker is slooww as it struggles to render all 2000+ emojis into the DOM&lt;/li&gt;
&lt;li&gt;Even in modern browsers there’s a noticeable delay&lt;/li&gt;
&lt;li&gt;The use of a large sprite sheet results in browsers having to reset their graphics context so switching back to a tab with the picker open results in a brief white screen&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Additionally, we really wanted to customise it in other ways:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Inject the search term as a prop to make it work with our message input&lt;/li&gt;
&lt;li&gt;To forward on keyboard events to move the selected emojis/trigger selection&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;What we didn’t need:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Customisable source (we opted for Twitter, we don’t need the overhead of supporting other emoji icons)&lt;/li&gt;
&lt;li&gt;Custom emojis (at least not yet)&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Moving to a virtualized list&lt;/h2&gt;
&lt;p&gt;We knew from performance profiling that the biggest issue is the rendering of the emoji list. 2000+ emojis on screen at once, not a good idea. We need a better way - a virtualised list.&lt;/p&gt;
&lt;p&gt;Simply put, a virtualised list only renders what is visible into the DOM on demand and tracks scroll position to trigger loading different parts of the list. This means the complete list is virtual and we render only a window.&lt;/p&gt;
&lt;p&gt;A well-established solution in the React world is &lt;a href=&quot;https://github.com/bvaughn/react-window&quot;&gt;react-window&lt;/a&gt;. It is a great project that handles most of the pain of managing a virtualised list or grid. There is also the possibility to get some vanilla JS to do this using an &lt;code class=&quot;language-text&quot;&gt;IntersectionObserver&lt;/code&gt; to trigger rendering “pages”.&lt;/p&gt;
&lt;p&gt;We are also fortunate that right now it’s pretty easy to make lightweight components using React hooks.&lt;/p&gt;
&lt;p&gt;Combining both react-window and hooks we sought to recreate the minimal feature set we wanted from emoji-mart:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Category icon buttons allow you to jump to categories and also highlight what category you’re currently viewing&lt;/li&gt;
&lt;li&gt;Inline category headers separating groups of emojis&lt;/li&gt;
&lt;li&gt;A larger preview of a highlighted emoji showing the shortcode, description and ASCII art equivalents&lt;/li&gt;
&lt;li&gt;A skin variation picker to select a skin tone&lt;/li&gt;
&lt;li&gt;Search to filter the emojis&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;A look at the finished picker&lt;/h2&gt;
&lt;p&gt;A first draft of this we threw together in about half a day - leveraging the emoji-mart data source so that we didn’t have to worry about compiling that. That is the power of leveraging hooks and react-window.&lt;/p&gt;
&lt;p&gt;We took that first draft, liked it and built it out over another couple of days to get it to where we wanted it for production.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/picker-tone-32f2c7748747621aabad37edd831bff1-0904b.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 398px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 103.26633165829145%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAENElEQVQ4y4WVSY/bRhCF9bvzC3I0kGtOOeTmg2EgcOxk7Alsy/Es2kWJEhdxFTeRlLhImhl/qaY8QW4RVOrWY3d19auqx15WZJjpmvJQEvgBYRiy3+9xXZfD4YDv+wRB0GGe53VYFEWkaUqe5ziO02Fqjdrbi/KQvn9NXu1Yr9aYlklZlpim2S3UNA1d19ntdtzd3XWYYRjdgQobj8f/BrDZbOilSUomp1VVJXboxsNh321UdsGry1xhsrl6nv/H6rruxl4cR2y8iMcnePp2Mfl2o/zKXB5cEJ4/j9+Nb90inp4uz47HI71DfabWf6WO52z134lDuc7kNZn3hUk/wZpHLG5cTK1Af/cby6s/ift3RF/vcKKYhdCUpBnWxu2i7u2rlpP3mbYIaLMZReZTpTOavU1i5pRhTugdSJIWb6rjzA0ay6fZBGRVI85K8lpG8ZPEMb1a+HGjnHxfMdPFiRCtjX22dsK15TEMEw7LAamxpo4+Uod98uCW3Llm59wSbd7TjO6p/3hJdTrRU4Rv072UTY0TZuzEsWNnZOGOftgwy87k2pR4ZVEmC4poKaNBGWlsYxcnWAldDoU1pG6Fw0rCLZYvZcGS2n5DE41pgz7HwqQwr2R+S+n3qZI7sqWUz3rJzjJIlxp716bUNSLXwh5+oT2dFYcNp1j+7IW76J4m31B4A9qdjb9dc8hMqngiuE5h6JS2TmYsSLQRhWOy02fkkU+0GtOoCNu2ZTyaEMUp809/sXU26Ldf8dc6q8VCCtbDGA1w9CXGdIg+uOnqNS9yEmfNdj1lKx2y1ma0qmwacWjNBmRJhD+9Ee5c3PEtsb0msFakoUe0log8C0cc2uN7iiQk8zdSWhaxOScJZI9E3BwlKXXT4K6EbMnuZiXXylJsUyeWErD0OYG0lKbfEsr19NVIsAllXkgpOZRFwVZ6uEwTtovJ9yuLw9FyRZTljCZTwihhNR8SBh7L4T2uYWJbU7Kty2o6YTPRiCUiazkn8j3W86nQFaAth5crV01LKBHuZFGR7+TUnCzaSqQJbpLhxXJ6LNF7IcGuRJfSCqX/B4bHLo3xbJNMbmfMJULx1Tsez9z8fY0t2YusdcelJyWhuNPWEp3I00LUZikJ0tyAe93EkDZ7P5jjSzJmkwlBkvDhU//Sy4VoWitkVqIWyg5KWaQ2Dy1UzbGb18cHmtOj1NmDiMg3zmJKFlRTqKQqlVH8Kdnr7b/L0bM8VVKXVRHTuq9pko2Y9PhWI7dnxPqQ2Zf3TD9/YPDhbefkWeqUn06+FNBIYi5Wy6IzbenxOP+Bh2AkZvBoX7EbvsX7+IqrX17w5ucXvPrpR86ieUoHn/eriKVTTtRCZl1X3cOLyTWb8wVv1PzY9amKSLXX8XzmdH7onDzvUQ3SOWySOW114VGB/28NRxkVX+q9ojCVDPU6UNg/5CUZ8YDCaO8AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Emoji picker with skin tone&quot;
        title=&quot;&quot;
        src=&quot;/static/picker-tone-32f2c7748747621aabad37edd831bff1-0904b.png&quot;
        srcset=&quot;/static/picker-tone-32f2c7748747621aabad37edd831bff1-7c835.png 148w,
/static/picker-tone-32f2c7748747621aabad37edd831bff1-11b18.png 295w,
/static/picker-tone-32f2c7748747621aabad37edd831bff1-0904b.png 398w&quot;
        sizes=&quot;(max-width: 398px) 100vw, 398px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/picker-preview-fa3fdceab79f58afba33fe9edcf15e3f-5d328.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 399px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 102.75689223057644%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAIAAADJt1n/AAAACXBIWXMAAA7DAAAOwwHHb6hkAAADU0lEQVQ4y41US2/cNhDWv+n/6z/oucf20EsuRR8w0iRFnLZ2W7tx4sSOY/ix2fVm1ytKlLR6kxTJGVI24nVnrboweuoHaDRPzlDkpwAQKl0AWill27aISBIAuq4jxTlHfnKSqZT6N0pOMoPOqjf5rsA2jvhsNqPsMAyNMYyx0WhEecfHx1SZZRl5qPj8/JzM5XJJaQGtkVXpsDCBFCEESfIPCrUaokO3pmmGNEKgpEp5enNzc3t7u7rD7f8DlQTWWq01zUMDD3JQCHAHvMdDnXLWnWkMfociL6IoChlL02wdRux73/f9ffYDieido5YBfRtrDW0D6EXoJJjOU/Dqum7qsiyonkw/PJ6Wo9g/VuCubuX0W5u/79gLzF6n4V4W7efsYHK2e7j9zV+Pv6jCPb34CapTNvlZFx/q0SMn5mr0te+vAxpBiwR0Y2QBpq6kaJSsW9mJ+vLj+Xj0zqjaiAyM6NoUjFRt7LADxah1AP0KJhuYndj4RZ/u8/FWNN+5ersFe89dc2Anz1x9jOyJa0Yu2sByasffY8P02SPvrwOgzbfM6Rpkgm0OKkGVGbk0gju6eQVztnIqdrZBFXotsWLOKFfOnesD8CvLNqA+AfYblod28qudbGF+gMlLWxwq9gtUJxA/9+0Yomcophj9iF1qyeOpGHsUMzSlLy/6loGIreBecq9S1EsnQ9QFyoUztZdzb5tezhwoLy/Xe0a/MovH9DFV9LvJ3zX8VZe+hvyNTXexOu74n7Y40WwT24kIN7vyopn8QLuIR985f0Wdr1w7djqzzUeQcdVyKyOnIhCX2GVdG7oug2bqdN5UM6tKlZ95aKE6czQ2+k+GbUI9svE+5KcuO7LFRKdHlh9gfWrSl7YaW75d06LZjpcLOhSvU4yfrs/Z4CdbvAcV6+gM8rnNLmwxN8up5WNQl7b+gGIBJSUkXXZgRazTV1Zmmu9YwIAujTz8XITbSaEWLGY8jZOMZ3kYpzFfLsKEJ8uI53Gc0Ix0rvfyhngSmOSPt199tjz6Mkry6cW4yJeCGNw2oiXK1FVZEJHpTdz2bk2MgRUOYX23iQ9dPtKq4kmaJBzXZOr9PQa9v4MbCPEAgQXor1fo+oGuD0n7HxDz6WcyJBD9SPkbkJuAO7JG2eEAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Emoji picker with preview&quot;
        title=&quot;&quot;
        src=&quot;/static/picker-preview-fa3fdceab79f58afba33fe9edcf15e3f-5d328.png&quot;
        srcset=&quot;/static/picker-preview-fa3fdceab79f58afba33fe9edcf15e3f-be17d.png 148w,
/static/picker-preview-fa3fdceab79f58afba33fe9edcf15e3f-7298c.png 295w,
/static/picker-preview-fa3fdceab79f58afba33fe9edcf15e3f-5d328.png 399w&quot;
        sizes=&quot;(max-width: 399px) 100vw, 399px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;The results are impressive considering the short time frame. The picker loads considerable faster and supports all the same use cases we had. We now have the full benefits of owning that:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;We can style it totally how we like&lt;/li&gt;
&lt;li&gt;We can add extra features specific to our use cases&lt;/li&gt;
&lt;li&gt;We can keep it as up to date as we need it (hello Unicode 12)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We want to take this open-source, but before we do we need to figure out a first-class data source and stop piggy-backing off of emoji-mart so we can control the Unicode support.&lt;/p&gt;
&lt;p&gt;I’ll take a deep-dive into the implementation in a few follow-up posts.&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Automate your memory profiling now]]></title><description><![CDATA[MindLink’s core product is designed to be a long running and stable server brokering between clients and chat systems. Stability is…]]></description><link>https://engineering.mindlinksoft.comautomate-your-memory-profiling-now</link><guid isPermaLink="false">https://engineering.mindlinksoft.comautomate-your-memory-profiling-now</guid><pubDate>Wed, 17 Apr 2019 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;MindLink’s core product is designed to be a long running and stable server brokering between clients and chat systems. Stability is paramount to ensure users have an optimal experience. We have a large number of automated tests, exploratory and planned manual QA processes and we dogfood our products with nightly builds.&lt;/p&gt;
&lt;p&gt;Combined, these strategies have ensured that our products continue to be scalable, reliable and high quality. But we can do better!&lt;/p&gt;
&lt;p&gt;Recently we discovered a memory leak late into the QA and release process and this made it out into customer hands. Not ideal, but these things happen. The leak itself has implications when lots of users log on and log out over time and will eventually cause the process to run out of memory.&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;By leveraging a memory profiler that provides programmatic access to snapshots you can automate memory performance testing to give you high confidence that you haven’t introduced any memory leaks into your code under reproducible scenarios.&lt;/p&gt;
&lt;p&gt;One such profiler for .Net code that’s free (as in beer) is &lt;a href=&quot;https://www.jetbrains.com/dotmemory/unit/&quot;&gt;dotMemory Unit&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;The story&lt;/h2&gt;
&lt;p&gt;The cause of the leak could only effectively be discovered through memory profiling and it was the result of several issues combining to cause the leak, if any one of those issues were alleviated the leak would not have occurred.&lt;/p&gt;
&lt;p&gt;If memory profiling is what it takes to notice these, or at least some manual observation, then there is always the chance that nobody actually checks (as what happened here).&lt;/p&gt;
&lt;p&gt;Can we do better than relying on somebody to remember to check, for every release? And to check all necessary scenarios manually? Not really…&lt;/p&gt;
&lt;h2&gt;The search for a tool&lt;/h2&gt;
&lt;p&gt;What we really want is a way to automate that profiling during the release pipeline and to flag memory leaks automatically.&lt;/p&gt;
&lt;p&gt;We could crudely invoke a process snapshot before and after an operation and then compare the memory, but that isn’t going to be reliable and how do we ensure that each scenario is actually running?&lt;/p&gt;
&lt;p&gt;Ideally we want programmatic access to a profiler so that we can:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;start profiling&lt;/li&gt;
&lt;li&gt;Execute the scenario&lt;/li&gt;
&lt;li&gt;Stop profiling&lt;/li&gt;
&lt;li&gt;Make assertions on the profiling session&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;As luck would have it, such magic exists! We leverage the .net framework and we are therefore fortunate to be able to benefit from the brilliant &lt;a href=&quot;https://www.jetbrains.com/dotmemory/unit/&quot;&gt;dotMemory Unit&lt;/a&gt; project by JetBrains (the makers of &lt;a href=&quot;https://www.jetbrains.com/resharper/&quot;&gt;ReSharper&lt;/a&gt; and &lt;a href=&quot;https://kotlinlang.org/&quot;&gt;Kotlin&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;Unlike its GUI profiling counterpart dotMemory, dotMemory Unit is free! And it lets you programmatically profile and make assertions on the profiling session.&lt;/p&gt;
&lt;p&gt;There are some caveats:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Tests have to be synchronous&lt;/li&gt;
&lt;li&gt;The test runner has to be called via the dotMemory Unit executable.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;That seems reasonable considering the power that it gives us. With dotMemory Unit we can write automated and isolated tests that ensure we don’t have memory leaks.&lt;/p&gt;
&lt;h2&gt;Writing a memory unit test&lt;/h2&gt;
&lt;p&gt;For our issue that snuck under the manual testing radar we knew the specific scenario that showed the issue, all we needed to do was log a user onto a chat system and then log them off again, basically the simplest scenario we could write - so that’s nice!&lt;/p&gt;
&lt;p&gt;Here’s an example &lt;a href=&quot;https://nunit.org/&quot;&gt;NUnit&lt;/a&gt; test:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-csharp&quot;&gt;&lt;code class=&quot;language-csharp&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;Test&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;LoggingOffCleansUpSession&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;LoggingOffCleansUpSessionAsync&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;Wait&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  dotMemory&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;Check&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;memory &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; Assert&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;That&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;memory&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token generic-method function&quot;&gt;GetObjectsOfType&lt;span class=&quot;token punctuation&quot;&gt;&amp;lt;&lt;/span&gt;ISession&lt;span class=&quot;token punctuation&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;ObjectCount&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; Is&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;EqualTo&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;async&lt;/span&gt; Task &lt;span class=&quot;token function&quot;&gt;LoggingOffCleansUpSessionAsync&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;var&lt;/span&gt; session &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;CreateSession&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token keyword&quot;&gt;await&lt;/span&gt; session&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;LogOnAsync&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token keyword&quot;&gt;await&lt;/span&gt; session&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;LogOffAsync&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;What we are doing here is creating a session, logging it on and then logging it off. Since the session is asynchronous and we want a clean test case we use an async method to perform the test steps and wait for that in the actual test to satisfy dotMemory unit’s synchronous constraint.&lt;/p&gt;
&lt;p&gt;Our first use of dotMemory Unit was to create a test over the entire session stack, since the memory leak we encountered was the combination of internal behaviour and how the session was wired together. This high level test will also flag any memory leaks specific to individual units.&lt;/p&gt;
&lt;p&gt;We are aiming to create memory tests for more isolated areas of our code so that we can have better coverage over different scenarios. It’s a work-in-progress, but certainly an improvement on relying on manual testing and something I think everybody should consider when stability and performance is important.&lt;/p&gt;
&lt;p&gt;In a future post I will discuss how we integrated dotMemory Unit tests into our Azure DevOps release pipeline.&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Hybrid Electron application deployment in the enterprise]]></title><description><![CDATA[Enterprises love control, that usually means they’re only going to install your software if there’s an MSI package that they can deploy…]]></description><link>https://engineering.mindlinksoft.comhybrid-electron-application-deployment-in-the-enterprise</link><guid isPermaLink="false">https://engineering.mindlinksoft.comhybrid-electron-application-deployment-in-the-enterprise</guid><pubDate>Thu, 14 Mar 2019 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Enterprises love control, that usually means they’re only going to install your software if there’s an MSI package that they can deploy through group policy.&lt;/p&gt;
&lt;p&gt;This means that if you want a lovely auto-updating client so that you don’t have to maintain backwards compatible server code you’re &lt;em&gt;stuck&lt;/em&gt;. Except there is a way forward!&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;By treating your Electron application as a browser shell and taking control of downloading the latest non-native assets from a target server (the “web app”), you can automatically sync the client version with the server version and do that securely by signing your non-native assets.&lt;/p&gt;
&lt;h2&gt;Background&lt;/h2&gt;
&lt;p&gt;It is inescapable that some form of forwards or backwards compatibility will be necessary, for MindLink we are forced to have a client that is backwards compatible with older servers since our mobile apps are app-store distributed and hence on a faster deployment cadence than the on-premise servers our customers deploy. If there’s a customer that is slow to deploy a new version, we don’t want all our other customers not getting the latest and greatest client.&lt;/p&gt;
&lt;p&gt;This means that we don’t need a backwards compatible server, and that’s brilliant because that keeps code lean and up to date.&lt;/p&gt;
&lt;p&gt;However, when we looked to package up our web app as a desktop application, suddenly we found a backwards compatible client is not enough. In an enterprise deployment often its the other way around, servers are updated first and then clients.&lt;/p&gt;
&lt;p&gt;That means we either need to also maintain a backwards compatible server, or find a way to keep clients in-sync.&lt;/p&gt;
&lt;p&gt;We know first hand the pain of keeping legacy code around for backwards compatibility - the choices available aren’t great when you’re talking about behavioural differences. Thats why we &lt;em&gt;really&lt;/em&gt; want to avoid it in the server.&lt;/p&gt;
&lt;h2&gt;The modern deployment approach&lt;/h2&gt;
&lt;p&gt;Modern client deployment techniques follow an auto-update process. If you’re trying to use a really old WhatsApp then it’ll force you to install the latest update. Mobile apps and associated app stores have instilled this approach into consumers, and the same trend is happening on desktops with most services now in the cloud. Maintaining a cloud service that has to be compatible with many different client versions is an engineering overhead best avoided.&lt;/p&gt;
&lt;p&gt;The classic auto-update approach is to replace the native installation. That’s great if you’re installing into user-space, but in an Enterprise world the app is natively installed and sysadmins don’t want an automatically installed native package that they haven’t got deployment control over.&lt;/p&gt;
&lt;p&gt;You could implement a native bootloader instead, so you pull down a native payload in the bootloader and then launch that. There are security concerns around this approach and it’s possible that the native code execution is blocked by some security policy.&lt;/p&gt;
&lt;p&gt;However there is a middle ground - do exactly what a browser does. After all, Enterprises don’t stop browsers from loading web sites or apps (provided they’re not blacklisted)!&lt;/p&gt;
&lt;h2&gt;The idea&lt;/h2&gt;
&lt;p&gt;In the Electron world you’re basically a browser renderer and engine without the native interop built-in (you implement those bits yourself). So here’s the idea:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Have a versioned native interface&lt;/li&gt;
&lt;li&gt;Load a default application payload into Chromium in an isolated context&lt;/li&gt;
&lt;li&gt;Fetch the right application payload from the server, stick it in local user app data and load that payload into the isolated context instead&lt;/li&gt;
&lt;li&gt;We can remember which payload we last used to skip always loading the default payload&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;This is basically what a browser does with a cache, only we are busting the cache manually and storing the payload manually.&lt;/p&gt;
&lt;p&gt;To ensure that we aren’t getting some nefarious payload from the server we will sign the payload and verify the signature using an embedded public key in the native payload.&lt;/p&gt;
&lt;p&gt;In normal circumstances, where the server and client are the same version the default payload is used and everything is awesome.&lt;/p&gt;
&lt;p&gt;When the server is upgraded the client sees there’s a newer payload available from the server, downloads it and uses that payload instead.&lt;/p&gt;
&lt;p&gt;The same is true of a client that accesses an older server - just download the older package (although a backwards-compatible client would just work).&lt;/p&gt;
&lt;p&gt;Wait, what about the native interop?! I hear you say.&lt;/p&gt;
&lt;p&gt;Well, we can’t overwrite that native interop with the payload without opening ourselves up to other attack vectors. Instead we version the native interop and ensure that:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Native interop maintains backwards compatibility&lt;/li&gt;
&lt;li&gt;Client maintains backwards compatibility with native interop&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Perhaps this sounds like more work than maintaining a backwards compatible server? We expect the native interop to change far less often than our server stack, and in addition all our backwards compatibility code remains client side instead of being spread between client and server.&lt;/p&gt;
&lt;h2&gt;Wrap-up&lt;/h2&gt;
&lt;p&gt;We didn’t come to this conclusion lightly, we brainstormed the approaches and overhead of maintaining a backwards compatible server and unanimously agreed that we really didn’t want to be doing that if we could avoid it. It would slow the release of new features and mean we’d have to maintain both a backwards compatible server and backwards compatible client - much nicer not to have to worry about that!&lt;/p&gt;
&lt;p&gt;This proposal is still being finalised and isn’t live yet. However, this approach essentially amounts to what a browser does, the only difference is we’re exposing some extra APIs for our “web app” to leverage.&lt;/p&gt;</content:encoded></item><item><title><![CDATA[The Symphony London Hackathon 2018]]></title><description><![CDATA[MindLink entered it’s first hackathon and placed second with the Town Hall bot. This post is a brief review of how it went! TL;DR MindLink…]]></description><link>https://engineering.mindlinksoft.comthe-symphony-london-hackathon-2018</link><guid isPermaLink="false">https://engineering.mindlinksoft.comthe-symphony-london-hackathon-2018</guid><pubDate>Mon, 01 Oct 2018 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;MindLink entered it’s first hackathon and placed second with the Town Hall bot. This post is a brief review of how it went!&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;MindLink entered the Symphony London Hackathon September 2018 and we placed second with Town Hall Bot!&lt;/li&gt;
&lt;li&gt;We had fun, learned a lot about Symphony and made a complex bot to improve transparency and company-wide relationships&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;The idea&lt;/h2&gt;
&lt;p&gt;This is the first hackathon MindLink has entered, after our success hopefully it won’t be our last!&lt;/p&gt;
&lt;p&gt;If you’re not familiar with Symphony as a platform, I suggest you go and check it out at &lt;a href=&quot;https://www.symphony.com&quot;&gt;https://www.symphony.com&lt;/a&gt;, in the most basic description it’s a secure real-time messaging platform that supports 1-1, multiparty and chat room messaging. There’s much more to it than that, but it sets the scene!&lt;/p&gt;
&lt;p&gt;Symphony has many APIs available, but we wanted to focus on user experience and that boils down to two main APIs:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Bot =&gt; a back-end application that can send and receive messages in 1-1, multiparty and chat room conversations,&lt;/li&gt;
&lt;li&gt;Client extension =&gt; a website that is embedded within the Symphony UX and can directly communicate with parts of the UX&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We wanted to incorporate both of these APIs in some form and tie them together for a unique and seamless user experience. We will cover the specifics of how we managed to tie them together in another post.&lt;/p&gt;
&lt;p&gt;MindLink has a lot of ideas from a long history in the collaboration and messaging space, but we kept coming back to the idea of a “Town Hall Bot”.&lt;/p&gt;
&lt;p&gt;Town Hall meetings are fundamentally about transparency, whether that’s within a company or with its shareholders. The larger a company gets the more silos form, the harder it is for everybody to move in the same direction. Town Halls are all about breaking down those barriers to keep everybody on the same page, that’s broadly applicable to every part of a company and so has a huge potential for positive business impact.&lt;/p&gt;
&lt;h2&gt;The hackathon&lt;/h2&gt;
&lt;p&gt;The hackathon itself was held at the BNP Paribas offices in Harewood Avenue, London. As can be expected from a banking group the offices are modern, professional and big! A large open atrium space was assigned for the hackathon and there were about 9 teams and 50 attendees including the Symphony cohort. Everbody from Symphony was welcoming, approachable and encouraging - exactly what you need for an event like this!&lt;/p&gt;
&lt;p&gt;Having never been to a hackathon before we were apprehensive, we knew we had the ideas and the expertise, but could we deliver in the time?!&lt;/p&gt;
&lt;p&gt;For the most part the day was spent with the MindLink team (myself, Dimitri Fadda and Jamie Matthews) diligently working away, making sure we kept regularly in sync. Our approach was simple - get a minimum viable product and then build upon its feature set. We didn’t break for lunch, but enjoyed the &lt;strong&gt;really good&lt;/strong&gt; sandwiches put on for the event while continuing to develop the bot. &lt;/p&gt;
&lt;p&gt;We were in a good place for most of the event, until the last hour and a half - that’s when everything is supposed to come together and work beautifully and it didn’t!&lt;/p&gt;
&lt;p&gt;Unfortunately we had an issue with a set of features that relied on sending IMs to users to notify them when their Town Hall questions are answered, we frantically tried to figure out the problem but ended up having to remove the feature on the day, which was disappointing.&lt;/p&gt;
&lt;h2&gt;The presentation&lt;/h2&gt;
&lt;p&gt;Having successfully created the bot and client extension it was up to me to present the idea to the room and make a compelling story. As we were down to the wire for implementation we didn’t spend a lot of time preparing a slide deck or rehearsing so I wasn’t feeling all the confident having seen other polished presentations.&lt;/p&gt;
&lt;p&gt;Fortunately the concept is relatable to almost anybody, so to tell a story about the impact of the idea isn’t the hard part, however the technical presentation wasn’t flawless. I predicted that PowerPoint would try to extend the display of the laptop and thus make it harder to then do a live demo so I turned off that “automatically extend my display” setting, only to find that didn’t work and the display was extended anyway :( So I had to flail around moving windows to the extended display so everyone could see =&gt; that’s not slick and despite my best effort to continue talking, not a great experience!&lt;/p&gt;
&lt;p&gt;All in all the presentation went OK, everything worked that should have worked and the audience didn’t look bewildered or bored afterward!&lt;/p&gt;
&lt;h2&gt;Grand finale&lt;/h2&gt;
&lt;p&gt;MindLink entered the hackathon to get some more exposure to the Symphony APIs and strengthen our relationship with Symphony, but I am also competitive so there was also a part of me vying to win one of the categories ;)&lt;/p&gt;
&lt;p&gt;We were up against some strong competition, everybody produced something that addressed a real need in a business area - whether that’s trading, dev-ops or client engagement. However, we had a couple of unique takes on the hackathon:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;We built for the whole business, not a specific set of users - anybody in any part of the business can be involved in a town hall meeting, there’s no specific expertise required and it’s fundamentally about getting everybody in the company getting their questions answered. I strongly believe that it has a huge business impact and one that is not easily quantified since it involves emotions and relationships and complex community systems.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;We leveraged two independent APIs and tied them together for a single user experience, using the Symphony identity and authentication model to have a mechanism that didn’t require external dependencies to authenticate a user.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We are very proud that our efforts were rewarded with second place for the &lt;em&gt;Most Cutting-Edge Technical Development&lt;/em&gt; category!&lt;/p&gt;
&lt;p&gt;Thank you to my fellow engineers - Dimitri and Jamie - and to Symphony and BNP Paribas for hosting the event!&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Visualising Skype for Business Persistent chat data]]></title><description><![CDATA[Persistent Chat allows the Skype For Business users to create topic-based discussion rooms that persist over time. This feature allows users…]]></description><link>https://engineering.mindlinksoft.comvisualising-skype-for-business-persistent-chat-data</link><guid isPermaLink="false">https://engineering.mindlinksoft.comvisualising-skype-for-business-persistent-chat-data</guid><pubDate>Thu, 09 Aug 2018 17:52:00 GMT</pubDate><content:encoded>&lt;p&gt;Persistent Chat allows the Skype For Business users to create topic-based discussion rooms that persist over time. This feature allows users within an organization to discuss either casual or business related subjects in chatrooms dedicated for a desired purpose. Logically, a lot of business relevant data for an organization is generated in those chatrooms so it would be great if we could extract that information and visualise it in some modern and cool charts, wouldn’t it? &lt;/p&gt;
&lt;h1&gt;Proof of concept&lt;/h1&gt;
&lt;p&gt;In this post we are going to show how easy it is to use Power Bi tool and to visualise some interesting statistics about the Persistent Chat usage in one of our dev environments. The goal with this exercise is to be able to display the 5 top users for any desired chatroom in a classic bar chart. &lt;/p&gt;
&lt;h1&gt;Enter Microsoft Power BI&lt;/h1&gt;
&lt;p&gt;Microsoft Power BI is a business intelligence cloud service that provides business users with tools for aggregating, analysing and visualising data. It did not take too long to get used to its simple interface as it is quite similar to other widely used Microsoft products.&lt;/p&gt;
&lt;p&gt;One great thing about this service is that it is possible to get data from multiple sources. For instance: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;File based data such as JSON or XML&lt;/li&gt;
&lt;li&gt;Datatabase data like SQL Server,  Oracle, MySQL and many more systems.&lt;/li&gt;
&lt;li&gt;Azure services such as SQL Data Warehouse, Azure HDInsight or Azure Data Lake Store.&lt;/li&gt;
&lt;li&gt;and many other online services like Facebook, SalesForce or MailChimp.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Another very good aspect about Power Bi is that the user has a very wide range of clean, neat and customisable charts to choose from. &lt;/p&gt;
&lt;p&gt;Bar, Column, Stacked, Clustered, Waterfall, Donut, Filled Map, Gauge, HeatMap, Word Cloud, amongst many others, can be easilly selected to represent the data that you want to visualise. Not surprisingly, many more type of charts can be downloaded from the Marketplace.&lt;/p&gt;
&lt;p&gt;There are two different versions: professional and free. The professional version has a paid subscription attached to it and it offers more storage, more frequent data refresh cycles and much more streaming data consuption.  &lt;/p&gt;
&lt;h1&gt;Persistent Chat Backend&lt;/h1&gt;
&lt;p&gt;Going back to Persistent Chat for Skype For Business, chat room and user data is persisted in a SQL Server database. Let’s look at some important tables for Lync Group Chat 2010, one of the predecessors of Skype For Business Persistent Chat, which has a very similar structure to Skype For Business 2015 Persistent Chat.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;em&gt;tblChat&lt;/em&gt; - contains all the messages sent in chatrooms.&lt;/li&gt;
&lt;li&gt;&lt;em&gt;tblNode&lt;/em&gt; - contains all the chatrooms and their respective categories that the chatrooms might be associated with. &lt;/li&gt;
&lt;li&gt;&lt;em&gt;tblPrincipal&lt;/em&gt; - contains the information about all the users that are enabled for Persistent Chat and the AD groups that they potentially are associated with.&lt;/li&gt;
&lt;li&gt;few more dozens of tables but they are out of scope for our exercise.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Shall we start doing our little proof of concept? &lt;/p&gt;
&lt;h1&gt;Preparing the DataSet and Relationships&lt;/h1&gt;
&lt;p&gt;Let’s start by opening the PowerBi Desktop application, which you can easilly download for free from the Microsoft website. First thing to do is to authenticate against it with an Office365 account.&lt;/p&gt;
&lt;p&gt;After authenticating the user is guided to the start page which has a form to create a new project.&lt;/p&gt;
&lt;p&gt;As expected the user is prompted with a menu that allows it to choose where to get its data from. In this case we want to get the data from the Persistent Chat database in our Skype For Business 2015 dev environment. After selecting &lt;em&gt;SQL Server database&lt;/em&gt; option the application ask the user to fill in the server details and the database name. In addition it is required to specify which Data Connectivity mode we prefer:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Import - extracts the data from your database and pulls it into the Power BI Desktop :&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Gives you the full suite of transformation and data manipulation in the Desktop.&lt;/li&gt;
&lt;li&gt;Consumes and pushes the data into the Power BI Azure backend&lt;/li&gt;
&lt;li&gt;Can only be refreshed up to 8x a day by setting up a scheduled refresh (In the Service)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Direct Query - leaves the data in the database and sends queries to pull the information:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Limits your ability to manipulate data in the Desktop (removes the Data section).&lt;/li&gt;
&lt;li&gt;Leaves the data in the SQL database.&lt;/li&gt;
&lt;li&gt;Is &lt;em&gt;live&lt;/em&gt;, no need to schedule a refresh.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this example we suggest to select the Import option as we want to be able to do transformation and data manipulation.&lt;/p&gt;
&lt;p&gt;Ok so now it’s time to specify the credentials to access the database and then press &lt;em&gt;Connect&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Now that we have tested that we can connect to the Persistent Chat database we are prompted with a navigator that allows the user to select which tables they want to import. We will select the three tables that were mentioned in the previous section: &lt;em&gt;tblNode&lt;/em&gt;, &lt;em&gt;tblPrincipal&lt;/em&gt; and &lt;em&gt;tblChat&lt;/em&gt;. Since &lt;em&gt;tblChat&lt;/em&gt; has thousands and thousands of rows it will take a while to load all the rows.&lt;/p&gt;
&lt;p&gt;Great! Once all the tables are loaded we will be taken to the project’s main page and on the left hand side there will be 3 icons:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Reports icon - where we place the actual charts and buttons for our report.&lt;/li&gt;
&lt;li&gt;Data icon - where the data is defined - queries, lookups and so on.&lt;/li&gt;
&lt;li&gt;Relationships icon - where the relationships between our entities are outlined.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Before doing anything else, maybe now it’s a good time to save your project. &lt;/p&gt;
&lt;p&gt;If you click on the Data icon you’ll be able to see imported tables and their respective data.&lt;/p&gt;
&lt;p&gt;But first we need to define the relationships between the tables! So click on the Relationships icon.&lt;/p&gt;
&lt;p&gt;We can see the three tables that we’ve loaded from our database connection. The goal is to clearly specify that: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;tblChat&lt;/em&gt;’s userId tells us which user sent the respective message so the &lt;em&gt;userId&lt;/em&gt; in &lt;em&gt;tblChat&lt;/em&gt; will correspond to the &lt;em&gt;prinId&lt;/em&gt; in &lt;em&gt;tblPrincipal&lt;/em&gt;. In order to establish that relationship just drag the &lt;em&gt;prinId&lt;/em&gt; field in &lt;em&gt;tblPrincipal&lt;/em&gt; to userId field in &lt;em&gt;tblChat&lt;/em&gt;. Now for a given &lt;em&gt;userId&lt;/em&gt; in &lt;em&gt;tblChat&lt;/em&gt; we will be able to know more about that user - more specifically - the prinName which corresponds to the user name that will be displayed in the chart.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;tblChat&lt;/em&gt;’s &lt;em&gt;channelId&lt;/em&gt; is referring to what group was the message sent in so in order to get the information about that group we need to drag the &lt;em&gt;tblNode&lt;/em&gt;’s nodeId to &lt;em&gt;tblChat&lt;/em&gt;’s &lt;em&gt;channelId&lt;/em&gt; field. &lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;It should look like this:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-1a87f.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 63.78482228626321%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAIAAAAmMtkJAAAACXBIWXMAAA7DAAAOwwHHb6hkAAACRUlEQVQoz32S7U/aUBTG+wejKKVluriwD2YOZEtwQshgIAhzyzL3luE+LBMMI/GTkgwWILy0lta+3ra0IBrcU/zkh3nSNLf3nt+5z3l6KI7jbdsyNMUgRDVMjVj2eEwsh5hja+zaNr4cLBx3ggN77JimNZ1MNN1wHIe6VDSDmLIk8PyFKErCSFI1TVI0UVIuZU1WVEVVsdZ1YzjkkDwaiYauXwgialGZ14l3bzMfPxTeH2Rf7eykUqnMm0wul9vby2Wz2Xw+n8tld3d3k8nkTjyeTqe9hEwGZ3hTXw5LvV5XFIU/jfrKyvLSkm9jY+P09PTs7Ly9iGarFQ6HfT6f3++vVCqNRqPT6fT7/UQiQVV+lS3LRifcoMmywdUAHYlGiWHIsjy7vr65uXFdNxqN0jQdDAZ5nocH2JzP58VikTqpHMESQkx++DfE0o/Xll++iBg6YGU6hTUTeBaLxRiGAd/r9RR4oKqoWCgUqOpx2XEmjuMO+831NXr90crzSAQYjlH+9vZ2Npttb2/fwVArLwJF9/f3PXgM1x2XG7SePlkFvLn5TJJEZEiShLcgCFtbW8wi+t7NCjYJIfASPX83LQvK+UGLYQIsy4ZCobX7cbcJuNvtgsREQJonGzAGAQ8MY4I00pAavB/sIiD7DiameXV15cHHP78ZBsGEdNvn6Iv9fwDutNsiJkmS0LMnu1b9MZ3ONE33bn4QDgQCHMfhLyIZRpZKJSqViH39VPp8WDwopdDVAzD0l8vlWu13tXpSr9fj8fg/ccI0KMOvYv8AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;alt text&quot;
        title=&quot;&quot;
        src=&quot;/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-fb8a0.png&quot;
        srcset=&quot;/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-1a291.png 148w,
/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-2bc4a.png 295w,
/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-fb8a0.png 590w,
/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-526de.png 885w,
/static/EntityRelationships-7eba4701c8b856839e8260901dd098e4-1a87f.png 1041w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;h1&gt;Creating the report&lt;/h1&gt;
&lt;p&gt;Now that the data is there we can start creating the report which will include a chart and a dropdown to select the chatroom that we wish to see the top 5 users that have sent more messages. First thing to do is to rename the the page for that report from &lt;em&gt;Page 1&lt;/em&gt; to &lt;em&gt;Top 5 Users in Chatroom&lt;/em&gt;.&lt;/p&gt;
&lt;h2&gt;Adding the slicer&lt;/h2&gt;
&lt;p&gt;By pressing the &lt;em&gt;Reports&lt;/em&gt; icon we are taken to the area where we can drop the slicer that will contain all the chatrooms in our Persistent Chat environment. The slicer can be formatted to look like a dropdown.&lt;/p&gt;
&lt;p&gt;The &lt;em&gt;Visualizations&lt;/em&gt; navigator has a list of predefined objects so we need to press the slicer icon.&lt;/p&gt;
&lt;p&gt;There are several properties that need to be filled so that the slicer displays all the chatrooms in a dropdown.&lt;/p&gt;
&lt;p&gt;First the filters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Field: Drag the &lt;em&gt;nodeName&lt;/em&gt; field from the &lt;em&gt;tblNode&lt;/em&gt; table.&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Filters: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Page level filters: This affects which rows will be shown in the dropdown.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;drag the &lt;em&gt;disabled&lt;/em&gt; field from &lt;em&gt;tblNode&lt;/em&gt; here and tick the &lt;em&gt;False&lt;/em&gt; box as we only want to show chatrooms that are enabled.&lt;/li&gt;
&lt;li&gt;drag the &lt;em&gt;nodeType&lt;/em&gt; field from &lt;em&gt;tblNode&lt;/em&gt; here and tick the &lt;em&gt;False&lt;/em&gt; box as we only want to filter out chatroom categories from the eligible chatrooms.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;And then the Formatting:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;First, click on the paint roller icon!&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Here you can define the visual aspects of the slicer.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;You can edit the title by switching the &lt;em&gt;Title&lt;/em&gt; on.&lt;/li&gt;
&lt;li&gt;Or edit edit the &lt;em&gt;Header&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;and change other properties as you please :)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To behave like a dropdown instead of a list hover the slicer’s header and there is the possibility to choose between &lt;em&gt;list&lt;/em&gt; or &lt;em&gt;dropdown&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Now you should be able to see the dropdown with all the valid chatrooms!&lt;/p&gt;
&lt;h2&gt;Adding the Clustered Column chart and linking it with the dropdown&lt;/h2&gt;
&lt;p&gt;It’s time to add the chart that will show us the top 5 users for a chatroom!&lt;/p&gt;
&lt;p&gt;The Clustered column chart will do just fine so in order to add it to our report simply press the report page and then click on the corresponding icon and place it next to the dropdown. &lt;/p&gt;
&lt;p&gt;Ok so now it is time to configure the chart! &lt;/p&gt;
&lt;p&gt;Our first goal is to link the chart with the dropdown so that every time we choose a different chatroom the chart component displays the top 5 users for the selected chatroom.&lt;/p&gt;
&lt;p&gt;In order to achieve this we need to :&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Highlight the slicer component.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Select the Format tab (in between Help and Data/Drill tabs). &lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Switch on Edit Interactions.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Filter controls are represented by the funnel icon on top of each component.      &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Initially all the Filter icons are selected.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Highlight both components and ensure the filter controls are selected for both.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Now we need to configure the chart accordingly:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Highlight the chart component and check the fields section on the right hand side.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Filters :&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Legend: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;This will determine what will be shown in the X axis. Since we want to display a bar for each of the top 5 users in the chatroom this has to show the user name. Don’t forget  that we have established the relationship between &lt;em&gt;tblPrincipal&lt;/em&gt; and &lt;em&gt;tblChat&lt;/em&gt;. So for each &lt;em&gt;tblChat&lt;/em&gt; row there is a userId associated with a message sent and that is linked with the respective user in &lt;em&gt;tblPrincipal&lt;/em&gt;. To show the sender’s user name we simply need to drag the prinName from &lt;em&gt;tblPrincipal&lt;/em&gt; to the Legend data field. We can rename the what is shown in the chart by clicking the arrow next to prinName and by clicking on &lt;em&gt;Rename&lt;/em&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Value:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;This will reflect what should be displayed in the Y axis. In our case we want a count of each occurences of a given &lt;em&gt;channelId&lt;/em&gt; in the &lt;em&gt;tblChat&lt;/em&gt;. This will give us a count of all messages in that chatroom. In order to achieve this we just need to drag the &lt;em&gt;channelId&lt;/em&gt; field in &lt;em&gt;tblChat&lt;/em&gt; and drop it in the &lt;em&gt;Value&lt;/em&gt; data field. Once that is done if you click the arrow next to it some options will appear and the one that we want is &lt;em&gt;Count&lt;/em&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;At the moment you should be able to change the chatroom and see a new graph being drawn every time you select a new chatroom. It is showing the number of messages sent by each user in the given chatroom. Now we want to limit it to only show 5 users and improve the looks of that chart! &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Visual level filters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;User (all) &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Click on &lt;em&gt;Filter type&lt;/em&gt; and select &lt;em&gt;Top N&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;Next to &lt;em&gt;Top&lt;/em&gt; type &lt;em&gt;5&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;Now drag &lt;em&gt;chatId&lt;/em&gt; from &lt;em&gt;tblChat&lt;/em&gt; to &lt;em&gt;By value&lt;/em&gt; section.&lt;/li&gt;
&lt;li&gt;Click on &lt;em&gt;Apply filter&lt;/em&gt; and it should only display the top 5 users in that chatroom!&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Format :&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Here it is where you can change the layout of the chart. For instance you can move the legend to be at the bottom instead of being at the top or you can add a title to any of the axis.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;More visual aspects can be changed such as the colour of the bars or add some nice labels that indicate the absolute number of messages sent to them. &lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Publish the report!&lt;/h2&gt;
&lt;p&gt;Now that we have done what we wanted we need to publish the report that we have just done to the cloud! &lt;/p&gt;
&lt;p&gt;In order to achieve that go to &lt;em&gt;File&lt;/em&gt; and click on the &lt;em&gt;Publish&lt;/em&gt; button! &lt;/p&gt;
&lt;p&gt;After the &lt;em&gt;success&lt;/em&gt; dialog click on the link provided and you will be redirected to the PowerBi portal so sign in using the same credentials as before and you should be able to see this report in your workspace.&lt;/p&gt;
&lt;p&gt; 
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-ea655.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 33.0488750969744%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAA4ElEQVQY032QSwrCMBCGvZwX8AIewJUnEF2oZ/AGIi5UqKArFbGK4ELBlYirolRIqEnMNA+nqe3Gx0+YTIb55pGCzaS1AZASIAaw1uRxpRTnXDqhwxh7ZioY885DR2mDF31wPGkkhQmhCHEulJN2QieHEwvsgna8PfUWB4fptAQhJIoi+6EMNkmeDLzLzu8MV935Hp+xg1GUUiHELxiNa3KdtKrNUqU9Wh9zGFcNwxCH/AJnC7smN69eaRTLtf7ykMNYGABwyS+wlEKKJ07FRCzJyZ9tBlP/HNyT/zfG/tULkqWQMQfVfkIAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;alt text&quot;
        title=&quot;&quot;
        src=&quot;/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-fb8a0.png&quot;
        srcset=&quot;/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-1a291.png 148w,
/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-2bc4a.png 295w,
/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-fb8a0.png 590w,
/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-526de.png 885w,
/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-fa2eb.png 1180w,
/static/Top5UsersChatroom-22c9b968eaa3647bea9669fcd5201502-ea655.png 1289w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;If you want to publish it to the Internet or to embed in a SharePoint or to a Powerpoint simply go to File and those options will be available!
&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;This was a very basic example on how we can combine a powerful visual tool like the Power BI with the Skype For Business Persistent Chat database to show us interesting information about a chatroom usage.&lt;/p&gt;
&lt;p&gt;Another good example that I was also working on was to create a word cloud &lt;em&gt;chart&lt;/em&gt; that is built of the most used words in a chatroom but unfortunately that is still work in progress as it requires a bit more advanced algorithms to gather the meaningful words present in messages that are sent in a chatroom. &lt;/p&gt;
&lt;p&gt;My experience with Power Bi was very positive as I found the tool quite powerful and easy to integrate with existing systems. It allows the user to get data from a long list of data sources and it offers a wide range of different charts that display the information the users need. With the R Script Editor,which I did not cover on this example, the more technical user will feel more empowered to use their scripting abilities to create and use their own Power Bi visuals. &lt;/p&gt;
&lt;p&gt;As a quick and last note I am definitely not an expert on Power Bi so if you found something that was wrong or could be explained better please leave a friendly comment :)&lt;/p&gt;
&lt;p&gt;Hopefully this article has sparked some curiosity about this tool! &lt;/p&gt;</content:encoded></item><item><title><![CDATA[Getting started with Python and the MindLink API.]]></title><description><![CDATA[Today, we’ll have a look at how I started coding in Python to work with the MindLink API. Although Python is an extremely popular language…]]></description><link>https://engineering.mindlinksoft.comgetting-started-with-python-and-the-mindlink-api</link><guid isPermaLink="false">https://engineering.mindlinksoft.comgetting-started-with-python-and-the-mindlink-api</guid><pubDate>Mon, 23 Jul 2018 17:10:00 GMT</pubDate><content:encoded>&lt;p&gt;Today, we’ll have a look at how I started coding in Python to work with the MindLink API.&lt;/p&gt;
&lt;p&gt;Although Python is an extremely popular language, we, or at least I, haven’t used too much of it. When we started investigating the capabilities of the &lt;a href=&quot;https://rasa.com/&quot;&gt;Rasa Core platform&lt;/a&gt; as part of our &lt;a href=&quot;../leveraging-rasa-to-build-on-premise-intelligent-chat-bots&quot;&gt;larger work around intelligent chat bots&lt;/a&gt;, we found out that they have a neat open source API, in Python. The next natural step for us was wanting to leverage the capabilities of the MindLink API (MLAPI for briefness) to provide the bot with voice and ears; while there are ways to use Rasa-Core in a language-agnostic way, we took the chance to dive into a bit of Python programming, beginners level.
This post will document how I went from virtually no Python knowledge to writing a simple bot harness that connects to the MLAPI and sends/receives messages, covering the following topics: &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;How to setup a basic Python dev environment on a Windows machine.&lt;/li&gt;
&lt;li&gt;How to create an application representing a bot that connects to the API and exposes methods to authenticate, retrieve and send messages.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;What we will not cover here is:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Setup of your MindLink API infrastructure.&lt;/li&gt;
&lt;li&gt;Provisioning of user, agents and channels.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You probably have noticed by now that the assumption is that you have some degree of familiarity with the fundamental concepts of the API - for more information please do visit our &lt;a href=&quot;https://wiki.mindlinksoft.com/tiki-index.php?page=MindLink+API#API_Developers_reference&quot;&gt;comprehensive developer reference wiki&lt;/a&gt;, and of some fundamental programming concepts. &lt;/p&gt;
&lt;p&gt;Also, in case it wasn’t clear enough from the blob above, &lt;strong&gt;I am no python expert&lt;/strong&gt; - so if you notice anything wrong or that could be done better please do feel free to leave a comment!&lt;/p&gt;
&lt;p&gt;By the way, you can find the code we’re writing here and more &lt;a href=&quot;https://github.com/mindlink/api-samples/tree/master/python-sample&quot;&gt;at our github&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Now, without any further ado:&lt;/p&gt;
&lt;h2&gt;1 Set up your dev environment&lt;/h2&gt;
&lt;p&gt;For briefness, the assumption is that we will configure our environment on a Windows machine. Using Linux or OSX will not be too dissimilar.&lt;/p&gt;
&lt;h3&gt;1.1 Install Python&lt;/h3&gt;
&lt;p&gt;First thing is to have a local installation of Python. Go to &lt;a href=&quot;https://www.python.org/downloads/&quot;&gt;https://www.python.org/downloads/&lt;/a&gt;, and download the latest stable version (3.6.5 at the time of writing).  Run the installer, and make sure you tick the “add to PATH” option. This will ensure that you can run the Python interpreter, pip, etc. from any folder.&lt;/p&gt;
&lt;h3&gt;1.2 Install your IDE of choice&lt;/h3&gt;
&lt;p&gt;While IDEs are like pizza toppings, everyone has their favourites and they are all great (except for pineapple), I have chosen to use &lt;a href=&quot;https://code.visualstudio.com/&quot;&gt;Visual Studio Code&lt;/a&gt;. It’s what we use for our client side TS/React development, it’s a great editor and a fantastic project. Simply download and run the installer.&lt;/p&gt;
&lt;h4&gt;1.2.1 Install the Python extension for VS Code&lt;/h4&gt;
&lt;p&gt;This will allow us to run the debugger from VS Code, set breakpoints, leverage autocompletion and more. Simply launch VS Code, go to the extensions tab (Ctrl-Shift-X) and search for Python, find the ms-python.python extension and install it. Restart VS Code to make the extension load.&lt;/p&gt;
&lt;h3&gt;1.3 Install Python Requests library&lt;/h3&gt;
&lt;p&gt;As I am sure you know, MLAPI is a REST api, meaning requests are made using HTTP. While Python has standard libraries that allow for HTTP handling, &lt;a href=&quot;http://docs.python-requests.org/en/master/&quot;&gt;requests&lt;/a&gt; is easy to use, widespread, comes highly recommended and is, according to them,“is the only Non-GMO HTTP library for Python, safe for human consumption.”, whatever that means.&lt;/p&gt;
&lt;p&gt;To install requests, simply run “pip install requests” in a cmd/powershell or VSCode terminal window.&lt;/p&gt;
&lt;p&gt;We’re now ready to code!&lt;/p&gt;
&lt;h2&gt;2 Get your details!&lt;/h2&gt;
&lt;p&gt;Before we start writing code we need to know few details about the MindLink API deployment that we’re going to use.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The API location URL, i.e. &lt;a href=&quot;http://mlapi.company.com:X&quot;&gt;http://mlapi.company.com:X&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;The agent ID for the agent that we will use.&lt;/li&gt;
&lt;li&gt;The username and password for the user backing the agent.&lt;/li&gt;
&lt;li&gt;The ID of a channel(s) that the agent is provisioned upon.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Again, if you are unsure of what any of those terms mean please do refer to our wiki.&lt;/p&gt;
&lt;p&gt;We can finally start writing some code! Find some place where you’ll write your files to, open your IDE, create a new file and off we go!&lt;/p&gt;
&lt;h2&gt;3 Getting a token&lt;/h2&gt;
&lt;p&gt;First thing we have to do is to get a token. This is done by performing a post request onto the authentication service, specifically on the Authentication/V1/Tokens resource, providing agent ID, username and password as the request’s payload. In our file, let’s define a function “authenticate” that takes in the URL of the service, agent ID, username, password and returns us the token.
The function will:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Build the URL of the service.&lt;/li&gt;
&lt;li&gt;Post the request to the service endpoint, with the necessary payload.
As a final note, to make the result easier to handle, we will specify via headers that we want the MLAPI server to return us JSON. This will make data very easy to deserialize, as the requests library has in-built support for JSON.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;import&lt;/span&gt; requests

&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;authenticate&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; username&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; password&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; agent&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    request_url &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Authentication/V1/Tokens&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;post&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;request_url&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; json &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Username&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; username&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Password&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; password&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;AgentId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; agent
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Let’s analyse what is going on in this code block.
The first line imports the requests library that we’ve previously installed on our environment, allowing us to use it in the scope of this file.&lt;/p&gt;
&lt;p&gt;We then define a function that takes in the aforementioned parameters.
The function begins by building the full URL of our request by appending the static part of the request address (the “/Authentication/V1/Tokens” bit) to the dynamic part (the provided protocol/server/port). &lt;/p&gt;
&lt;p&gt;After this, we create and forward a post request. The first parameter is the URL we just created, and then we provide the request payload which is automatically serialized in JSON by the requests library - all we have to do is to match the attributes to the parameters.&lt;/p&gt;
&lt;p&gt;The next parameter for our requests is the headers. We specify that we want to accept JSON, which will cause the API server to JSON encode the response data.&lt;/p&gt;
&lt;p&gt;Do note that the request is immediately fired once the .post() method is called, hence why we assign its result to the “response” parameter.&lt;/p&gt;
&lt;p&gt;Provided that the request was completed succesfully, our response will contain a JSON-encoded API token. To “extract” it, we simply use the .json() method onto the response object.&lt;/p&gt;
&lt;p&gt;OK, how do we actually use what we just wrote though? Below the previous code block, let’s add:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;token &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; authenticate&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;http://apiserver.domain.com:8081&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;userdomain\\username&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;password&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;agentId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Here`s my token!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;We can now run our program. 2 ways of doing this, either &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;open a cmd/powershell/terminal instance in the folder where our file resides and run python my_file.py&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-87d31.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 59.241706161137444%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAA7DAAAOwwHHb6hkAAABCElEQVQoz+2STU7DMBCFHTt2FMEmBFdqkoKRIA03QJSGLsslOQASG35aICSBtlFaLvWwvYEFiAq6ZPFp3nispyfNkMura9xOJ7iZ3uOxLDBvGyxWLWa6Gt28LTFftlpXaFa11WZevtYoXioUdYmn6hl3DxMUsxrkMEsRdyMopbDX60HtK/TTvu0PNFl2jPQoRRJLRN0dxFGidYKOlAjDEEEQQO5KcJdjfDEGEVs+GGNgrgtKKTjnEELYat5t7wkQQn7kdHgGQoW71meD4zhfYsKYeT46B2EeX9vwOyijtg5HuU7ouf+GfzbcxFLYJ0OxgYQO/Tgbvu2Dcn3UOqkx/w3c96zhST7AO2mBLFiZfgKrAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Running Python via PS&quot;
        title=&quot;&quot;
        src=&quot;/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-fb8a0.png&quot;
        srcset=&quot;/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-1a291.png 148w,
/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-2bc4a.png 295w,
/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-fb8a0.png 590w,
/static/cmd_run_py-d4ae5f47947f8767601cce6b7a7e15e4-87d31.png 844w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;or by leveraging the Python running and debugging capabilities of our IDE. In VSCode simply go to the debug menu and press “Start (Without) Debugging”. A dropdown menu to choose the running environment will open, and you can safely choose “Python”. In the terminal tab you’ll see the command VSCode actually runs, and hopefully our output.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/vscode_run-d1de836a4ba530d11766017a22e6c694-ef9ea.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 54.270833333333336%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABZUlEQVQoz52RaW7CQAyFcwIQEoVss2WZLISGXf1X0t6g96FQEUpv/eoMYpWq0v745IznPXvsWOvtJ9brDVard2w2H6h3O2y3taGu76PR7si333/BkqFGr9cHZwzc8+C7LrrdLlqt1p9ot9vodDqwZvMFlJIo8xSz8hFaazqrf2PleQYhJOI4hpISgjOCX8EJRhP8RqOjgjnCMERKL0viCFGgEFDhS6QQRnwPVpZl8H3f7JAxDv+n7hewm29G/qPOFGwSx/GkoPH4GdNZ3GBywtxL2ltAE8pmXYRVPJbQ5QT5dIFi/oTBeAKVDYgCKh9CJNQwicF0ciBNwMPmTJPFKaKiRDqaGo/UKe2wGCKINXSWGyIyKeqogtBEIRW9RBygn3eIZ3yzJn7SWGkcwus/wHf68O0eGEXu2td4DmFf4FzknBOMtFZWvUFMniHnr5CzFzjjCv3REvYdNDp3vISYVeDTyvi/AcmcRkOwkYvvAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;VSCode debugger&quot;
        title=&quot;&quot;
        src=&quot;/static/vscode_run-d1de836a4ba530d11766017a22e6c694-fb8a0.png&quot;
        srcset=&quot;/static/vscode_run-d1de836a4ba530d11766017a22e6c694-1a291.png 148w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-2bc4a.png 295w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-fb8a0.png 590w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-526de.png 885w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-fa2eb.png 1180w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-08f6a.png 1770w,
/static/vscode_run-d1de836a4ba530d11766017a22e6c694-ef9ea.png 1920w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;h3&gt;3.1 Let’s handle some errors&lt;/h3&gt;
&lt;p&gt;One of the problems with the code we just wrote, is that it deals poorly with errors. It always tries to deserialize the response, which can fail horribly if the request was not successful. So let’s add some error handling/reporting.&lt;/p&gt;
&lt;p&gt;Instead of just returning the response.json(), we can add a conditional statement that will try to deserialize and return the token &lt;strong&gt;only&lt;/strong&gt; if the server has given us a 200 response code (OK). We can also take this a step further by making the function print to console the status code and reason of an erroneous response. The second part of our function will now look like:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;    &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Something went wrong!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;reason&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;If we now run our program, the output will look a lot less scary and more useful, even when something goes wrong!&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-59922.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 54.06673618352451%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABbklEQVQoz5WRWU7DQBBE5wZEQnFMmNXr2IlDWBI4ABJwAk6SxPmAqxc1kxiExPrxVO5Rd/Vi0e/32PV7bDYbbLdb7BkH+r7/M7tdH2teX14hXFFhkqawRsNqBS0lkiTBaDTCycko6m8MeePxGOJmtYK1FovW42rRoSzLGDvn/kWoybIMYtY0MAy8b1AWBYwx0JpQldZQSkV0+Cb6G0JOqBUNDfMsR+trNFWBuixQHcnZObOG5zA/mg2GQWnYHsZ1AccuQxI7Uj/iI2pQFXMOG8j3TUTjPeT5OYsPBtZ8MYFhcXh/J5zEQlGty5DlPFUYhoOJbnmJ9nqN+eqO3MIvr6D8HKrpoNoFoc5njFvoqqLW0DxRQNIs5+2b7gJZVcPRWMy6BUrfRnI+2qKEsu4zZoBT6cDxhxHJNaUMK+uI8IXDNDmFSscRfZbATCf/Qk9TMoEior5/hlo/QN89RdKbByTXfyfkq9tHyPUjDOvfAGDsRPDvOeTQAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;With some error handling&quot;
        title=&quot;&quot;
        src=&quot;/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-fb8a0.png&quot;
        srcset=&quot;/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-1a291.png 148w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-2bc4a.png 295w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-fb8a0.png 590w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-526de.png 885w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-fa2eb.png 1180w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-08f6a.png 1770w,
/static/auth_with_error_handling-aa594c2505e2e5f0214d436046820a10-59922.png 1918w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;It’s now obvious that the issue here is to do with wrong credentials.&lt;/p&gt;
&lt;h2&gt;4 Getting messages&lt;/h2&gt;
&lt;p&gt;Now that we have a token, we can use it to do actual collaboration!
Let’s write a function that will retrieve the last X messages from a channel.&lt;/p&gt;
&lt;p&gt;In our existing file:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;get_messages&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channel_id&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; count&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    request_url &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Channels/{}/Messages&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channel_id&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;get&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        request_url&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        params &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;take&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; count&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Something went wrong!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;reason&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;As you can see, the method is quite similar to the “authenticate” one that we wrote previously; notable differences are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The request URL, we now want to access the &lt;a href=&quot;https://wiki.mindlinksoft.com/tiki-index.php?page=MindLink+API#Collaboration_REST_API&quot;&gt;Collaboration service&lt;/a&gt;, and in it the &lt;a href=&quot;https://wiki.mindlinksoft.com/tiki-index.php?page=MindLink+API#Channels_id_Messages_resource&quot;&gt;Messages resource of a specific channel&lt;/a&gt;, identified by its ID.&lt;/li&gt;
&lt;li&gt;We are using a GET request, not a POST.&lt;/li&gt;
&lt;li&gt;We have to specify a “take” parameter with the number of messages that we want to retrieve.&lt;/li&gt;
&lt;li&gt;We have to provide the token as our authorization header.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We can now update our code to use the new function. As we have to reuse the host, username, password and agent id values, we can store them in variables, i.e.:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;host &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;http://localhost:8081&apos;&lt;/span&gt;
user_name &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;domain\\user&apos;&lt;/span&gt;
password &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;super_secret_password&apos;&lt;/span&gt;
agent &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;agent_1&apos;&lt;/span&gt;

token &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; authenticate&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; user_name&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; password&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; agent&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Here`s my token!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

messages &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; get_messages&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;chat-room:d343ec64-c867-4edf-9daf-84f25a3c8ed4&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;here are my messages!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; messages&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;If we now run the program, the output should look something like:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-2c446.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 55.67542213883677%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABL0lEQVQoz42SWXLDIAyGuUfYMZsTUjtO0mnvf6+/Eo4zaaed+uFDSAihBWGtQy0Vp1oQhwHBe4QQYIyBUupftNZdeu/6XeFoaa2hpIhIgQIfkHTOPS/shZMQxlqMJSNSIO9sP5BSfnt9b5ZrQFpSTKg5UUDXjczmtDe7zVfElHBpR7QcweV7gsvtTjv4GVjwILjkQhkO3DvLw5DQ/OIOetlbiwgRhohEWZZCUybJPVWMYQwkPSAfsutkZ53PpTZwdD+PR1jnO+J8WXD7+Oxc7u9o1zvOt3ecSJZpRpknVJJ1Ic60nxeM8xWpTchvZJsWZLInCprqCDFQyZky4yxjjH0wK5bKX+k6/ddXvcuHrrV6DlPwF5GHw4o8rP3YUA/kL6hXH/X0FcoF7Mb+YXvhC80CMFSZfTQKAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Get messages result&quot;
        title=&quot;&quot;
        src=&quot;/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-fb8a0.png&quot;
        srcset=&quot;/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-1a291.png 148w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-2bc4a.png 295w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-fb8a0.png 590w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-526de.png 885w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-fa2eb.png 1180w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-08f6a.png 1770w,
/static/get_messages-8a8a4ce52af32aa104e553b1759e93bd-2c446.png 2132w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;This looks perhaps a bit confusing…time to look at those messages using the debugger. Set a breakpoint on the line where we print the messages, launch the application and inspect the “messages” object.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/messages-59cfc6a26e7ba1545eef50766ecd0432-e19cf.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 67.17373899119295%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAA7EAAAOxAGVKw4bAAABiklEQVQ4y42T6W6jMBSFeY02eAMvGDBL2BKkGc38Gc37v9DptZNWUduk+fHpWl4OdzlkQgjYyiOEgLqq0NUera/QVA62LKALQhtCoygUGGN3yfMcmSTBkh6O44imrlGoyyNOCM4Rz6WUifjxdEb798jeL3Ztg8raJKIkiZAwvwrcEh89yvIqKGF1idpoeEPRalgSV0p9K/ZzhlJRHxu0bQvvPRT1LWYnbsp9RuxD0DmH2hl0TqOhDGMf4xB0qVM0xiTRpwQLyuY4HWm6VZpooS49FdcLcXJ5frgRYw/JYp8M2cISJmYVo7FwZCFrXRrOZUASh0O0Bn9IFr0VgkfXGIxdhdA6zNOK/fQb87zitJ2xbTvGYSAXkMU0Jz+yu2S6jIIafZAYgkJHcTrO+LP+xTKdMPYjpmFC35Pxa5bwnn2sP5NxLqiUPJUTeXk9INQD/u3/sa871mnGeVnJpz1enylZiPdG86s1crRNj335hW3ZsJ/PJLrQXgDjPw8m+/I73azZJ+IDxh7b5g2lJmuCfA67TwAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Messages&quot;
        title=&quot;&quot;
        src=&quot;/static/messages-59cfc6a26e7ba1545eef50766ecd0432-fb8a0.png&quot;
        srcset=&quot;/static/messages-59cfc6a26e7ba1545eef50766ecd0432-1a291.png 148w,
/static/messages-59cfc6a26e7ba1545eef50766ecd0432-2bc4a.png 295w,
/static/messages-59cfc6a26e7ba1545eef50766ecd0432-fb8a0.png 590w,
/static/messages-59cfc6a26e7ba1545eef50766ecd0432-526de.png 885w,
/static/messages-59cfc6a26e7ba1545eef50766ecd0432-fa2eb.png 1180w,
/static/messages-59cfc6a26e7ba1545eef50766ecd0432-e19cf.png 1249w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;Now it makes a bit more sense. We have an array with 2 messages, and if we expand the first node we can see all the properties of each message object. We can use what we have just learned to prettify the output of our application. We’ll iterate through each message and print some properties. Simply replace the line printing messages for:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; message &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; messages&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;message ID: &apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Id&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos; Alert? &apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;IsAlert&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos; Sender &apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;SenderId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos; Text &apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Text&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Notice how we’re displaying the text instead of using the message parts. Message parts are a very powerful modelling tool that remove the coupling between what a part of a message looks like and what it represents. They are a more advanced topic though, and we’ll come back to them in another episode.&lt;/p&gt;
&lt;h2&gt;5 Sending messages&lt;/h2&gt;
&lt;p&gt;We are now ready to send our first message! To do that:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;send_message&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channel_id&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; content&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; is_alert&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    request_url &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Channels/{}/Messages&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channel_id&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;post&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        request_url&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        json &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Text&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; content&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;IsAlert&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; is_alert
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Something went wrong!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;reason&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;As you have probably noticed, the method doesn’t look too different from the previous ones; in fact you can probably see a pattern beginning to take shape! Anyway, the way it works is pretty self explanatory; do however note how we use a POST request!&lt;/p&gt;
&lt;p&gt;Now, to call him from our application we’ll move the chatroom ID to a variable to the previous block to avoid duplication, and then wel’ll add a call to the method:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token comment&quot;&gt;# snip&lt;/span&gt;
agent &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;agent_1&apos;&lt;/span&gt;
chat &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;chat-room:d343ec64-c867-4edf-9daf-84f25a3c8ed4&apos;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;# snip&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;sending a message&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
send_message&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; chat&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;hello world&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token comment&quot;&gt;#True for alerts, False for regular messages&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;That easy!&lt;/p&gt;
&lt;h2&gt;6 Getting events&lt;/h2&gt;
&lt;p&gt;The final piece of this initial puzzle is how to get events. Events are the mechanism that the MindLink API uses to relay in real time what is happening on channels. Again, for more information &lt;a href=&quot;https://wiki.mindlinksoft.com/tiki-index.php?page=MindLink+API#Events_resource&quot;&gt;check the dedicated section on the Wiki&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The method to get message events looks something like:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;get_events&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; token&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channel_id&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; last_event_id&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    request_url &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Events&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    parameters &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;last-event&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; last_event_id&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;types&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;message&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;channels&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;channel_id&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;regex&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token string&quot;&gt;&apos;origins&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;remote&apos;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;get&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        request_url&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        params &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; parameters&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Something went wrong while getting events!&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;reason&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Note how we need to keep track of the last event ID - that can be extracted by simply iterating through the events and comparing their id; this could be done in the get_events function our outside of it. A function to do that could look like:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;get_latest_event_id&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;events&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;

    last_event_id &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; event &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; events&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        event_id &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;EventId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; event_id &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; last_event_id&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
            last_event_id &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; event_id        
    
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; last_event_id&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;While events can be retrieved individually, the streaming paradigm implies that what we probably want to do is to continuously ask for new requests, i.e. poll. To do that we would need few things:
- keep track of the last event ID, so we only ask for newer events
- introduce a loop with control
- run this event pump in a separate thread, or use asynchronous requests, so that it does not block our bot.
To see a sample of that, everything we discussed here and much much more, do check out our sample &lt;a href=&quot;https://github.com/mindlink/api-samples/tree/master/python-sample/sample2&quot;&gt;on our github&lt;/a&gt;.
The sample introduces a second module that contains all the various API methods that we have seen here in an instantiable class, so that we reduce duplication (no need to provide host and token for each request) and our application logic is less coupled with the API.&lt;/p&gt;
&lt;h2&gt;7 Wrapping up&lt;/h2&gt;
&lt;p&gt;Hopefully if you have made it this far you’ll have realised how simple it is to begin writing collaborative tools using the MindLink API, even when starting anew with a programming language as we did here. Naturally we have just begun to scratch the surface of the capabilities of the platform, so stay tuned for more!&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Parallelizing our CI tests]]></title><description><![CDATA[In this second “improving our CI tests” blog we’ll complete our journey by: Enabling parallel NUnit test execution at the text-fixture…]]></description><link>https://engineering.mindlinksoft.comparallelizing-our-ci-tests</link><guid isPermaLink="false">https://engineering.mindlinksoft.comparallelizing-our-ci-tests</guid><pubDate>Thu, 05 Jul 2018 13:10:00 GMT</pubDate><content:encoded>&lt;p&gt;In this second “improving our CI tests” blog we’ll complete our journey by:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Enabling parallel NUnit test execution at the text-fixture granularity&lt;/li&gt;
&lt;li&gt;Fixing some issues in our CI tests stack that prevented parallelisation&lt;/li&gt;
&lt;li&gt;Reducing our CI test run time by a further 10%&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Background&lt;/h2&gt;
&lt;p&gt;The MindLink .NET CI/test stack looks like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;TFS 2018&lt;/li&gt;
&lt;li&gt;NUnit&lt;/li&gt;
&lt;li&gt;Moq&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;See the previous &lt;a href=&quot;../improving-our-ci-tests&quot;&gt;blog post&lt;/a&gt; about how we enabled test project-level parallelism and upgraded to NUnit 3.10.&lt;/p&gt;
&lt;h2&gt;Parallelising Take 2&lt;/h2&gt;
&lt;p&gt;NUnit 3 now supports running individual tests in parallel. This is at a different level of granularity to the project-level scope supported by the external test runners. We can choose which groups of tests are able to run in parallel by attaching &lt;code class=&quot;language-text&quot;&gt;[Parallelizable]&lt;/code&gt; attributes &lt;a href=&quot;https://github.com/nunit/docs/wiki/Parallelizable-Attribute&quot;&gt;at the correct level and scope&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This has some interesting implications for your tests. With project-level granularity, your tests still run in the same environment as they would do otherwise - basically just entire isolated environments running in parallel.&lt;/p&gt;
&lt;p&gt;Now, with test-level granularity &lt;em&gt;your tests&lt;/em&gt; need to make sure that they can run whilst other tests are running. This of course &lt;em&gt;should&lt;/em&gt; be the case for correctly designed Unit and in-memory component tests (and correctly designed code under test).&lt;/p&gt;
&lt;p&gt;In our previous post, project-level parallelism gave us a 2x speed-up (on a 2-processor build server). In theory, turning on test-level parallelisation isn’t going to give us any greater speed up - there are still 2 cores to perform the synchronously-running tests, regardless of who and how is scheduling them.&lt;/p&gt;
&lt;p&gt;However, we decided to look into this anyway because:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Unless the project-level distribution of the tests is &lt;em&gt;exactly equal&lt;/em&gt; across test projects, there will always be the case where there’s one test project left running after all the others have completed. At this stage, we will benefit from test-level parallelisation.&lt;/li&gt;
&lt;li&gt;Developers running the tests individually from Resharper/Visual Studio will benefit from the test-level parallelisation.&lt;/li&gt;
&lt;li&gt;We &lt;em&gt;should&lt;/em&gt; be able to turn on parallelisation - if our tests are written correctly - so it’s an interesting exercise to try.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Turning on Parrallelisation&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;At this point in the blog I have typed the word “parallel” so many times I’m forgetting how to spell it.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;We need to decide at what level we can enable parallelisation. We have a couple of options, based on where we apply the &lt;code class=&quot;language-text&quot;&gt;[Parallelizable]&lt;/code&gt; attribute, and what scope we define.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Per-test - all tests can run parallel with each other&lt;/li&gt;
&lt;li&gt;Per-test-fixture - all test fixtures can run parallel with each other, but tests inside run sequentially&lt;/li&gt;
&lt;li&gt;Per-namespace - tests in different namespaces can be run in parallel, but tests inside the same namespace run sequentially&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The answer to the above depends on how you’ve written your tests. I think we’re in the same boat as most NUnit teams in that our test fixtures have state that is reset in each &lt;code class=&quot;language-text&quot;&gt;[SetUp]&lt;/code&gt; method (i.e. we have private fields in each test fixture class). On the up-side, each test depends on that state and that state alone.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-c#&quot;&gt;&lt;code class=&quot;language-c#&quot;&gt;[TestFixture]
public sealed class MyTestFixture
{
    private Mock&amp;lt;IMyDependentInterface&amp;gt; mockMyDependentInterface;

    private MyClass myClass;

    [SetUp]
    public void SetUp()
    {
        this.mockMyDependentInterface = new Mock&amp;lt;IMyDependentInterface&amp;gt;();

        this.myClass = new MyClass(this.mockMyDependentInterface.Object);
    }

    [Test]
    public void DoingSomethingWhenSomethingDoesSomething()
    {
        this.mockMyDependentInterface.Setup(...);

        Assert.That(
            this.myClass.DoSomething(),
            Is.EqualTo(Expected));
    }
}&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;I should point out that the goal with this work is to enable parallelization:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;With the minimum amount of work/changes/risk possible.&lt;/li&gt;
&lt;li&gt;Zero changes to the actual code (i.e. outside of the tests).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If we can’t achieve parallelization without either of the above being true, then we’ll abandon the exercise.&lt;/p&gt;
&lt;p&gt;This leads me to the conclusion that we can run all test-fixtures in parallel, but tests inside each fixture must be run in sequence. This isn’t to say we can’t change individual test fixtures - or write new fixtures - to support per-test parallelization in future.&lt;/p&gt;
&lt;p&gt;So to do this, I apply the &lt;code class=&quot;language-text&quot;&gt;[Parallelizable]&lt;/code&gt; to each of our test projects, at the assembly level. The Scope parameter is instructing that each test-fixture can be run in parallel, but no two tests inside each fixture can run at the same time.&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;[assembly: Parallelizable(ParallelScope.Fixtures)]&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;And low-and-behold, when I run chunks of test fixtures in Resharper, I can actually see multiple fixtures running in parallel.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/parallel-running-00849fbe03616d34da4f76653ea0a107-6eb6a.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 547px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 15.904936014625228%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAADCAIAAAAcOLh5AAAACXBIWXMAAA7DAAAOwwHHb6hkAAAAlUlEQVQI12XJ0QqCMACF4b3/W0QYFHhTe4O60ASRXM41ajO3JJeFbpYVYZE3gvDdnP8AuFVwI8drNHOotYytFbE9NnWZ7YvftUB5j5qH2cgJJh6C/wLad/M0JT+T/JSIPc3lUWVC31T7egw0n0YHzA0TvyvA1PdLcQ136MBYjEkUYcaTVEhVFJU22tR9ZaUjjklKu/kFxN+eas7S9E4AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Test Platform Settings&quot;
        title=&quot;&quot;
        src=&quot;/static/parallel-running-00849fbe03616d34da4f76653ea0a107-6eb6a.png&quot;
        srcset=&quot;/static/parallel-running-00849fbe03616d34da4f76653ea0a107-d1e32.png 148w,
/static/parallel-running-00849fbe03616d34da4f76653ea0a107-c5651.png 295w,
/static/parallel-running-00849fbe03616d34da4f76653ea0a107-6eb6a.png 547w&quot;
        sizes=&quot;(max-width: 547px) 100vw, 547px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;So with that, I push and that’s it!&lt;/p&gt;
&lt;h2&gt;Problems&lt;/h2&gt;
&lt;p&gt;Except it never is.&lt;/p&gt;
&lt;h3&gt;Problem Number 1 - Static Variables&lt;/h3&gt;
&lt;p&gt;The first thing that happens is 7 tests in the same namespace fail with the same error:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-82ab5.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 25.832223701731028%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAFCAYAAABFA8wzAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAAx0lEQVQY03WOCw+CMAyE9///nG9NVF5KkIewAQKGwc52oDEmNvlyt6bdVWitofWAvu9hjMG7vv2/MmbEOA5fjBD5/YYodBC4B2TJBaqIUaoUSsaQeTy9ZUK9hHoT/JZ5REd0n/A3onmUNHwH67OrrXZtTVQ/WqNtZt+xL+1FvyVUkSFNQkrO5oUJXmjZN9Wn/5w/mqgoXOFRF6TSKoeJNPbhuxtcvB288wq+s7a4pyU8qyvruRfQHCvPOceFVSYM9rj6W9zCM15Ko31p80vLQwAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Nullreference Exception&quot;
        title=&quot;&quot;
        src=&quot;/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-fb8a0.png&quot;
        srcset=&quot;/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-1a291.png 148w,
/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-2bc4a.png 295w,
/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-fb8a0.png 590w,
/static/nullreference-exception-1-233509de7469615d5ed114cd00fe2c85-82ab5.png 751w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;I then come to the realisation: The tests that have failed are testing what can perhaps be described as &lt;em&gt;not the most up-to-date&lt;/em&gt; part of our codebase. In fact, there’s a particular set of static variables that the code under test relies apon.&lt;/p&gt;
&lt;p&gt;In real life, these variables get initialized when the application starts up. In the test world, each test fixture initializes and tears down these static variables - and worse, each test fixture initializes the variables with &lt;em&gt;slightly different values&lt;/em&gt;. &lt;/p&gt;
&lt;p&gt;Because the fixtures are now running concurrently, they’re overwriting the variables and it’s causing havoc with the code under test, and the mocked dependencies.&lt;/p&gt;
&lt;p&gt;I have a couple of options to fix this - re-write the code under test to avoid the static variables, or fix the way the variables are initialized during the test runs. Removing the static variables breaks our rule that we shouldn’t be making any changes to the actual code. We’ll do this eventually at some point no doubt, but for now the least risk is to lift the initialization of these variables out of the test fixtures themselves. &lt;/p&gt;
&lt;p&gt;NUnit has a &lt;code class=&quot;language-text&quot;&gt;[OneTimeSetUp]&lt;/code&gt; attribute that denotes a method used to initialize test-fixtures across a whole namespace. We already use this mechanism to alleviate the boilerplate set up of some our “cross-cutting” concerns, like logging. Introducing a new method just for this troublesome namespace allows me to lift the initialization of these static variables into one shared place.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-c#&quot;&gt;&lt;code class=&quot;language-c#&quot;&gt;[OneTimeSetUp]
public override void TestFixtureSetUp
{
    base.TestFixtureSetUp();

    ChannelService.ChatConnectionChatRoomPrefix = DummyChatConnectionChatRoomPrefix;
    ChannelService.ChatConnectionUserPrefix = DummyChatConnectionUserPrefix;
}&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;I also remove the equivalent code from each of the offending child fixtures, and update the tests/mocks to understand these new constant values. If anything, this does more closely mirror how the code runs in real life.&lt;/p&gt;
&lt;h3&gt;Problem Number 2 - Cross-Cutting Dependencies&lt;/h3&gt;
&lt;p&gt;So I push the previous fix and all of a sudden more problems start occurring.&lt;/p&gt;
&lt;p&gt;In particular, loads of errors like the below:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-e402a.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 25.36945812807882%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAFCAYAAABFA8wzAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAA0UlEQVQY021R2Q6CMBDs//+XD+LxYkQBNR4gprSggLcw7i4YSbTJpLszk91Oqs5lgWNmkVqN++2K1/OBp+De4tHBt6+rCnVd/0DlJwN92EDHaxi9hU1CZHYPa0KkJhJwL1yrpaTxcD48pHvU5VIiz4+y9WNoTN+t6HCMqn0d/vhUtPPhTnoI5kP4MwdLb4SlP8KCbu5914E37Tc61QtvSPoYwWxA+gCrYNz6+titXaiyyCgCRzpQTIpoY6obcG2TvegmiaS2hvtYwJzwoof0Fxpvmeh65CzcvB8AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Nullreference Exception&quot;
        title=&quot;&quot;
        src=&quot;/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-fb8a0.png&quot;
        srcset=&quot;/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-1a291.png 148w,
/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-2bc4a.png 295w,
/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-fb8a0.png 590w,
/static/nullreference-exception-2-b08f6ad7278e791b3b3da23b5cda4768-e402a.png 812w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;I come to the second realisation - we have cross-cutting behaviour that’s being mocked out individually in each test fixture. The test fixtures are again overwriting each other’s attempts at this global mocking.&lt;/p&gt;
&lt;p&gt;Just to take a step back for a second:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Our code models certain cross-cutting concerns via a “service locator” pattern&lt;/li&gt;
&lt;li&gt;The “service locator” is a global static instance. The locator’s implementation can vary, but it’s initialized when the application starts and you just always presume it’s there, and that you can fetch various services from it. So for example I can retrieve an ICatService implementation by going &lt;code class=&quot;language-text&quot;&gt;var myCatService = ServiceLocator.Instance.GetService&amp;lt;ICatService&amp;gt;()&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;This is about as awkward as it sounds. We have concrete dependencies on the ServiceLocator.Instance. We have no real control over which instances end up with references to which service implementation.&lt;/li&gt;
&lt;li&gt;We’ve generally removed this pattern, but it is still used in a number of places for obtaining access to our &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; instance. This is pretty much used by anything that wants to schedule a callback - immediately on the threadpool, or as a timed callback etc.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;So we need to make sure this mechanism works when running our tests. This means in each test setup we have to:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Set the &lt;code class=&quot;language-text&quot;&gt;ServiceLocator.Instance&lt;/code&gt; guy as a stubbed &lt;code class=&quot;language-text&quot;&gt;IServiceLocator&lt;/code&gt; implementation that’ll let us register a &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; instance for retrieval by the code under test&lt;/li&gt;
&lt;li&gt;Register a mock &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; with the stub service locator. We typically set up the &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt;’s mocked behaviour as part of this (for instance, simulating the callback). The code under test will then retrieve this instance from the service locator instance.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;You can probably see where the problem is - test fixtures are competing for the mocking of the service locator instance.&lt;/p&gt;
&lt;p&gt;Unfortunately this pattern is in a lot of places in the code, and it’s just not feasible to refactor away from this pattern in one fell swoop.&lt;/p&gt;
&lt;p&gt;To fix this, we somehow need to mock a static instance, but keep how we’ve mocked that instance unique in the context of each parallely running test fixture. This sounds like something thready - but it’s worse than that: we use &lt;code class=&quot;language-text&quot;&gt;async/await&lt;/code&gt; a lot in the code, so tests don’t even necessarily run on their own thread. Rather, they run in their own async execution context.&lt;/p&gt;
&lt;p&gt;At this point I give up and go to the pub.&lt;/p&gt;
&lt;hr&gt;
&lt;blockquote&gt;
&lt;p&gt;About 3 pints in and it dawns on me.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The first observation is that I need to stop each test-fixture overwriting the &lt;code class=&quot;language-text&quot;&gt;ServiceLocator.Instance&lt;/code&gt; singleton. This means we need to initialize this guy (with &lt;em&gt;something&lt;/em&gt;) once per test project.&lt;/p&gt;
&lt;p&gt;Now - what do we set the global instance as? My second observation is that whatever it is, I need a way for each test fixture to register their own &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; instance with it, and for their code-under-test to see the same instance. So there needs to be a logical affinity between the test-code registering the service instance, and the code-under-test fetching the service instance. And this affinity needs to flow across async control flow.&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;System.Threading.AsyncLocal&amp;lt;T&amp;gt;&lt;/code&gt; to the rescue!&lt;/p&gt;
&lt;p&gt;So I come up with this as my new global test fixture setup:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-c#&quot;&gt;&lt;code class=&quot;language-c#&quot;&gt;public static readonly AsyncLocal&amp;lt;ISchedulerService&amp;gt; SchedulerServiceAsyncLocal = new AsyncLocal&amp;lt;ISchedulerService&amp;gt;();

[OneTimeSetUp]
public virtual void TestFixtureSetUp()
{
    var mockServiceLocator = new Mock&amp;lt;IServiceLocator&amp;gt;();

    mockServiceLocator.Setup(sl =&amp;gt; sl.GetObjectOfType&amp;lt;ISchedulerService&amp;gt;())
        .Returns(() =&amp;gt; SchedulerServiceAsyncLocal.Value);

    ServiceLocator.Instance = mockServiceLocator.Object;
}&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;To clarify:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;I’ve created a global &lt;code class=&quot;language-text&quot;&gt;AsyncLocal&lt;/code&gt; object that all the test fixtures can see and set their own &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; instance on.&lt;/li&gt;
&lt;li&gt;I’ve created a mocked &lt;code class=&quot;language-text&quot;&gt;ServiceLocator&lt;/code&gt; object using &lt;a href=&quot;https://github.com/moq/moq4&quot;&gt;Moq&lt;/a&gt; that will serve requests for &lt;code class=&quot;language-text&quot;&gt;ISchedulerService&lt;/code&gt; instances by fetching the current value from the &lt;code class=&quot;language-text&quot;&gt;AsyncLocal&lt;/code&gt;. This mock &lt;code class=&quot;language-text&quot;&gt;ServiceLocator&lt;/code&gt; is initialized once per test project and shared across all concurrently running test fixtures.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each test in a test fixture then registers its mock instance a la:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-c#&quot;&gt;&lt;code class=&quot;language-c#&quot;&gt;var mockSchedulerService = new Mock&amp;lt;ISchedulerService&amp;gt;();
SetUpFixture.SchedulerServiceAsyncLocal.Value = mockSchedulerService.Object;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This means that:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Each test can fiddle about with their own ISchedulerService mock instance, isolated from tests in other fixtures&lt;/li&gt;
&lt;li&gt;All code under test (running in the same async execution flow) will get the same instance as the corresponding set up code registered.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;So we have the exact same static-mocking behaviour as before, but our test fixtures can run in parallel!&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;I’m actually strangely pleased with myself that I came up with this&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Concluding&lt;/h2&gt;
&lt;p&gt;So anyway, after that change all the tests started reliably passing. I don’t think the problems we found were actually too bad on a codebase of our size. Work to resolve the static references in the older bits of our codebase continues.&lt;/p&gt;
&lt;p&gt;And now our build timings:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Unit tests - 1:44 (this actually isn’t any faster than our result from the &lt;a href=&quot;../improving-our-ci-tests&quot;&gt;first blog post&lt;/a&gt;, as predicted. Though this is a couple of weeks later, and we do have a few more tests now)&lt;/li&gt;
&lt;li&gt;Behaviour tests - 1:36 (trimmed a minute off here - there’s more of an uneven distribution of tests between projects in this step)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;And running from Resharper is now faster than lightning!&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Mixin that memory leak]]></title><description><![CDATA[This is a story of how, without much effort, mixins can give you memory leaks all over your shiny stuff. TL;DR Mixins  share the properties…]]></description><link>https://engineering.mindlinksoft.commixin-that-memory-leak</link><guid isPermaLink="false">https://engineering.mindlinksoft.commixin-that-memory-leak</guid><pubDate>Thu, 05 Jul 2018 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;This is a story of how, without much effort, mixins can give you memory leaks all over your shiny stuff.&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Mixins &lt;strong&gt;share the properties with their target&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Conflicts between property names with different semantics can lead to memory leaks or undefined behaviour&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Ideally don’t use mixins&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you have to, either:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Make sure the mixin is using unique property names, or symbols as properties for ‘private’ members&lt;/li&gt;
&lt;li&gt;Make it clear when a property is accessible to a target and when it isn’t&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;The Story&lt;/h2&gt;
&lt;p&gt;Performance is important to us, we heavily optimize our clients to get the most out of the browser for fast rendering, we also regularly perform memory profiling to ensure we don’t leak all over the place.&lt;/p&gt;
&lt;p&gt;After introducing a new implementation of our rich text input (moving away from DraftJS on mobile devices because &lt;a href=&quot;/why-draftjs-doesnt-work-on-android&quot;&gt;it doesn’t work properly&lt;/a&gt;) I did a quick memory profiling session over it and found just what I hoped I wouldn’t - a leak all over our nice new input.&lt;/p&gt;
&lt;p&gt;The problem is that leak wasn’t in the new code, it was old… We missed it. We need to get better at making sure we don’t miss it in future, that’s a journey we’re taking.&lt;/p&gt;
&lt;p&gt;The memory snapshot showing the leak looks like this:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-0a6eb.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 42.90865384615385%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAJABQDASIAAhEBAxEB/8QAGAAAAgMAAAAAAAAAAAAAAAAAAAECAwX/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAAB2m5JUMP/xAAXEAADAQAAAAAAAAAAAAAAAAAAARAR/9oACAEBAAEFAsMjjP/EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8BP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8BP//EABYQAAMAAAAAAAAAAAAAAAAAAAAg4f/aAAgBAQAGPwIq/wD/xAAbEAACAgMBAAAAAAAAAAAAAAAAATFBEVGRcf/aAAgBAQABPyFe+mOg0k76SZRM/9oADAMBAAIAAwAAABDU/wD/xAAWEQADAAAAAAAAAAAAAAAAAAAAARH/2gAIAQMBAT8QiIj/xAAXEQADAQAAAAAAAAAAAAAAAAAAAREx/9oACAECAQE/EK8Kz//EABwQAAMAAQUAAAAAAAAAAAAAAAABEZEQMVGBsf/aAAgBAQABPxBVmFj5uRMRO+gW08T/2Q==&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;The leak itself&quot;
        title=&quot;&quot;
        src=&quot;/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-f8fb9.jpg&quot;
        srcset=&quot;/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-e8976.jpg 148w,
/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-63df2.jpg 295w,
/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-f8fb9.jpg 590w,
/static/the-leak-b24b251c7680bea9bcd83642a9000b7a-0a6eb.jpg 832w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;That event handler &lt;code class=&quot;language-text&quot;&gt;&amp;quot;interactivitychanged&amp;quot;&lt;/code&gt; on the &lt;code class=&quot;language-text&quot;&gt;InteractivityMonitor&lt;/code&gt; shouldn’t be there, it should have been removed as part of the destruction of the view model.&lt;/p&gt;
&lt;p&gt;My first instinct was that this must be because we’re just not destroying the view model at all. A quick debug later, nope that’s not it.&lt;/p&gt;
&lt;p&gt;Next on the list is that we may be destroying it, but is that actually cleaning up the event handlers? Yes, yes it is…&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-typescript&quot;&gt;&lt;code class=&quot;language-typescript&quot;&gt;&lt;span class=&quot;token comment&quot;&gt;/**
 * Destroys the interactable mixin.
 */&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;destroyInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;un&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;interactivitychanged&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;handleInteractivityMonitorChanged&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;token comment&quot;&gt;// &amp;lt;-- Yupp we&apos;re removing the listener.&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;destroy&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;So what’s going on? The debug session showed that the &lt;code class=&quot;language-text&quot;&gt;InteractivityMonitor&lt;/code&gt; that we’re cleaning up has no registered handlers, but the memory profiler is definitive in its report that there is one there…&lt;/p&gt;
&lt;p&gt;That means we’re looking at different instances of the same type and so the answer lies in who we add the listener to and remove the listener from.&lt;/p&gt;
&lt;p&gt;Let’s start from the leaky view model:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-typescript&quot;&gt;&lt;code class=&quot;language-typescript&quot;&gt;&lt;span class=&quot;token comment&quot;&gt;/**
 * Represents an interface for the constructor configuration.
 */&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;interface&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;IComposeMessageInputViewModelConfig&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token comment&quot;&gt;/**
     * The interactivity monitor.
     */&lt;/span&gt;
    interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; FcfJS&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;viewmodel&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;AbstractInteractivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token comment&quot;&gt;/** -snip- **/&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token comment&quot;&gt;/**
 * Represents the view model for a compose message input view.
 */&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;ComposeMessageInputViewModel&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;extends&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;FcfJS&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;viewmodel&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;MessageInputViewModel&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token comment&quot;&gt;/**
     * Initializes the interactable mixin.
     *
     * @param config The configuration object.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; initializeInteractable&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;config&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; IInteractableConfig&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&gt;&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Destroys the interactable mixin.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; destroyInteractable&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&gt;&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/** -snip- **/&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * The interactivity monitor.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; FcfJS&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;viewmodel&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;AbstractInteractivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Initializes a new instance of this class.
     *
     * @param config The initial configuration.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;constructor&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;config&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; IComposeMessageInputViewModelConfig&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;super&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        &lt;span class=&quot;token comment&quot;&gt;/** -snip- **/&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;initializeInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; config&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Destroys this instance.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;destroy&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token comment&quot;&gt;/** -snip- **/&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getInteractivityChangedEvent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;un&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        
        &lt;span class=&quot;token keyword&quot;&gt;super&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;destroy&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;destroyInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/** -snip- **/&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Updates the interactivity monitor following a change in active conversation.
     *
     * @param interactivityMonitor The interactivity monitor for the conversation.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;updateInteractivityMonitor&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; AbstractInteractivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getInteractivityChangedEvent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;un&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
            &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;destroy&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getInteractivityChangedEvent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;on&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;handleMessageSendabilityChanged&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token function&quot;&gt;mixin&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ComposeMessageInputViewModel&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; Interactable&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;default&lt;/span&gt; ComposeMessageInputViewModel&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;We see that he mixes in an &lt;code class=&quot;language-text&quot;&gt;Interactable&lt;/code&gt;. The view model himself adds and removes a listener symmetrically, so that’s fine. He does have an interactivity monitor and through his lifetime changes it based on the conversation it represents, but that’s used for something else, not listening on the leaky event.&lt;/p&gt;
&lt;p&gt;Enter the &lt;code class=&quot;language-text&quot;&gt;Interactable&lt;/code&gt; mixin.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-typescript&quot;&gt;&lt;code class=&quot;language-typescript&quot;&gt;&lt;span class=&quot;token comment&quot;&gt;/**
 * Represents an interface for the constructor configuration.
 */&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;interface&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;IInteractableConfig&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token comment&quot;&gt;/**
     * The interactivity monitor.
     */&lt;/span&gt;
    interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; AbstractInteractivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token comment&quot;&gt;/**
 * Represents a simple mixin that determines whether the view model is interactable.
 */&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;default&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;Interactable&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token comment&quot;&gt;/**
     * The interactivity monitor.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; AbstractInteractivityMonitor &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;null&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Sets whether the property is interactable or not.
     *
     * @param name The property name.
     * @param value Whether the property is interactable or not.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; setProperty&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;name&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;any&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; value&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;any&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&gt;&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Gets whether the property is interactable or not.
     *
     * @param name The property name.
     * @returns `true` if the property is interactable, `false` otherwise.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; getProperty&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;name&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;any&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&gt;&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;any&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Add properties.
     *
     * @param name The property name.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; addProperties&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;name&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;any&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&gt;&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Initializes the interactable mixin.
     *
     * @param config The initial configuration.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;initializeInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;config&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; IInteractableConfig&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;addProperties&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            isInteractable&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;false&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; config&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;on&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;interactivitychanged&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;handleInteractivityMonitorChanged&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;setIsInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;isInteractive&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Destroys the interactable mixin.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;destroyInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;un&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;interactivitychanged&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;handleInteractivityMonitorChanged&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;interactivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;destroy&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Handles when the interactable state changes.
     *
     * @param _ The new interactable state.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;protected&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;handleInteractableChanged&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;_&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;boolean&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Handles when the interactable state of the monitor changes.
     *
     * @param _ The interactivity monitor.
     * @param isInteractive The new interactable state.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;protected&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;handleInteractivityMonitorChanged&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;_&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; AbstractInteractivityMonitor&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; isInteractive&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;boolean&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;setIsInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;isInteractive&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Gets a value indicating whether the view model is interactable.
     *
     * @returns `true` if the view model is interactable, `false` otherwise.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;public&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;getIsInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;boolean&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getProperty&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;isInteractable&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;token comment&quot;&gt;/**
     * Sets whether the view model is interactable.
     *
     * @param isInteractable Whether the view model is interactable.
     */&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;private&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;setIsInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;isInteractable&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token builtin&quot;&gt;boolean&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getIsInteractable&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;===&lt;/span&gt; isInteractable&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;setProperty&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;isInteractable&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; isInteractable&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;handleInteractableChanged&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;isInteractable&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Looking at him he seems all fine and dandy - initializing adds the listener, destroying removes it. Great, no problem here…&lt;/p&gt;
&lt;p&gt;…&lt;/p&gt;
&lt;p&gt;Hold on… What’s that right there? Can you see it? The damn interactivity monitor property has the same name in the mixin as in the view model! That’s a problem.&lt;/p&gt;
&lt;p&gt;When we mix something in what we’re actually doing is &lt;em&gt;copying&lt;/em&gt; implementation from the mixin to the target. There’s only a single instance with both the mixin and target properties, that means that things with the same name in both types refer to &lt;strong&gt;the same thing&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;What’s happening here is that the &lt;code class=&quot;language-text&quot;&gt;Interactable&lt;/code&gt; mixin is adding the event listener to the initial instance of the interactivity monitor, then during its lifetime the view model is assigning to itself a new interactivity monitor. By the time the &lt;code class=&quot;language-text&quot;&gt;Interactable&lt;/code&gt; is asked to clean up, the shared property is referring to a different instance and the event listener is still on the old instance that we no longer have a reference to! LEAK! We have found you and now we have to patch you up!&lt;/p&gt;
&lt;p&gt;The proper way to do this is to stop using mixins, they really aren’t great for a number of reasons - this is just one of them. However, that’s quite a big change that we will schedule for another time. For now we can avoid this issue by renaming the mixin property so that it’s unique (though there’s no guarantee unless we use symbols for properties).&lt;/p&gt;
&lt;p&gt;With the property renamed the leak is gone :)&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Improving our CI tests]]></title><description><![CDATA[This is the story of how we: Enabled parallel NUnit test execution Migrated our CI test stack to a reproduceable and self-contained…]]></description><link>https://engineering.mindlinksoft.comimproving-our-ci-tests</link><guid isPermaLink="false">https://engineering.mindlinksoft.comimproving-our-ci-tests</guid><pubDate>Thu, 28 Jun 2018 13:10:00 GMT</pubDate><content:encoded>&lt;p&gt;This is the story of how we:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Enabled parallel NUnit test execution&lt;/li&gt;
&lt;li&gt;Migrated our CI test stack to a reproduceable and self-contained configuration&lt;/li&gt;
&lt;li&gt;Reduced our CI test run time by 65%&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Background&lt;/h2&gt;
&lt;p&gt;The MindLink .NET CI/test stack looks like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;TFS 2018&lt;/li&gt;
&lt;li&gt;NUnit&lt;/li&gt;
&lt;li&gt;Moq&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For our main product repository, we have a CI build that runs on every server-side change. This builds our .NET components and then runs our tests. The tests are split between what we call “Unit Tests” and “Behaviour Tests”. The “Behaviour Tests” are still isolated in-memory tests, but run over chunky bits of the stack across component-level test seams. I’ll argue with you about what these should be called another time.&lt;/p&gt;
&lt;p&gt;Our timings looked like this:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Unit Tests - 18982 tests - 7:56 minutes&lt;/li&gt;
&lt;li&gt;Behaviour Tests - 1769 tests - 3:34 minutes&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-dd63b.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 44.60616438356164%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABZ0lEQVQoz32SSU/DMBCF8+c5oQIS914ocODImQtLBYfyAxASCKmoSZzFzub9MXZKoVB1pKdxRvGXNzNJ2r6HlAbGuCjvHUTLwaocrE6RsiWy8hMFz1CKHFXDUIgMjKfrc45WcpKIShwBBqWgtEWIZqgxvT3A1dMprhdTzO4nOLs7xMX8GLOHCc6D5kdRoRby5ePJRkmASAK2nYaxHorOi9cbvLw/I199IFu9ISVVxRI1OQ3qyFnXFlF9zGzMVE88fHQoWk3ZwjkPBLOU4nlPOEdj0hrWGCgpYy1BBErUQqIhl0obNE1FX20ghz5e8t7/UwitaVR0d+g7sHxFOzBIrLPo+hE20HJGYI2ioGETNFwKsQtojI6QgT5csCw6TRRZblpFcxyX4kFbFhUEr2n7EtbaDfA7fhzqCN0Cbr0U5kZb57xEXZWxnf1AFaEByLJ1y3/bCP9h37fUNt8J+/1srYmQMOcAD/Uv6TqwtOYiPtMAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Build Summary&quot;
        title=&quot;&quot;
        src=&quot;/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-fb8a0.png&quot;
        srcset=&quot;/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-1a291.png 148w,
/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-2bc4a.png 295w,
/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-fb8a0.png 590w,
/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-526de.png 885w,
/static/build-summary-initial-f6ac8293b032f89d16e6b8ece138dcc4-dd63b.png 1168w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;Whilst I’m sure there are definitely worse build times out there, a total time of 16 minutes definitely has room for improvement.&lt;/p&gt;
&lt;p&gt;And yes, we can no doubt improve the 4:30 time for the core build, but that’s a different story.&lt;/p&gt;
&lt;h2&gt;Parallelising Take 1&lt;/h2&gt;
&lt;p&gt;Time to look at our test run configuration.&lt;/p&gt;
&lt;p&gt;I was aware that the TFS Test runner will run tests in parallel - at the granularity of each test DLL. Our build servers have 2 processor cores. My presumption was that we could just enable parallelisation on our test task config, the test runner would farm off our 8 or so test DLLs to run in parallel (2 at a time), and we’d get a 2x speed up.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-dfab2.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 63.0272952853598%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABB0lEQVQ4y51T2W6DMBD0//9iH9smTQMYX2vwNdk1tJX6BKw0soSsYY61ouAxjU/GAGM0SimQaa0x6n4eh6K4wloHIo/gLSgEJs2ote7EODXKU4KxFusSESNBFMe4kV4ZtS4BM1smCqzQIackulhhuQSV0orAqn6ykxG7TdDOQ0l237d3jI87nB7h5qnbvQq1cG5CNj8fmL5uGO6fiOSvE4rv9q9Ksfxr+yS4lKWvTORS5JSWJdcecrlSyroTsvVOKoT8bVO5XZLCjkK9fXgMk+25ee92la6vkHO2/0gu1oNQ2niM8uxmDS0tW4OFl1xIcs5d4anFLjltitiqZ0WiLPXl/pszb/kFKSz/MeElCzAAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Original Test Settings&quot;
        title=&quot;&quot;
        src=&quot;/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-fb8a0.png&quot;
        srcset=&quot;/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-1a291.png 148w,
/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-2bc4a.png 295w,
/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-fb8a0.png 590w,
/static/original-test-settings-801a7cd9d090f2f3f9dc34883e194046-dfab2.png 806w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;To my surprise/dismay - the “Run tests in parallel” box was already checked. So why are the tests still so slow? Is the parallelism not actually working?&lt;/p&gt;
&lt;p&gt;I unchecked the box and queued another run. Same test timings. So what’s going on?&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://blogs.msdn.microsoft.com/devops/2016/10/10/parallel-test-execution/&quot;&gt;This Blog&lt;/a&gt; helped me on my way (sort of). The parallel test run behaviour is a convoluted function of both the TFS GUI settings and the test .runsettings file. &lt;/p&gt;
&lt;p&gt;Our .runsettings looked like&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;&amp;lt;MaxCpuCount&amp;gt;0&amp;lt;/MaxCpuCount&amp;gt;&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;which apparently enables parallelisation by default.&lt;/p&gt;
&lt;p&gt;Removing the .runsettings file from the test config altogether gave me what I was expecting - The unit tests alone now took 13 minutes (just under twice as long as before).&lt;/p&gt;
&lt;p&gt;In hindsight, I could observe the tests running in parallel as the test runner launched. This is the log with parallelisation enabled:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-6b4b8.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 19.06614785992218%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAA0klEQVQY0yWQ6W6DMBCEef/nq5qAbXxgCLSlHDZ2EyGmu86P0R6Wvpl1pXUDZyWcUeicRt9bjA9P6jBQ//M9YltnHMeOnAJSivjLR1HOEc9nwnWdAK5SK6XuaNsaWtUEVnCuxWPoinyn4b0p8N/5CyGsOOKOGDYkMmA4zzkxOL+BBUYyWqD3FtPYk0EDqyWskbQzWCkhAxgUtqWAw75gXWbEuBUY6zxfqIT4hJI3GIZYStc7SHGDqN/7pv6geifTplyh6I37aRoKONE38Omc8kXQf4dVKtbXMrUbAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Parallel Test Execution&quot;
        title=&quot;&quot;
        src=&quot;/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-fb8a0.png&quot;
        srcset=&quot;/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-1a291.png 148w,
/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-2bc4a.png 295w,
/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-fb8a0.png 590w,
/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-526de.png 885w,
/static/test-execution-parallel-632cd4fb77c20ee0dceb4f5a20b32343-6b4b8.png 1028w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;And this is the log with parallelisation disabled:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-cccd3.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 19.414634146341463%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAAyElEQVQY01WQ226DMBBE+f8fbDC+hkvcJE2aYggUwqntplX7MNrRSjtztIU1FU1rsUayt5K6Nviu5hDVRH86dlyvZ0K4MYQP7mNgHPo8p2n49fN8Bx4U1giaxmC0wNmKrnXRV7nAWZX3qfTt7BnHnnWZWZYp68enua6fbNtCYXQZqTRK7mKAxB8ahHhBVru4E2gl8L6LB9/HieKftjUHZT1ioHOK9klV7w3H1xYbqbQsUap8UpaZMr3gcjkR+ht9/57fkMj+FnwByoQs6eHe7PwAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Sequential Test Execution&quot;
        title=&quot;&quot;
        src=&quot;/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-fb8a0.png&quot;
        srcset=&quot;/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-1a291.png 148w,
/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-2bc4a.png 295w,
/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-fb8a0.png 590w,
/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-526de.png 885w,
/static/test-execution-sequential-ce046a022d9acebe8c8739111dc3a0ac-cccd3.png 1025w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;You can see it processing two DLL test containers at a time on test launch, and then subsequent test result logs overlap.&lt;/p&gt;
&lt;p&gt;So - good news is: I’ve got to the bottom of what’s making the tests run in parallel. Bad news is: our tests runs are taking a long time even though they’re already running in parallel.&lt;/p&gt;
&lt;h2&gt;Time for some housekeeping&lt;/h2&gt;
&lt;p&gt;All this talk of test configuration got me thinking about our build dependencies.&lt;/p&gt;
&lt;p&gt;We’re on the same path as a lot of teams with this one: we have several build servers that each have some global build dependencies installed. This means:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When we make changes to the build, we need to make the same changes across all build servers&lt;/li&gt;
&lt;li&gt;If we upgrade a build dependency for one branch, typically all the other branches start failing to build&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Our end goal however is:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Remove as many global build agent dependencies as possible&lt;/li&gt;
&lt;li&gt;Define all build dependencies in source code as versioned package dependencies&lt;/li&gt;
&lt;li&gt;Dynamically install and configure the build environment as part of the build itself&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;So couple of things to fix with our test configurations:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;.runSettings file&lt;/strong&gt; - This guy was defined at a well known path on the build server, and had to be kept in sync between all servers. He seems to have been superceded by the TFS test task configuration, and he wasn’t doing much anyway (other than making things more confusing). So time for him to go. Parallel execution of tests is enabled by the TFS task config.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Visual Studio Test Platform&lt;/strong&gt; - This was installed as part of Visual Studio on the build server. Ultimately we want to reduce our dependency on needing Visual Studio on the build server, and let the agents upgrade themselves as required. We switched this to be installed by the TFS build agent. &lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-7492f.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 15.254237288135593%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAADCAYAAACTWi8uAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAASklEQVQI152OyQ3AMAzDsv+0dXwV9aEm3aARwKcIDpqKOQkqDDNBxIPuxumGiIK3UBnuhsz8hKeM23fhBWGCm6KrUEta9ZP1yQy8Vp3sy6GLE94AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Test Platform Settings&quot;
        title=&quot;&quot;
        src=&quot;/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-fb8a0.png&quot;
        srcset=&quot;/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-1a291.png 148w,
/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-2bc4a.png 295w,
/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-fb8a0.png 590w,
/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-526de.png 885w,
/static/test-platform-settings-7aeaadfbf3558d7adba6434515e0f183-7492f.png 1003w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;This means adding another bootstrap step in the build tasks flow. In practice this task caches the installation between runs, so little extra overhead is added to the build process.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-02305.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 15.062111801242233%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAADCAYAAACTWi8uAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAAkElEQVQI14XPOQrCQABA0dz/LAGvYC0KCqLZTEKSWbJvM4FU38FasXjth++1okWLmixPKasSpRVSSYQzr+vHYsxf/ThgrMFLzgWn4xX/4HO7P1DtQCE1pWpohpm6n5jM/tPoLNtOPy0uvOFlUUMUK4JMEqaS4CWIc80zKUmKBtltVK1x7FeqtyRi5RJ27sbyBnWz4CLUpDyrAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Test Platform Installer Task&quot;
        title=&quot;&quot;
        src=&quot;/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-fb8a0.png&quot;
        srcset=&quot;/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-1a291.png 148w,
/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-2bc4a.png 295w,
/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-fb8a0.png 590w,
/static/test-platform-installer-task-425b1c27823ceeb1ddce7d6b4a86f188-02305.png 644w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;&lt;strong&gt;NUnit Test Adaptors&lt;/strong&gt; - We had these installed globally on each build server using the Visual Studio extension installer. I think this was the right decision when we intially set up the build servers, but the installing-as-nuget-dependency seems like the right way for us to go now. This will give us one less global dependency.&lt;/p&gt;
&lt;p&gt;To do this, you just need to add the adaptor package to your test project’s NuGet package dependencies. Something does feel a little odd about adding another dependency &lt;em&gt;just because the build server needs it&lt;/em&gt;, but that’s ultimately the shift in thinking towards self-describing builds.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-de96f.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 4.058441558441558%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAABCAYAAADeko4lAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAARUlEQVQI1wXBAQqAIBAAwf7/SvHy0DMElRJK3WaO5hyfZSw8SIAUNyIb7+FuAy5jxUSxgUYYpcMpLFUQz0yZbBPVTa0vP/jQS+DoIDiuAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;NUnit Test Nuget Package&quot;
        title=&quot;&quot;
        src=&quot;/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-fb8a0.png&quot;
        srcset=&quot;/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-1a291.png 148w,
/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-2bc4a.png 295w,
/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-fb8a0.png 590w,
/static/nunit-test-adaptor-nuget-725b45e7859bf262a5a083a41b902afd-de96f.png 616w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;I’m also not in love with the fact that you technically only need to do this for &lt;em&gt;one project in the solution&lt;/em&gt; (I assume just so that when NuGet solution-restores everything it’s present in the packages folder), but I’ll let that one go for simplicity.&lt;/p&gt;
&lt;h2&gt;Upgrading NUnit&lt;/h2&gt;
&lt;p&gt;At this point I also decided to upgrade our test dependencies. We were using NUnit 3.6.0 at this stage. I updated our NuGet package config to bump this to the latest 3.10.0 libraries.&lt;/p&gt;
&lt;p&gt;This did cause a couple of compilation errors, but nothing that wasn’t easily fixed with equivalent NUnit API constructs. I pushed and waited (16 minutes) for the build to finish.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;But the build only took 9 minutes&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-4adfa.png&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 35.762331838565025%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAAA7DAAAOwwHHb6hkAAABEElEQVQoz4VRuU7EMBT0N/IFnH8ADTSI5Q8o4BfouCQatqahRFuDFljIsZvEd5xkeHY2RwFaS6PxGz+Nx8/s8HoLF09HuJwe42p6grPbXZzebBN2cH5/QNgPPCGe3O1tBHuePeDl9RHx/A3x5ztWiw+kX3MU0Td0lvaolQS0QrMGjAr1oMnAzFkLazWUNtAk2NLAVSWcsyhp38Fr1QglnfseV9lWq11gJmQejJqmwabVddR1Dc7zlosccbSAMSacMakKCE4QnG70yVww/w+doTE67JUUWC2TvmZcZCSkSOIfpElEI1FtmpFBn3Bde5Zk5JfgbULlZ+wNleIhoW8YG/w1gsFwSKjJKM+W9A/tk38B4DEI8UM5hqMAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Improved Build Time&quot;
        title=&quot;&quot;
        src=&quot;/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-fb8a0.png&quot;
        srcset=&quot;/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-1a291.png 148w,
/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-2bc4a.png 295w,
/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-fb8a0.png 590w,
/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-526de.png 885w,
/static/improved-build-time-43e45d56121fbcdf3e40bf8e4fc4162b-4adfa.png 892w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;That’s right. The Unit tests now ran in 100 seconds, the Behaviour tests down to 2 minutes. You can see the difference between the build times from the build summary graph (the first two builds are before we upgraded NUnit).&lt;/p&gt;
&lt;p&gt;After some digging, I &lt;em&gt;think&lt;/em&gt; it’s &lt;a href=&quot;https://github.com/nunit/nunit/issues/2217&quot;&gt;this issue&lt;/a&gt; that was the culprit. Indeed, even when I run from Resharper in Visual Studio, the tests now run lightning fast.&lt;/p&gt;
&lt;h2&gt;Concluding&lt;/h2&gt;
&lt;p&gt;I didn’t expect this investigation to turn out like this - our problems were solved just by upgrading our NUnit dependency.&lt;/p&gt;
&lt;p&gt;On the other hand, we did clean up our test configuration and dependencies, and double-checked that we’re running our tests in parallel.&lt;/p&gt;
&lt;p&gt;Moral of the story is:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Make sure you’re on at least NUnit 3.8 so your tests run correctly!&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Next time: Further test parallelisation&lt;/p&gt;</content:encoded></item><item><title><![CDATA[End-To-End Encryption with the Signal Protocol]]></title><description><![CDATA[What is the Signal Protocol? The Signal Protocol (formerly TextSecure Protocol) defines a cryptographic encryption protocol for secure end…]]></description><link>https://engineering.mindlinksoft.comend-to-end-encryption-with-the-signal-protocol</link><guid isPermaLink="false">https://engineering.mindlinksoft.comend-to-end-encryption-with-the-signal-protocol</guid><pubDate>Wed, 20 Jun 2018 10:15:00 GMT</pubDate><content:encoded>&lt;h2&gt;What is the Signal Protocol?&lt;/h2&gt;
&lt;p&gt;The Signal Protocol (formerly TextSecure Protocol) defines a cryptographic encryption protocol for secure end-to-end encryption (E2EE). It was created by Open Whisper Systems for use in their secure messaging app TextSecure but has since been released, open-source, to the wider community. It has become hugely popular amongst consumer applications and has earned its name as the de facto standard for securing messaging and voice and video communications. Some of the largest adopters include Google Allo, Facebook Messenger and WhatsApp, but more and more organizations are embracing E2EE everyday as concerns surrounding privacy, security and surveillance increasingly find themselves centre-stage in the public eye.&lt;/p&gt;
&lt;p&gt;In simple terms, E2EE allows us to ensure our data is only visibile to its intended recipient; the data is encrypted on the client on the user’s own device before it is even sent over the wire. This means that all the infrastructure in the middle (routing, messaging servers, databases etc.) will never get to see your data in its raw form! And this is great — it means we can completely eliminate the element of trust between client and server so users no longer have to worry about whether their data is truly safe in the hands of their messaging provider. More importantly though, it means that any third party trying to intercept a user’s communications in transit will surely fail!&lt;/p&gt;
&lt;h2&gt;Give me an example.&lt;/h2&gt;
&lt;p&gt;As an example let’s say that user A wants to send a message to user B over MindLink — here is how a typical Signal Protocol exchange goes:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;User A and User B log on for the first time and generate new asymmetric key-pairs. They publish their public keys to a server.&lt;/li&gt;
&lt;li&gt;User A downloads user B’s keys from the server.&lt;/li&gt;
&lt;li&gt;He creates a new secure messaging session.&lt;/li&gt;
&lt;li&gt;Then encrypts the message.&lt;/li&gt;
&lt;li&gt;Sends the message over MindLink.&lt;/li&gt;
&lt;li&gt;User B receives the messages and starts a matching secure messaging session, decrypting the message.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;Ok, now let’s unpack this a little…&lt;/h2&gt;
&lt;p&gt;1 . When logging in to MindLink for the first time, each user generates several sets of keys: A long-term identity key-pair, a medium-term signed prekey pair and several ephemeral keys. The public keys of these pairs are packaged up into what’s known as a prekey bundle. This bundle is then sent to a Key Exchange server for storage and dissemination.&lt;/p&gt;
&lt;p&gt;2 . User A decides to send a message to user B for the first time. User A starts by requesting user B’s prekey bundle from the Key Exchange.&lt;/p&gt;
&lt;p&gt;3 . With this, user A builds a new session based on both their keys and user B’s keys in what is referred to as the &lt;strong&gt;Extended Triple Diffie-Hellman (X3DH) key agreement protocol&lt;/strong&gt;. &lt;/p&gt;
&lt;p&gt;Note: This sounds like a mouthful, and while I likely won’t do it justice here you can find very detailed information about X3DH and indeed the other defining features of the Signal Protocol at the official website, &lt;a href=&quot;https://signal.org/docs/&quot;&gt;https://signal.org/docs/&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The Diffie-Hellman (or more specifically the &lt;strong&gt;elliptic-curve Diffie-Hellman&lt;/strong&gt;) protocol describes a novel way of deriving a shared secret amongst two parties over an insecure channel. Using some clever mathematics it can be shown that two users sharing some initial public data and some hidden secret data can exchange a sequence of transformations based on that data and arrive at entirely the same conclusion - the shared secret - without anyone who might be observing this exchange being able to do the same! This shared secret then becomes the starting point of an encrypted session between the two users.&lt;/p&gt;
&lt;p&gt;4 . User A encrypts a message using their new session.&lt;/p&gt;
&lt;p&gt;This relies on Signal Protocol’s famous &lt;a href=&quot;https://signal.org/docs/specifications/doubleratchet/&quot;&gt;Double Ratchet algorithm&lt;/a&gt; which combines two types of ratchet - a symmetric-key ratchet and a Diffie-Hellman ratchet - to achieve a means of deriving a series of unique message encryption keys that are totally dissociated from any previous or future encryption keys. This means that if an attacker were to compromise one the Double Ratchet keys, they could not then go on to decrypt any other messages. It is this feature of dynamically evolving cryptography that affords Signal its uber-secure reputation.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;symmetric ratchet&lt;/strong&gt; uses the shared secret key from the X3DH protocol in combination with user B’s ephemeral keys to generate two symmetric keys: a root key (a secret only the users A and B can posses) and a sending chain key. These keys are passed through a &lt;strong&gt;key derivation function (KDF)&lt;/strong&gt; to derive unique message keys for each message that is sent. When user A sends a message to user B, user A will advance their sending key chain by one step, generating a new sending chain key and a message encryption key, and on receipt of this message user B will advance their receiving key chain one step and generate the corresponding message decryption key. This is good but if an attacker were to steal user B’s sending and receiving chain key they could potentially calculate every future message key and thus every future message. This is where the other half of the &lt;strong&gt;Double Ratchet&lt;/strong&gt; comes in…&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;Diffie-Hellman ratchet&lt;/strong&gt; is “interleaved” with the symmetric ratchet and the output periodically initialises a new set of sending and receiving chain keys and a brand new root key. This means the attacker’s stolen keys are useless after only a single two-way message exchange (the “period” of the DH ratchet)!&lt;/p&gt;
&lt;p&gt;This all sounds quite complicated… and that’s because it is! I would encourage anyone who’s really interested to go away and read and this very technical, very in-depth &lt;a href=&quot;https://eprint.iacr.org/2016/1013.pdf&quot;&gt;analysis of Signal&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The message itself contains the encrypted ciphertext payload along with the outputs of the DH ratchet at each stage and some additional information for the recipient to advance their receiving key chain correctly. But this is also a special case - being the first message sent in a new session, user A also straps their prekey bundle to the encrypted message so user B can derive a complementary session.&lt;/p&gt;
&lt;p&gt;5 . User A sends the encrypted payload over the wire!&lt;/p&gt;
&lt;p&gt;6 . User B receives the encrypted message payload. They recognise that they don’t have a matching session for the message, but they DO realise that there’s a prekey bundle attached to the message that they can establish one with. After going through the same steps as user A in #3 and #4 user B ends up with the decrypted plaintext message.&lt;/p&gt;
&lt;h2&gt;Wrapping it up&lt;/h2&gt;
&lt;p&gt;So that’s it! There are plenty of details left out here but this should hopefully offers a decent high-level overview of the kind of flow and processes you can expect to see in a typical Signal Protocol offering. To get started with Signal you can play with their APIs &lt;a href=&quot;https://github.com/signalapp&quot;&gt;available here&lt;/a&gt; - they have support for C, JavaScript and Java currently.&lt;/p&gt;
&lt;p&gt;For further interest regarding implementation of the protocol, I’ve created a simple browser-based demonstration using the JavaScript Signal APIs &lt;a href=&quot;https://github.com/Jamie-Matthews/Signal-Protocol-Demo&quot;&gt;which you can find on GitHub&lt;/a&gt;. The demo creates two “users” and runs through the steps outlined in this post to exchange some messages between them. Enjoy!&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Creating a private Ethereum blockchain and using it as a model]]></title><description><![CDATA[Recently there’s been a lot of fascination surrounding blockchains, and how they can be used to improve long-standing industry technologies…]]></description><link>https://engineering.mindlinksoft.comcreating-a-private-ethereum-blockchain-and-using-it-as-a-model</link><guid isPermaLink="false">https://engineering.mindlinksoft.comcreating-a-private-ethereum-blockchain-and-using-it-as-a-model</guid><pubDate>Wed, 06 Jun 2018 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Recently there’s been a lot of fascination surrounding blockchains, and how they can be used to improve long-standing industry technologies. Because of this, we’ve been investigating the applications of blockchains in messaging platforms. I’ve developed a blockchain-based messaging web-app which treats the blockchain as the main data model to gain some hands-on experience working with a blockchain.&lt;/p&gt;
&lt;p&gt;In this post I’ll detail my process and rationale behind creating a private Ethereum blockchain and a web-app which interacts with it, with the hopes of guiding anyone who’d like to try it for themselves. No Ethereum experience necessary!&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://github.com/mindlink/sample-blockchain-messenger&quot;&gt;You can check out my proof-of-concept web-app here&lt;/a&gt;, which contains the finished web-app and a simple readme with instructions for running the Ethereum network.&lt;/p&gt;
&lt;h2&gt;Creating a private Ethereum network&lt;/h2&gt;
&lt;p&gt;The first thing that’s needed to start playing around with Ethereum is a way to leverage the Ethereum protocol, which gives access to the tools needed to interact with Ethereum networks. There are C++ and Python implementations of the protocol, but I’ll be using Geth (the Go implementation) as it can be interacted with through the CLI, making it quick and easy to use. Even though I’m using Geth, any implementation can be used as they should provide similar capabilities.&lt;/p&gt;
&lt;p&gt;Before we can create the Ethereum network, we’ll first need to create its blockchain (starting with a genesis block).&lt;/p&gt;
&lt;h3&gt;Creating the genesis block&lt;/h3&gt;
&lt;p&gt;A blockchain is a data structure which builds upon itself over time, which means that it needs an initial structure to build upon - the genesis block! This block is defined by a genesis state file, which describes both the block’s parameters and some parameters which affect the operation of the Ethereum network.&lt;/p&gt;
&lt;p&gt;I initialized the genesis block using a state file named &lt;code class=&quot;language-text&quot;&gt;genesis.json&lt;/code&gt; in my working directory. Here’s what my &lt;code class=&quot;language-text&quot;&gt;genesis.json&lt;/code&gt; file contains:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-json&quot;&gt;&lt;code class=&quot;language-json&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;config&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;  
        &lt;span class=&quot;token property&quot;&gt;&quot;chainId&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;731&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; 
        &lt;span class=&quot;token property&quot;&gt;&quot;homesteadBlock&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;difficulty&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;0x400&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;gasLimit&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;0x8000000&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;alloc&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This is a fairly minimal genesis state file, as I’ve only defined the parameters that I need for my particular setup. I’ll give some explanation on what each parameter does:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;config&lt;/code&gt;: An object containing the configuration of Ethereum. It doesn’t configure anything specific to the genesis block itself, but it defines how the Ethereum network will run.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;chainId&lt;/code&gt;: Identifies which blockchain Ethereum will be using. There are a few different important chain IDs, like the ID of the mainnet’s blockchain (1). I chose a random 3 digit ID.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;homesteadBlock: 0&lt;/code&gt;: Ensures that we’ll be using the latest release of Ethereum, homestead.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;difficulty&lt;/code&gt;: Determines how difficult it is for a block to be mined. The value is the reciprocal of the probability that mining the block succeeds (in hexadecimal). The value I chose (0x400, 1024 in denary) is fairly low, as quicker mining means that transactions can be committed to the blockchain with less latency.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;gasLimit&lt;/code&gt;: The maximum number of computations any following block can support (in hexadecimal), chosen as an arbitrarily high number that shouldn’t be reached.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;alloc&lt;/code&gt;: An object allowing you to allocate Ether to specific accounts. I didn’t pre-allocate any Ether, as my blockchain was set up to have no transaction fees and my transactions won’t contain any Ether.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;Initializing the blockchain&lt;/h3&gt;
&lt;p&gt;Once you have a genesis state file, setting up the blockchain is easy-peasy. I ran the following command in my working directory to create the blockchain based on my genesis state file in a directory named &lt;code class=&quot;language-text&quot;&gt;blockchain&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;geth -datadir ./blockchain init ./genesis.json&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;If all goes well, the output looks something like this:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Maximum peer count                       ETH&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;25 LES&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 total&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;25
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Allocated cache and &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; handles         database&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;my-working-dir&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;\blockchain\geth\chaindata cache&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;16 handles&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;16
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Writing custom genesis block
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Persisted trie from memory database      nodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 size&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B time&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s gcnodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 gcsize&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B gctime&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s livenodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 livesize&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Successfully wrote genesis state         database&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;chaindata                                                                  hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;d1a12d…4c8725
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:14&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Allocated cache and &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; handles         database&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;my-working-dir&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;\blockchain\geth\lightchaindata cache&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;16 handles&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;16
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:15&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Writing custom genesis block
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:15&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Persisted trie from memory database      nodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 size&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B time&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s gcnodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 gcsize&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B gctime&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s livenodes&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 livesize&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0.00B
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:12:15&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Successfully wrote genesis state         database&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;lightchaindata                                                                  hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;d1a12d…4c8725&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;h3&gt;Creating the network&lt;/h3&gt;
&lt;p&gt;After creating the blockchain data, running a private network for the blockchain is simple thanks to Geth. Here’s the command I used to initialize my private Ethereum network (replacing &amp;#x3C;device name&gt; with my device’s name):&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;geth --datadir ./blockchain --networkid 7314 --ws --wsorigins &lt;span class=&quot;token string&quot;&gt;&quot;http://localhost:8080,http://&amp;lt;device name&gt;&quot;&lt;/span&gt; --wsapi&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;eth,web3,personal,miner&quot;&lt;/span&gt; --gasprice 0&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;I’ll explain why I passed those specific parameters below (&lt;a href=&quot;https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options&quot;&gt;see here for a full list of viable options&lt;/a&gt;):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;datadir&lt;/code&gt;: The directory containing the blockchain data (created by initializing the blockchain).&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;networkid&lt;/code&gt;: Identifies the Ethereum network, I chose a random 4 digit ID to minimise clashing with other networks (&lt;a href=&quot;https://ethereum.stackexchange.com/questions/17051/how-to-select-a-network-id-or-is-there-a-list-of-network-ids/17101#17101&quot;&gt;see here for a list of public network IDs&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;ws&lt;/code&gt;: Enables the WS-RPC server, which allows (insecure) websocket connections to be opened with this server. I’ll be using websockets to communicate between the web-app and the blockchain as it can run easily in a browser. To use secure websockets, you’d need a proxy node or similar which responds to secure websocket connections and forwards requests, as Ethereum doesn’t provide support for secure websockets.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;wsorigins&lt;/code&gt;: A list of origins from which to accept websocket requests. I’ll be serving my web-app on port 8080, so requests from that port needed to be accepted or else the app wouldn’t be able to communicate with the network. My device’s name also needs to be accepted so that I can connect to the network through the Geth console. Passing &lt;code class=&quot;language-text&quot;&gt;*&lt;/code&gt; as a parameter will allow any connection, but this is less secure as it allows anyone access to the network.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;wsapi&lt;/code&gt;: A list of APIs to allow over websocket communication. I need &lt;code class=&quot;language-text&quot;&gt;web3&lt;/code&gt; to expose the Ethereum protocol to my web-app, &lt;code class=&quot;language-text&quot;&gt;eth&lt;/code&gt; to manage transactions, &lt;code class=&quot;language-text&quot;&gt;personal&lt;/code&gt; to manage accounts, and &lt;code class=&quot;language-text&quot;&gt;miner&lt;/code&gt; to mine transactions.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;gasprice&lt;/code&gt;: The minimum gas price of a transaction. Transaction fees are calculated as &lt;code class=&quot;language-text&quot;&gt;gasprice * gaslimit&lt;/code&gt; and paid in Ether. As I want no transaction fees on my transactions, I set the gas price to 0 to allow fee-free transactions to be sent/mined.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With this, the private Ethereum network is running and transactions can be sent and mined to the blockchain! When running successfully, your ouput should look something like this:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Maximum peer count                       ETH&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;25 LES&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 total&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;25
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Starting peer-to-peer node               instance&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;Geth/v1.8.10-stable-eae63c51/windows-amd64/go1.10.2
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Allocated cache and &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; handles         database&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;my-working-dir&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;\blockchain\geth\chaindata cache&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;768 handles&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1024
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Initialised chain configuration          config&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;{ChainID: 731 Homestead: 0 DAO: &amp;lt;nil&gt; DAOSupport: false EIP150: &amp;lt;nil&gt; EIP155: &amp;lt;nil&gt; EIP158: &amp;lt;nil&gt; Byzantium: &amp;lt;nil&gt; Constantinople: &amp;lt;nil&gt; Engine: unknown}&quot;&lt;/span&gt;
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Disk storage enabled &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; ethash caches   dir&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;my-working-dir&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;\blockchain\geth\ethash count&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;3
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Disk storage enabled &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; ethash DAGs     dir&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;C:\AppData\Ethash                                                     count&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;2
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Initialising Ethereum protocol           versions&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;[63 62]&quot;&lt;/span&gt; network&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;7314
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Loaded &lt;span class=&quot;token function&quot;&gt;most&lt;/span&gt; recent local header          number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;d1a12d…4c8725 td&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1024
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Loaded &lt;span class=&quot;token function&quot;&gt;most&lt;/span&gt; recent local full block      number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;d1a12d…4c8725 td&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1024
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Loaded &lt;span class=&quot;token function&quot;&gt;most&lt;/span&gt; recent local fast block      number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;d1a12d…4c8725 td&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1024
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Regenerated local transaction journal    transactions&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 accounts&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:55&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Starting P2P networking
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:57&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; UDP listener up                          self&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;enode://939c12fbde8d140e55800327c8b5fcd71a81a32647b95a89840580f67628cf4a650974af1c28588c1df476400cbaa6d514b23d0d14919f04ece8da52e6f7fcf5@&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;::&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;:30303
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:57&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; RLPx listener up                         self&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;enode://939c12fbde8d140e55800327c8b5fcd71a81a32647b95a89840580f67628cf4a650974af1c28588c1df476400cbaa6d514b23d0d14919f04ece8da52e6f7fcf5@&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;::&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;:30303
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:57&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; IPC endpoint opened                      url&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;\\.\pipe\geth.ipc
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;13:45:57&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; WebSocket endpoint opened                url&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;ws://127.0.0.1:8546&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;h3&gt;Interacting with the network&lt;/h3&gt;
&lt;p&gt;With the server running, the network should be accessible through Geth. You can attach a Geth console to the network with the following command:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;geth attach ws://localhost:8546&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This attaches to the network through the websocket served at port 8546 (the default port for WS-RPC) and provides a javascript console for interacting with it.&lt;/p&gt;
&lt;p&gt;If you see the error message: &lt;code class=&quot;language-text&quot;&gt;Fatal: Unable to attach to remote geth: bad status&lt;/code&gt; accompanied with the following warning in the network output: &lt;code class=&quot;language-text&quot;&gt;WARN [06-19|13:58:58] origin &amp;#39;http://&amp;lt;device name&amp;gt;&amp;#39; not allowed on WS-RPC interface&lt;/code&gt;, then you need to pass the given device name to the &lt;code class=&quot;language-text&quot;&gt;-wsorigins&lt;/code&gt; option when running your network.&lt;/p&gt;
&lt;p&gt;Now that we have a console attached to the network, we’ll need to create an account before we can do certain things on the network (like mining and sending transactions). This can be done with the function &lt;code class=&quot;language-text&quot;&gt;personal.newAccount()&lt;/code&gt;, which prompts twice for a passphrase and then outputs the new account’s address (which acts like an ID for our purposes). The address you are allocated should look something like this: &lt;code class=&quot;language-text&quot;&gt;0xedfc0cea37e9ed3bc7927a81f0dcd393bc57c0e6&lt;/code&gt;.&lt;/p&gt;
&lt;h3&gt;Sending and mining a transaction&lt;/h3&gt;
&lt;p&gt;To test that the network is set up correctly, we’ll send a dummy transaction and mine it. First, your account needs to be unlocked so that you can send a transaction from it. Calling the function &lt;code class=&quot;language-text&quot;&gt;personal.unlockAccount(&amp;quot;&amp;lt;your account address&amp;gt;&amp;quot;)&lt;/code&gt; and entering the passphrase when prompted will unlock the given account, otherwise an error will be thrown.&lt;/p&gt;
&lt;p&gt;Next, we’ll send a transaction. The function &lt;code class=&quot;language-text&quot;&gt;ethereum.sendTransaction({from: &amp;quot;&amp;lt;address&amp;gt;&amp;quot;})&lt;/code&gt; will send a transaction on the network from a given address. For the full details of what the configuration object can hold &lt;a href=&quot;https://github.com/ethereum/wiki/wiki/JavaScript-API#web3ethsendtransaction&quot;&gt;see here&lt;/a&gt;, but for now you only need to provide the required &lt;code class=&quot;language-text&quot;&gt;from&lt;/code&gt; field to send a basic transaction. The ouput from the function is the transaction hash (i.e. all of the transaction data hashed together).&lt;/p&gt;
&lt;p&gt;After sending a transaction, you can check that it’s visible to the network by calling &lt;code class=&quot;language-text&quot;&gt;eth.getBlock(&amp;quot;pending&amp;quot;)&lt;/code&gt;, which outputs the &lt;code class=&quot;language-text&quot;&gt;pending&lt;/code&gt; block (a virtual block containing transactions which haven’t been committed to the blockchain). If the transactions list present in the block contains the dummy transaction’s hash, that means the transaction is waiting to be mined. You can mine the transaction by calling &lt;code class=&quot;language-text&quot;&gt;miner.start(&amp;lt;number-of-threads&amp;gt;)&lt;/code&gt; (but remember to call &lt;code class=&quot;language-text&quot;&gt;miner.stop()&lt;/code&gt; unless you want to be mining forever). While mining, the network output should look like this:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:39&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Starting mining operation
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:39&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Commit new mining work                   number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 txs&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 uncles&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 elapsed&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:43&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Successfully sealed new block            number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0d5b6c…978671
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:43&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; 🔨 mined potential block                  number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;1 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0d5b6c…978671
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:43&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Commit new mining work                   number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;2 txs&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 uncles&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0 elapsed&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;0s
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:43&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; Successfully sealed new block            number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;2 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;62f2bd…351578
INFO &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;06-19&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt;14:34:43&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; 🔨 mined potential block                  number&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;2 hash&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;62f2bd…351578**&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;After mining for a (very) short while, the transaction should be committed to the blockchain. You can check individual blocks with &lt;code class=&quot;language-text&quot;&gt;eth.getBlock(&amp;lt;block-number&amp;gt;)&lt;/code&gt; to find the transaction manually (it should be in block 1 if you didn’t mine until after sending the transaction), or you can loop through the possible block numbers to find any non-empty &lt;code class=&quot;language-text&quot;&gt;transactions&lt;/code&gt; array. If all has gone well, you should find the same transaction hash you saw before in one of the blocks’ &lt;code class=&quot;language-text&quot;&gt;transactions&lt;/code&gt; arrays. This means that your network is set up correctly!&lt;/p&gt;
&lt;h3&gt;Creating a lazy miner&lt;/h3&gt;
&lt;p&gt;The web-app we’ll be making will use transactions as a vehicle for data, so we’ll need to mine new blocks whenever transactions are sent or the data will never be stored on the blockchain. We don’t really want to run a miner non-stop as this is a waste of energy, so I’ve written the script &lt;code class=&quot;language-text&quot;&gt;lazyMine.js&lt;/code&gt; to subscribe to updates to the set of pending transactions and mine them when necessary:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;checkPendingTransactions&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getBlock&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;pending&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;transactions&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;length &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;mining&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;Mining pending transactions...\n&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        miner&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;start&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;!&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;mining&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

        miner&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;stop&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;Mining stopped.\n&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;filter&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;latest&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; checkPendingTransactions&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;filter&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;pending&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; checkPendingTransactions&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token function&quot;&gt;checkPendingTransactions&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;I’ll break down how this script works now:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The function &lt;code class=&quot;language-text&quot;&gt;checkPendingTransactions&lt;/code&gt; checks if any transactions are pending, and then:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;If transactions are pending and we’re not already mining, it starts mining on 1 thread.&lt;/li&gt;
&lt;li&gt;If no transactions are pending and we’re mining, it stops mining.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;eth.filter&lt;/code&gt; causes the given function to be called on the specified block (&lt;code class=&quot;language-text&quot;&gt;latest&lt;/code&gt; is the latest block added to the blockchain, &lt;code class=&quot;language-text&quot;&gt;pending&lt;/code&gt; is the virtual block of non-committed transactions) when that block is updated.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When the &lt;code class=&quot;language-text&quot;&gt;latest&lt;/code&gt; block updates it means that new transactions may have been mined, so we may have to stop mining if no pending transactions are left.&lt;/li&gt;
&lt;li&gt;When the &lt;code class=&quot;language-text&quot;&gt;pending&lt;/code&gt; block updates it means that new transactions may have been sent, so we may have to start mining.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;We then call &lt;code class=&quot;language-text&quot;&gt;checkPendingTransactions&lt;/code&gt; to mine any non-committed transactions when the script is first run.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We can run this script in a Geth console attached to the network using the following command:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;geth attach ws://localhost:8546 --preload ./lazyMine.js&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;You can test if the script is working correctly with a similar method to how we tested that the network was set up correctly. I recommend leaving this script running in the background for now, as it won’t mine anything until transactions are actually sent.&lt;/p&gt;
&lt;p&gt;Now that we have our blockchain and Ethereum network fully set up, it’s time to make the web-app that’ll communicate with it!&lt;/p&gt;
&lt;h2&gt;Creating the web-app&lt;/h2&gt;
&lt;p&gt;Since we’re interested in messaging here at MindLink, I’ll be building an MVC messaging app which uses the blockchain as a model. I won’t explain all of the nitty-gritty implementation details behind the app, but I’ll explain everything you should need to interact with the Ethereum network. If you want to see the full codebase for my web-app you can access it &lt;a href=&quot;https://github.com/mindlink/sample-blockchain-messenger&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;h3&gt;Server and packaging&lt;/h3&gt;
&lt;p&gt;I used &lt;a href=&quot;https://webpack.js.org/&quot;&gt;webpack&lt;/a&gt; to package my web-app, mainly as we use it for development of MindLink and so anything I create in webpack is potentially compatible with our systems. It also has some convenient features/plugins, like &lt;a href=&quot;https://webpack.js.org/configuration/dev-server/&quot;&gt;webpack-dev-server&lt;/a&gt;, which will help speed up development of the app.&lt;/p&gt;
&lt;h3&gt;Web-app overview&lt;/h3&gt;
&lt;p&gt;I’ve written a very simple messaging web-app, which allows messages to be represented by individual transactions on the blockchain. In order to stay focused on Ethereum, the web-app is a barebones html/javascript implementation with packages needed to run the server in webpack and nothing more. The application has the following features:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;It can unlock accounts with a given account’s address &amp;#x26; passphrase&lt;/li&gt;
&lt;li&gt;It subscribes to blockchain updates (i.e. new transactions)&lt;/li&gt;
&lt;li&gt;It can list all of the accounts on the network&lt;/li&gt;
&lt;li&gt;It displays new transactions as messages&lt;/li&gt;
&lt;li&gt;It can send messages in the form of transactions&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;When the web-app is first accessed, the user must provide their account’s credentials, which will be used to unlock the account (or create a new one). The account address will also be used as the &lt;code class=&quot;language-text&quot;&gt;from&lt;/code&gt; field when sending messages. &lt;em&gt;Make sure not to input any important credentials, as they are not encrypted at this stage!&lt;/em&gt; After this, the user is greeted with a basic messaging application, containing a message list and a message input, as shown in the screenshot below.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;./messaging-app-screenshot.png&quot; alt=&quot;The messaging web-app&quot;&gt;&lt;/p&gt;
&lt;h3&gt;Interacting with Ethereum in Javascript&lt;/h3&gt;
&lt;p&gt;In order to access the Ethereum protocol in Javascript, I’m using &lt;a href=&quot;https://github.com/ethereum/web3.js/tree/1.0&quot;&gt;Web3&lt;/a&gt; - a package which provides a Javascript implementation for Ethereum. Thankfully it’s very easy to set it up! You can import the package through &lt;code class=&quot;language-text&quot;&gt;npm&lt;/code&gt; (or in my case, &lt;code class=&quot;language-text&quot;&gt;yarn&lt;/code&gt;) and then import and instantiate it wherever you need to use it, like so:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;import&lt;/span&gt; Web3 &lt;span class=&quot;token keyword&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;web3&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; web3 &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;new&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;Web3&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;ws://localhost:8546&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This instantiates a new Web3 object, which you can use to interact with the Ethereum network through the given websocket. We can call methods on &lt;code class=&quot;language-text&quot;&gt;web3&lt;/code&gt; to interact with the network.&lt;/p&gt;
&lt;p&gt;If you’re not using websockets, you can find out what arguments to pass the Web3 constructor &lt;a href=&quot;https://web3js.readthedocs.io/en/1.0/include_package-core.html#providers&quot;&gt;here&lt;/a&gt;. The reason I decided on using websockets is that my application code is run entirely on the client’s browser, which means that we can’t use IPC as it requires the NPM &lt;code class=&quot;language-text&quot;&gt;net&lt;/code&gt; package (which isn’t browser compatible).&lt;/p&gt;
&lt;h3&gt;Ethereum account management with Web3&lt;/h3&gt;
&lt;p&gt;In order to send transactions, we’ll need to unlock an account first (just like when we sent a transaction from the Geth console). To do this with Web3, you’ll need to obtain the user’s credentials and then pass them to the following function:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;personal&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;unlockAccount&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;address&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; passphrase&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This is an asynchronous function (as are any which require a response from the network), so if you need the result before continuing then make sure to use &lt;code class=&quot;language-text&quot;&gt;await&lt;/code&gt; beforehand. The function will throw an error if the wrong credentials are given, so you should ideally have some error handling in place if your credentials are input by the user.&lt;/p&gt;
&lt;p&gt;If you don’t already have an account set up (or you just want to create a new account), you can do this with Web3 as well! First you’ll need to obtain a passphrase, and then you can pass it onto the following async function (which returns the new account address if successful):&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;personal&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;newAccount&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;passphrase&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;It can also be useful to get a list of all accounts on the Ethereum network, allowing users to select addresses when needed instead of typing them (as they’re not human-readable). In my messaging web-app, I used this to allow the user to select the recipient of their message from a drop-down box. You can get a list of accounts in the Ethereum network with the following async function:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;personal&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getAccounts&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;h3&gt;Sending transactions with Web3&lt;/h3&gt;
&lt;p&gt;Sending transactions with Web3 is quick and easy! First you’ll need to construct a transaction object, which contains the fields used by the transaction (&lt;a href=&quot;https://web3js.readthedocs.io/en/1.0/web3-eth.html#sendtransaction&quot;&gt;see here for the full list of available fields&lt;/a&gt;). The only required field is the &lt;code class=&quot;language-text&quot;&gt;from&lt;/code&gt; field, but I’ll go through all of the fields I used in my web-app to treat a transaction as a message-sending mechanism:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;from&lt;/code&gt;: The address the transaction is sent from (must be unlocked). I use the account the user provided for this, and treat it as the message’s sender field.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;to&lt;/code&gt;: The address the transaction is sent to. I use this as the message’s recipient field, and obtain the value from a drop-down box (populated by all account addresses in the network) which the user can select.&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;value&lt;/code&gt;: The amount of Ether transferred for the transaction. I set this value to 0 so that all messaging can be effectively free (and therefore unlimited).&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;data&lt;/code&gt;: This field can take any hexadecimal string, which is great for sending additional data!&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To send my messages in the &lt;code class=&quot;language-text&quot;&gt;data&lt;/code&gt; field, I first create an object with a &lt;code class=&quot;language-text&quot;&gt;message&lt;/code&gt; field (which contains the message text). This is useful as if I wanted to send any other data along with the message text, I could simply add another field. I then use &lt;code class=&quot;language-text&quot;&gt;JSON.stringify&lt;/code&gt; to convert the object into a string, and then &lt;code class=&quot;language-text&quot;&gt;web3.utils.asciiToHex&lt;/code&gt; to convert the string to hexadecimal. This final value is what I pass in to the transaction’s &lt;code class=&quot;language-text&quot;&gt;data&lt;/code&gt; field.&lt;/p&gt;
&lt;p&gt;Once you have your transaction object, you can send the transaction with the following asynchronous function:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;sendTransaction&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;transaction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This returns the transaction hash upon completion. The transaction will have to be mined after this, so it won’t appear on the blockchain for a little while. If you still have the lazy miner running then it should appear quickly with no outside intervention. How convenient!&lt;/p&gt;
&lt;h3&gt;Receiving transactions with Web3&lt;/h3&gt;
&lt;p&gt;Receiving transactions requires a little more work than sending them. There’s no function to list all transactions, so instead we’ll have to make our own! In order to get all the transactions from the blockchain, we’ll have to iterate over each block and process the transactions array they contain. Here’s the method I use to process transactions (which I’ll explain below):&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;async&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;getMessages&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; startBlockNumber &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; lastBlockRead &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; endBlockNumber &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;await&lt;/span&gt; web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getBlockNumber&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;var&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; startBlockNumber&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;&amp;lt;=&lt;/span&gt; endBlockNumber&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;var&lt;/span&gt; block &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;await&lt;/span&gt; web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;getBlock&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;i&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;block &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&amp;amp;&lt;/span&gt; block&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;transactions &lt;span class=&quot;token operator&quot;&gt;!==&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;null&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; tx &lt;span class=&quot;token keyword&quot;&gt;of&lt;/span&gt; block&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;transactions&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token function&quot;&gt;handleTransaction&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tx&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
            &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
            lastBlockRead &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; block&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;number&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Firstly, the function checks which block number it should start from (using a variable &lt;code class=&quot;language-text&quot;&gt;lastBlockRead&lt;/code&gt;). This is useful as there’s no point re-processing blocks you’ve already processed, so it’s good to keep track of where you got up to. The variable is initialized at -1 (as the first block number is 0). The function then checks where it can read up to using &lt;code class=&quot;language-text&quot;&gt;web3.eth.getBlockNumber&lt;/code&gt;, which returns the highest block number on the blockchain. You could also read blocks up until a block is undefined, and use that as a loop termination condition.&lt;/p&gt;
&lt;p&gt;The function then iterates over the blocks in the blockchain, getting each one with &lt;code class=&quot;language-text&quot;&gt;web3.eth.getBlock(i, true)&lt;/code&gt;. The first parameter to &lt;code class=&quot;language-text&quot;&gt;getBlock&lt;/code&gt; is the block number, and the second is a boolean which determines whether the transactions array should contain transaction objects (&lt;code class=&quot;language-text&quot;&gt;true&lt;/code&gt;) or hashes (&lt;code class=&quot;language-text&quot;&gt;false&lt;/code&gt;). We need the transaction objects to process the data they store, so we must pass in &lt;code class=&quot;language-text&quot;&gt;true&lt;/code&gt;. Once we have the block, the function simply loops through the available transactions and processes them.&lt;/p&gt;
&lt;p&gt;This is great for manually processing all unread transactions, but what if we want to process transactions as they occur? For this, we’ll need to subscribe to certain blockchain events, so that this function is called whenever a new block is committed. To do this we can use the following asynchronous function:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-js&quot;&gt;&lt;code class=&quot;language-js&quot;&gt;web3&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;eth&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;subscribe&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;newBlockHeaders&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; callback&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This function takes a string and a callback function. The string defines what kind of events we are subscribing to - &lt;code class=&quot;language-text&quot;&gt;&amp;quot;newBlockHeaders&amp;quot;&lt;/code&gt; means the callback is called whenever new blocks are committed (&lt;a href=&quot;https://web3js.readthedocs.io/en/1.0/web3-eth-subscribe.html&quot;&gt;see here for the list of event types&lt;/a&gt;). The callback is called whenever an event of the given kind occurs, and is passed a block header object or an error. I pass &lt;code class=&quot;language-text&quot;&gt;getMessages&lt;/code&gt; as my callback function so that the list of messages updates whenever a new block is committed.&lt;/p&gt;
&lt;p&gt;With that, you should have all you need to send and receive transactions from a web-app! If you’d like to see my blockchain-integrated web-app, &lt;a href=&quot;https://github.com/mindlink/sample-blockchain-messenger&quot;&gt;click here&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;As you can see, once you have an Ethereum network and blockchain set up it’s surprisingly easy to interact with it from a web-app. Everything you need to use the blockchain as a model is provided through Web3, as you can commit and read back data in the form of transactions. This makes it quite easy to use and quick to set up.&lt;/p&gt;
&lt;p&gt;When it comes to the efficacy of an Ethereum blockchain as a data models, there are some obvious problems. The main issue is latency, as any transactions being sent to the blockchain must be mined first, which is not particularly quick in comparison to more conventional data stores.&lt;/p&gt;
&lt;p&gt;Another big issue I found was that all data on the blockchain is freely available to anyone with access to the network, and so any user (with or without an account) could read any other user’s data easily. While there are ways to work around this restriction, there are methods built into conventional data stores which handle this issue for you, so this makes using a blockchain as a store look much less appetizing.&lt;/p&gt;
&lt;p&gt;I’ve yet to find any practical benefits to using the blockchain as a model. While there are unique systems like mining/transaction fees and smart contracts, I’m not sure that they provide any tangible benefit that couldn’t be replicated on other types of model.&lt;/p&gt;
&lt;p&gt;Overall, while this was a worthwhile proof-of-concept to gain experience working with Ethereum, it seems like there aren’t many reasons to use the blockchain as a model in the first place.&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Custom content in a Skype for Business conversation]]></title><description><![CDATA[At MindLink we like to take current trends in messaging and see if we can get more out of Skype for Business (SfB) by supporting them. This…]]></description><link>https://engineering.mindlinksoft.comcustom-content-in-a-skype-for-business-conversation</link><guid isPermaLink="false">https://engineering.mindlinksoft.comcustom-content-in-a-skype-for-business-conversation</guid><pubDate>Mon, 21 May 2018 10:15:00 GMT</pubDate><content:encoded>&lt;p&gt;At MindLink we like to take current trends in messaging and see if we can get more out of Skype for Business (SfB) by supporting them. This week with our intern Daniel I’ve been investigating how to display rich custom content i.e. cards, in channels while having fallback content for other clients that don’t understand the rich content.&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;We have made it possible to bring rich content cards as seen in many popular trendy chat platforms to existing Skype for Business deployments, further extending the value of SfB!&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Getting custom data inside a SfB message&lt;/h2&gt;
&lt;p&gt;Skype for Business channels are known as Persistent Chat groups in SfB speak. They support sending Rich Text Format (RTF), but the native clients will reject unrecognised/malformed RTF and display a very specific subset of the RTF content. This is also true for 1-1 conversations.&lt;/p&gt;
&lt;p&gt;Fortunately RTF is pretty well documented, what we need is some way to send RTF that can have some non-RTF content and a fallback if that content isn’t supported.&lt;/p&gt;
&lt;p&gt;Enter the &lt;code class=&quot;language-text&quot;&gt;\object&lt;/code&gt; block! The RTF specification describes this as a block for representing OLE objects, where the object is a base64 encoded OLE stream and there are other blocks for defining the type and importantly the RTF “result” should the object not be supported. See the &lt;a href=&quot;http://www.biblioscape.com/rtf15_spec.htm#Heading50&quot;&gt;RTF Specification&lt;/a&gt; for more details.&lt;/p&gt;
&lt;p&gt;By using our framework we were able to insert a test object to see how the SfB client reacts to such a definition. As expected and conforming to the RTF specification the client renders only the RTF result of the object. The format we used to achieve this within an RTF message was:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-text&quot;&gt;&lt;code class=&quot;language-text&quot;&gt;{\object \objemb {*\objclass MindLinkCard} {*\objdata &amp;lt;data&amp;gt;} {\result &amp;lt;fallback&amp;gt;}}&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Using this as a starting point we defined a common representation for structured JSON data that we would encode as the object data (replacing &lt;code class=&quot;language-text&quot;&gt;&amp;lt;data&amp;gt;&lt;/code&gt; above). We then extended our core application to allow the sending of a restricted set of rich content cards, ensuring that the fallback RTF result is populated with a textual description of the content, replacing &lt;code class=&quot;language-text&quot;&gt;&amp;lt;fallback&amp;gt;&lt;/code&gt; above e.g. inserting a support ticket might render “see ticket &lt;a href=&quot;https://support.company.com/ticket/AT-123%22&quot;&gt;https://support.company.com/ticket/AT-123”&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;We then enhanced our client to be able to send these rich content cards and display rich content in the message history instead of the fallback text!&lt;/p&gt;
&lt;p&gt;What’s more is that we made this extensible, so it’s possible to load client extensions to send other rich content cards! This opens up the possibility for more intuitive line of business workflows right within your conversations.&lt;/p&gt;
&lt;p&gt;Expect to see us leverage this new extensibility in sample integrations with other line-of-business applications.&lt;/p&gt;
&lt;h2&gt;Example use case - Giphy&lt;/h2&gt;
&lt;p&gt;While we work on more advanced integrations, here’s an example popular integration with &lt;a href=&quot;https://www.giphy.com&quot;&gt;Giphy&lt;/a&gt; that is a very simple showcase of what these new capabilities can provide. Imagine linking this into line-of-business applications that show dynamic data or allow actions to be taken with the click of a button from right within the conversation.&lt;/p&gt;
&lt;p&gt;Our work-in-progress extensibility API allows an extension to load a popup for composing a message card, as shown below.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/popup-a6c66713b004436b5ee92bfae218fe0b-0de24.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 68.31896551724138%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAOABQDASIAAhEBAxEB/8QAGAAAAgMAAAAAAAAAAAAAAAAAAAQCAwX/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAAB2IMUiY2S/wD/xAAaEAEAAgMBAAAAAAAAAAAAAAACAQMAEBIT/9oACAEBAAEFAuBjisH1qnVtfZIkx//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8BP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8BP//EABsQAAIBBQAAAAAAAAAAAAAAAAARARASMjOh/9oACAEBAAY/AsYHZBq5RNCZ/8QAHBAAAwABBQAAAAAAAAAAAAAAAAERITFhceHx/9oACAEBAAE/IfLKDHhC6ESjFSyulElMVsf/2gAMAwEAAgADAAAAEPAv/8QAFREBAQAAAAAAAAAAAAAAAAAAACH/2gAIAQMBAT8QR//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABsQAQEAAwADAAAAAAAAAAAAAAERACFBMcHh/9oACAEBAAE/EJO3runLReoHL3DFNTWkPF94uEmDyDY4GXQhfrP/2Q==&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Composing Popup&quot;
        title=&quot;&quot;
        src=&quot;/static/popup-a6c66713b004436b5ee92bfae218fe0b-f8fb9.jpg&quot;
        srcset=&quot;/static/popup-a6c66713b004436b5ee92bfae218fe0b-e8976.jpg 148w,
/static/popup-a6c66713b004436b5ee92bfae218fe0b-63df2.jpg 295w,
/static/popup-a6c66713b004436b5ee92bfae218fe0b-f8fb9.jpg 590w,
/static/popup-a6c66713b004436b5ee92bfae218fe0b-85e3d.jpg 885w,
/static/popup-a6c66713b004436b5ee92bfae218fe0b-0de24.jpg 928w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;The extension can insert any number of pre-defined card message parts, below you can see the Giphy extension insert an image card.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/composing-c7fd938a1177e779b5eab47b1e87d16e-99062.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 24.54642475987193%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAFABQDASIAAhEBAxEB/8QAFwABAAMAAAAAAAAAAAAAAAAAAAECBf/EABQBAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhADEAAAAdyQoD//xAAWEAEBAQAAAAAAAAAAAAAAAAAAATH/2gAIAQEAAQUCmK//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAEDAQE/AT//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/AT//xAAUEAEAAAAAAAAAAAAAAAAAAAAQ/9oACAEBAAY/An//xAAYEAADAQEAAAAAAAAAAAAAAAAAESFRMf/aAAgBAQABPyHSkwV8P//aAAwDAQACAAMAAAAQcA//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAEDAQE/ED//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/ED//xAAaEAEAAgMBAAAAAAAAAAAAAAABABEhQVFx/9oACAEBAAE/EDQMvWp4xAhpP//Z&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Composing&quot;
        title=&quot;&quot;
        src=&quot;/static/composing-c7fd938a1177e779b5eab47b1e87d16e-f8fb9.jpg&quot;
        srcset=&quot;/static/composing-c7fd938a1177e779b5eab47b1e87d16e-e8976.jpg 148w,
/static/composing-c7fd938a1177e779b5eab47b1e87d16e-63df2.jpg 295w,
/static/composing-c7fd938a1177e779b5eab47b1e87d16e-f8fb9.jpg 590w,
/static/composing-c7fd938a1177e779b5eab47b1e87d16e-85e3d.jpg 885w,
/static/composing-c7fd938a1177e779b5eab47b1e87d16e-99062.jpg 937w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;When the message is sent the MindLink client will then display the card in full detail:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/inline-7cdee083e5832a3299a12c77916dbea4-4e39d.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 67.95698924731182%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAOABQDASIAAhEBAxEB/8QAGAAAAgMAAAAAAAAAAAAAAAAAAAQBAgX/xAAUAQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIQAxAAAAHYF2SC4f/EABkQAAMBAQEAAAAAAAAAAAAAAAACAwETEv/aAAgBAQABBQLmh4Q5oVkz1lmpPT//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAEDAQE/AT//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/AT//xAAbEAACAwADAAAAAAAAAAAAAAABEQACMhASkf/aAAgBAQAGPwLA8mKzInYJJIwVsWRx/8QAGhAAAwEAAwAAAAAAAAAAAAAAAAERIUHR8f/aAAgBAQABPyGGC8gv1iY/nhk7W1jV4f/aAAwDAQACAAMAAAAQQM//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAEDAQE/ED//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/ED//xAAdEAEAAwABBQAAAAAAAAAAAAABABEhMWFxgZHR/9oACAEBAAE/EMA9KY6njGs1djGpqEZtreHWYW0Hh35HsSFT/9k=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Inline conversation&quot;
        title=&quot;&quot;
        src=&quot;/static/inline-7cdee083e5832a3299a12c77916dbea4-f8fb9.jpg&quot;
        srcset=&quot;/static/inline-7cdee083e5832a3299a12c77916dbea4-e8976.jpg 148w,
/static/inline-7cdee083e5832a3299a12c77916dbea4-63df2.jpg 295w,
/static/inline-7cdee083e5832a3299a12c77916dbea4-f8fb9.jpg 590w,
/static/inline-7cdee083e5832a3299a12c77916dbea4-85e3d.jpg 885w,
/static/inline-7cdee083e5832a3299a12c77916dbea4-4e39d.jpg 930w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;When the card is rendered in a Skype for Business client it shows the fallback text described by the extension. For Giphy integration that’s simply the link to the GIF, for other integrations that may be a combination of text and links.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/sfb-06f330df972c5f0a7dcf7473dfde6534-1f48b.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 479px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 119.6242171189979%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAYABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAECAwX/xAAWAQEBAQAAAAAAAAAAAAAAAAABAgP/2gAMAwEAAhADEAAAAfXzuVn2E3YJQP8A/8QAGBAAAwEBAAAAAAAAAAAAAAAAAQIREBL/2gAIAQEAAQUCa2HGWnjQbn//xAAWEQEBAQAAAAAAAAAAAAAAAAAQESH/2gAIAQMBAT8BLp//xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAXEAADAQAAAAAAAAAAAAAAAAAAITDh/9oACAEBAAY/AkbD/8QAHRABAAEDBQAAAAAAAAAAAAAAAQAQETEhgZHR8f/aAAgBAQABPyEruhd1hnhwscvFRwHcp//aAAwDAQACAAMAAAAQzPi+/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAh/9oACAEDAQE/EE2KAT//xAAWEQEBAQAAAAAAAAAAAAAAAAABECH/2gAIAQIBAT8QIGLP/8QAGhABAAMBAQEAAAAAAAAAAAAAAQARMSFBYf/aAAgBAQABPxDqCFfEOVKRqVCWcqCBxhq4oC1MEUD0Ngj5P//Z&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;Skype for Business fallback&quot;
        title=&quot;&quot;
        src=&quot;/static/sfb-06f330df972c5f0a7dcf7473dfde6534-1f48b.jpg&quot;
        srcset=&quot;/static/sfb-06f330df972c5f0a7dcf7473dfde6534-e4839.jpg 148w,
/static/sfb-06f330df972c5f0a7dcf7473dfde6534-9f8fc.jpg 295w,
/static/sfb-06f330df972c5f0a7dcf7473dfde6534-1f48b.jpg 479w&quot;
        sizes=&quot;(max-width: 479px) 100vw, 479px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;We are very excited to be developing this new set of features for our customers, enabling next generation functionality without requiring a whole new platform!&lt;/p&gt;</content:encoded></item><item><title><![CDATA[Leveraging Rasa to build on-premise intelligent chat bots]]></title><description><![CDATA[In this post we will explain and demonstrate how to leverage  Rasa  to develop intelligent chatbots and securely deploy them on premise…]]></description><link>https://engineering.mindlinksoft.comleveraging-rasa-to-build-on-premise-intelligent-chat-bots</link><guid isPermaLink="false">https://engineering.mindlinksoft.comleveraging-rasa-to-build-on-premise-intelligent-chat-bots</guid><pubDate>Fri, 18 May 2018 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;In this post we will explain and demonstrate how to leverage &lt;a href=&quot;https://rasa.com/&quot;&gt;Rasa&lt;/a&gt; to develop intelligent chatbots and securely deploy them on premise. Chatbots are one of the most direct ways to integrate with existing chat platforms. Whether you require a simple bot that replies to specific queries by fetching some information, or a more complex NPL-powered interactive agent, chatbots provide a way to integrate external services and functions into the collaboration experience of the chat platform users.&lt;/p&gt;
&lt;h2&gt;Chatbot development since 2016&lt;/h2&gt;
&lt;p&gt;Modern chatbot development for the most part leverages the capabilities of cloud APIs such as Microsoft &lt;a href=&quot;https://azure.microsoft.com/en-gb/services/cognitive-services/language-understanding-intelligent-service/&quot;&gt;LUIS&lt;/a&gt; and Google &lt;a href=&quot;https://dialogflow.com/&quot;&gt;DialogFlow&lt;/a&gt; that tend to abstract away from the developer most of the underlying complexities of natural language understanding and response selection tasks. This makes developing chatbot integration significantly easier than it would have been previously, where either more rudimentary approaches where necessary or intelligent chatbot development was significantly harder. This unfortunately means that this useful level of abstraction is unavailable to enterprise environments that have strict compliance or security requirements.&lt;/p&gt;
&lt;p&gt;The use of cloud APIs that can read and store the information exchanged can lead to sensitive information being leaked to unintended parties, as well as not being as easlily traceable and archivable for compliance purposes. This means that many existing chatbot integrations that use these APIs are unsuitable for use on premise, and in-house development teams are prevented from using them as well. In order to achieve the same rich user experience talking to intelligent chatbots the in-house team has a lot more work to do, which may make these integrations unreasonably complex and time consuming to develop.&lt;/p&gt;
&lt;h2&gt;Enter Rasa&lt;/h2&gt;
&lt;p&gt;Rasa is the only natural language understanding and chatbot development API that is open source. The API itself can be deployed on premise in a controlled environment, with each message becoming easily traceable and subject to the same security constrains as any other message exchanged over the chat system. By using Rasa it becomes relatively easy to analyse natural language messages sent by chat users for intent categorisation and to build intelligent integrations with external systems.&lt;/p&gt;
&lt;p&gt;We’re now going to take a look at a very simple bot built through Rasa and how to connect it to the MindLink API.&lt;/p&gt;
&lt;h2&gt;The interesting bit&lt;/h2&gt;
&lt;p&gt;For this example we’re going to use one of the tutorials for Rasa Core, and implement a simple group-listening bot connection to MindLink that will read all messages sent in a particular chatroom and reply with some information. It should be possible to resuse the connection code for other bots in the same manner as it is used here. We’ll implement the whole thing in Python as that is the simplest way to write bots against Rasa, but mutatis mutandis you should be able to re-implement the connection in your favorite language. Similarly to how the MindLink API works, you can setup the Rasa API as a REST service and use any language.&lt;/p&gt;
&lt;p&gt;I’m going to assume you have Rasa Core installed locally, and you have your local Python environment setup (I’m assuming Python3, but the connection code is relatively straightforward and should not be too hard to port to Python2).&lt;/p&gt;
&lt;h3&gt;The anathomy of an intelligent chat bot&lt;/h3&gt;
&lt;p&gt;There are two independent pieces of a Rasa bot: an “interpreter” or intent-extractor and a dialogue model. The first is a mechanism to “parse” natural language input and extract the user intent, then feed this to the dialogue model together with raw natural language input. It is trained over a set of pre-classified data, and uses this training to classify new inputs. Similarly, the dialogue model is built from a set of response templates, actions and the same intents recognized by the interpreter. It is also trained over sample dialogues that represent possible conversations.&lt;/p&gt;
&lt;p&gt;Both modular pieces are implemented in a “data-driven” fashion: to develop an intelligent chatbot a significant amount of training data is required for both the intent extractor and the dialogue model. Interactive training modes, where you basically have a dialogue with the interpreter/dialogue model and give it feedback allowing it to adjust its responses, are available out of the box. Both components are also independent from each other (you can use alternative dialogue models/intent extractor with them) and allow very fine tuning of the resulting trained models (if you know what you’re doing). Again, compelling chatbots are always the product of repeated training over a significant body of data. Depending on the application, there may be large data sets already publically available (e.g. healthcare etc).&lt;/p&gt;
&lt;h3&gt;Great, but how do I actually put these together?&lt;/h3&gt;
&lt;p&gt;Let’s focus on the intent extractor first. We’ll define a set of intents in a markdown file with some initial associated data: &lt;code class=&quot;language-text&quot;&gt;intents.md&lt;/code&gt;. This is the file that basically defines how the intent extractor is going to interpret the input&lt;sup id=&quot;fnref-1&quot;&gt;&lt;a href=&quot;#fn-1&quot; class=&quot;footnote-ref&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-markdown&quot;&gt;&lt;code class=&quot;language-markdown&quot;&gt;&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:greet&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; hey
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; hello
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; hi

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:goodbye&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; bye
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; goodbye

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:mood_affirm&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; yes
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; correct

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:mood_deny&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; no
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; nope

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:mood_great&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; very good
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; great
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; amazing
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; happy

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; intent:mood_unhappy&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; I am sad
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; super sad
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; sad
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; very sad
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; unhappy
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; so saad
&lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; so sad&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Secondly, we’ll need two separate files for the dialogue model, one to define the things it recognizes and the actions it can take, and another with example conversations that exemplify when to take an action. First we define a domain: &lt;code class=&quot;language-text&quot;&gt;domain.yml&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-yml&quot;&gt;&lt;code class=&quot;language-yml&quot;&gt;intents:
  - greet
  - goodbye
  - mood_affirm
  - mood_deny
  - mood_great
  - mood_unhappy

actions:
- utter_greet
- utter_cheer_up
- utter_did_that_help
- utter_happy
- utter_goodbye

templates:
  utter_greet:
  - text: &amp;quot;Hey! How are you?&amp;quot;
    buttons:
    - title: &amp;quot;great&amp;quot;
      payload: &amp;quot;great&amp;quot;
    - title: &amp;quot;super sad&amp;quot;
      payload: &amp;quot;super sad&amp;quot;

  utter_cheer_up:
  - text: &amp;quot;Here is something to cheer you up:&amp;quot;
    image: &amp;quot;https://i.imgur.com/nGF1K8f.jpg&amp;quot;

  utter_did_that_help:
  - text: &amp;quot;Did that help you?&amp;quot;
  - text: &amp;quot;Was that helpful?&amp;quot;

  utter_happy:
  - text: &amp;quot;Great carry on!&amp;quot;

  utter_goodbye:
  - text: &amp;quot;Bye&amp;quot;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;As you can see the domain also contains the response templates the bot can respond with.
Then we define sample conversations &lt;code class=&quot;language-text&quot;&gt;stories.md&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-markdown&quot;&gt;&lt;code class=&quot;language-markdown&quot;&gt;&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; happy path               &lt;/span&gt;&lt;span class=&quot;token comment&quot;&gt;&amp;lt;!-- name of the story - just for debugging --&gt;&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; greet
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_greet
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_great               &lt;span class=&quot;token comment&quot;&gt;&amp;lt;!-- user utterance, in format _intent[entities] --&gt;&lt;/span&gt;
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_happy

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; sad path 1               &lt;/span&gt;&lt;span class=&quot;token comment&quot;&gt;&amp;lt;!-- this is already the start of the next story --&gt;&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; greet
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_greet             &lt;span class=&quot;token comment&quot;&gt;&amp;lt;!-- action of the bot to execute --&gt;&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_unhappy
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;cheer&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;up
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;did&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;that_help
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_affirm
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_happy

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; sad path 2&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; greet
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_greet
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_unhappy
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;cheer&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;up
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;did&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;that_help
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_deny
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_goodbye

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; say goodbye&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; goodbye
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_goodbye

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; no greeting happy&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_great
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter_happy

&lt;span class=&quot;token title important&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;##&lt;/span&gt; no greeting sad&lt;/span&gt;
&lt;span class=&quot;token list punctuation&quot;&gt;*&lt;/span&gt; mood_unhappy
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;cheer&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;up
  &lt;span class=&quot;token list punctuation&quot;&gt;-&lt;/span&gt; utter&lt;span class=&quot;token italic&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;did&lt;span class=&quot;token punctuation&quot;&gt;_&lt;/span&gt;&lt;/span&gt;that_help&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Each story is an independent example of a complete conversation. As you can see, the dialogue model doesn’t understand natural language, it undertstands intents. So the raw input needs to be piped through the intent extractor and then into the dialogue model.&lt;/p&gt;
&lt;p&gt;Now that we’ve defined our data in files we need to train (“compile” if you like) the actual models from the data. You can use config files or pass additional parameters to the python scripts:&lt;/p&gt;
&lt;p&gt;nlu_config.json&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-json&quot;&gt;&lt;code class=&quot;language-json&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token property&quot;&gt;&quot;pipeline&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;spacy_sklearn&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
  &lt;span class=&quot;token property&quot;&gt;&quot;path&quot;&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;./models/interpreter&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
  &lt;span class=&quot;token property&quot;&gt;&quot;data&quot;&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;./data/intents.md&quot;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;Training the intent extractor:&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;python -m rasa_nlu.train -c nlu_config.json --fixed_model_name current&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Training the dialogue model:&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;python -m rasa_core.train -s data/stories.md -d domain.yml -o models/dialogue --epochs 300&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Hopefully, you’ve got a working bot at this point. You can talk to it over the command line&lt;sup id=&quot;fnref-2&quot;&gt;&lt;a href=&quot;#fn-2&quot; class=&quot;footnote-ref&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;:&lt;/p&gt;
&lt;p&gt;&lt;code class=&quot;language-text&quot;&gt;python -m rasa_core.run -d models/dialogue -u models/interpeter/default/current&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Now we are ready to wire this up to the MindLink API.&lt;/p&gt;
&lt;h3&gt;A simple Python MindLink API connection&lt;/h3&gt;
&lt;p&gt;We’re going to define a simple API connection class. This class will be responsible for interfacing with MindLink, relaying messages received to a consumer (which will be the bot in our example) and allowing for sending messages.&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;ApiConnection&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;authenticate&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        requestUrl &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Authentication/V1/Tokens&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;post&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
            requestUrl&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            json &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Username&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;username&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Password&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;password&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;AgentId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;agent
            &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;RequestHeaders&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;token &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;token

    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;getMessages&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; count&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        requestUrl &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Channels/{}/Messages&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        parameters &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
            &lt;span class=&quot;token string&quot;&gt;&apos;take&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; count&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

        response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;get&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
            requestUrl&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            params &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; parameters&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
            &lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;sendMessage&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        requestUrl &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Channels/{}/Messages&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;post&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
            requestUrl&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            json &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Text&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; json&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;dumps&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;message&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;IsAlert&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;False&lt;/span&gt;
            &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
            headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
            &lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
            &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Failed to send message to channel&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;_getEvents&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; callback&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;running &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;True&lt;/span&gt;

        &lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;running&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
            requestUrl &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;{}/Collaboration/V1/Events&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;host&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

            parameters &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;last-event&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;last_event&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;types&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;message&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;channels&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;channelId&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;regex&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                &lt;span class=&quot;token string&quot;&gt;&apos;origins&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;remote&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
            &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

            response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; requests&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;get&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
                requestUrl&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                params &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; parameters&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                headers &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
                    &lt;span class=&quot;token string&quot;&gt;&apos;Accept&apos;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;application/json&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
                    &lt;span class=&quot;token string&quot;&gt;&apos;Authorization&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&apos;FCF {}&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token builtin&quot;&gt;format&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;token&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
                    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

            &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;status_code &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
                &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Failed to get new messages from channel&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
                &lt;span class=&quot;token keyword&quot;&gt;continue&lt;/span&gt;

            &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; event &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;json&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
                eventId &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;EventId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
                &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;eventId &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;last_event&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
                    self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;last_event &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; eventId

                &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Sender&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;localUserId&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
                    message &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Message&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Sender&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Content&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Time&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; event&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;ChannelId&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
                    callback&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;message&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;startStreaming&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; callback&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;requestThread &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; threading&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;Thread&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;target&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;_getEvents&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; args &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; callback&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;requestThread&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;start&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;stopStreaming&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;self&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
        self&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;running &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;False&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;h4&gt;What’s all this?&lt;/h4&gt;
&lt;p&gt;Let’s breakdown the methods on by one. First off, the &lt;code class=&quot;language-text&quot;&gt;authenticate&lt;/code&gt; should be relatively self-explanatory, it makes a request to the authentication endpoint of the MindLink API in order to log on an agent. It’s going to keep a hold of the received token and send it back with each subsequent request (this needs to be renewed if it expires).&lt;/p&gt;
&lt;p&gt;The &lt;code class=&quot;language-text&quot;&gt;getMessages&lt;/code&gt; method is going to read a certain number of messages from the messages endpoint for a specific channel. Similarly with the &lt;code class=&quot;language-text&quot;&gt;sendMessage&lt;/code&gt; method.&lt;/p&gt;
&lt;p&gt;Then we get to the more complicated bits: &lt;code class=&quot;language-text&quot;&gt;_getEvents&lt;/code&gt; is a perpetually running method that repeatedly makes get requests to the events endpoint of the API, fetching new events for a given channel ID. This method is invoked from the public &lt;code class=&quot;language-text&quot;&gt;startStreaming&lt;/code&gt; method on a separate thread, and will keep that thread running forever until the &lt;code class=&quot;language-text&quot;&gt;stopStreaming&lt;/code&gt; method is invoked. As you can see, what happens with each event received is not defined by this class, but needs to be forwarded by the caller in the form of a callback.&lt;/p&gt;
&lt;p&gt;Now that we’ve covered the core connection code, we’ll build a basic example of how to use it.&lt;/p&gt;
&lt;h3&gt;MindLink + Rasa&lt;/h3&gt;
&lt;p&gt;Let’s make a python script entry point file, this is the actual sauce that feeds the message content to the bot and posts back a respone:&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-python&quot;&gt;&lt;code class=&quot;language-python&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;handleMessage&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;message&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    response &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; agent&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;handle_message&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;message&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;content&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    connection&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;sendMessage&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; response&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; __name__ &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;__main__&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Loading Rasa agent (this may take a while).&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    agent &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Agent&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;load&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;dialogue_model&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; interpreter_model&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;print&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Creating connection to MindLink.&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    connection &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; ApiConnection&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        u&lt;span class=&quot;token string&quot;&gt;&apos;http://localhost:8081&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        u&lt;span class=&quot;token string&quot;&gt;&apos;userdomain\\moodbot&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        u&lt;span class=&quot;token string&quot;&gt;&apos;some password&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        u&lt;span class=&quot;token string&quot;&gt;&apos;moodbot_agent&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        u&lt;span class=&quot;token string&quot;&gt;&apos;contact:moodbot@userdomain.local&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    connection&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;authenticate&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    connection&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;startStreaming&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        channelId&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;lambda&lt;/span&gt; _&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; message&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; handleMessage&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;message&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

    &lt;span class=&quot;token builtin&quot;&gt;input&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&apos;Type anything to stop.\r\n&apos;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    connection&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;stopStreaming&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This will “start” the API connection on a given channel, with some preconfigured agent settings and a given channel ID (in a real application these may be passed as command line arguments, read from file etc). For each message received in the channel, this will be forwarded to the Rasa bot and the response will be sent back to the channel.&lt;/p&gt;
&lt;h2&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;We covered a fair amount of ground in this post, explaining why Rasa is interesting for enterprise development, touching on core Rasa bot development concepts and demonstrating a basic way to wire everything up the MindLink API.&lt;/p&gt;
&lt;div class=&quot;footnotes&quot;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;This is a condensed version of &lt;a href=&quot;https://core.rasa.com/tutorial_basics.html&quot;&gt;this short tutorial&lt;/a&gt;.&lt;/p&gt;
&lt;a href=&quot;#fnref-1&quot; class=&quot;footnote-backref&quot;&gt;↩&lt;/a&gt;
&lt;/li&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;Note that we’re using extremely limited sample data here, the classification and response selection performance is almost certainly going to be disappointing. A lot of data is necessary before satisfactory performance is achieved.&lt;/p&gt;
&lt;a href=&quot;#fnref-2&quot; class=&quot;footnote-backref&quot;&gt;↩&lt;/a&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;</content:encoded></item><item><title><![CDATA[Why DraftJS doesn't work on Android]]></title><description><![CDATA[DraftJS  is a popular open source rich text editing library for React maintained by Facebook. It allows developers to decorate input text as…]]></description><link>https://engineering.mindlinksoft.comwhy-draftjs-doesnt-work-on-android</link><guid isPermaLink="false">https://engineering.mindlinksoft.comwhy-draftjs-doesnt-work-on-android</guid><pubDate>Sun, 22 Apr 2018 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;&lt;a href=&quot;https://draftjs.org/&quot;&gt;DraftJS&lt;/a&gt; is a popular open source rich text editing library for React maintained by Facebook. It allows developers to decorate input text as the user types, and provides a way to embed metadata into the decorated text (e.g. a URL.) Also supports commands, keyboard shortcuts …&lt;/p&gt;
&lt;p&gt;All of these rich development features come with a caveat however: mobile browsers are not officially supported. This means that if you’re developing your mobile applications as wrapped web applications via &lt;a href=&quot;https://cordova.apache.org/&quot;&gt;Cordova&lt;/a&gt; (or &lt;a href=&quot;https://phonegap.com/&quot;&gt;Phonegap&lt;/a&gt;, or &lt;a href=&quot;https://ionicframework.com/&quot;&gt;Ionic&lt;/a&gt;) then you’re probably out of luck.&lt;/p&gt;
&lt;p&gt;This post is about &lt;em&gt;why&lt;/em&gt; the way Draft works is in principle incompatible with the Google virtual keyboard which is the default on certain Android devices. Supporting multiple mobile operating systems with the same code-base is one of the main reasons why to use Cordova (or similar) in the first place, so you are likely to want to use the same rich text editing library across them all&lt;sup id=&quot;fnref-1&quot;&gt;&lt;a href=&quot;#fn-1&quot; class=&quot;footnote-ref&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;h2&gt;An eventful experience&lt;/h2&gt;
&lt;p&gt;The core React execution model involves deciding what the state of the DOM is in memory first, then pushing the new state down onto the DOM. As a React component, the &lt;a href=&quot;https://github.com/facebook/draft-js/blob/master/src/component/base/DraftEditor.react.js&quot;&gt;DraftEditor&lt;/a&gt; follows the same model.&lt;/p&gt;
&lt;p&gt;DraftJS essentially works by creating a &lt;code class=&quot;language-text&quot;&gt;contenteditable&lt;/code&gt; div and &lt;a href=&quot;https://github.com/facebook/draft-js/blob/05b2b4c83c39163697dccf7a3fa28fe04eb8e008/src/component/base/DraftEditor.react.js#L387&quot;&gt;attaching multiple input-handling events&lt;/a&gt; to it. Leaving all of the more advanced stuff aside (drag events, keyboard shortcuts, input metadata), browsers will normally fire events describing the plain text user input from the editable div. DraftJS will handle these events and create its own virtual version of the text contents and eventually push an update down into the editable div, replacing the DOM text node contents that the browser initially rendered on user input with its rich styled markup and embedded metadata.&lt;/p&gt;
&lt;p&gt;Because of the amount of detail Draft looks after, it does not represent user input (even when it’s all plain text) as a plain DOM text node, but will create separate blocks (strictly speaking divs) to represent different lines. Additionally, any “styled” text is treated as its own block&lt;sup id=&quot;fnref-2&quot;&gt;&lt;a href=&quot;#fn-2&quot; class=&quot;footnote-ref&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;This means Draft relies on the information about the typed text forwarded by the browser through events in order to construct its internal state.&lt;/p&gt;
&lt;h2&gt;Enter GBoard&lt;/h2&gt;
&lt;p&gt;&lt;a href=&quot;https://play.google.com/store/apps/details?id=com.google.android.inputmethod.latin&quot;&gt;GBoard&lt;/a&gt; is the default virtual keyboard on first-party Android models (Pixel, Nexus etc.) It supports various fancy features such as gif, emojis, multiple languages etc. It is also available on virtually every Android device (although the default on Samsung devices is probably going to be Samsung-made) and even iOS.&lt;/p&gt;
&lt;p&gt;There are well known issues due to how this keyboard communicates with mobile Chrome: the keyboard does not trigger detailed text input information via browser events (see &lt;a href=&quot;https://github.com/facebook/draft-js/issues/1077&quot;&gt;these&lt;/a&gt; &lt;a href=&quot;https://github.com/ianstormtaylor/slate/issues/725&quot;&gt;discussions&lt;/a&gt;), and there’s multiple issues stemming from text suggestions and auto-complete. &lt;strong&gt;Basically, all of the stuff DraftJS relies on is simply not there&lt;/strong&gt;. This means Draft has to “guess” what the state of the content editable div is at every turn based on the limited information that is forwarded via events.&lt;/p&gt;
&lt;p&gt;What’s worse, this information is sometimes inconsistent with the state of the text node rendered in the DOM: it is possible for the internal state of Draft to be inconsistent with what is shown in the DOM, so if you read the Draft state when extracting input information you may be reading something different from what the user is seeing on the screen. For example: entering newlines does not produce different block level elements in the editor (which, as mentioned, is how Draft represents newlines) but sometimes results in a single block with newline character: ”&lt;code class=&quot;language-text&quot;&gt;&amp;lt;div&amp;gt;hello\nthere&amp;lt;/div&amp;gt;&lt;/code&gt;”. Because of this, both the content state and the selection state become detached from the DOM when newlines are entered, resulting in a complete mess upon backspace (note that some of these issues are also present with the Samsung keyboard on some devices.)&lt;/p&gt;
&lt;h2&gt;Not-so-easy solution&lt;/h2&gt;
&lt;p&gt;These issues with GBoard are long standing and developers outside of DraftJS users have brought them up with the &lt;a href=&quot;https://bugs.chromium.org/p/chromium/issues/detail?id=118639#c260&quot;&gt;Chromium project&lt;/a&gt;. There’s no solution in sight.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The core of the issue is that the way the virtual keyboard interacts with the browser changes contents of the DOM text node inside the content editable block but does not forward edit events as accurately as most other input methods do. This means DraftJS is in principle incompatible with standard Android virtual keyboards as input methods because it relies on these events to update its virtual state.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;DraftJS would need to operate in a completely different “mode” on Android with GBoard: instead of relying on events, passively read the state of the &lt;code class=&quot;language-text&quot;&gt;contenteditable&lt;/code&gt; element and update the internal state accordingly. A host of issues awaits down this path: what event(s) would we wait for before reading the state? Is there some input gesture that would trigger no events? Detecting the running browser and OS is possible, but how do we detect GBoard? Should we maintain two separate modes of operation for Draft, one event-based, another passive, or stick to the passive model indefinitely? Are there any performance consequences? Would the user experience be acceptable or would the timing of the styling effects create awkward delay effects?&lt;/p&gt;
&lt;h2&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;DraftJS maintainers promise to eventually address the mobile issues, but this isn’t a priority at the moment, in fact it has not been a priority for &lt;a href=&quot;https://github.com/facebook/draft-js/issues/102&quot;&gt;around a couple of years now&lt;/a&gt;. Ultimately, DraftJS is not production-ready for mobile Android&lt;sup id=&quot;fnref-3&quot;&gt;&lt;a href=&quot;#fn-3&quot; class=&quot;footnote-ref&quot;&gt;3&lt;/a&gt;&lt;/sup&gt;. If you want to re-use as much of your code as possible throughout your web and mobile clients, DraftJS simply isn’t the solution you need right now. The other alternative is to reject Draft as a way to decorate your mobile input, or investigate alternatives. What is the point of having a rich text decorated input when basic plain text input is unacceptably broken?&lt;/p&gt;
&lt;h3&gt;References&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;DraftJS contributors. &lt;a href=&quot;https://draftjs.org/docs/advanced-topics-issues-and-pitfalls.html#content&quot;&gt;&lt;em&gt;DraftJS docs&lt;/em&gt;&lt;/a&gt;. 2018. (and generally Android specific issues on &lt;a href=&quot;https://github.com/facebook/draft-js/issues?q=is%3Aissue+is%3Aopen+android&quot;&gt;the DraftJS repository&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;ReactJS contributors. &lt;a href=&quot;https://reactjs.org/docs/hello-world.html&quot;&gt;&lt;em&gt;ReactJS docs&lt;/em&gt;&lt;/a&gt;. 2018.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;footnotes&quot;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;Originally posted &lt;a href=&quot;https://niccoloterreri.com/why-draftjs-does-not-work-on-android&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;a href=&quot;#fnref-1&quot; class=&quot;footnote-backref&quot;&gt;↩&lt;/a&gt;
&lt;/li&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;For that matter, there is &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Editable_content#Differences_in_markup_generation&quot;&gt;no standard&lt;/a&gt; on how browsers represent content of a &lt;code class=&quot;language-text&quot;&gt;contenteditable&lt;/code&gt; block.&lt;/p&gt;
&lt;a href=&quot;#fnref-2&quot; class=&quot;footnote-backref&quot;&gt;↩&lt;/a&gt;
&lt;/li&gt;
&lt;li id=&quot;fn-3&quot;&gt;
&lt;p&gt;To be clear, DraftJS maintainers do not “owe” anyone mobile Android support. This is only a word of caution to developers thinking of using Draft in their Android mobile projects.&lt;/p&gt;
&lt;a href=&quot;#fnref-3&quot; class=&quot;footnote-backref&quot;&gt;↩&lt;/a&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;</content:encoded></item><item><title><![CDATA[Publishing to GitHub Pages from TFS]]></title><description><![CDATA[Recently we decided it was time to publish our own blog to showcase some of what our talented engineers do and find interesting. To do this…]]></description><link>https://engineering.mindlinksoft.compublishing-to-github-pages-from-tfs</link><guid isPermaLink="false">https://engineering.mindlinksoft.compublishing-to-github-pages-from-tfs</guid><pubDate>Fri, 13 Apr 2018 10:09:00 GMT</pubDate><content:encoded>&lt;p&gt;Recently we decided it was time to publish our own blog to showcase some of what our talented engineers do and find interesting. To do this with the existing tools and as cheaply and quickly as possible we needed to link Team Foundation Server (TFS) to GitHub Pages.&lt;/p&gt;
&lt;h2&gt;TL;DR&lt;/h2&gt;
&lt;p&gt;To deploy from TFS to GitHub pages:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Install the &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; npm module &lt;/li&gt;
&lt;li&gt;Define an npm script to run gh-pages e.g. &lt;code class=&quot;language-text&quot;&gt;“deploy”: “gh-pages -d public”&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Use TFS secure variables to inject a GitHub personal token into &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; call &lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Combined this means defining a build step of the form&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;npm run deploy — -r https://$(GH_TOKEN)@github.com/user/repository.git&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;The &lt;code class=&quot;language-text&quot;&gt;—&lt;/code&gt; (that’s a double &lt;code class=&quot;language-text&quot;&gt;-&lt;/code&gt;) tells npm that it should forward what follows as additional script arguments &lt;/li&gt;
&lt;li&gt;The &lt;code class=&quot;language-text&quot;&gt;-r&lt;/code&gt; is the &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; argument for defining an alternative repository &lt;/li&gt;
&lt;li&gt;The &lt;code class=&quot;language-text&quot;&gt;$(GH_TOKEN)&lt;/code&gt; will be replaced by TFS with the value of the variable called &lt;code class=&quot;language-text&quot;&gt;GH_TOKEN&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;Onward to a continuously deployed blog&lt;/h2&gt;
&lt;p&gt;Here at MindLink we use Microsoft Team Foundation Server to host our source in a git repository, manage the build and release process and track our iterations. It’s all nice and integrated!&lt;/p&gt;
&lt;p&gt;We wanted to keep ownership of the blogging platform as seems to be the trend, as opposed to using a service like &lt;a href=&quot;https://www.medium.com&quot;&gt;medium&lt;/a&gt;. The main reason for this is that by having control we also gain the ability to change where we publish and how we publish in future.&lt;/p&gt;
&lt;p&gt;GitHub pages struck us as the perfect platform for a low cost (&lt;em&gt;free&lt;/em&gt;) way to publish a blog and is widely used for such a purpose. It also has a simple way to publish a site - simply push a static site to a specific repository (or gh-pages branch in a project repository). You can read more about GitHub pages &lt;a href=&quot;https://pages.github.com/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;What we wanted was a simple way to write a blog post and automatically publish it using the tools our engineers use every day.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;This will give it a low barrier to entry and encourage all the development team to contribute!&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;As TFS was our existing git repository host and build system it made sense to leverage that for the blog source and release mechanism. Getting TFS to publish a built static site to GitHub is not so straight forward.&lt;/p&gt;
&lt;p&gt;With a static site generator (&lt;a href=&quot;https://www.gatsbyjs.org/&quot;&gt;gatsby&lt;/a&gt; in this case) it’s easy to take a source tree and build a static site. That’s step 1.&lt;/p&gt;
&lt;p&gt;To take that static site and publish it to GitHub when you’re not already in a GitHub repository is less straight-forward, but still perfectly doable!&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Use the &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; npm module to publish &lt;/li&gt;
&lt;li&gt;Use an npm script to define the standard &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; script &lt;/li&gt;
&lt;li&gt;Get TFS build to run the npm script and inject the GitHub credentials and target repository &lt;/li&gt;
&lt;li&gt;Setup the build triggers to either run on every commit, schedule it or do something else &lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;I’d probably also suggest splitting the build and release phases if you’re worried about publishing potentially broken sites. Since TFS has a fairly powerful build and release mechanism you can get a process that works for you!&lt;/p&gt;
&lt;h2&gt;The details&lt;/h2&gt;
&lt;p&gt;I’m going to assume the following:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;You already have a script to build the static site (maybe an npm script or gulp or grunt process) &lt;/li&gt;
&lt;li&gt;The static site is built to &lt;code class=&quot;language-text&quot;&gt;/public&lt;/code&gt; in the root directory &lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Setup gh-pages npm script&lt;/h3&gt;
&lt;p&gt;The first step is to install the node modules we will need (I’m using yarn here but it will work just as well with npm install): &lt;code class=&quot;language-text&quot;&gt;yarn add gh-pages&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;With that installed we’ll now specify an npm script to publish the built static site (you could just as easily do this with a task runner like gulp or grunt instead, the trick is in injecting the GitHub key):&lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-json&quot;&gt;&lt;code class=&quot;language-json&quot;&gt;&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    ...&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;scripts&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        ...&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token property&quot;&gt;&quot;deploy&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;gh-pages -d public&quot;&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;This tells &lt;code class=&quot;language-text&quot;&gt;gh-pages&lt;/code&gt; to run using only the “public” folder. With only this configuration it will add a branch called gh-pages to the current repository.&lt;/p&gt;
&lt;h3&gt;Get TFS Build to run the script&lt;/h3&gt;
&lt;p&gt;For the second step we need to tell TFS build to run this script and target a GitHub repository instead!&lt;/p&gt;
&lt;p&gt;Get yourself a personal access token from GitHub by going to &lt;a href=&quot;https://github.com/settings/tokens&quot;&gt;github token settings for your account&lt;/a&gt; and creating a new access token. Make sure you give it permission to read and write to public repositories. Copy that token and don’t lose it!&lt;/p&gt;
&lt;p&gt;Now go to your TFS build configuration and select the variables tab. Add the token as a secure variable with any name you like - here we use &lt;code class=&quot;language-text&quot;&gt;GH_TOKEN&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-eeb2a.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 30.13270882123341%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAGABQDASIAAhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAAEF/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/aAAwDAQACEAMQAAAB34FB/8QAFhAAAwAAAAAAAAAAAAAAAAAAABAR/9oACAEBAAEFAiL/xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAEDAQE/AT//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/AT//xAAWEAADAAAAAAAAAAAAAAAAAAAAEDH/2gAIAQEABj8CKv/EABgQAQEAAwAAAAAAAAAAAAAAABEBABBh/9oACAEBAAE/IQyqqnNf/9oADAMBAAIAAwAAABADz//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8QP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABsQAQACAgMAAAAAAAAAAAAAAAEAETFBYZGh/9oACAEBAAE/EAYX3GEGNKp8ic1P/9k=&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;TFS Variables&quot;
        title=&quot;&quot;
        src=&quot;/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-f8fb9.jpg&quot;
        srcset=&quot;/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-e8976.jpg 148w,
/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-63df2.jpg 295w,
/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-f8fb9.jpg 590w,
/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-85e3d.jpg 885w,
/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-d1924.jpg 1180w,
/static/tfs-variables-576181a66ba553dcf1f9105718fcbac9-eeb2a.jpg 1281w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;p&gt;Next add a new build step to run our deploy script as below:&lt;/p&gt;
&lt;p&gt;
  &lt;a
    class=&quot;gatsby-resp-image-link&quot;
    href=&quot;/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-d6369.jpg&quot;
    style=&quot;display: block&quot;
    target=&quot;_blank&quot;
    rel=&quot;noopener&quot;
  &gt;
  
  &lt;span
    class=&quot;gatsby-resp-image-wrapper&quot;
    style=&quot;position: relative; display: block; ; max-width: 590px; margin-left: auto; margin-right: auto;&quot;
  &gt;
    &lt;span
      class=&quot;gatsby-resp-image-background-image&quot;
      style=&quot;padding-bottom: 46.334310850439884%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAJABQDASIAAhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAQAF/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/aAAwDAQACEAMQAAAB3mQmP//EABQQAQAAAAAAAAAAAAAAAAAAACD/2gAIAQEAAQUCX//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8BP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8BP//EABQQAQAAAAAAAAAAAAAAAAAAACD/2gAIAQEABj8CX//EABUQAQEAAAAAAAAAAAAAAAAAAAEQ/9oACAEBAAE/IRpf/9oADAMBAAIAAwAAABCAD//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8QP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABgQAAIDAAAAAAAAAAAAAAAAAAAQETFB/9oACAEBAAE/EMCVR//Z&apos;); background-size: cover; display: block;&quot;
    &gt;
      &lt;img
        class=&quot;gatsby-resp-image-image&quot;
        style=&quot;width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;&quot;
        alt=&quot;TFS Build Step&quot;
        title=&quot;&quot;
        src=&quot;/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-f8fb9.jpg&quot;
        srcset=&quot;/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-e8976.jpg 148w,
/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-63df2.jpg 295w,
/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-f8fb9.jpg 590w,
/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-85e3d.jpg 885w,
/static/tfs-build-step-c2afb440c28667ee9f1dad68b0d9466c-d6369.jpg 1023w&quot;
        sizes=&quot;(max-width: 590px) 100vw, 590px&quot;
      /&gt;
    &lt;/span&gt;
  &lt;/span&gt;
  
  &lt;/a&gt;
    &lt;/p&gt;
&lt;div class=&quot;gatsby-highlight&quot;&gt;
      &lt;pre class=&quot;language-text&quot;&gt;&lt;code class=&quot;language-text&quot;&gt;Type: npm
Run: custom
Arguments: deploy — -b master -r https://$(GH_TOKEN)@github.com/MindLink/MindLink.github.io.git&lt;/code&gt;&lt;/pre&gt;
      &lt;/div&gt;
&lt;p&gt;That’s it! TFS build will run the npm script “deploy” and pass additional arguments:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;-b master&lt;/code&gt; =&gt; publish to the master branch of the repository&lt;/li&gt;
&lt;li&gt;&lt;code class=&quot;language-text&quot;&gt;-r https://$(GH_TOKEN)@&amp;lt;repository&amp;gt;&lt;/code&gt; =&gt; publish to a repository using the personal token (which allows you to publish without a user/pass) &lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Now whenever you trigger the build it will publish a new version of your static site to GitHub Pages.&lt;/p&gt;
&lt;h3&gt;Wrap up&lt;/h3&gt;
&lt;p&gt;This is the exactly how we have started doing it. Right now our team has total freedom to push updates to the blog with continuous deployment, in future that may change to a more managed release process, but our aim is to get our content out with as few steps as possible!&lt;/p&gt;
&lt;p&gt;If you’re interested the full process for somebody to post an article to our blog is:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Checkout the blog source &lt;/li&gt;
&lt;li&gt;Add an author yaml entry if they’ve not already got one &lt;/li&gt;
&lt;li&gt;Add a new post folder with a unique name &lt;/li&gt;
&lt;li&gt;Add a markdown article with front matter (author, title, date and tags) &lt;/li&gt;
&lt;li&gt;Run it locally first if you like &lt;/li&gt;
&lt;li&gt;Commit and push &lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We could distil this further, but we shall see how we get on! A couple of ideas are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;don’t require a separate folder, so you just create the markdown article and push &lt;/li&gt;
&lt;li&gt;Support drafts so you can push without publishing, but still view in develop &lt;/li&gt;
&lt;/ul&gt;</content:encoded></item></channel></rss>