-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathimplementation.html
527 lines (445 loc) · 28.5 KB
/
implementation.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta content="width=device-width, initial-scale=1.0" name="viewport">
<title>Implementation - TrueLearn</title>
<meta content="" name="description">
<meta content="" name="keywords">
<!-- Favicons -->
<link href="assets/img/logos/TL-favicon.png" rel="icon">
<link href="assets/img/logos/TL-logo.png" rel="apple-touch-icon">
<!-- Google Fonts -->
<link href="https://fonts.googleapis.com/css?family=Open+Sans:300,300i,400,400i,700,700i|Raleway:300,400,500,700,800|Montserrat:300,400,700" rel="stylesheet">
<!-- Vendor CSS Files -->
<link href="assets/vendor/aos/aos.css" rel="stylesheet">
<link href="assets/vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<link href="assets/vendor/bootstrap-icons/bootstrap-icons.css" rel="stylesheet">
<link href="assets/vendor/boxicons/css/boxicons.min.css" rel="stylesheet">
<link href="assets/vendor/glightbox/css/glightbox.min.css" rel="stylesheet">
<link href="assets/vendor/swiper/swiper-bundle.min.css" rel="stylesheet">
<!-- Template Main CSS File -->
<link href="assets/css/style.css" rel="stylesheet">
<!-- =======================================================
* Template Name: Reveal - v4.10.0
* Template URL: https://bootstrapmade.com/reveal-bootstrap-corporate-template/
* Author: BootstrapMade.com
* License: https://bootstrapmade.com/license/
======================================================== -->
</head>
<body>
<!-- ======= Header/top bar ======= -->
<header id="header" class="d-flex align-items-center">
<div class="container d-flex justify-content-between">
<div id="logo">
<h1><a href="index.html">True<span>Learn</span></a></h1>
<!-- Uncomment below if you prefer to use an image logo -->
<!-- <a href="index.html"><img src="assets/img/logo.png" alt=""></a>-->
</div>
<nav id="navbar" class="navbar">
<ul>
<li class="dropdown"><a href="index.html"><span>Home</span> <i class="bi bi-chevron-down"></i></a>
<ul>
<li><a class="nav-link scrollto" href="index.html#about">Abstract</a></li>
<li><a class="nav-link scrollto" href="index.html#portfolio">Portfolio Video</a></li>
<li><a class="nav-link scrollto" href="index.html#timeline">Project Timeline</a></li>
<li><a class="nav-link scrollto" href="index.html#team">Team</a></li>
</ul>
</li>
<li><a class="nav-link" href="requirements.html">Requirements</a></li>
<li><a class="nav-link" href="research.html">Research</a></li>
<li><a class="nav-link" href="design.html">System Design</a></li>
<li><a class="nav-link" href="implementation.html">Implementation</a></li>
<li><a class="nav-link" href="testing.html">Testing</a></li>
<li><a class="nav-link" href="evaluation.html">Evaluation</a></li>
<li class="dropdown"><a href="appendices.html"><span>Appendices</span><i class="bi bi-chevron-down"></i></a>
<ul>
<li><a class="nav-link scrollto" href="appendices.html#documentation">Documentation</a> </li>
<li><a class="nav-link scrollto" href="appendices.html#development-blog">Development Blog</a></li>
<li><a class="nav-link scrollto" href="appendices.html#development-videos">Monthly Videos</a> </li>
<li><a class="nav-link scrollto" href="appendices.html#license">License</a> </li>
<li><a class="nav-link scrollto" href="appendices.html#privacy-policy">Privacy policy</a> </li>
</ul>
</li>
</ul>
<i class="bi bi-list mobile-nav-toggle"></i>
</nav><!-- .navbar -->
</div>
</header><!-- End Header -->
<!-- ======= Banner ======= -->
<section id="banner">
<div class="banner-content" data-aos="fade-up">
<h2>Implementation</h2>
</div>
</section>
<!-- End Banner -->
<main>
<section id="implementation">
<div class="container" data-aos="fade-up">
<div class = "section-header">
<h2 id="model-implementation">Models</h2>
<p>Models of the learner and learning events are essential parts of the TrueLearn library as they form the basis of the classification and prediction algorithms. These representations are found in <code>truelearn.models</code> .</p>
</div>
<div class = "row">
<div>
<h3>Representing Knowledge</h3>
<p>The Knowledge Component (KC) is the atomic unit of knowledge used in the TrueLearn library. It represents a Wikipedia topic with an associated <strong>mean</strong> and <strong>variance</strong>. The mean represents the estimated skill associated with that KC. The variance represents the confidence of the estimation.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/kc.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>As KC is the atomic unit of knowledge, we design a class <code>Knowledge</code> to group and store KCs by their <code>id</code>.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/knowledgev1.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>The <code>Knowledge</code> class is used to model the learner and event.</p>
<p>For the learner, the KC in <code>Knowledge</code> estimates how well the learner knows about this topic. For the event, the KC in <code>knowledge</code> estimates how well the learning event covers the topic.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/dataclasses-events.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Extendable Knowledge Components</h3>
<p>To ensure developers can freely extend the KC class based on their application, we avoid referring to <code>KnowledgeComponent</code> directly in <code>Knowledge</code>. Instead, we define an interface <code>BaseKnowledgeComponent</code> that specifies the public APIs for all KCs.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/baseKC.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>With the interface above, the implementation of <code>Knowledge</code> does not need to refer to any concrete implementation.</p> </div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/knowledge-final.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Updating and Transferring Knowledge</h3>
<p>As a family of Bayesian algorithms, Truelearn classifiers, which utilise the <code>truelearn.models</code>, need to update the prior estimation of the existing KCs and create new KCs in the learner model based on the given learning event.</p>
<p>Thus, we specify two additional APIs in <code>BaseKnowledgeComponent</code> that all KCs need to implement.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/basekc-update-clone.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
</div>
<div class="section-header">
<h2 id="classifier-implementation">Classifiers</h2>
<p>The <code>truelearn.learning</code> sub-package implements all the classifiers in the truelearn library.</p> </div>
<div>
<h3>Dependencies</h3>
<ul>
<li>
<p>trueskill: a rating system among game players. It was developed by Microsoft Research and has been used on Xbox LIVE for ranking and matchmaking services. This system quantifies players’ TRUE skill points by the Bayesian inference algorithm.</p>
</li>
<li>
<p>mpmath: a free (BSD licensed) Python library for real and complex floating-point arithmetic with arbitrary precision.</p>
</li>
</ul>
</div>
<div class="row">
<div>
<h3>Public Interface Walkthrough</h3>
<p>Based on what we outlined in our study, we designed a <code>BaseClassifier</code> class to ensure a consistent interface across all our classifiers.</p>
<p>The class <code>BaseClassifier</code> follows the naming convention of the scikit-learn library:</p>
<ul>
<li>The <code>fit</code> method is used to train the classifier</li>
<li>The <code>predict</code> and <code>predict_proba</code> methods use the classifier to make predictions.</li>
<li>The <code>get_params</code> and <code>set_params</code> methods are used to get or set the parameters of the classifier.</li>
</ul>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/base-classifier.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Defining Parameter Constraints</h3>
<p>When the developer initializes the classifier, they need to pass in the parameters for the classifier. Then, the classifier implementer usually needs to implement checks inside classifiers to verify that the given parameters have the correct type and value. To simplify this process and provide a simple yet powerful way to validate parameters, we used parameter constraints.</p>
<p>There are three kinds of ways to express different constraints on parameters. <code>TypeConstraint</code> ensures the type of the parameters must match one of the types in the <code>TypeConstraint</code> class. <code>ValueConstraint</code> checks if the value of the parameters matches one of the values in the <code>ValueConstraint</code> class. <code>FuncConstraint</code> ensures that the parameters satisfy some user-defined functions.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/constraints.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>Then, in all classifiers, we define a class attribute <code>_parameter_constraints</code>, which maps parameter names to a constraint or a list of constraints.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/param-constraints.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>When the classifier object is initialized, or its parameters are changed via <code>set_params</code>, the classifier runs a check to verify that all parameters satisfy the constraints.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/validate-params.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Getting and Setting Parameters</h3>
<p>To reduce code redundancy, we implement <code>get_params</code> and <code>set_params</code> in <code>BaseClassifier</code>. We use the Python reflection mechanism via <code>hasattr</code>, <code>getattr</code> and <code>setattr</code> to do this, as we need to support the inspection and modification of any classifier object at runtime.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/get-set-params.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Training the Classifier</h3>
<p>The core part of truelearn.learning is three classifiers (<code>KnowledgeClassifier</code>, <code>NoveltyClassifier</code> and <code>InterestClassifier</code>).</p>
<p>The methods used to train them are not identical but follow a similar pattern. Thus, we apply the template pattern when implementing <code>fit</code>. First, these three classifiers have a <code>positive_only</code> mode, during which the classifiers update the learner model iff the learner engages in the given event <code>x</code>. Second, besides the learner model, the classifiers also update some statistics related to the model via <code>self.__update_engagement_stats</code>, which is always invoked regardless of the <code>positive_only</code> mode.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/fit.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>In <code>__update_knowledge_representation</code>, we also apply the strategy pattern as this process can be divided into three parts:</p>
<ul>
<li>based on the given KCs in the event model, create (in case the corresponding KCs are not in the learner model) or select KCs from the learner model</li>
<li>update the selected KCs</li>
<li>commit the changes to the learner model</li>
</ul>
<p>It turns out that part 1 and part 3 are the same across these 3 classifiers. They only differ in how they update the selected KCs (the <code>self._generate_ratings</code> method). Thus, we define the following template for <code>__update_knowledge_representation</code>.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/update-knowledge.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>The method left to each classifier is <code>self._generate_ratings</code> since each classifier relies on different hypotheses presented in the TrueLearn paper and updates the selected KCs differently.</p>
</div>
<div>
<h3>Predicting Learner Engagement</h3>
<p>All three classifiers rely on decision threshold to output binary classification. Thus, we can define <code>predict</code> method like this:</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/predict.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>For <code>predict_proba</code>, since each classifier relies on different assumptions, they have different ways to evaluate the matching quality between the learner and the learning event. It's worth noting that the return value of <code>self._eval_matching_quality</code> is between 0 and 1.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/predict-proba.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
</div>
<div class="section-header">
<h2 id="datasets">Loading Datasets</h2>
<p>Truelearn comes with a datasets sub-package to load datasets. These datasets are useful to quickly illustrate the behaviour of the various classifiers implemented in truelearn.</p>
</div>
<div>
<h3>Dependencies</h3>
<ul>
<li>PEEK Dataset: a large, novel and open-source dataset of learners engaging with educational videos</li>
</ul>
</div>
<div class = "row">
<div>
<h3>Downloading the PEEK dataset</h3>
<p>Since the size of the PEEK dataset is over 3MB, it is not feasible for us to embed it into truelearn as it would unnecessarily bloat our library. Therefore, we implement a download method to download the PEEK dataset from Github and use the <code>sha256sum</code> method to check the integrity of the downloaded file.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/download-peek.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>To avoid unnecessary downloading, we implement a simple cache mechanism based on sha256. We only download the file if the file does not exist or if the hash of the file does not match the expected hash.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/caching.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Parsing the PEEK dataset mapping</h3>
<p>The PEEK Dataset contains an <code>id_to_wiki_metadata_mapping.csv</code> file which stores a mapping from Wikipedia id to its title, description, and url. It looks like this:</p>
<pre><code>id,url,title,description
0,"https://en.wikipedia.org/wiki/""Hello,_World!""_program","""Hello, World!"" program",Traditional beginners' computer program
...</code></pre>
<p>Based on this format, we designed a <code>__build_mapping</code> method:</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/build-mapping.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Parsing the PEEK dataset events</h3>
<p>PEEK Dataset contains two other files <code>train.csv</code> and <code>test.csv</code>, which have the same format as shown below.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<br>
<img src="assets/img/implementation/peek-datasets-column.png" class="img-fluid" alt="">
<p style="text-align: center"><strong>[1]</strong></p>
</div>
<div class = "col-lg-3"></div>
<div>
<p>[1] S. Bulathwela, M. Perez-Ortiz, E. Novak, E. Yilmaz, and J. Shawe-Taylor, “PEEK: A Large Dataset of Learner Engagement with Educational Videos,” arXiv.org, 2021. https://arxiv.org/abs/2109.03154 (accessed Apr. 09, 2023).
</p>
</div>
<div>
<p>By using this format, we can write two methods <code>__restructure_line</code> and <code>__restructure_data</code>.</p>
<p>In <code>__restructure_line</code>, we:</p>
<ol>
<li>Unpack the data</li>
<li>Convert them to the correct types</li>
<li>Construct KCs, <code>Knowledge</code> and <code>EventModel</code></li>
<li>Return the <code>(learner_id, event_model, label)</code> tuple</li>
</ol>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/restructure-line.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>In <code>__restructure_data</code>, we</p>
<ol>
<li>Restructure each line by using <code>__restructure_line</code></li>
<li>Append the returned <code>EventModel</code> and label to a dictionary</li>
<li>Return the dictionary in list format</li>
</ol>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/restructure-data.png" class="img-fluid" alt="">
</div>
</div>
<div class="section-header">
<h2 id="datasets">Visualisations</h2>
<p>The <code>truelearn.utils.visualisations</code> package equips users with tools to generate various visualisations. </p>
</div>
<div>
<h3>Dependencies</h3>
<ul>
<li>Matplotlib: the most popular Python graphing library for generating visualisations. Provides advanced customisation options for plots.</li>
<li>Plotly: a Python graphing library built on top of Matplotlib. Provides fewer plots but charts generated with Plotly are interactive and can be exported to HTML.</li>
</ul>
<p>Due to interactivity being a requirement for our project, we decided to use Plotly whenever possible and use Matplotlib only to generate charts that were not possible with Plotly.</p>
</div>
<div class = "row">
<div>
<h3>BasePlotters</h3>
<p>Initially we decided to write a series of plotters, with each plotter providing methods to generate a different visualisation and we designed a <code>BasePlotter</code> class to define the core methods across all plotters. These methods are: </p>
<ul>
<li><code>_standardise_data</code> and <code>_get_kc_details</code>: to clean and standardise the data, converting a Knowledge object into something that could be used by Plotly and Matplotlib.</li>
<li><code>savefig</code>: to save the visualisation to an external file.
</ul>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/baseplotter.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<p>Since we had to generate visualisations with both Plotly and Matplotlib, we also created the <b>BasePlotlyPlotter</b> and <b>BaseMatplotlibPlotter</b> classes that inherit from <b>BasePlotter</b> and extend its functionality by adding methods unique to their respective graphing libraries. This way, we ensured that all current and future plotters would share the same interface regardless of the library they relied on.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/plotly-matplotlib-base.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Preparing the data</h3>
<p>In the <code>_standardise_data</code> method, we convert the given <code>Knowledge</code> object into a Python dictionary via the <code>knowledge_to_dict</code> function and we extract information on the desired topics from all the knowledge components.</p>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/visualisations-prepare-data.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Plotting the data</h3>
<p>The contents of the <code>plot</code> method vary for each plotter, but they all follow the same structure:</p>
<ol>
<li>Initially, <code>_standardise_data</code> is called inside <code>plot</code> to clean the data.</li>
<li>The data returned from <code>_standardise_data</code> is filtered to obtain the desired information to visualise in each chart.</li>
<li>Using the additional arguments passed in for customisation by the user (e.g. history), the visualisation is generated and stored in <code>self</code>.</li>
</ol>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<br>
<img src="assets/img/implementation/visualisations-plot.png" class="img-fluid" alt="">
</div>
<div class = "col-lg-3"></div>
<div>
<h3>Saving the visualisation</h3>
<p>Implementing this feature was straightforward since it's the same regardless of the plotter. The only difference came when considering whether the plotter relied on Matplotlib or Plotly. Making this distinction is important as different libraries provide different formats to export to. For example, with Plotly it is possible to export to HTML but that is not the case for Matplotlib. For now, the plotters are based on either Matplotlib and Plotly, so there are only 2 variations of the <code>savefig</code> method:</p>
<ul>
<li>The Matplotlib one:</li>
</ul>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/matplotlib-savefig.png" class="img-fluid" alt="">
</div>
<div>
<ul>
<li>and the Plotly one:</li>
</ul>
</div>
<div class = "col-lg-3"></div>
<div class = "col-lg-6 content">
<img src="assets/img/implementation/plotly-savefig.png" class="img-fluid" alt="">
</div>
</div>
<div class="section-header">
<h2>Additional Information</h2>
<p>For more information concerning our implementation, please refer to the <a href="https://truelearn.readthedocs.io/en/latest/dev/design_considerations.html">Design Considerations</a> section of our documentation.</p>
</div>
</div>
</section>
</main>
<!-- ======= Footer ======= -->
<footer id="footer">
<div class="container">
<div class="copyright">
© Copyright <strong>TrueLearn</strong>. All Rights Reserved
</div>
<div class="credits">
<!--
All the links in the footer should remain intact.
You can delete the links only if you purchased the pro version.
Licensing information: https://bootstrapmade.com/license/
Purchase the pro version with working PHP/AJAX contact form: https://bootstrapmade.com/buy/?theme=Reveal
-->
<div class = "text-center">Designed by Karim Djemili</div>
Theme from <a href="https://bootstrapmade.com/">BootstrapMade</a>
</div>
</div>
</footer><!-- End Footer -->
<a href="#" class="back-to-top d-flex align-items-center justify-content-center"><i class="bi bi-arrow-up-short"></i></a>
<!-- Vendor JS Files -->
<script src="assets/vendor/aos/aos.js"></script>
<script src="assets/vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<script src="assets/vendor/glightbox/js/glightbox.min.js"></script>
<script src="assets/vendor/isotope-layout/isotope.pkgd.min.js"></script>
<script src="assets/vendor/swiper/swiper-bundle.min.js"></script>
<!-- Template Main JS File -->
<script src="assets/js/main.js"></script>
</body>
</html>