conclusion.tex

\chapter{Conclusions and Future Work}
\label{sec:conclusion}

This dissertation attempted to demonstrate the benefits of using model checking and synthesis to generate correct-by-construction driver assistance systems through an application using a cognitive architecture. This required a model of a human driver in a cognitive architecture to be implemented in PRISM's modelling language. Since Salvucci established his driver model in ACT-R in \cite{salvucci_1}, this dissertation focused on the abstraction process to convert the integrated continuous model into a discrete-time Markov chain. This was done using pre-simulation and Monte Carlo simulation techniques, which allowed the generated models to be relatively small in terms of state space and transitions. Afterwards, several designs were considered in terms of the capabilities and placement of the driver assistance system within the model. These options ranged from suggestions at the decision making level to correcting linear or steering acceleration. They were evaluated carefully, and the best performing one was then more extensively compared to the human driver model previously obtained. The results showed a significant performance increase (in terms of the metrics used) by the introduction of the driver assistance system.

Most of the expected contributions were achieved. The modelling of a complex scenario (a 2-lane highway and the interactions that arise with various profiles of drivers e.g. follow, crash or overtake another vehicle) through the abstraction of Salvucci's human driver model presented in \cite{salvucci_1} was achieved through the use of efficient pre-simulation and Monte Carlo simulation techniques and automation of the model writing which lead to compact models (e.g. in \cite{lam}, Lam considers a one way road with at most $270m$, and \cite{games} considers similarly simple scenarios through its model assumptions). For a road length of $500m$, none of the initial conditions considered had a model with more than 700 states, an achievement given the original complexity of the scenario and the constraints of the ACT-R architecture. The establishment of insightful metrics in terms of safety and liveness and their application to multi-objective synthesis of possible ADAS was another central achievement of the dissertation (e.g. in \cite{lam}, only single-objective safety centric metrics are considered). Furthermore, it should be noted that, as shown in Chapter~\ref{sec:results}, the final models (human driver and ADAS) obtained were also manageable in terms of size - with none exceeding 1 million states - an accomplishment as it shows the methodology's flexibility towards even more complex models. These two main achievements combined pave the way for the establishment of a general framework which can be used to tackle similar driving situations (e.g. intersections, multilane highways with more vehicles or urban driving), another aim of the dissertation. 

Despite the achievements and the contributions of this work, there are several limitations to the methodology introduced. The first limitation is the building time of the models. While the models built have a relatively small state space, the building time is quite high for both the human driver model and the full system (human and ADAS), as shown in Section~\ref{sec:res_state}. This is mostly due to the simulation and the tables which result out of this process. While these reduce the state space of the final model, they make the model description quite big (particularly compared to other similarly sized models found in \cite{lam, games}). The model checking tool, in turn, then takes quite some time to process the description files, leading to high building times for the models. Despite this, the time can be reduced by the selection of the right tool and engine (e.g. the sparse engine in Storm outperforms all the engines PRISM supports for the human driver model, but PRISM's hybrid engine outperforms all of Storm's engines in the full model). 

A second limitation has to do with the tools available. As the reader might have noticed, the models presented in Section~\ref{sec:add_exp} were built considering a road segment of $400m$ instead of the $500m$ assumed throughout the rest of the dissertation. This came from the fact that the adversary generation options in PRISM (\texttt{-adv} or \texttt{-advmdp}) make use of the LP engine which, as of right now, is not capable of solving a problem as big as this would have been for the length of $500m$ (despite the MDP having under $500,000$ states). Another limitation of the methodology has to do with the lack of support for the model checking of conditional properties in MDPs in both PRISM and Storm (despite the latter being able to verify them in DTMCs). While PRISM does not support the generation of Pareto curves for more than two objectives, PRISM-games - an extension of PRISM to stochastic multi-player games \cite{prism_games} - does so. However, this tool does not use a symbolic representation of the model and, as such, is quite limited in terms of the size of the models it supports.

The final limitation presented is related to the underlying assumptions made and the usefulness of the driver assistance systems generated. There were quite some assumptions made throughout the dissertation which are not necessarily true. While the control laws that Salvucci presents in \cite{salvucci_1} are the result of corrections made to his original laws presented in \cite{salvucci_0} through the use of real data he collected, most of the assumptions used in this dissertations (e.g. driver profiles) are not. As such, while it is still possible to compare between situations and discuss the advantages of the driver assistance system, it is questionable to what degree the ADAS synthesised in the course of this dissertation are actually accurate and useful. 

In the future, the model description might be reduced through pre-computation and elimination of the guards that are never satisfied within the modules, which make up a significant portion of the file. Furthermore, a data driven approach should be followed to validate the results obtained and correct the assumptions made about the drivers. Once the models are accurate according to the real world data, it is possible to deploy the solutions obtained.

Extensions of this work might include implementations of similar driving situations using the ideas presented in terms of abstraction and driver assistance in order to validate the approach. A framework to formalise and generalise the process of obtaining the driver assistance systems can also be the focus of future research in this area.