Publications

Papers

2024

1. 

   Toward Unified Practices in Trajectory Prediction Research on Drone Datasets

   Theodor Westny, Björn Oskarsson , and Erik Frisk

   arXiv preprint arXiv:2405.00604, 2024

   Abs Bib PDF Code

   The availability of high-quality datasets is crucial for the development of behavior prediction algorithms in autonomous vehicles. This paper highlights the need for standardizing the use of certain datasets for motion forecasting research to simplify comparative analysis and proposes a set of tools and practices to achieve this. Drawing on extensive experience and a comprehensive review of current literature, we summarize our proposals for preprocessing, visualizing, and evaluation in the form of an open-sourced toolbox designed for researchers working on trajectory prediction problems. The clear specification of necessary preprocessing steps and evaluation metrics is intended to alleviate development efforts and facilitate the comparison of results across different studies. The toolbox is available at: https://github.com/westny/dronalize.

   @article{westny2024dronalize,
     title = {Toward Unified Practices in Trajectory Prediction Research on Drone Datasets},
     author = {Westny, Theodor and Oskarsson, Bj{\"o}rn and Frisk, Erik},
     journal = {arXiv preprint arXiv:2405.00604},
     year = {2024},
   }

2. 

   Diffusion-Based Environment-Aware Trajectory Prediction

   Theodor Westny, Björn Oskarsson , and Erik Frisk

   arXiv preprint arXiv:2403.11643, 2024

   Abs Bib PDF

   The ability to predict the future trajectories of traffic participants is crucial for the safe and efficient operation of autonomous vehicles. In this paper, a diffusion-based generative model for multi-agent trajectory prediction is proposed. The model is capable of capturing the complex interactions between traffic participants and the environment, accurately learning the multimodal nature of the data. The effectiveness of the approach is assessed on large-scale datasets of real-world traffic scenarios, showing that our model outperforms several well-established methods in terms of prediction accuracy. By the incorporation of differential motion constraints on the model output, we illustrate that our model is capable of generating a diverse set of realistic future trajectories. Through the use of an interaction-aware guidance signal, we further demonstrate that the model can be adapted to predict the behavior of less cooperative agents, emphasizing its practical applicability under uncertain traffic conditions.

   @article{westny2024diffusion,
     title = {Diffusion-Based Environment-Aware Trajectory Prediction},
     author = {Westny, Theodor and Oskarsson, Bj{\"o}rn and Frisk, Erik},
     journal = {arXiv preprint arXiv:2403.11643},
     year = {2024},
   }

3. 

   Stability-Informed Initialization of Neural Ordinary Differential Equations

   Theodor Westny, Arman Mohammadi , Daniel Jung , and Erik Frisk

   In Proceedings of the 41st International Conference on Machine Learning , 2024

   Abs Bib PDF Code

   This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver’s corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications.

   @inproceedings{westny2023stability,
     title = {Stability-Informed Initialization of Neural Ordinary Differential Equations},
     author = {Westny, Theodor and Mohammadi, Arman and Jung, Daniel and Frisk, Erik},
     booktitle = {Proceedings of the 41st International Conference on Machine Learning},
     pages = {52903--52914},
     year = {2024},
   }

2023

1. 

   MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction with Neural ODEs

   Theodor Westny, Joel Oskarsson , Björn Olofsson , and Erik Frisk

   IEEE Transactions on Intelligent Vehicles, 2023

   Abs Bib PDF Code

   Enabling resilient autonomous motion planning requires robust predictions of surrounding road users’ future behavior. In response to this need and the associated challenges, we introduce our model, titled MTP-GO. The model encodes the scene using temporal graph neural networks to produce the inputs to an underlying motion model. The motion model is implemented using neural ordinary differential equations where the state-transition functions are learned with the rest of the model. Multi-modal probabilistic predictions are provided by combining the concept of mixture density networks and Kalman filtering. The results illustrate the predictive capabilities of the proposed model across various data sets, outperforming several state-of-the-art methods on a number of metrics.

   @article{westny2023graph,
     journal = {IEEE Transactions on Intelligent Vehicles},
     title = {{MTP-GO}: Graph-Based Probabilistic Multi-Agent Trajectory Prediction with Neural {ODEs}},
     year = {2023},
     volume = {8},
     number = {9},
     pages = {4223-4236},
     author = {Westny, Theodor and Oskarsson, Joel and Olofsson, Bj{\"o}rn and Frisk, Erik},
     doi = {10.1109/TIV.2023.3282308},
   }

2. 

   Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction

   Theodor Westny, Joel Oskarsson , Björn Olofsson , and Erik Frisk

   In Intelligent Vehicles Symposium (IV) , 2023

   Abs Bib PDF Code

   Given their flexibility and encouraging performance, deep-learning models are becoming standard for motion prediction in autonomous driving. However, with great flexibility comes a lack of interpretability and possible violations of physical constraints. Accompanying these data-driven methods with differentially-constrained motion models to provide physically feasible trajectories is a promising future direction. The foundation for this work is a previously introduced graph-neural-network-based model, MTP-GO. The neural network learns to compute the inputs to an underlying motion model to provide physically feasible trajectories. This research investigates the performance of various motion models in combination with numerical solvers for the prediction task. The study shows that simpler models, such as low-order integrator models, are preferred over more complex, e.g., kinematic models, to achieve accurate predictions. Further, the numerical solver can have a substantial impact on performance, advising against commonly used first-order methods like Euler forward. Instead, a second-order method like Heun’s can greatly improve predictions.

   @inproceedings{westny2023evaluation,
     title = {Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction},
     author = {Westny, Theodor and Oskarsson, Joel and Olofsson, Bj{\"o}rn and Frisk, Erik},
     booktitle = {Intelligent Vehicles Symposium (IV)},
     year = {2023},
     doi = {10.1109/IV55152.2023.10186615},
   }

3. 

   Analysis of Numerical Integration in RNN-Based Residuals for Fault Diagnosis of Dynamic Systems

   Arman Mohammadi , Theodor Westny, Daniel Jung , and Mattias Krysander

   IFAC-PapersOnLine, 2023

   22nd IFAC World Congress

   Abs Bib PDF

   Data-driven modeling and machine learning are widely used to model the behavior of dynamic systems. One application is the residual evaluation of technical systems where model predictions are compared with measurement data to create residuals for fault diagnosis applications. While recurrent neural network models have been shown capable of modeling complex non-linear dynamic systems, they are limited to fixed steps discrete-time simulation. Modeling using neural ordinary differential equations, however, make it possible to evaluate the state variables at specific times, compute gradients when training the model and use standard numerical solvers to explicitly model the underlying dynamic of the time-series data. Here, the effect of solver selection on the performance of neural ordinary differential equation residuals during training and evaluation is investigated. The paper includes a case study of a heavy-duty truck’s after-treatment system to highlight the potential of these techniques for improving fault diagnosis performance.

   @article{mohammadi2023analysis,
     title = {Analysis of Numerical Integration in {RNN}-Based Residuals for Fault Diagnosis of Dynamic Systems},
     author = {Mohammadi, Arman and Westny, Theodor and Jung, Daniel and Krysander, Mattias},
     journal = {IFAC-PapersOnLine},
     volume = {56},
     number = {2},
     pages = {2909-2914},
     year = {2023},
     note = {22nd IFAC World Congress},
   }

2022

1. 

   Uncertainties in Robust Planning and Control of Autonomous Tractor-Trailer Vehicles

   Theodor Westny, Björn Olofsson , and Erik Frisk

   In 15th International Symposium on Advanced Vehicle Control (AVEC’22) , 2022

   Abs Bib PDF

   To study the effects of uncertainty in autonomous motion planning and control, an 8-DOF model of a tractor-semitrailer is implemented and analyzed. The implications of uncertainties in the model are then quantified and presented using sensitivity analysis and closed-loop simulations. The analysis reveals that the significance of various model parameters varies depending on the specific scenario under investigation. By using sampling-based closed-loop predictions, uncertainty bounds on state variable trajectories are determined. Our findings suggest the potential for the inclusion of our method within a robust predictive controller or as a driver-assistance system for rollover or lane departure warnings.

   @inproceedings{westny2022robust,
     title = {Uncertainties in Robust Planning and Control of Autonomous Tractor-Trailer Vehicles},
     author = {Westny, Theodor and Olofsson, Björn and Frisk, Erik},
     booktitle = {15th International Symposium on Advanced Vehicle Control (AVEC'22)},
     year = {2022},
   }

2021

1. 

   Vehicle Behavior Prediction and Generalization Using Imbalanced Learning Techniques

   Theodor Westny, Erik Frisk , and Björn Olofsson

   In International Conference on Intelligent Transportation Systems (ITSC) , 2021

   Abs Bib PDF Code

   The use of learning-based methods for vehicle behavior prediction is a promising research topic. However, many publicly available data sets suffer from class distribution skews which limits learning performance if not addressed. This paper proposes an interaction-aware prediction model consisting of an LSTM autoencoder and SVM classifier. Additionally, an imbalanced learning technique, the multiclass balancing ensemble is proposed. Evaluations show that the method enhances model performance, resulting in improved classification accuracy. Good generalization properties of learned models are important and therefore a generalization study is done where models are evaluated on unseen traffic data with dissimilar traffic behavior stemming from different road configurations. This is realized by using two distinct highway traffic recordings, the publicly available NGSIM US-101 and I80 data sets. Moreover, methods for encoding structural and static features into the learning process for improved generalization are evaluated. The resulting methods show substantial improvements in classification as well as generalization performance.

   @inproceedings{westny2021vehicle,
     title = {Vehicle Behavior Prediction and Generalization Using Imbalanced Learning Techniques},
     author = {Westny, Theodor and Frisk, Erik and Olofsson, Bj{\"o}rn},
     booktitle = {International Conference on Intelligent Transportation Systems (ITSC)},
     pages = {2003--2010},
     year = {2021},
     organization = {IEEE},
     doi = {10.1109/ITSC48978.2021.9564948},
   }

Theses

2024

1. Context-Aware Behavior Prediction for Autonomous Driving

   Theodor Westny

   Dec 2024

   No. 2419

   Abs PDF

   Autonomous vehicles (AVs) are set to transform transportation by providing safer, more efficient, and accessible mobility solutions. However, deploying AV systems requires designers to ensure these vehicles can navigate complex, dynamic traffic environments safely and precisely. A vital component of this capability is the ability to predict the behavior of surrounding road users, yet achieving reliable predictions is a complex task. This thesis investigates several challenges in trajectory and intention prediction for autonomous driving, focusing on contextual awareness, probabilistic modeling, and differential motion constraints. A primary focus of this thesis is context awareness, which includes interaction-aware (agent-to-agent) and environment-aware (road-to-agent) modeling. Early approaches to context awareness involved manually crafting interaction features. While effective, these methods rely on predefined heuristics and often scale poorly as environmental complexity increases. To address these limitations, the thesis adopts a graph-based approach that offers greater flexibility and expressiveness. By constructing relational graphs, graph neural networks can be used to learn agent interactions and environmental context in a data-driven manner. The thesis proposes several context-aware models and provides an extensive evaluation of their mechanisms, highlighting their overall impact on prediction performance. Another core theme of this thesis is addressing the inherent uncertainty and non-determinism of traffic environments. This involves creating models that provide probabilistic predictions. Given the multimodal nature of road-traffic agent behavior, it is also important to design methods that offer multiple candidate predictions for a single condition, enabling AVs to account for different possible future outcomes. This thesis proposes several context-aware frameworks that leverage probabilistic modeling, illustrating how ensemble methods, mixture density networks, and diffusion-based generative models can be adapted to provide uncertainty estimates and multimodal predictions. While neural networks are well-suited for capturing the complex dynamics of traffic, they do not inherently ensure that outputs conform to physical laws, which poses risks in safety-critical applications. To address this, the thesis incorporates differential motion constraints into the prediction framework to ensure that predicted trajectories are accurate, physically feasible, and robust to noise. In addition to improving prediction accuracy, it is demonstrated how this integration enhances interpretation, extrapolation, and generalization capabilities. The use of neural ordinary differential equations is a central component of this thesis, providing a data-driven approach to modeling the motion of agents—such as pedestrians—that are challenging to describe using physical laws or rule-based methods. The thesis investigates their application beyond motion prediction to a wide array of sequence modeling tasks, analyzing the effects of numerical integration techniques, stability regions, and initialization methods on model performance. A key contribution is the proposal of a stability-informed initialization (SII) technique, which significantly enhances model convergence, training stability, and prediction accuracy across various learning benchmarks.

2023

1. Data-Driven Interaction-Aware Behavior Prediction for Autonomous Vehicles

   Theodor Westny

   May 2023

   No. 1960

   Abs PDF

   Future progress toward the realization of fully self-driving vehicles still re-quires human-level social compliance, arguably dependent on the ability to accurately forecast the behavior of surrounding road users. Due to the inter-connected nature of traffic participants, in which the actions of one agent can significantly influence the decisions of others, the development of behavior pre-diction methods is crucial for achieving resilient autonomous motion planning. As high-quality data sets become more widely available and many vehicles already possess significant computing power, the possibility of adopting a data-driven approach for motion prediction is increasing. The first contribution is the design of an intention-prediction model based on autoencoders for highway scenarios. Specifically, the method targets the problem of data imbalance in highway traffic data using ensemble methods and data-sampling techniques. The study shows that commonly disregarded information holds potential use for improved prediction performance and the importance of dealing with the data imbalance problem. The second contribution is the development of a probabilistic motion pre-diction framework. The framework is used to evaluate various graph neural network architectures for multi-agent prediction across various traffic scenarios. The graph neural network computes the inputs to an underlying motion model, parameterized using neural ordinary differential equations. The method additionally introduces a novel uncertainty propagation approach by combining Gaussian mixture modeling and extended Kalman filtering techniques. The third contribution is attributed to the investigation of combing data-driven models with motion modeling and methods for numerical integration. The study illustrates that improved prediction performance can be achieved by the inclusion of differential constraints in the model, but that the choice of motion model as well as numerical solver can have a large impact on the prediction performance. It is also shown that the added differential constraints improve extrapolation properties compared to complete black-box approaches. The thesis illustrates the potential of data-driven methods and their usability for the behavior prediction problem. Still, there are additional challenges and interesting questions to investigate—the main one being the investigation of their use in autonomous navigation frameworks.

2020

1. Modeling and Lateral Control of Tractor-Trailer Vehicles during Aggressive Maneuvers

   Carl Hynén , and Theodor Westny

   Linköping University, The Division of Automatic Control , May 2020

   MSc. Thesis

   Abs PDF

   In the last decades, the development of self-driving vehicles has rapidly increased. Improvements in algorithms, as well as sensor and computing hardware have led to self-driving technologies becoming a reality. It is a technology with the potential to radically change how society interacts with transportation. One crucial part of a self-driving vehicle is control schemes that can safely control the vehicle during evasive maneuvers. This work investigates the modeling and lateral control of tractor-trailer vehicles during aggressive maneuvers. Models of various complexity are used, ranging from simple kinematic models to complex dynamic models, which model tire slip and suspension dynamics. The models are evaluated in simulations using TruckMaker, which is a high fidelity vehicle simulator. Several lateral controllers are proposed based on Model predictive control (MPC) and linear-quadratic (LQ) control techniques. The controllers use different complex prediction models and are designed to minimize the path-following error with respect to a geometric reference path. Their performance is evaluated on double lane change maneuvers of various lengths and with different longitudinal speeds. Additionally, the controllers’ robustness against changes in trailer mass, weight distribution, and road traction is investigated. Extensive simulations show that dynamic prediction models are necessary to keep the control errors small when performing maneuvers that result in large lateral accelerations. Furthermore, to safely control the tractor-trailer vehicle during high speeds, it is a necessity to include a model of the trailer dynamics. The simulation study also shows that the proposed LQ controllers have trouble to evenly balance tractor and trailer deviation from the path, while the MPC controllers handle it much better. Additionally, a method for approximately weighting the trailer deviation is shown to improve the performance of both the LQ and MPC controllers. Finally, it is concluded that an MPC controller with a dynamic tractor-trailer model is robust against model errors, and can become even more robust by tuning the controller weights conservatively.