An Evaluation Methodology for VANET Applications Combining Simulation and Multi-sensor Experiments

Dominique Seydel, Sebastian Bittl, Jakob Pfeiffer, Josef Jiru, Hanno Beckmann, Kathrin Frankl, Bernd Eissfeller


Wireless vehicular networks are in the wake of mass deployment both in Europe and the USA. These networks introduce a new promising source of information about vehicular environments usable by cooperative advanced driver assistance systems (ADAS). However, development and evaluation of such ADAS is still challenging. Thus, we propose a methodology for their development and evaluation process. It is applied to evaluate the fulfillment of requirements on position accuracy information within the communicated data sets. Accuracy requirements are only roughly defined and not sufficiently evaluated in real world environments. This holds especially for GNSS (Global Navigation Satellite Systems) optimized for maximum integrity of obtained positions, which is required for safety critical ADAS to increase robustness and reliability. Our main goal is to determine whether position accuracy provided by GNSS is sufficient for cooperative ADAS. Thereby, we find that pure GNSS input cannot fulfill position accuracy requirements in most test cases.


  1. Behrisch, M., Bieker, L., Erdmann, J., and Krajzewicz, D. (2011). SUMO - Simulation of Urban MObility: An Overview. In The Third International Conference on Advances in System Simulation, pages 63-68.
  2. Berger, C. and Rumpe, B. (2012). Engineering Autonomous Driving Software. In Experience from the DARPA Urban Challenge. Springer, pages 243-272.
  3. Blackman, S. S. (2004). Multiple Hypothesis Tracking for Multiple Target Tracking. IEEE Aerospace and Electronic Systems Magazine, 19(1):5-18.
  4. Broggi, A. et al. (2012). High Performance MultiTrack Recording System for Automotive Applications. In International Journal of Automotive Technology, 13(1):pages 123-132.
  5. Chrisofakis, E., Junghanns, A., Kehrer, C., and Rink, A. (2011). Simulation-based development of automotive control software with Modelica. In 8th International Modelica Conference, pages 20-22.
  6. Cohda Wireless (2013). MK4a V2X Evaluation Kit. online: lessMK4a.pdf. accessed: 28.10.2015.
  7. Christopher, T. (2009). Analysis of dynamic scenes: Application to driving assistance. Doctoral dissertation, Institut National Polytechnique de Grenoble-INPG.
  8. Hanzlik, A. (2013). Simulation-based Application Software Development in Time-Triggered Communication Systems. International Journal of Software Engineering & Applications, 4(2):75-92.
  9. Harding, J., Powell, G. R., Yoon, R. F., J., Doyle, C., Sade, D., Lukuc, M., Simons, J., and Wang, J. (2014). Vehicle-to-Vehicle Communications: Readiness of V2V Technology for Application. Technical Report DOT HS 812 014, Washington, DC: National Highway Traffic Safety Administration.
  10. Hofmann-Wellenhof, B. and Lichtenegger, H. and Wasle, E. (2007). GNSS - Global Navigation Satellite Systems: GPS, GLONASS, Galileo, and more. Springer.
  11. Huckle, T. and Schneider, S. (2006). Numerische Methoden: Eine Einführung für Informatiker, Naturwissenschaftler, Ingenieure und Mathematiker. Springer Berlin Heidelberg, 2nd edition.
  12. Joerer, S., Segata, M., Bloessl, B., Cigno, R. L., Sommer, C., and Dressler, F. (2012). To crash or not to crash: Estimating its likelihood and potentials of beacon-based IVC systems. IEEE Vehicular Networking Conference (VNC), pages 25-32.
  13. Kaplan, E.D. and Hegarty, C.J. (2005). Understanding GPS: Principles and Applications. Artech House mobile communications series, Artech House.
  14. Kluge, T. (2011). Library for Cubic Spline interpolation in C++. online: accessed: 06.11.2015.
  15. Lytrivis, P., Thomaidis, G. and Amditis, A. (2008). Cooperative path prediction in vehicular environments. In 11th International Conference on Intelligent Transportation Systems (ITSC), pages 803-808.
  16. Madas, D., Nosratinia, M., Keshavarz, M., Sundstrom, P., Philippsen, R., Eidehall, A. and Dahlen, K. M. (2013). On path planning methods for automotive collision avoidance. In Intelligent Vehicles Symposium (IV), pages 931-937.
  17. Misra, P. and Enge, P. (2001). Global positioning system: signals, measurements, and performance. GangaJamuna Press.
  18. Navipedia (2015). The collaborative GNSS encyclopaedia. online: accessed: 06.11.2015.
  19. Riley, G. F. and Henderson, T. R. (2010). The ns-3 Network Simulator. In Wehrle, K., Günes, M., and Gross, J., editors, Modeling and Tools for Network Simulation, pages 15-34. Springer Berlin Heidelberg.
  20. M. Rondinone et al (2013). iTETRIS: A modular simulation platform for the large scale evaluation of cooperative ITS applications. In Simulation Modelling Practice and Theory 34, pages 99-125.
  21. Roscher, K., Bittl, S., Gonzalez, A. A., Myrtus, M., and Jiru, J. (2014). ezCar2X: Rapid-Prototyping of Communication Technologies and Cooperative ITS Applications on Real Targets and Inside Simulation Environments. In 11th Conference Wireless Communication and Information, pages 51 - 62.
  22. Roscher, K., Jiru, J., Gonzalez, A., and Heidrich, W. (2013). ezCar2X: A Modular Software Framework for Rapid Prototyping of C2X Applications. In 9th ITS European Congress.
  23. Sharp, G., Lin, P.-C., and Komsuoglu, H. (2003). Ground Truth Measurement System. online: accessed: 01.10.2015.
  24. Sommer, C., German, R., and Dressler, F. (2011). Bidirectionally coupled network and road traffic simulation for improved IVC analysis. IEEE Transactions on Mobile Computing, 10.1: pages 3-15.
  25. B. Strasser et al. (2010). Networking of test and simulation methods for the development of advanced driver assistance systems (ADAS). In 4. Tagung Sicherheit durch Fahrerassistenz.
  26. Takasu, T. (2009). RTKLIB: Open Source Program Package for RTK-GPS, FOSS4G.
  27. Stiller, C., Frber, G. and Kammel, S. (2007). Cooperative cognitive automobiles. In IEEE Intelligent Vehicles Symposium, pages 215-220.
  28. Streit, R. L., and Luginbuhl, T. E. (1995). Probabilistic multi-hypothesis tracking. Naval Underwater Systems Center Newport RI.
  29. Tan, H.-S., and Huang, J. (2006). DGPS-based vehicleto-vehicle cooperative collision warning: Engineering feasibility viewpoints. IEEE Transactions on Intelligent Transportation Systems, 7.4: pages 415-428.
  30. Voigtländer, P. (2008). ADTF: Framework for Driver Assistance and Safety Systems. In FISTA World Automotive Congress.
  31. Wegener, A., Piórkowski, M., Raya, M., Hellbrück, H., Fischer, S. and Hubaux, J.-P. (2008). TraCI: An Interface for Coupling Road Traffic and Network Simulators. 11th Communications and Networking Simulation Symposium, pages 155-163.
  32. Winner, H. and Hakuli, S. and Wolf, G. (2009). Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort. ATZ/MTZ-Fachbuch, Vieweg+Teubner Verlag.
  33. Yan, G., Olariu, S., and Weigle, M. C. (2008). Providing VANET Security through Active Position Detection. Computer Communications, Mobility Protocols for ITS/VANET, 31(12):2883-2897.
  34. Ziegler, J., Bender, P., Dang, T. and Stiller, C. (2014). Trajectory planning for Bertha A local, continuous method. In Intelligent Vehicles Symposium Proceedings, pages 450-457.

Paper Citation

in Harvard Style

Seydel D., Bittl S., Pfeiffer J., Jiru J., Beckmann H., Frankl K. and Eissfeller B. (2016). An Evaluation Methodology for VANET Applications Combining Simulation and Multi-sensor Experiments . In Proceedings of the International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS, ISBN 978-989-758-185-4, pages 213-224. DOI: 10.5220/0005799802130224

in Bibtex Style

author={Dominique Seydel and Sebastian Bittl and Jakob Pfeiffer and Josef Jiru and Hanno Beckmann and Kathrin Frankl and Bernd Eissfeller},
title={An Evaluation Methodology for VANET Applications Combining Simulation and Multi-sensor Experiments},
booktitle={Proceedings of the International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS,},

in EndNote Style

JO - Proceedings of the International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS,
TI - An Evaluation Methodology for VANET Applications Combining Simulation and Multi-sensor Experiments
SN - 978-989-758-185-4
AU - Seydel D.
AU - Bittl S.
AU - Pfeiffer J.
AU - Jiru J.
AU - Beckmann H.
AU - Frankl K.
AU - Eissfeller B.
PY - 2016
SP - 213
EP - 224
DO - 10.5220/0005799802130224