Unity Game Engine- Why do we need it for testing?

Sure – exciting partial successes can be recorded. Be it Kopernikus Automotive with its Automated Valet Parking, about which you can find an interview here, at whose tech demo at the IAA Mobility 2021 you could actually see unmanned cars. Or ELMO Rent from Estonia, which would like to realize remotely driving rental vehicles in the first step and independently to the customer in the second step. The latter example in particular also highlights one of the problems that cannot be overlooked in the industry that wants to drive autonomously: How do we develop the autonomous vehicle? How do we test these vehicles, for whose autonomous driving concept countless tests must simulate equally countless traffic situations so that the vehicle can learn? How do we implement this, when driving in real traffic is simply impossible? How do we reconcile all the traffic situations, sensors, artificial intelligences, the occurrence of situations that are difficult to recreate, in a way that allows real testing and thus opens the door to progress? It is not for nothing that the motto of Cognizant Mobility is: “Develop and test today what will be on the road tomorrow”. This motto gives rise to a compelling imperative: We have to test, and we have to test today, or nothing will run tomorrow.

So it needs environments where the automotive industry can test. Environments with as much realism as possible, with scenarios that can be recreated down to the smallest detail, so that A.I.s can learn, data can be collected and analyzed, so that testing can be automated to move forward, literally and figuratively. This is where – finally – the Unity game engine comes into play. Well, actually, these and other engines that emerged from the gaming sector and are now able to solve our problems. Today, however, we are focusing on the area of the Unity Game Engine, have interviewed expert specialists for this purpose, and today we present an exciting article that deals with the implementation of development requirements for virtualized solutions.

The use of the Unity Game Engine in testing

The automotive industry has been a bit left out in the cold when it comes to developing autonomous driving functions: Before something can hit the road, it has to be tested. You would have to test a car that can, for example, recognize a dangerous situation and brake, but under conditions that are as realistic as possible. This is where the Unity Game Engine presents itself as a solution: using the experience of the games industry, developers create a simulation environment. Of course, the data needed for this must first be collected, for example through test-set recordings of real road conditions. This data is collected and subsequently a model is trained with it. After segmenting and classifying the data, the model is further evaluated and fed with parameters so that a Unity model can be created. This model can be refined in collaboration with scenario designers until it meets all requirements: What should the environment look like? How much traffic, which car models, speeds, road traffic regulations, weather – the parameters are countless and should be determined in advance in as much detail as possible; the more realistic the test scenario will be. The advantages are obvious: everything can be entered and, often more importantly, also updated. Regular “DLC “s (Downloadable Content) are conceivable, analogous to the practices of the games industry, which add functions, parameters and Unity models.

Interested parties can therefore initially imagine the Unity Game Engine as an empty stage: In the first step, Unity designers create the landscape with the customer as it is required for testing. This alone is almost without alternative as a starting point for initial tests. In further steps, the vehicle models can be entered down to the last detail – every sensor can be taken into account, every function implemented and tested. This makes it possible to quickly identify the test cases relevant for testing and to optimize them more quickly. After all, the right question is also important to get a good answer, to put it casually: The more precise the requirement, the more result-rich the test. Test cases can be found faster, test data sets can be cleaned up effectively and last, but not least, the triage process, the root cause identification process, can be shortened dramatically. And every tester knows: development time is hard cash and particularly attractive if it can be shortened without sacrificing quality.

Examples of the use of the Unity Game Engine in the mobility industry

Testing, whether fully or partially automated (or, in the worst case, done manually) brings one thing with it, and that is data. This data must be classified and evaluated, and that includes troubleshooting. Classic customers in the automotive industry need an average of 5-10 days to find the cause of errors that have occurred in development. While simple errors can be found quickly, uncommon and small (but still relevant) errors are often not unlike the oft-cited search for a needle in a haystack. If a vehicle exhibits an error, the question arises as to whether the fault lies with the vehicle, the program, or the interface? Testing autonomous driving functions in a purpose-built Unity environment also requires this step – but only once. The causes of errors are entered so that the system gradually learns and can work predictively over time. Through certain algorithms such as the “One Nearest Neighbor” (which has also been successfully used in medical technology by Cognizant Mobility), the training model gradually learns to identify faults with similar patterns. At this point, we also recommend our article on the ticket pre-analysis system “HEIDI”, which has been successfully used by the data science experts around Dr. Daniel Isemann for quite some time.

Besides the automotive industry, other branches of mobility are also taking advantage of testing in the Unity Game Engine: Aircraft technology is leading the way here, as both pilot training and the entire training for the tower and similar specializations in flight operations have been handled almost entirely in virtual environments for years, mostly based on the Unity Game Engine.

This also makes sense in the automotive industry. After developing huge simulated cities, it is only a small step to add real images, as mentioned before from test landscapes (e.g. intersections or street sections) into Unity. The game becomes reality, entertainment becomes simulation in which to test. For example, a video taken in traffic can either be directly imported into the Unity system or recorded via an interface. The advantage seems immediately obvious here as well: The data can be sifted and adjusted first, and there are no more falsifications due to direct uploading. One verifies the data – which can also be, for example, raw data from vehicle occupants and their desired behavior – manipulates it as desired, and captures the scenarios after the camera has been virtualized. Thus, testing in the Unity Game Engine brings together the landscape, the vehicle model and the driver, and the most complex test scenarios are possible, in which the desired data can now be captured.

This is certainly where the challenge of the industry lies: the significant amounts of data that are being generated. Here, no test vehicle drives over a test track equipped with sensors, where the time is stopped. Test cases can be run by the thousands in the simulated virtual environment, automated tests can run any number of times, in parallel or sequentially, and can be continuously adjusted by a trained A.I. model. This results in large amounts of data (“Big Data”) that must be verified. The tests must run as completely automated as possible, repositories want to be built. It’s almost ironic that the biggest challenge of our time for autonomous driving is not a lack of data, but a surplus. Evaluating this data and optimizing it for real-world road use is one of the most important steps.

Unity Game Engine and its data – What is possible, what are the dangers?

One of the problems of testing autonomous driving features, whether inside or outside of the Unity game engine, is the lack of a standard. To date, there are no existing, real-world benchmarks, as there is no nationwide cooperation among OEMs, and many situations cannot be easily determined and cataloged. Many autonomous driving functions could already be tested, but many manufacturers do not want to give out their own data, even to cooperating market competitors. A necessary step will therefore be the involvement of the legislator and its helpers such as the TÜV.

Sure, reference scenarios already exist in some cases, such as DEKRA’s large training grounds that let cars drive autonomously without errors and virtualize their data. All the vehicle’s sensors whose data need to be recorded can thus be transferred to the model after the corresponding test vehicle drives for an hour, records all the data and sends that to the Unity interface. This makes it easy to test technical aspects. But how do we test special scenarios? The classic example, not only in autonomous driving but in artificial intelligence in general, is certainly that of the autonomous vehicle that has to decide in a dangerous situation whether to run over an old lady or a small child, since only left or right is possible. Who decides that and establishes a reference for such situations?

Also, every autonomously driving vehicle will have to be manually controlled at some point – end-to-end driving in a completely autonomous flow is more than just pie in the sky. So what happens if the driver has to intervene in 30 seconds, but his condition does not allow this because the driver has fallen asleep or lost consciousness?

These are scenarios that cannot be solved without a virtual implementation of test scenarios in game engines, since they cannot be tested in the real case. Sociologists certainly say “for ethical reasons”, but in purely mathematical terms these situations do not occur often enough in real life to be relevant for a statistical test scenario. Here, the data collections of the simulated environments provide the necessary groundwork and the test opportunity for scenarios that cannot be tested in real life – but for which autonomous driving vehicles must be absolutely prepared.

What makes Cognizant Mobility a reliable partner in automated testing in Unity Game Engine?

Already in the physics lecture one learns that there is only one variable – safety. If you don’t have security for your data, you can’t prove anything. This applies more than ever to the testing of autonomous driving functions, also, no, especially in virtual environments. The first step in successful testing is therefore always 100% data validation. Completeness, consistency, reliability of test cases – these are elementary building blocks of successful testing projects that are error-free, fully documented, always set up in the same way and repeatable. Unlike in the school math problem, there are no points for “consequential errors” in testing: if a structural break occurs, the probability of errors increases, and with it the number of bad results.

Cognizant Mobility certainly has experience in the careful handling of data and highly efficient testing. In cooperation with BMW, the company has been working for years with ECU, a tool for controlling test sequences, manufactured by the company Tracetronic. A test, such as whether a driver is braking or accelerating, can be created for any desired operation. These test cases can be connected in series and the files can always be reused, even in virtual environments. For example, Cognizant Mobility was able to qualify to work with the Unity Game Engine quite some time ago. Synergies will also continue to develop in the area of A.I.-based testing . For example, a function is conceivable that selects suitable test cases, narrows down the defects (and finds similar defects), or that could even write its own test case after a defect in the software so that it can be tested better in the future. This goes as far as complete error analysis by the A.I., which in theory could draw conclusions without the need for a repository (i.e., library). Wonderful dreams of the future, however, conceivable, realizable.

The advantage here is also obvious and runs like a thread through the entire article: Data can be collected from test cases that no longer rely on real-world scenarios in which testing is hardly possible, not possible, or at least not possible without interference. These data go into terrabytes, petabytes even, which due to their sheer mass also allow any error to occur that is only conceivable: All the more important the thorough and error-free validation of all data to make the test case efficient and meaningful. What happens when this is not the case, or when all contingencies are not considered, were the tragic accidents at Boeing. There, of course, the systems were tested according to FAA specifications, otherwise no certification would have taken place. Meanwhile, the case of a broken sensor was not planned for, so that no rescue from this situation was possible either in testing or in the real event. If the case “grandmother or child?” is not tested in the autonomous vehicle, the vehicle will not be able to decide in real operations and, in the worst case, will run over both pedestrians. It is imperative to think about these cases, and this requires a partner who not only has the competence on the development side, but also has extensive experience in the area of testing and is rooted in the automotive industry. Please feel free to contact us via our contact form at this link for first information, or subscribe to our newsletter to stay up to date with new developments.

Cognizant Mobility has been working with high-level customers from many industries in the area of testing autonomous driving functions, including in the Unity Game Engine, for many years. Among them are medical players (which we are unfortunately not allowed to name here for data protection reasons) and big names from the automotive sector from BMW to Audi, from VW to Aston Martin. We are working in the field of connected home, the acceptance of sensor technology and, of course, research in the field of autonomous driving.

Certainly – not all of it is highly complex, but an important insight is that the first step is not only necessary to move, but also to gain experience: If you don’t know what statics is, you can’t build a bridge, or in other words, if you don’t have experience in the connected home, you’re going to have a hard time with sensors in a vehicle. It always starts with a sensor, the complexity follows from the number of sensors – Connected Car is in the end the supreme discipline, where many practical tools rank, but few solution providers are present. The oft-cited “fool with a tool” who is on the wrong track but keeps hammering away is also present in autonomous testing, and to anticipate one thing: Often, these are pioneers who are urgently needed in the industry. At some point, however, solutions have to be found that do not work without standards and that have to be predefined by testing experts. Knowing that the car has to go down the road is not always enough. What kind of road are we looking at? What kind of coating? What kind of weather? What happens when someone pours nails on the road? Only with the necessary experience can the right questions be asked.

Science fiction: Will we control vehicles from the couch until they drive themselves?

It sounds incredibly tempting – until autonomous end-to-end driving is a reality, vehicles can still be controlled remotely, and virtual and extensive testing procedures ensure the necessary safety in the event of any imponderables. So do I steer my truck into the central warehouse from my cell phone, and if someone runs into the road, the truck brakes automatically? Sounds great!

A nice and practical example that not everything is pie in the sky is the “intelligent axle”, which is being developed by Daimler in the field of commercial vehicles and has been researched for a long time. The concept behind it is simple: 10 trucks are equipped and coupled with intelligent software and an electronic axle. Only the first vehicle has a driver who controls his vehicle. Following him, the other nine trucks do what the lead truck does. So, only one controller per 10 trucks is needed, automation is achieved quickly and efficiently and will be able to reach the roads in the near future. In a next step, it could be considered whether a driver is required or, similar to the “ELMO Rent” concept mentioned at the beginning, remote control is conceivable.

Similar examples concern the control of drones in a network, in which a “guide drone” also dictates what the other devices in the network imitate – an initial vision of the completely networked road, in which all vehicles accelerate identically at the same time and red traffic lights, if still necessary at all, no longer represent bottlenecks in rush hour. With the advance of virtualization, unlimited networking (5G, 6G) and time-delayed signal transmission, things will become possible that currently still sound like science fiction but will not remain so.

Automated testing of such functions in virtual environments is a great, and at the same time indispensable, preliminary step for this: Without testing in simulated realities, it will not be possible to bring these visions to the road. That’s why Cognizant Mobility continues to work and research in this area and invites you to approach us and join us in walking down a piece of the virtual road, strolling a bit towards the future by doing what we can: Developing and testing together what will be driving on the road tomorrow.

And if it’s a round of “Farming Simulator” by then, we’ll be there, of course.