Notifications
Article
SpaceY
Updated 6 years ago
6.4 K
16
0
SpaceY
This project demonstrates how machine learning algorithm can be used to teach a rocket to take off from one planet, then fly and land on another one.
The following video demonstrates the final result (builds are attached at the end of this article):
Simulation Description
- Game area is a rectangular area that contains two planets - Planet 1 and Planet 2.
- Planet 1 is randomly positioned in a small central part of the game area, Planet 2 is placed randomly in the remaining space.
- Planets have gravity fields, they are denoted by circles around the planets.
- At the beginning of each session rocket is located on Planet 1.
- Rocket has six engines that apply different accelerations (they are denoted by red arrows in the picture below) at different positions.
Success \ Goal
The goal is to take off the rocket from Planet 1 and to land it on Planet 2. Landing is considered successful when both landing legs touch Planet 2.
Failures
The following situations are considered as failures:
- Rocket hits any of the planets with its hull.
- Rocket moves out of the game area.
- Rocket starts flying too fast (faster then some predefined constant). This condition is applied to ML-Agent simulation, when rocket is controlled manually by a player this condition is removed.
- Rocket starts rotating to fast (faster then some predefined constant). This condition is also applied only to ML-Agent simulation, when rocket is controlled manually by a player this condition is also removed.
Training
ML-Agent's brain uses discrete action space type - 6 actions are used for different engines, and one action is a "do nothing" action.
The training was performed in a few separate steps:
- The model was trained on a medium size field without curriculum learning until it reached a good enough success rate.
- Then the training was continued using curriculum learning - on each step the size of the game area was increased until all curriculum learning lessons were completed.
- After that the training was continued with modified Hyperparameters ("beta" was reduced).
After these training sessions were conducted, the algorithm was able to achieve ~91-93% success rate on a large game field.
Manual mode
Simulation can be switched to manual control mode by pressing "Brain: ML-Agent" button at the top of the screen. In the manual mode use the following keys to control the rocket:
- "Q" - upper left side engine;
- "E" - upper right side engine;
- "A" - bottom left side engine;
- "D" - bottom right side engine;
- "S" - main bottom engine;
- "W" - top stop engine.
Conclusion
The training was performed on an old mobile Geforce GTX 460M graphics card and it was rather slow. It is obvious that the above-described training process can be improved in many ways.
Unity Machine Learning tools open wide opportunities for game developers.
References
- Source code
- Demo video
You can download Windows and Mac builds here:
- SpaceY_Windows_ver.0.03.zip
- SpaceY_Mac_ver.0.03.app.zip
