Notifications
Article
Metal Warfare - Real Time Strategy game
Updated 6 years ago
11.5 K
27
9
Metal Warfare - Real Time Strategy game, special edition for AI & ML challenging
Metal Warfare is an RTS game I designed for both entertainment and education purpose. The education will be focused on teaching player how to build their own Artificial Intelligence (AI) and challenge online with other player or AI. The final milestone of the game will let player using Machine Learning (ML) to train their AI.
As the release of Unity ML-Agents, I will have some showcases for you on how to do Machine Learning for your AI on an real time strategy game.
The game is built from simple to complex, at very first levels on the game, player will have a Base, attack enemy by "Missile Launcher" and defense by "Anti-Air":
Let have a look at the game level we selected to do training:
After checking the Unity ML-agents example and our game level, we find out that 3 things we need to add into the training model:
1. The Idle action: Almost the time, player need to wait for enough power to do the action. There is no sense we need an action for those moments and we do not need any learning in that case also.
2. Action filtering: Let say at position (1,1) on the map, you already have a building, so the action to build another building on that position should be invalid, we should filter it out. Of course, you can think of letting ML study it. But just think back, we have many other interesting parts on the game need the ML to learn right?
3. Delay rewards for the action: We have action launch a missile, we only know the result of that only when the missile come to the destination. Let make the learning smarter, we will store the action and update the rewards when the missile done his job.
Our learning model will look like this:
The states array will content 5 parts:
Action ID - rewards for previous action if any - is Idle or not - available action - learning states
The learning states will be separated and joined with the observation of the mini map for learning, Other parts of the states will be used for filtering out the actions.
The rewards for the game we decided will be:
Lose the game: -1.0f
Lose 1 building: -0.01f
Win the game: +1.0f
Plan missile to building: +0.0003f
Plan missile to fog: +0.00015f
Anti defense success 1 missile: +0.0001f
Successfully clear 1 fog: +0.001f
Successfully cause damage to an enemy building: +0.002f
Successfully destroy 1 building: +0.008f
(20% rewards will be account into the launcher building action)
This is how the game was played after some training:
On the training process, we find out an epic situation that the ML can learn: There is a time when ML nearly lose, only have 1 launcher when enemy has 1 anti and 3 launchers, but the ML notice that the enemy HQ only have 1 HP remain. It choose to put that final missile direct to enemy HQ and expect the 10% that anti will miss. Luckily, it work! here is the victory screenshot:
Ok, now let have a look at our normal game play:
So now we have the marine, the AOE missile,... and more. How the training could be? Is it amazing if the marine know how to go out of turret range and attack other building, and when there are enough number of marines, they will come together to take down the turret?
Here is my shared GitHub: https://github.com/ccthien/MetalWarfareML
Thank you for reading my article. Hope you enjoy.