Glorious Alpha Two Testers!
Phase I of Alpha Two testing will occur on weekends. Each weekend is scheduled to start on Fridays at 10 AM PT and end on Sundays at 10 PM PT. Find out more here.
Check out Alpha Two Announcements here to see the latest Alpha Two news and update notes.
Our quickest Alpha Two updates are in Discord. Testers with Alpha Two access can chat in Alpha Two channels by connecting your Discord and Intrepid accounts here.
Phase I of Alpha Two testing will occur on weekends. Each weekend is scheduled to start on Fridays at 10 AM PT and end on Sundays at 10 PM PT. Find out more here.
Check out Alpha Two Announcements here to see the latest Alpha Two news and update notes.
Our quickest Alpha Two updates are in Discord. Testers with Alpha Two access can chat in Alpha Two channels by connecting your Discord and Intrepid accounts here.
Entrust us with your AI, we will make them warriors (Reinforcement learning)
Heljy
Member
First of all, I know that this idea has no place in the short term, and that it should apply to a limited field.
I have been wondering for a long time why we do not use the large player bases of MMORPGs, to apply reinforcement learning to the evolution of NPC behaviors.
Because it's not my job nor my skill, here is two (cursed) exemple of reinforcement learning :
https://www.youtube.com/watch?v=qnFCescnO1Q
https://www.youtube.com/watch?v=pJPdW8WWAso
Concretely my idea would be to create an arena system (M&B 2 Bannerlords like), where the player could face NPCs for a fee, to the delight of the people (Warrior nodes?). Only, these NPCs would have the Reinforcement Learning system, and would evolve over the fights. At the beginning of the implementation, NPCs would therefore have a Vanilla behavior, but would tend to become more complex and more unique over time.
The objective here is not to apply these new behaviors to the whole game, for performance reasons, but also because the behavior of an AI must be predictable in PVE.
On the other hand, it could be applied to certain creatures in the world, such as camp bosses and their bodyguards, or rare entities.
The interest of this method is that depending on the reward parameters given to the AI, several types of behaviors could emerge. For example:
- Aggressive: based on outdraws, proximity to target
- Protector: based on survivability of close allies
- Defensive: based on own survivability
- Evasion: based on distance to target and survivability
Etc.
I also find it very fun that players can participate in forging these behaviors, while enjoying an arena (which is always very cool)
I have been wondering for a long time why we do not use the large player bases of MMORPGs, to apply reinforcement learning to the evolution of NPC behaviors.
Because it's not my job nor my skill, here is two (cursed) exemple of reinforcement learning :
https://www.youtube.com/watch?v=qnFCescnO1Qhttps://www.youtube.com/watch?v=pJPdW8WWAsoConcretely my idea would be to create an arena system (M&B 2 Bannerlords like), where the player could face NPCs for a fee, to the delight of the people (Warrior nodes?). Only, these NPCs would have the Reinforcement Learning system, and would evolve over the fights. At the beginning of the implementation, NPCs would therefore have a Vanilla behavior, but would tend to become more complex and more unique over time.
The objective here is not to apply these new behaviors to the whole game, for performance reasons, but also because the behavior of an AI must be predictable in PVE.
On the other hand, it could be applied to certain creatures in the world, such as camp bosses and their bodyguards, or rare entities.
The interest of this method is that depending on the reward parameters given to the AI, several types of behaviors could emerge. For example:
- Aggressive: based on outdraws, proximity to target
- Protector: based on survivability of close allies
- Defensive: based on own survivability
- Evasion: based on distance to target and survivability
Etc.
I also find it very fun that players can participate in forging these behaviors, while enjoying an arena (which is always very cool)
0
Comments
The idea of PvE content isn't to make it so hard that it is impossible. If that were the goal, developers can already do it.
To me, this is something that would be a great idea as a stand alone game, but I just haven't yet seen the point of it in an MMORPG.
there is already something similar in place. also you don't really need any machine learning techniques for that.
it could also get out of control, to the point where players cant even defeat the enemies. however, that can be adjusted.
That would be cool!
Corrupted players to be hunted by NPC bounty hunters, even a zerg of them if players defend as zergs.
And world bosses to really destroy the world. Like a real boss.
you can also do that without machine learning T_T
if they are useless, its because the programmers made them like that. an npc can still be useless if it uses machine learning...
you can add interesting behaviour without machine learning and its faster to do so. the only thing machine learning could help with, would be predicting the next user input(which is already done to some extent without machine learning, this is mostly to fight ping) and reacting accordingly. but because computers are faster than human hands, you can simply wait for the user input and then react accordingly with whatever action the programmer wants.
try playing one of the old mortal combat games in snes in the hardest difficulty. this demonstrates what I just said perfectly, how the ai will just counter you without needing to predict your moves. the game basically just changes the order in which the inputs are resolved, prioritizing the CPU "inputs" over the player.