Entrust us with your AI, we will make them warriors (Reinforcement learning)

First of all, I know that this idea has no place in the short term, and that it should apply to a limited field.

I have been wondering for a long time why we do not use the large player bases of MMORPGs, to apply reinforcement learning to the evolution of NPC behaviors.
Because it's not my job nor my skill, here is two (cursed) exemple of reinforcement learning :

https://www.youtube.com/watch?v=qnFCescnO1Q

https://www.youtube.com/watch?v=pJPdW8WWAso

Concretely my idea would be to create an arena system (M&B 2 Bannerlords like), where the player could face NPCs for a fee, to the delight of the people (Warrior nodes?). Only, these NPCs would have the Reinforcement Learning system, and would evolve over the fights. At the beginning of the implementation, NPCs would therefore have a Vanilla behavior, but would tend to become more complex and more unique over time.

The objective here is not to apply these new behaviors to the whole game, for performance reasons, but also because the behavior of an AI must be predictable in PVE.
On the other hand, it could be applied to certain creatures in the world, such as camp bosses and their bodyguards, or rare entities.

The interest of this method is that depending on the reward parameters given to the AI, several types of behaviors could emerge. For example:
- Aggressive: based on outdraws, proximity to target
- Protector: based on survivability of close allies
- Defensive: based on own survivability
- Evasion: based on distance to target and survivability
Etc.

I also find it very fun that players can participate in forging these behaviors, while enjoying an arena (which is always very cool)

Find more posts tagged with

Comments

Noaani

The question I have for all suggestions like this is; why?

The idea of PvE content isn't to make it so hard that it is impossible. If that were the goal, developers can already do it.

To me, this is something that would be a great idea as a stand alone game, but I just haven't yet seen the point of it in an MMORPG.

Korela

wow ai so cool!! poggers!! i am BatChesting everywhere!! hype!!!!11

Dygz

Shitty McShitface

Depraved

Heljy wrote: »

First of all, I know that this idea has no place in the short term, and that it should apply to a limited field.

I have been wondering for a long time why we do not use the large player bases of MMORPGs, to apply reinforcement learning to the evolution of NPC behaviors.
Because it's not my job nor my skill, here is two (cursed) exemple of reinforcement learning :
https://www.youtube.com/watch?v=qnFCescnO1Q
https://www.youtube.com/watch?v=pJPdW8WWAso

Concretely my idea would be to create an arena system (M&B 2 Bannerlords like), where the player could face NPCs for a fee, to the delight of the people (Warrior nodes?). Only, these NPCs would have the Reinforcement Learning system, and would evolve over the fights. At the beginning of the implementation, NPCs would therefore have a Vanilla behavior, but would tend to become more complex and more unique over time.

The objective here is not to apply these new behaviors to the whole game, for performance reasons, but also because the behavior of an AI must be predictable in PVE.
On the other hand, it could be applied to certain creatures in the world, such as camp bosses and their bodyguards, or rare entities.

The interest of this method is that depending on the reward parameters given to the AI, several types of behaviors could emerge. For example:
- Aggressive: based on outdraws, proximity to target
- Protector: based on survivability of close allies
- Defensive: based on own survivability
- Evasion: based on distance to target and survivability
Etc.

I also find it very fun that players can participate in forging these behaviors, while enjoying an arena (which is always very cool)

there is already something similar in place. also you don't really need any machine learning techniques for that.

it could also get out of control, to the point where players cant even defeat the enemies. however, that can be adjusted.

Otr

Depraved wrote: »

it could also get out of control, to the point where players cant even defeat the enemies. however, that can be adjusted.

That would be cool!
Corrupted players to be hunted by NPC bounty hunters, even a zerg of them if players defend as zergs.
And world bosses to really destroy the world. Like a real boss.

Heljy

IMO that's the point. As an exemple, NPCs in PVP are always useless, unless they have huge HP & damage. But some interesting behaviors could make them more dangerous, without abused statistics.

Depraved

Otr wrote: »

Depraved wrote: »

it could also get out of control, to the point where players cant even defeat the enemies. however, that can be adjusted.

That would be cool!
Corrupted players to be hunted by NPC bounty hunters, even a zerg of them if players defend as zergs.
And world bosses to really destroy the world. Like a real boss.

you can also do that without machine learning T_T

Heljy wrote: »

IMO that's the point. As an exemple, NPCs in PVP are always useless, unless they have huge HP & damage. But some interesting behaviors could make them more dangerous, without abused statistics.

if they are useless, its because the programmers made them like that. an npc can still be useless if it uses machine learning...

you can add interesting behaviour without machine learning and its faster to do so. the only thing machine learning could help with, would be predicting the next user input(which is already done to some extent without machine learning, this is mostly to fight ping) and reacting accordingly. but because computers are faster than human hands, you can simply wait for the user input and then react accordingly with whatever action the programmer wants.

try playing one of the old mortal combat games in snes in the hardest difficulty. this demonstrates what I just said perfectly, how the ai will just counter you without needing to predict your moves. the game basically just changes the order in which the inputs are resolved, prioritizing the CPU "inputs" over the player.

Otr

Depraved wrote: »

Otr wrote: »

Depraved wrote: »

it could also get out of control, to the point where players cant even defeat the enemies. however, that can be adjusted.

That would be cool!
Corrupted players to be hunted by NPC bounty hunters, even a zerg of them if players defend as zergs.
And world bosses to really destroy the world. Like a real boss.

you can also do that without machine learning T_T

Yes. I wouldn't want AoC to use AI. Just complex algorithms which the developers understand and can balance.

Depraved

nah simple > complex xd

Child Item