Greetings, glorious adventurers! If you're joining in our Alpha One spot testing, please follow the steps here to see all the latest test info on our forums and Discord!

Introducing True AI for next-level boss fights & PvE

Hi Everyone

I really like playing PvE. But i do find that the monsters are kinda static & predictable. So i was thinking.. How can we improve this? With Artificial Inteligence or Machine Learning this is totally possible today. With upgraded programming languages and new dedicated hardware for machine learning (NVIDIA Tensor cores, AMD Radeon Instinct). It is possible to write the MIND of a creature and let it act on its own. This would give a real dynamic feeling and a lot of diversification.

Today, we do not really have AI's in game. A certain mob has a set of predefined code with a lot of parameters and attributes to calculate what the next move will be. Some games like League of legends and Civilization 6 may say they have AI, but I hardly doubt that. You could call it "a very smart Bot". But thats it.

Now what defines an AI? It is something that is 'aware' off itself. It is aware it has arms and legs, and it could learn to walk by trial & error. An AI at it's earliest stage is an empty head and it has the capacity to learn along the way. It stores the information & iterates the action to get a reward (desired result). It will reach the result because it has learned doing it himself, not because we coded it that way.

An example: Dungeon boss, Location: on top of a volcano.
It is a dragon. It has 2 feet, 2 arms with claws, it has wings, a tail,  energy, can breath fire, has a certain hight etc.. It could gain energy over time and when it dmg's enemies.
So it learnes by doing, Trial and error. But it needs a reward system. ex: The dragon needs to feel good. You could implement the following to adjust 'the feelings':

It can scratch enemies (+3), hide behind a rock (+1), fly (+1), regenerate (+1), create dust (+1), receive dmg (-3), below 75% HP (-5), below 50HP (-5). Breath fire (use energy, multi-player dmg, +7), when it leaves its territory (-100 u don't want it leaving the volcano, it's a raid boss), kill enemy (+10) ETC..  Now the dragon will make it's own decisions what it will do next based on the rewards it might receive. depending on the experience off the AI

Im sugesting raid bosses because it is easy to implement, just 1 creature. In the end you can do it for general mobs as well.

Please follow the next links to learn more about AI in games. Deep Q learning, reïnforcement learning and more. Let your imagination run wild and see the possibilities. I truly believe this will be huge in the gaming industry!

https://youtu.be/A5eihauRQvo
https://youtu.be/HBAUeJkFMH0
https://youtu.be/79pmNdyxEGo

Thanks for taking the time to read this. I look forward to your responses. What do you think?

Cheers

«1

Comments

  • ArchivedUserArchivedUser Guest
    edited September 2017
    Even if we assume this would really work (without any issues), one problem I see with this is the period until mobs would learn.

    Players who would play in this period would fight "dumb" mobs and bosses, and gain unfair advantage. Players who would join the game later on would have much more difficult time, and would have hard time reaching the spot where endgame players are, players who used easy path to gear up and do everything.

    There would be a big gap between new players and old due to this.

    And as time would move on, mobs would continually become better and smarter, and each new generation of players that would join the game would have even harder time, and gap between player generations would become higher and higher.
  • ArchivedUserArchivedUser Guest
    edited September 2017
    @Gothix
    nah that wouldn't be a problem, the ai would be pre trained by developers and its development will be frozen at some point.
    I recently found a paper about hindsight experience replay, that might improve the required time to train the mob by alot!
    https://arxiv.org/pdf/1707.01495.pdf
    (in summary: it enables the ai to learn from mistakes//usualy ai can only learn if it did something right and its reward score increases)

    The real issue would be the constant required computing power for every mob on every server i guess? Because computing a ai is much more demanding than narrow ki with just a handful of set rules...
  • leamese said:

    Please follow the next links to learn more about AI in games. Deep Q learning, reïnforcement learning and more. Let your imagination run wild and see the possibilities. I truly believe this will be huge in the gaming industry!
    Thanks for posting this. I have also posted similar discussions before and completely agree with you. Deep learning technology is not only cheap to implement but it is becoming increasingly plug-and-play.

    While Intrepid haven't said anything specific, what Steven did say about AI sounds promising.
    Source: DJ Tech Live 5/5/17 Livestream
  • Gothix said:
    Even if we assume this would really work (without any issues), one problem I see with this is the period until mobs would learn.

    Players who would play in this period would fight "dumb" mobs and bosses, and gain unfair advantage. Players who would join the game later on would have much more difficult time, and would have hard time reaching the spot where endgame players are, players who used easy path to gear up and do everything.

    There would be a big gap between new players and old due to this.

    And as time would move on, mobs would continually become better and smarter, and each new generation of players that would join the game would have even harder time, and gap between player generations would become higher and higher.
    The deep learning neural networks can be tuned to ensure the AI skill remains within acceptable parameters. The best time to do this will of course be in the alpha and beta phases, however training data can be continuously provided to the AI from the live servers to ensure that AI controlled entities achieve the best balance with the player base.

    In a way, one the deep learning systems are in place, keeping them updated and relevant would require less effort than reprogramming traditional algorithmic based AI.

    Intrepid are in discussions with Google about hosting Ashes of Creation on the Google Cloud Platform. That platform offers a sophisticated deep learning engine called the Google Cloud Machine Learning Engine. This is based on their leading edge tensor flow technology.
  • but im not even sure if people would want a smart ai within the game...
    It would probably ignore the tank, 
    position himself behind the players with most dmg output 24/7
    or focus healers.

    The real improvement would come through greater skill pools for mobs,
    if the mob only has 2 attacks he wont be abel to do something else with a smart ai either.
    With smart ai the implementation of dodging/blocking/positioning/cc skills for mobs would add a whole new difficulty layer however.
  • Uao said:
    but im not even sure if people would want a smart ai within the game...
    It would probably ignore the tank, 
    position himself behind the players with most dmg output 24/7
    or focus healers.

    The real improvement would come through greater skill pools for mobs,
    if the mob only has 2 attacks he wont be abel to do something else with a smart ai either.
    With smart ai the implementation of dodging/blocking/positioning/cc skills for mobs would add a whole new difficulty layer however.
    In Ashes, a tank really does have to block incoming attacks, melee really do need to have good armor and avoidance skills and ranged classes actually need to stay at range in case the boss turns on them. This is exactly the approach you need for PvP.

    Smarter bosses and mobs would in general have to follow the same game physics, so I wouldn't expect too many encounters where a boss could just plough through the tank(s) straight to the healers or DPS

    Caravans, sieges and open world dungeons and raids, are all PvX encounters, meaning that PvP and PvE combat is happening at the same time. It would be very difficult to have two entirely different rulesets apply to combat when both PvP and PvE combat are happening at the same time. 

    Having mob behavior closer to human behavior will mean that skills, gear and stats won't have to compensate as much for PvE combat against "dumb" or predictable bosses and mobs. This could actually mean less work tuning game systems in the long run.
  • ArchivedUserArchivedUser Guest
    edited September 2017
    Uao said:
    but im not even sure if people would want a smart ai within the game...
    It would probably ignore the tank, 
    position himself behind the players with most dmg output 24/7
    or focus healers.

    The real improvement would come through greater skill pools for mobs,
    if the mob only has 2 attacks he wont be abel to do something else with a smart ai either.
    With smart ai the implementation of dodging/blocking/positioning/cc skills for mobs would add a whole new difficulty layer however.
     :D This is exactly what I was thinking. Eventually after they learned to nuke the healers first.

    One day you would step into the raid instance to find every mob waiting at the front door and gank you during the loading screen.

    Or corpse camping us at graveyards.
  • ArchivedUserArchivedUser Guest
    edited September 2017
    @lexmax
    I was talking about mobs that get the ability to use dodging skills etc.
    Ofc players have these anyway.

    Edit: I prob misread ur point.
    I agree that it would decrease work in the long run, especialy if they buy google's solutions.
    But mob behavior thats closer to human will still be similar to usual ki if the mob only has its auto attack and maybe 1-2 special ability as in most games.
    Especialy for tab target heavy combat, since it makes positioning less important..

    @BlackCat75
    That would be funny/horrible  :D
    But these extreme situations can be prevented by adding appropriate rules/negative rewards.
  • Really hoping for thorough AI inside the game.
    Intelligent mobs are a huge step towards a living world, that can keeps us on our toes, and at the same time make the usage of bots really hard.
  • So Intrepid Studio instead of Cyberdyne Systems!

  • But yeah, I do agree, I would also love smart enemies in Ashes. Any way Intrepid decides to implement this.

    So the content is challenging and tactical and not just mind numbing repetitive clicking of few skills without any further thought given into it.
  • ArchivedUserArchivedUser Guest
    edited September 2017
    Nice thread.  Nice thought.

    Do you think we should also have a "tricking/deceiving" system for the same AI Bosses, like luring them into a point where the boss mob is forced to attack with their AOE(for example), because the boss will "think" it'll be more efficient to attack with AOE, so we can gain advantage out of that. Or should it be just too unpredictable no matter what we try, thus blocking every upcoming move instantly without a plan?
  • It should be like playing a chess VS super computer.
  • I don't think the boss starting to focus healers is a bad thing though. That would cause healers to gear up and make them become more self-reliant instead of being this little gem that needs to be protected at all costs.
  • @HeckinBamboozled
    Well thats not necessarily bad, but why would someone play a tank if the boss attacks the healer anyway. And intrepid want tanks to be a thing.

    Personaly i dont like the general concept of a tank anyway.
    All classes should avoid incoming dmg by dodging/parry/etc.
    There could be classes that endure some more dmg but not too diffrent.

    Because forcing the enemy to only attack the only player with super armor(who is also the only player that doesnt represent any threat towards the enemy) is a weird game mechanic that is totaly illogical and imo doesnt provide fun either.
  • ArchivedUserArchivedUser Guest
    edited September 2017
    @Uao I get what you mean. It's just that sometimes, tanks can die as well or make a mistake and that shouldn't neccessarily cause the complete downfall of the group but that situation should be able to be saved. That's why I prefer healers who can also take a bit of damage, so they can actually efficiently revive everyone instead of being a 1HKO.
    Because you also brought up the logic aspect - many games have healing aggro, which depending on the game can actually amount to quite a lot. That also makes more sense logically since they're supposed to provide life-saving support. An AI might figure that out as well.
    If the tank can just keep aggro all the time without much effort, that defeats the thrill.
  • @HeckinBamboozled
    Ofc tanks can die aswell, but the ai will figure out that its best chances to survive are somehow like:
    attack healer that is about to cast revive>attack low health player>attack player with lowest armor>attack healer>=attack highest dps>position behind highest dps>= run away > attack random dps>>attack tank.
    So as long as you dont physicaly prevent the ai from doing anything but attacking the tank, it wouldn't do so..  :D 

    healing aggro doesnt really solve the problem: tank aggro remains illogical even if there is a logical reason to healing aggro.


    As i think about it... true AI would turn pve into pvp similar fights so tanks will be useless unless you put in another reward function to reward dmging the tank... but why even create a smart ai in the first place if it has to do the same thing dumb ki would do anyway ...

    Classic tanks are just doomed in the future?
  • @Uao The AI would probably still be limited by stats, though, so you still have the chance of gearing up enough to survive a smart boss. But yeah, if it gets too intelligent, then one simple mistake by anyone can cause trouble with the boss. It's a risky concept.
  • Just because the AI got upgraded doesn't mean the tank role becomes useless and can't exist. You just need to change tanks so that their is more to them then the magical threat meter. Give them ways to actually defend their team, control their targets movements, and punish their enemies for not attacking them. 
  • @McStackerson
    Thats exactly why i said 'classic' tanks ;) 
  • Uao said:

    Classic tanks are just doomed in the future?
    In Ashes we have action targeting and player/entity collision and a PvX environment; where it will not be uncommon for players to be engaged in PvP at the same time they are fighting a PvE battle. Tanks and other classes in Ashes will undoubtedly have to play PvE differently than other MMOs, even if we have no advanced AI at all.

    Adding human-like AI behavior into the above, probably won't change class roles and playstyles far beyond what is already necessary for PvP.
    • Tanks will need to be well armored, physically block attacks (Bulwark), taunt (Hatred, Impale), silence/stun (Shield Might), mitigate damage to other players (Cover), etc. I think tanks in Ashes will have a similar necessary role in both PvP and PvE.

    • Melee classes will also need excellent armor and will need to use their defensive abilities, because smart AI bosses will look for the weakest targets to attack. Melee in Ashes will likely play a role in blocking attacks directed towards ranged classes.

    • Ranged classes will actually need to stay at range from their enemies. Distance and physical barriers created by spells or simply body blocking from the melee classes will be significant, just like in PvP. Ranged will also need to be vigilant of the boss or other mobs attacking them directly; using their mobility (Mage's Escape, Fleeting Shot) and defensive skills (After Image, Ice Prison, Snare) to avoid danger. 
    To me, this is a way more immersive style of combat for all of the classes, rather than solely memorizing timing and ability sequences of "dumb" AI bosses and mobs in other MMORPGS.
  • Dave Mark is the person to follow for game AI. The now defunct Storybricks was also the next big step for AI in mmorpgs.  Sadly so far as we know, nobody has picked up the baton.

    http://www.gdcvault.com/play/1012410/Improving-AI-Decision-Modeling-Through 
  • I'd love all AoEs to be lethal. Meaning they all be one shot kills for all players (including tanks).


    I would do a combination of classic and modern.

    Boss should do some "basic attacks" towards target with AGRO, this target would (ideally) be tank, tank would be armored and able to survive this basic attacks if, of course, he is been healed by capable healer.

    In addition to this basic attacks (classic tanking mode) boss should do AoEs on other targeted areas, some AoEs to predetermined areas, some AoEs to areas where most real threat to him lies "most dangerous DPSers, healers etc".

    All this AoEs would have graphs drawn and all players would have limited time to exit the danger area where AoE would soon strike. All AoEs would be 1 shot kills (even for tanks).

    So raid that attacks boss should have all players knowing to play their role (tank, heal, DPS) in classic sense (be tanky enough, heal enough,do enough DPS). In addition to that all players would need to avoid predetermined AoEs (know tactics and movement paths), and also all players would need to have good reaction / instantaneous thinking, by reacting to un-predetermined boss AoEs, that boss choses to cast at current area of interest.

    This, plus being able to react well enough to do combos that Intrepid already introduced (and hopefully with reworked "timer bar") would ensure game is sufficiently skill based, and interesting enough.

    Perhaps add some dynamic boss random moves that would play a bit differently in each time you face him, and fights would hardly become boring soon.

    Added that some of these bosses would be in open world dungeons with players hopefully auto flagged for PvP (like in sieges), and you would have plenty of fun, that would persist through time.
  • Gothix said:
    I'd love all AoEs to be lethal. Meaning they all be one shot kills for all players (including tanks).


    I would do a combination of classic and modern.

    Boss should do some "basic attacks" towards target with AGRO, this target would (ideally) be tank, tank would be armored and able to survive this basic attacks if, of course, he is been healed by capable healer.

    In addition to this basic attacks (classic tanking mode) boss should do AoEs on other targeted areas, some AoEs to predetermined areas, some AoEs to areas where most real threat to him lies "most dangerous DPSers, healers etc".

    All this AoEs would have graphs drawn and all players would have limited time to exit the danger area where AoE would soon strike. All AoEs would be 1 shot kills (even for tanks).

    So raid that attacks boss should have all players knowing to play their role (tank, heal, DPS) in classic sense (be tanky enough, heal enough,do enough DPS). In addition to that all players would need to avoid predetermined AoEs (know tactics and movement paths), and also all players would need to have good reaction / instantaneous thinking, by reacting to un-predetermined boss AoEs, that boss choses to cast at current area of interest.

    This kinda sounds like how Wildstar was before they went free to play and nerfed everything. Getting hit by an enemy telegraph in that game, particularly in dungeons and raids, was almost a guaranteed death for the player. "Basic attacks" also hit pretty hard too requiring Healers to give Tanks almost constant attention, so good dps learned to evade and dodge properly so as not to over stress the healer. I loved Wildstar's dungeons pre-nerf. 
  • Putting all Wildstars problems aside, I can say that playing that game was pretty fun experience for me. I have plenty nice memories.
  • @Gothix
    Tera still has similar pve combat and its by far the best ive ever seen.
  • Uao said:
    @Gothix
    Tera still has similar pve combat and its by far the best ive ever seen.
    Ah, I've never played Tera.

    I loved telegraphs in Wildstar and in original Secret World combat, it took some skills to win the fights and not just mind numbing key smashing. :) That's why i mentioned it.
  • Uao said:
    but im not even sure if people would want a smart ai within the game...
    It would probably ignore the tank, 
    position himself behind the players with most dmg output 24/7
    or focus healers.

    The real improvement would come through greater skill pools for mobs,
    if the mob only has 2 attacks he wont be abel to do something else with a smart ai either.
    With smart ai the implementation of dodging/blocking/positioning/cc skills for mobs would add a whole new difficulty layer however.
    Storybricks or similar Tagging system seems like it would be great.
    Too much work for Ashes, but we might be getting a simplified implementation of the SOE version of StoryBricks.

    AI wouldn't be able to ignore the Tank any more than players can ignore a Tank's threat abilities.
    AI should ideally have the same ranges of abilities as players.
  • @Gothix
    The thing that bothers me with wildstar is that they used these telgraphs way too much. Its more of a replacement for proper animations smh.

    In tera you had these telegraphs for most boss attacks in normal mode and in hard mode you simply die if you dont pay attention to the actual animation of the boss.

    Also tera has a ton of i-frame and dodging skills that add alot to the gameplay while wildstar only has this small ability bar with ~11 skills.

    But no need to start playing tera, they successfully made every possible other part of the game so bad that no sane person would want to play the game for a long time..  :'(
  • leamese said:
    An example: Dungeon boss, Location: on top of a volcano.
    It is a dragon. It has 2 feet, 2 arms with claws, it has wings, a tail,  energy, can breath fire, has a certain hight etc.. It could gain energy over time and when it dmg's enemies.
    So it learnes by doing, Trial and error. But it needs a reward system. ex: The dragon needs to feel good. You could implement the following to adjust 'the feelings':

    It can scratch enemies (+3), hide behind a rock (+1), fly (+1), regenerate (+1), create dust (+1), receive dmg (-3), below 75% HP (-5), below 50HP (-5). Breath fire (use energy, multi-player dmg, +7), when it leaves its territory (-100 u don't want it leaving the volcano, it's a raid boss), kill enemy (+10) ETC..  Now the dragon will make it's own decisions what it will do next based on the rewards it might receive. depending on the experience off the AI
    Isn't this just a regular fitness function?  I wouldn't be surprised if most games are using something similar for their mobs.  You're also missing the important negative cost of becoming Skynet.
    Uao said:
    The thing that bothers me with wildstar is that they used these telgraphs way too much. Its more of a replacement for proper animations smh.
    Removing the ambiguity is a good thing though.  When your raid wipes in Wildstar, you know exactly what bad stuff you stood in.  I cringe every time I watch the PAX PvE demo, the only way the tank knows he's out of the boss's breath is that he isn't taking damage from it.  A proper telegraph would make things absolutely clear.
    Uao said:
    Also tera has a ton of i-frame and dodging skills that add alot to the gameplay while wildstar only has this small ability bar with ~11 skills.
    No iframes in Wildstar, knowing exactly where things will happen because of the telegraphs let you avoid damage by actually avoiding it, and crafting a skill set to maximize your damage/threat and tankiness/healing while bringing the required utility is part of the appeal.



Sign In or Register to comment.