Passer à la navigation principale Passer à la recherche Passer au contenu principal

Hierarchical Cooperative Multi-Agent Reinforcement Learning with AlphaZero Skill Execution

  • Université de Liège

Résultats de recherche: Chapitre dans un livre, un rapport, des actes de conférences › Contribution à une conférence › Revue par des pairs

Résumé

Cooperative multi-agent reinforcement learning (MARL) for team-level tactical problems requires agents that can both discover reusable behavioral abstractions (skills) and execute each skill with primitive actions. We present \emph{AZ-HMARL}, a hierarchical framework that assigns discrete skills via QMIX and executes them with a per-agent AlphaZero planner operating on real environment dynamics. To prevent skill collapse, we derive an intrinsic reward that maximizes a variational lower bound on the mutual information $\mathrm{MI}(z, \tau)$ between skills and trajectories. A segment-level decoder $p_\phi(z|\tau)$ is trained to predict the assigned skill $z$ from the observed trajectory $\tau$, and its log-probability serves as the intrinsic reward.

Integrating this segment-level reward with AlphaZero introduces a structural reward inconsistency: the decoder reward is unavailable during MCTS tree search, creating a mismatch between the tree backup and the value-head training target. We formalize this inconsistency through a value consistency gap metric and propose a step-level entropy proxy that is fully MCTS-consistent. Experiments on several SMAX scenarios demonstrate that the entropy proxy achieves superior win rates and lower value inconsistency compared to the decoder baseline, though with reduced skill discriminability. In heterogeneous teams, we observe emergent role specialization where different unit types execute different skills. This arises purely from the intrinsic reward signal without explicit role supervision.
langue originaleAnglais
titreMulti-Agent Systems - 23rd European Conference, EUMAS 2026, Proceedings
Sous-titre23rd European Conference, EUMAS 2026, Malmö, Sweden, September 23-25, 2026, Proceedings
rédacteurs en chefFabian Lorig, Paul Davidsson, Franziska Klügl, Valérie Camps, Juan Carlos Nieves Sanchez, Elsy Kaddoum
EditeurSpringer Science and Business Media Deutschland GmbH
Pages54-72
Nombre de pages19
VolumeLNCS Volume 16829
ISBN (Electronique)978-3-032-39395-1
ISBN (imprimé)978-3-032-39394-4
Les DOIs
étatPublié - 23 sept. 2026
Evénement23rd European Conference on Multi-Agent Systems, EUMAS 2026 - Malmo, Sucde
Durée: 23 sept. 2026 → 25 sept. 2026

Série de publications

NomLecture Notes in Computer Science
Volume16829 LNCS
ISSN (imprimé)0302-9743
ISSN (Electronique)1611-3349

Une conférence

Une conférence23rd European Conference on Multi-Agent Systems, EUMAS 2026
Pays/TerritoireSucde
La villeMalmo
période23/09/26 → 25/09/26

Empreinte digitale

Examiner les sujets de recherche de « Hierarchical Cooperative Multi-Agent Reinforcement Learning with AlphaZero Skill Execution ». Ensemble, ils forment une empreinte digitale unique.

Contient cette citation