Learning and Information in Stochastic Networks and Queues

AboutSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked InEmail Go to Section HomeINFORMS TutORials in Operations ResearchEmerging Optimization Methods and Modeling Techniques with Applications Learning and Information in Stochastic Networks and QueuesNeil Walton, Kuang XuNeil Walton, Kuang XuPublished Online:18 Oct 2021https://doi.org/10.1287/educ.2021.0235AbstractWe review the role of information and learning in the stability and optimization of queueing systems. In recent years, techniques from supervised learning, online learning, and reinforcement learning have been applied to queueing systems supported by the increasing role of information in decision making. We present observations and new results that help rationalize the application of these areas to queueing systems. We prove that the MaxWeight policy is an application of Blackwell’s approachability theorem. This connects queueing theoretic results with adversarial learning. We then discuss the requirements of statistical learning for service parameter estimation. As an example, we show how queue size regret can be bounded when applying a perceptron algorithm to classify service. Next, we discuss the role of state information in improved decision making. Here, we contrast the roles of epistemic information (information on uncertain parameters) and aleatoric information (information on an uncertain state). Finally, we review recent advances in the theory of reinforcement learning and queueing, as well as provide discussion of current research challenges. Video of this TutORial from the 2021 INFORMS Annual Meeting, held virtually October 24, 2021, is available at https://youtu.be/9Hep40ZuKpk. Your Access Options Login Options INFORMS Member Login Nonmember Login Purchase Options Save for later Item saved, go to cart Tutorials in OR, TutorialsNew $20.00 Add to cart Tutorials in OR, TutorialsNew Checkout Other Options Token Access Insert token number Claim access using a token Restore guest access Applies for purchases made as a guest Previous Back to Top Next FiguresReferencesRelatedInformation Emerging Optimization Methods and Modeling Techniques with ApplicationsOctober 2021 Article Information Metrics Information Published Online:October 18, 2021 Copyright © 2021, INFORMSCite asNeil Walton, Kuang Xu (2021) Learning and Information in Stochastic Networks and Queues. INFORMS TutORials in Operations Research null(null):161-198. https://doi.org/10.1287/educ.2021.0235 Keywordsqueuequeueing networklearninginformationreinforcement learningBlackwell approachabilityonline convex optimizationPDF download