3 Presentations

• 
  04:00 PM - 04:30 PM

  Robust Markov Decision Processes

  • Wolfram Wiesemann, presenter, Imperial College London

  Markov decision processes (MDPs) are powerful tools for decision making in uncertain dynamic environments. However, the solutions of MDPs are of limited practical use due to their sensitivity to distributional model parameters, which are typically unknown and have to be estimated by the decision maker. To counter the detrimental effects of estimation errors, we consider robust MDPs that offer probabilistic guarantees in view of the unknown parameters. To this end, we assume that an observation history of the MDP is available. Based on this history, we derive a confidence region that contains the unknown parameters with a pre-specified probability 1-beta. Afterwards, we determine a policy that attains the highest worst-case performance over this confidence region. By construction, this policy achieves or exceeds its worst-case performance with a confidence of at least 1-beta. Our method involves the solution of tractable conic programs of moderate size.

• 
  04:30 PM - 05:00 PM

  Optimal control of Weakly Connected Markov Decision Processes

  • Panos Parpas, presenter, Imperial College London

  Weakly connected Markov processes are used to model stochastic dynamics across different scales.
  They are widely used in electrical engineering, finance and molecular dynamics simulations.
  In many of these applications it is becoming increasingly important to both efficiently simulate these processes as well as to control them.
  Classical algorithms for the control of Markov Processes cannot be directly applied to weakly connected Markov processes.
  The problem with existing approaches is that they exhibit an extremely slow convergence rate due to the existence of multiscale effects.
  We show why existing algorithms are slow, and propose a new class of algorithms with more favourable convergence properties and computational complexity.
  In our approach we use spectral graph theory to derive a hierarchy of models that are valid at different resolutions.
  We then propose a polynomial time algorithm that uses the finest resolution model only when required.
  The rate of convergence of the algorithm is discussed as well as its complexity.

• 
  05:00 PM - 05:30 PM

  General Linear Formulations of Stochastic Dominance Criteria

  • Milos Kopa, presenter, Charles University in Prague, MFF
  • Thierry Post, Koc University, Istanbul, Turkey

  We develop and implement linear formulations of general N-th order Stochastic Dominance criteria for discrete probability distributions. Our approach is based on a piece-wise polynomial representation of utility and its derivatives and can be implemented by solving a relatively small system of linear inequalities. This approach allows for comparing a given prospect with a discrete set of alternative prospects as well as for comparison with a polyhedral set of linear combinations of prospects. We also derive a linear dual formulation in terms of lower partial moments and co-lower partial moments. An empirical application to historical stock market data suggests that the passive stock market portfolio is highly inefficient relative to actively managed portfolios for all investment horizons and for nearly all investors.