When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-states in visually rich and dynamic 3D environments. More precisely, we show that the learned representation captures the layout of the environment as well as the position and orientation of the agent. Our experiments show that the model substantially improves data-efficiency on a number of reinforcement learning (RL) tasks compared with strong model-free baseline agents. We find that predicting multiple steps into the future (overshooting), in combination with an expressive generative model, is critical for stable representations to emerge. In practice, using expressive generative models in RL is computationally expensive and we propose a scheme to reduce this computational burden, allowing us to build agents that are competitive with model-free baselines.

AIMS The long-term outcomes of biolimus-eluting stents (BESs) with biodegradable polymer as compared with bare-metal stent (BMS) in patients with ST-segment elevation myocardial infarction (STEMI) remain unknown. METHODS AND RESULTS We performed a 5-year clinical follow-up of 1157 patients (BES: N = 575 and BMS: N = 582) included in the randomized COMFORTABLE AMI trial. Serial intracoronary imaging of stented segments using both intravascular ultrasound (IVUS) and optical coherence tomography performed at baseline and 13 months follow-up were analysed in 103 patients. At 5 years, BES reduced the risk of major adverse cardiac events [MACE; hazard ratio (HR) 0.56, 95% confidence interval (CI): 0.39-0.79, P = 0.001], driven by lower risks for target vessel-related reinfarction (HR 0.44, 95% CI: 0.22-0.87, P = 0.02) and ischaemia-driven target lesion revascularization (HR 0.41, 95% CI: 0.25-0.66, P < 0.001). Definite stent thrombosis (ST) was recorded in 2.2% and 3.9% (HR 0.57, 95% CI: 0.28-1.16, P = 0.12) with no differences in rates of very late definite ST (1.3% vs. 1.6%, P = 0.77). Optical coherence tomography showed no difference in the frequency of malapposed stent struts at follow-up (BES 0.08% vs. BMS 0.02%, P = 0.10). Uncovered stent struts were rarely observed but more frequent in BES (2.1% vs. 0.15%, P < 0.001). In the IVUS analysis, there was no positive remodelling in either group (external elastic membrane area change BES: -0.63 mm2, 95% CI: -1.44 to 0.39 vs. BMS -1.11 mm2, 95% CI: -2.27 to 0.04, P = 0.07). CONCLUSION Compared with BMS, the implantation of biodegradable polymer-coated BES resulted in a lower 5-year rate of MACE in patients with STEMI undergoing primary percutaneous coronary intervention. At 13 months, vascular healing in treated culprit lesions was almost complete irrespective of stent type. CLINICAL TRIAL REGISTRATION http://www.clinicaltrials.gov. Unique identifier: NCT00962416.