Dimensionality reduction is often used for visualization without considering their understanding by users. Multidimensional scaling, for instance, provides an arbitrarily-oriented visualization. However, users can be integrated into the loop to provide clues about their understanding of the visualization. In this paper, we propose an interactive probabilistic multidimensional scaling (iPMDS) approach to compute the visualization with the lowest information loss while taking the information provided by users into account. We show that a more interpretable visualization can be obtained after interacting with the visualization while keeping a good dimensionality reduction accuracy.