High-contrast imaging instruments are today primarily limited by non-common path aberrations appearing between the scientific and wavefront sensing arms. These aberrations can produce quasi-static speckles in science images that are difficult to distinguish from exoplanet signatures. With the help of recent advances in deep learning, we have developed in previous works a method that implements convolutional neural networks to estimate pupil-plane phase aberrations from point spread functions (PSF). Here we take it a step further by incorporating into the deep learning architecture the optical propagation occurring inside the instrument. The motivation behind this is to give physical meaning to the models and to improve their robustness to various conditions. We explore how an autoencoder architecture that contains a differentiable optical simulator as the decoder can be used for that task. Because this unsupervised learning approach reconstructs the PSFs, it is not required to know the true wavefront aberrations in order to train the models, which is particularly promising for on-sky applications. In our simulator, we consider using a vortex coronagraph for high-contrast imaging, and we also exploit its properties to provide an alternative type of phase diversity with a 100% science duty cycle.