A focal plane wavefront sensor offers major advantages to AO, including removal of non-common-path error and providing sensitivity to blind modes (such as island modes). But simply using the observed PSF is not sufficient for wavefront correction, as only the intensity, not phase, is measured. Here we demonstrate the use of a multimode fibre and mode converter (photonic lantern) to directly measure the wavefront phase and amplitude at the focal plane. Starlight is injected into a multimode fibre at the image plane, with the combination of modes excited within the fibre a function of the phase and amplitude of the incident wavefront. The fibre undergoes an adiabatic transition into a set of multiple, single-mode outputs, such that the distribution of intensities between them encodes the incident wavefront. The mapping (which may be strongly non-linear) between spatial modes in the PSF and the outputs is stable but must be learned. This is done by a deep neural network, trained by applying random combinations of spatial modes to the deformable mirror. Once trained, the neural network can instantaneously predict the incident wavefront for any set of output intensities. Moreover, by making the lantern partially mode selective (hybrid mode selective) the flux injected into a specific output can be optimised over a broad bandwidth, providing optimum injection into a single fibre for spectroscopy. We demonstrate the successful reconstruction of a wavefront made of randomly varying Zernike terms and Kolmogorov screen, and its ability to reconstruct low-wind-effect modes.