The paper presents an end-to-end learning-based autonomous driving system designed solely with a neural network through supervised learning, which has been deployed on a model vehicle equipped with a LiDAR. Input of the convolutional neural network are the point clouds from the LiDAR and outputs are the requested drive torques and steering angles. For the training of the neural network, the required sensor and actuator data were recorded by remotely controlling the model vehicle. With supervised learning, the end-to-end neural network learns the safety-relevant rules only implicitly through examples in the training data. In order to ensure the safety of model vehicle controlled by the neural network, the paper introduces a software safety architecture based on runtime monitoring. Once the runtime monitoring component detects a violation of any predefined safety rule, it will select an appropriate strategy in order to transfer the vehicle into a safe state.