Memristors are two-terminal memory devices that can change the conductance state and store analog values. Thanks to their simple structure, suitability for high-density integration, and non-volatile characteristics, memristors have been intensively studied as synapses in artificial neural network systems. Memristive synapses in neural networks have theoretically better energy efficiency compared with conventional von Neumann computing processors. However, memristor crossbar array-based neural networks usually suffer from low accuracy because of the non-ideal factors of memristors such as non-linearity and asymmetry, which prevent weights from being programmed to their targeted values. In this article, the improvement in linearity and symmetry of pulse update of a fully CMOS-compatible HfO2-based memristor is discussed, by using a second-order memristor effect with a heating pulse and a voltage divider composed of a series resistor and two diodes. We also demonstrate that the improved device characteristics enable energy-efficient and fast training of a memristor crossbar array-based neural network with high accuracy through a realistic model-based simulation. By improving the memristor device’s linearity and symmetry, our results open up the possibility of a trainable memristor crossbar array-based neural network system that possesses great energy efficiency, high area efficiency, and high accuracy at the same time.