Recently an incredible progress has been achieved in the hardware implementation of artificial neural networks utilizing resistive switching memory (RRAM) technology relying on the voltage induced formation and degradation of conducting filaments within an insulator matrix. As an example, 128x64 memristor crossbar arrays were built and successfully applied for efficient image processing and machine learning tasks. The active volume of such devices is much smaller than in conventional semiconductor architectures, it can even reach the ultimate, atomic size-scales. In the talk I will review the recent progress in RRAM research, along with our results on the physical understanding of atomic-sized artificial synapses.