Tropical geometry is a relatively recent field in mathematics and computer science combining elements of algebraic geometry and polyhedral geometry. It has recently emerged in the study of deep neural networks (DNNs) and other machine learning systems. In this talk we will first summarise introductory ideas and tools of tropical geometry and its underlying max-plus algebra. Then, we will focus on how this new set of tools can aid in the analysis, design and understanding of several classes of neural networks and other machine learning systems, including DNNs with piecewise-linear (PWL) activations, morphological neural networks, and nonlinear regression with PWL functions. Our coverage will include studying the representation power, training and pruning of these networks and regressors under the lens of tropical geometry and algebra. More information and related papers can be found in http://robotics.ntua.gr.