I examine the publications of Carl Neumann (1832–1925) on electrodynamics, which constitute a major part of his work and which illuminate his approach to mathematical physics. I show how Neumann contributed to physics at an important stage in its development and how his work led to a polemic with Hermann Helmholtz (1821–1894). Neumann advanced and extended the ideas of the Königsberg school of mathematical physics. His investigations were aimed at founding a mathematically exact physical theory of electrodynamics, following the approach of Carl G.J. Jacobi (1804–1851) on the foundation of a physical theory as outlined in Jacobi’s lectures on analytical mechanics. Neumann’s work also shows how he clung to principles that impeded him in appreciating and developing new ideas such as those on field theory that were proposed by Michael Faraday (1791–1867) and James Clerk Maxwell (1831–1879).