So to fix it, I went to Revisiting Programming Fonts to select the fonts I wanted: Consolas on Windows, and Inconsolata on Mac. I installed those as you usually would in each respective operating system (emacs will use the system available fonts).
And then I put this in my .emacs file:
(if (string= (symbol-name system-type) "windows-nt") (set-default-font "-outline-Consolas-normal-normal-normal-mono-18-*-*-*-c-*-iso8859-1") (modify-frame-parameters nil '((wait-for-wm . nil)))) (if (string= (symbol-name system-type) "darwin") (set-default-font "-outline-Inconsolata-normal-normal-normal-mono-18-*-*-*-c-*-iso8859-1") (modify-frame-parameters nil '((wait-for-wm . nil))))
Now, I get Consolas font on Windows, and Inconsolata on Mac, and whatever was my default on Ubuntu that I didn't have to set manually!
If you don't know what fonts are available, it turns out you could find out in emacs by doing M-x set-face-font, then choose default, then hit the tab key twice to see the completion listing. Scroll through the listing until you see the font name you want. After noting down the name, just hit C-g several times to exit out of set-face-font so you could make the selection permanent by making the above adjustment in your .emacs file.